Digital face model construction method and apparatus, and electronic device
By acquiring facial images and template facial parameters, and combining them with a component template database and an adaptation loss function, the problem of unmodeled facial tissues such as hair, eyes, and teeth in existing technologies is solved, resulting in more realistic and vivid digital facial models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JINGDONG TECH HLDG CO LTD
- Filing Date
- 2023-02-01
- Publication Date
- 2026-04-14
AI Technical Summary
Existing methods for building digital facial models cannot effectively model facial structures such as hair, eyes, and teeth, resulting in models that are not realistic or vivid enough.
By acquiring the target face image, determining the face reconstruction parameters and template face parameters, and combining them with the component template database, a digital facial model including face-related tissues is constructed. The parameters are then optimized using a component classification network and an adaptation loss function to generate a digital facial model that conforms to human anatomy.
It achieves accurate modeling of facial features such as hair, eyes, and teeth, making digital facial models more realistic and vivid, and improving the realism and visual appeal of the models.
Smart Images
Figure CN116030196B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus and electronic device for constructing a digital facial model. Background Technology
[0002] When constructing digital facial models, mainstream optical scanners often cannot scan and reconstruct facial features such as hair, eyes, and teeth due to their unique characteristics. Therefore, commonly used 3D facial models do not model these areas.
[0003] When building digital facial models, the absence of eye models greatly limits the flexibility and freedom of eye movements; the absence of hair will cause visual incongruity; and in scenarios where digital humans need to communicate, the modeling of teeth is also indispensable.
[0004] Therefore, proposing a method for constructing a digital facial model that includes facial associations is an urgent problem to be solved. Summary of the Invention
[0005] This disclosure provides a method, apparatus, and electronic device for constructing a digital facial model, which addresses the shortcomings of existing technologies in constructing digital facial models, such as the lack of modeling of facial related tissues like hair, eyes, and teeth, resulting in digital facial models that are not realistic or vivid enough. The method and apparatus provide a digital facial model construction method that includes facial related tissues, making the digital facial model more realistic and vivid.
[0006] This disclosure provides a method for constructing a digital facial model, including:
[0007] Acquire the target face image;
[0008] Determine the face reconstruction parameters based on the target face image;
[0009] The appropriate face reconstruction parameters are determined based on the face reconstruction parameters and the preset template face parameters corresponding to the target face image.
[0010] The parameters of the target component 3D model are determined based on the target face image and a preset component template database including face-related tissues.
[0011] The target digital facial model is constructed based on the adapted face reconstruction parameters and the parameters of the target component 3D model.
[0012] According to the digital facial model construction method provided in this disclosure, the step of determining facial reconstruction parameters based on the target facial image includes:
[0013] Initial face reconstruction parameters are generated based on the target face image and a comprehensive loss is calculated, wherein the comprehensive loss includes one or more of image loss, depth loss, identity perception loss, key point error loss and regularization loss;
[0014] The initial face reconstruction parameters are optimized based on the comprehensive loss to obtain the face reconstruction parameters.
[0015] According to the digital facial model construction method provided in this disclosure, the template face parameters are obtained through the following steps:
[0016] The average face shape parameters are calculated based on multiple pre-acquired sets of face shape parameters;
[0017] Template face parameters are generated based on the average face shape parameters, preset face base parameters, and preset shape coefficients.
[0018] According to the digital facial model construction method provided in this disclosure, the step of determining the appropriate facial reconstruction parameters based on the facial reconstruction parameters and preset template facial parameters corresponding to the target facial image includes:
[0019] Based on the template face parameters, determine the template face key points and template face reference key points;
[0020] The key points of the target face are determined based on the face reconstruction parameters;
[0021] Face adaptation is performed based on the template face key points, the template face reference key points, the target face key points, and the preset adaptation loss function to obtain adapted face reconstruction parameters.
[0022] According to the digital facial model construction method provided in this disclosure, the component template database is obtained through the following steps:
[0023] Obtain the parameters of the component's 3D model;
[0024] The parameters of the component's 3D model are adapted and adjusted based on the template face parameters to obtain the parameters of the adapted component's 3D model.
[0025] The component category is labeled according to the parameters of the 3D model of the adapter component;
[0026] A component template database is generated based on the parameters of the 3D model of the adapted component and the corresponding component category.
[0027] According to the digital facial model construction method provided in this disclosure, the step of determining the parameters of the target component based on the target face image and a preset component template database including face-related tissues includes:
[0028] The target face image is preprocessed to obtain a target face component image;
[0029] The target face component image is input into a pre-trained component classification network to obtain the component category;
[0030] Match the target component 3D model to the component template database according to the component category and determine the parameters of the target component 3D model.
[0031] According to the digital facial model construction method provided in this disclosure, the adaptive face reconstruction parameters include adaptive facial key points and adaptive facial triangles, and the parameters of the target component 3D model include target component key points and target component triangles.
[0032] The step of constructing the target digital facial model based on the adapted face reconstruction parameters and the parameters of the target component 3D model includes:
[0033] The key points of the target digital facial model are determined based on the adapted facial key points and the key points of the target component.
[0034] The triangle of the target digital face model is determined based on the adapted face triangle and the target component triangle.
[0035] The target digital facial model is generated based on the key points and triangular faces of the target digital facial model.
[0036] This disclosure also provides a digital facial model building apparatus, including:
[0037] The acquisition unit is used to acquire the target face image;
[0038] A construction unit is configured to: determine face reconstruction parameters based on the target face image; determine adaptive face reconstruction parameters based on the face reconstruction parameters and preset template face parameters corresponding to the target face image; determine parameters of a target component 3D model based on the target face image and a preset component template database including face-related tissues; and construct a target digital facial model based on the adaptive face reconstruction parameters and the parameters of the target component 3D model.
[0039] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described digital facial model construction methods.
[0040] This disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the digital face model construction method as described above.
[0041] The digital facial model construction method, apparatus, and electronic device disclosed herein acquire a target face image, determine face reconstruction parameters based on the target face image, and determine adaptive face reconstruction parameters based on the face reconstruction parameters and preset template face parameters corresponding to the target face image. This achieves face adaptation between the target face and the template face. The adapted face reconstruction parameters make the subsequently generated digital facial model conform to human anatomy and be more realistic. Furthermore, the method determines the parameters of a target component 3D model based on the target face image and a preset component template database including face-related tissues. It then constructs a target digital facial model based on the adaptive face reconstruction parameters and the parameters of the target component 3D model, thus determining the target component 3D model corresponding to the target face. Finally, it constructs a digital facial model including face-related tissues based on the parameters of the target component 3D model and the adaptive face reconstruction parameters, making the digital facial model more realistic and vivid. Attached Figure Description
[0042] To more clearly illustrate the technical solutions in this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0043] Figure 1 This is a flowchart illustrating the digital facial model construction method disclosed herein;
[0044] Figure 2 This is a training diagram of the component classification network provided in this publication;
[0045] Figure 3 This is a schematic diagram of the digital facial model construction process disclosed herein;
[0046] Figure 4 This is a schematic diagram of the structure of the digital facial model building device disclosed herein;
[0047] Figure 5 This is a schematic diagram of the structure of the electronic device provided in this disclosure. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this disclosure.
[0049] This disclosure provides a method for constructing a digital facial model, such as Figure 1 As shown, it includes steps S11-S15.
[0050] S11. Obtain the target face image.
[0051] S12. Determine the face reconstruction parameters based on the target face image.
[0052] S13. Determine the appropriate face reconstruction parameters based on the face reconstruction parameters and the preset template face parameters corresponding to the target face image.
[0053] S14. Determine the parameters of the target component 3D model based on the target face image and a preset component template database including face-related tissues.
[0054] S15. Construct a target digital facial model based on the adapted face reconstruction parameters and the parameters of the target component 3D model.
[0055] The execution order of steps S13 and S14 can be set according to actual needs. Step S13 can be executed first and then step S14, or step S14 can be executed first and then step S13.
[0056] Among them, face-related organizations include, but are not limited to, hairstyles, eyeballs, and teeth, which are related to the face.
[0057] In this embodiment, by acquiring a target face image, determining face reconstruction parameters based on the target face image, and determining adaptive face reconstruction parameters based on the face reconstruction parameters and preset template face parameters corresponding to the target face image, face adaptation between the target face and the template face is achieved. The adapted face reconstruction parameters make the subsequently generated digital facial model conform to human anatomy and be more realistic. The parameters of the target component 3D model are determined based on the target face image and a preset component template database including face-related tissues. The target digital facial model is constructed based on the adaptive face reconstruction parameters and the parameters of the target component 3D model, thus determining the target component 3D model corresponding to the target face. Finally, a digital facial model including face-related tissues is constructed based on the parameters of the target component 3D model and the adaptive face reconstruction parameters, making the digital facial model more realistic and vivid.
[0058] According to the digital facial model construction method provided in this disclosure, step S12 specifically includes S121-S122.
[0059] S121. Generate initial face reconstruction parameters based on the target face image and calculate the comprehensive loss.
[0060] Specifically, image scanning and recognition can be performed based on the target face image to generate initial face reconstruction parameters. A comprehensive loss can be calculated based on the target face image and the initial face reconstruction parameters. The comprehensive loss function includes one or more of the following: image loss, depth loss, identity perception loss, key point error loss, and regularization loss.
[0061] In one example, the overall loss can be determined using the following formula 1:
[0062]
[0063]
[0064] in, Indicates overall loss; ω represents the image loss of the target face image. rgb This represents the weight value corresponding to the image loss; ω represents the depth error loss between the depth map of the target face image and the depth map of the target face image. dep This represents the weight value corresponding to the depth error loss; ω represents the identity perception loss, which involves capturing and abstracting the identity information of a person in a target face image. id This represents the weight value corresponding to the identity perception loss; ω represents the keypoint error loss between the keypoints in the target face image and the projected coordinates of the keypoints in the initial face reconstruction parameters. lan This represents the weight value corresponding to the keypoint error loss; ω represents the regularization loss for regularizing the target face image. reg This represents the weight value corresponding to the regularization loss.
[0065] The methods for determining image loss, depth loss, identity perception loss, keypoint error loss, and regularization loss, as well as the corresponding weight values for each loss, can be selected according to actual needs, without any limitations. For example, if the target face image does not have depth information, the depth error loss between its depth map and the target face image can be disregarded, and ω can be used as the weight value. dep Set to 0.
[0066] S122. Optimize the initial face reconstruction parameters based on the comprehensive loss to obtain the face reconstruction parameters.
[0067] In one example, after calculating the comprehensive loss, the initial face reconstruction parameters can be iteratively optimized based on the comprehensive loss. After initially optimizing the initial face reconstruction parameters based on the comprehensive loss, the comprehensive loss is recalculated, and the initial face reconstruction parameters are further optimized based on the comprehensive loss until a preset condition is met to stop the iterative optimization. The preset condition can be set according to actual needs, including but not limited to stopping the iterative optimization when a preset number of iterations is reached, or stopping the iterative optimization when the comprehensive loss is less than a preset threshold.
[0068] In this embodiment of the disclosure, initial face reconstruction parameters are generated based on the target face image and a comprehensive loss is calculated. The reconstruction similarity between the initial face reconstruction parameters and the actual corresponding target face is quantified into a comprehensive loss. The initial face reconstruction parameters are then optimized based on the comprehensive loss, thereby enabling the rapid and accurate determination of face reconstruction parameters that have high similarity and accuracy to the actual target face.
[0069] According to the digital facial model construction method provided in this disclosure, the template face parameters are obtained through the following steps S131-S132.
[0070] S131. Calculate the average face shape parameters based on multiple pre-acquired face shape parameters.
[0071] Specifically, multiple sets of facial shape parameters can be obtained in advance. These facial shape parameters include, but are not limited to, parameters used to represent facial features, such as the type of face, the distance between the eyes, the height of the bridge of the nose, and the length, width, and concavity of the face.
[0072] The average face shape parameter is obtained by averaging multiple sets of face shape parameters. The average face shape parameter can better represent the shape of an unknown face.
[0073] S132. Generate template face parameters based on the average face shape parameters, preset face basic parameters, and preset shape coefficients.
[0074] Specifically, in one example, the template face parameters can be obtained using the following formula 2:
[0075]
[0076] Where s represents the template face parameters, Represents the basic parameters of a face, x shap This represents the shape factor.
[0077] The basic facial parameters and shape coefficients can be set according to actual needs, and there are no restrictions on them.
[0078] In this embodiment of the disclosure, the average face shape parameter is calculated by pre-acquired multiple sets of face shape parameters. The average face shape parameter can better represent the shape of an unknown face, thereby making the template face parameter generated based on the average face shape parameter, the preset face basic parameters, and the preset shape coefficient conform to the human body structure and have a better ability to represent the shape of the face.
[0079] According to the digital facial model construction method provided in this disclosure, step S13 specifically includes S133-S135.
[0080] S133. Determine the key points of the template face and the reference key points of the template face based on the template face parameters.
[0081] S134. Determine the key points of the target face based on the face reconstruction parameters.
[0082] S135. Perform face adaptation based on the template face key points, the template face reference key points, the target face key points, and the preset adaptation loss function to obtain adapted face reconstruction parameters.
[0083] The execution order of steps S133 and S134 can be set according to actual needs. Step S133 can be executed first and then step S134, or step S134 can be executed first and then step S133.
[0084] Specifically, key points can be annotated globally on the template face corresponding to the template face parameters to obtain the template face key points. Reference key points can be annotated and numbered for the eyes, mouth, and top of the head region of the template face corresponding to the template face parameters to obtain the template face reference key points. Based on the face reconstruction parameters, the target face key points, such as the key points of the eyes, mouth, and top of the head region of the target face, can be directly determined. The rotation, translation, and scaling parameters in the alignment sub-coordinate are optimized to align the template face reference key points with the target face key points.
[0085] In one example, the parameters for adapting face reconstruction can be determined using the following formulas 3, 4, and 5:
[0086]
[0087] L2=|f j -Φ(v j )|2 (4)
[0088]
[0089] in, This represents the adaptation loss function. f represents the set of coordinates of key facial features in the template.j L2 represents the coordinates of the template face reference points, L2 represents the distance between the template face reference points and the target face reference points, and Φ(v) represents the distance between the template face reference points and the target face reference points. j ) represents the target facial landmark v j The coordinates after the Φ transformation, where R represents the rotation matrix, t represents the translation matrix, and s represents the scaling factor. T This represents the zero vector of size 1*3.
[0090] In this embodiment, template facial key points and template facial reference key points are determined based on template facial parameters, and target facial key points are determined based on facial reconstruction parameters. Facial adaptation is performed based on the template facial key points, template facial reference key points, target facial key points, and a preset adaptation loss function to obtain adapted facial reconstruction parameters. This aligns the template facial reference key points and the target facial key points, resulting in a more realistic digital facial model that conforms to human anatomy.
[0091] According to the digital facial model construction method provided in this disclosure, the component template database is obtained through the following steps S141-S144.
[0092] S141. Obtain the parameters of the component's 3D model.
[0093] Specifically, the parameters of a component can be determined by creating a 3D model of the component using various modeling software, or by determining the parameters of an imported 3D model of the component. The 3D model of the component can include, but is not limited to, components of the human face and head, such as teeth, eyeballs, and hairstyles.
[0094] S142. Adjust the parameters of the component 3D model according to the template face parameters to obtain the parameters of the adapted component 3D model.
[0095] Specifically, due to the diverse acquisition methods, the parameters of the obtained component 3D models are located in their respective source coordinate systems and have different sizes. They need to be adapted and adjusted with the template face parameters. The adapted component 3D model parameters can be in the same coordinate system and have the same scale as the template face parameters after adaptation and adjustment.
[0096] S143. Label the corresponding component category according to the parameters of the 3D model of the adapter component.
[0097] Specifically, the parameters of the 3D model of the adapter component are labeled with its component category. The component category can be used to distinguish which specific part of the human face and head it belongs to, and can further distinguish the features of the corresponding part.
[0098] The component category can be set according to actual needs. In one example, the corresponding component category can be labeled as a subclass of "curly hair" in the "hairstyle" category based on the parameters of the 3D model of the adapter component. "Hair" and "curly hair" can be used as the component category of the parameters of the 3D model of the adapter component.
[0099] S144. Generate a component template database based on the parameters of the 3D model of the adapting component and the corresponding component category.
[0100] In this embodiment, by acquiring the parameters of the component's 3D model and adapting them to the template face parameters, the parameters of the component's 3D model are obtained, ensuring that the adapted component's 3D model and the face template have the same scale. This facilitates the construction of a digital facial model using the face reconstruction parameters determined by the face template, resulting in a more realistic and vivid digital facial model. The corresponding component category is labeled based on the parameters of the adapted component's 3D model. A component template database is generated based on the parameters of the adapted component's 3D model and the corresponding component category, achieving unified management of the parameters of the adapted component's 3D model. This facilitates subsequent determination of the target component's 3D model's parameters based on the component category, and then, according to actual needs, selection of specific target component's 3D model parameters to generate the target digital facial model.
[0101] According to the digital facial model construction method provided in this disclosure, step S14 specifically includes steps S145-S147.
[0102] S145. The target face image is preprocessed to obtain a target face component image.
[0103] Specifically, the target face image can be preprocessed, including but not limited to face parsing data processing methods such as downsampling and image segmentation. In one example, the target face image can be segmented based on components such as hair, eyes, and teeth to obtain a target face component image.
[0104] S146. Input the target face component image into a pre-trained component classification network to obtain the component category.
[0105] Specifically, the target face component image is input into a pre-trained component classification network, which then identifies and classifies the target face component image. The component classification network can be trained and its output component categories can be set according to actual needs.
[0106] In one example, such as Figure 2As shown, an initial component classification network is trained based on component image samples of various hairstyles such as straight hair, curly hair, and wavy hair, which are pre-labeled. The initial component classification network extracts the depth features of the component image samples, determines the component category based on the depth features, and optimizes the initial component classification network based on the component category and the pre-labeled tags to obtain the component classification network.
[0107] S147. Match the target component 3D model in the component template database according to the component category and determine the parameters of the target component 3D model.
[0108] Specifically, after determining the component category based on the target face component image, a matching component 3D model with the same component category can be matched in the component template database as the target component 3D model, and the parameters of the matching component 3D model with the same component category can be determined as the parameters of the target component 3D model.
[0109] In this embodiment, the target face image is preprocessed to obtain a target face component image. This target face component image is then input into a pre-trained component classification network to determine the component category. Preprocessing reduces the significant computational effort required for the subsequent pre-trained component classification network to recognize and classify the original target face image. Utilizing the pre-trained component classification network directly to recognize and classify the target face component image improves the efficiency of recognition and classification. Based on the component category, the target component's 3D model is matched against a component template database, and the parameters of the target component's 3D model are determined. This enables rapid determination of the target component's 3D model parameters, improving matching accuracy and efficiency.
[0110] According to the digital facial model construction method provided in this disclosure, the adaptive face reconstruction parameters include adaptive facial key points and adaptive facial triangles, and the parameters of the target component 3D model include target component key points and target component triangles.
[0111] Step S15 specifically includes S151-S153.
[0112] S151. Determine the key points of the target digital facial model based on the adapted facial key points and the target component key points.
[0113] S152. Determine the triangle of the target digital face model based on the adapted face triangle and the target component triangle.
[0114] S153. Generate a target digital face model based on the key points of the target digital face model and the triangular facets of the target digital face model.
[0115] The execution order of steps S151 and S152 can be set according to actual needs. Step S151 can be executed first and then step S152, or step S151 can be executed first and then step S152.
[0116] Specifically, in one example, after determining the parameters for adapting the face reconstruction and the parameters of the target component 3D model, the key points and triangular faces of the target digital face model can be determined using the following formulas 6 and 7:
[0117] V = V f ∪V e ∪V t ∪V h (6)
[0118] F = F f ∪F e ∪F t ∪F h (7)
[0119] Where V represents the key points of the target digital facial model, V f V e V t and V h These represent the key points of the adapted face, the key points of the target components corresponding to the eyes, the key points of the target components corresponding to the teeth, and the key points of the target components corresponding to the hairstyle; F represents the triangle of the target digital facial model. f F e F t and F h These represent the triangles corresponding to the face, eyes, teeth, and hairstyle, respectively.
[0120] In this embodiment, the key points of the target digital face model are determined based on the key points of the adapted face and the key points of the target component. The triangular face of the target digital face model is determined based on the triangular face of the adapted face and the triangular face of the target component. Since the key points and triangular face of the target digital face model already contain the key points and triangular face information of the adapted face and the target component, the target digital face model can be finally generated based on the key points and triangular face of the target digital face model. The target digital face model can be accurately generated through the information of key points and triangular face.
[0121] In one example based on the above embodiments, such as Figure 3 As shown, the digital facial model construction provided in this disclosure includes two stages: template creation and face reconstruction and completion.
[0122] In the template creation stage: Generate template face parameters, adapt and adjust the pre-acquired component 3D model parameters according to the template face parameters to obtain the adapted component 3D model parameters, and generate a template library based on the template face parameters and the adapted component 3D model parameters.
[0123] In the face reconstruction and completion stage: the component categories of the target face are determined based on the target face image, and the parameters of the target component 3D model are matched in the template library according to the component categories; the face reconstruction parameters are determined based on the target face image, and the face is adapted to obtain the adapted face reconstruction parameters based on the face reconstruction parameters and the template face parameters; the face is assembled to obtain the target digital face model based on the adapted face reconstruction parameters and the parameters of the target component 3D model.
[0124] The digital face model construction apparatus provided in the embodiments of this disclosure is described below. The digital face model construction apparatus described below can be referred to in correspondence with the digital face model construction method described above.
[0125] This disclosure also provides a digital facial model building apparatus, such as Figure 4 As shown, it includes:
[0126] Acquisition unit 41 is used to acquire the target face image;
[0127] The construction unit 42 is used to determine face reconstruction parameters based on the target face image; determine adaptive face reconstruction parameters based on the face reconstruction parameters and preset template face parameters corresponding to the target face image; determine parameters of the target component 3D model based on the target face image and a preset component template database including face-related tissues; and construct a target digital face model based on the adaptive face reconstruction parameters and the parameters of the target component 3D model.
[0128] In this embodiment, by acquiring a target face image, determining face reconstruction parameters based on the target face image, and determining adaptive face reconstruction parameters based on the face reconstruction parameters and preset template face parameters corresponding to the target face image, face adaptation between the target face and the template face is achieved. The adapted face reconstruction parameters make the subsequently generated digital facial model conform to human anatomy and be more realistic. The parameters of the target component 3D model are determined based on the target face image and a preset component template database including face-related tissues. The target digital facial model is constructed based on the adaptive face reconstruction parameters and the parameters of the target component 3D model, thus determining the target component 3D model corresponding to the target face. The digital facial model including face-related tissues is constructed based on the parameters of the target component 3D model and the adaptive face reconstruction parameters, making the digital facial model more realistic and vivid.
[0129] According to the digital facial model building apparatus provided in this disclosure, the building unit 42 is specifically used for:
[0130] Initial face reconstruction parameters are generated based on the target face image and a comprehensive loss is calculated, wherein the comprehensive loss includes one or more of image loss, depth loss, identity perception loss, key point error loss and regularization loss;
[0131] The initial face reconstruction parameters are optimized based on the comprehensive loss to obtain the face reconstruction parameters.
[0132] According to the digital facial model building apparatus provided in this disclosure, the building unit 42 is further used for:
[0133] The average face shape parameters are calculated based on multiple pre-acquired sets of face shape parameters;
[0134] Template face parameters are generated based on the average face shape parameters, preset face base parameters, and preset shape coefficients.
[0135] According to the digital facial model building apparatus provided in this disclosure, the building unit 42 is specifically used for:
[0136] Based on the template face parameters, determine the template face key points and template face reference key points;
[0137] The key points of the target face are determined based on the face reconstruction parameters;
[0138] Face adaptation is performed based on the template face key points, the template face reference key points, the target face key points, and the preset adaptation loss function to obtain adapted face reconstruction parameters.
[0139] According to the digital facial model building apparatus provided in this disclosure, the building unit 42 is further used for:
[0140] Obtain the parameters of the component's 3D model;
[0141] The parameters of the component's 3D model are adapted and adjusted based on the template face parameters to obtain the parameters of the adapted component's 3D model.
[0142] The component category is labeled according to the parameters of the 3D model of the adapter component;
[0143] A component template database is generated based on the parameters of the 3D model of the adapted component and the corresponding component category.
[0144] According to the digital facial model building apparatus provided in this disclosure, the building unit 42 is specifically used for:
[0145] The target face image is preprocessed to obtain a target face component image;
[0146] The target face component image is input into a pre-trained component classification network to obtain the component category;
[0147] Match the target component 3D model to the component template database according to the component category and determine the parameters of the target component 3D model.
[0148] According to the digital facial model construction apparatus provided in this disclosure, the adaptive face reconstruction parameters include adaptive facial key points and adaptive facial triangles, and the parameters of the target component 3D model include target component key points and target component triangles; the construction unit 42 is specifically used for:
[0149] The key points of the target digital facial model are determined based on the adapted facial key points and the key points of the target component.
[0150] The triangle of the target digital face model is determined based on the adapted face triangle and the target component triangle.
[0151] The target digital facial model is generated based on the key points and triangular faces of the target digital facial model.
[0152] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a digital facial model construction method, which includes: acquiring a target face image and template face parameters; determining face reconstruction parameters based on the target face image; determining adaptive face reconstruction parameters based on the face reconstruction parameters and the template face parameters; determining parameters of a target component 3D model based on the target face image and a component template database; and constructing a target digital facial model based on the adaptive face reconstruction parameters and the parameters of the target component 3D model.
[0153] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this disclosure, essentially, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0154] On the other hand, this disclosure also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, enable the computer to perform the digital facial model construction method provided by the above methods. The method includes: acquiring a target face image and template face parameters; determining face reconstruction parameters based on the target face image; determining adaptive face reconstruction parameters based on the face reconstruction parameters and the template face parameters; determining parameters of a target component three-dimensional model based on the target face image and a component template database; and constructing a target digital facial model based on the adaptive face reconstruction parameters and the parameters of the target component three-dimensional model.
[0155] In another aspect, this disclosure also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the digital facial model construction methods provided above. The method includes: acquiring a target face image and template face parameters; determining face reconstruction parameters based on the target face image; determining adaptive face reconstruction parameters based on the face reconstruction parameters and the template face parameters; determining parameters of a target component 3D model based on the target face image and a component template database; and constructing a target digital facial model based on the adaptive face reconstruction parameters and the parameters of the target component 3D model.
[0156] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.
Claims
1. A method for constructing a digital facial model, characterized in that, include: Acquire the target face image; Determine the face reconstruction parameters based on the target face image; The appropriate face reconstruction parameters are determined based on the face reconstruction parameters and the preset template face parameters corresponding to the target face image. The parameters of the target component 3D model are determined based on the target face image and a preset component template database including face-related tissues. Construct a target digital facial model based on the adapted face reconstruction parameters and the parameters of the target component 3D model; The adaptive face reconstruction parameters include adaptive face key points and adaptive face triangles, and the parameters of the target component 3D model include target component key points and target component triangles. The step of constructing the target digital facial model based on the adapted face reconstruction parameters and the parameters of the target component 3D model includes: The key points of the target digital facial model are determined based on the adapted facial key points and the key points of the target component. The triangle of the target digital face model is determined based on the adapted face triangle and the target component triangle. The target digital facial model is generated based on the key points and triangular faces of the target digital facial model.
2. The digital facial model construction method according to claim 1, characterized in that, The step of determining face reconstruction parameters based on the target face image includes: Initial face reconstruction parameters are generated based on the target face image and a comprehensive loss is calculated, wherein the comprehensive loss includes one or more of image loss, depth loss, identity perception loss, key point error loss and regularization loss; The initial face reconstruction parameters are optimized based on the comprehensive loss to obtain the face reconstruction parameters.
3. The digital facial model construction method according to claim 1, characterized in that, The template face parameters are obtained through the following steps: The average face shape parameters are calculated based on multiple pre-acquired sets of face shape parameters; Template face parameters are generated based on the average face shape parameters, preset face base parameters, and preset shape coefficients.
4. The digital facial model construction method according to claim 1, characterized in that, The step of determining the appropriate face reconstruction parameters based on the face reconstruction parameters and the preset template face parameters corresponding to the target face image includes: Based on the template face parameters, determine the template face key points and template face reference key points; The key points of the target face are determined based on the face reconstruction parameters; Face adaptation is performed based on the template face key points, the template face reference key points, the target face key points, and the preset adaptation loss function to obtain adapted face reconstruction parameters.
5. The digital facial model construction method according to claim 1, characterized in that, The component template database is obtained through the following steps: Obtain the parameters of the component's 3D model; The parameters of the component's 3D model are adapted and adjusted based on the template face parameters to obtain the parameters of the adapted component's 3D model. The component category is labeled according to the parameters of the 3D model of the adapter component; A component template database is generated based on the parameters of the 3D model of the adapted component and the corresponding component category.
6. The digital facial model construction method according to claim 5, characterized in that, The step of determining the parameters of the target component based on the target face image and a preset component template database including face-related structures includes: The target face image is preprocessed to obtain a target face component image; The target face component image is input into a pre-trained component classification network to obtain the component category; Match the target component 3D model to the component template database according to the component category and determine the parameters of the target component 3D model.
7. A digital facial model construction device, characterized in that, include: The acquisition unit is used to acquire the target face image; A construction unit is configured to: determine face reconstruction parameters based on the target face image; determine adaptive face reconstruction parameters based on the face reconstruction parameters and preset template face parameters corresponding to the target face image; determine parameters of a target component 3D model based on the target face image and a preset component template database including face-related tissues; and construct a target digital facial model based on the adaptive face reconstruction parameters and the parameters of the target component 3D model. The adaptive face reconstruction parameters include adaptive face key points and adaptive face triangles, and the parameters of the target component 3D model include target component key points and target component triangles; the construction unit is specifically used for: The key points of the target digital facial model are determined based on the adapted facial key points and the key points of the target component. The triangle of the target digital face model is determined based on the adapted face triangle and the target component triangle. The target digital facial model is generated based on the key points and triangular faces of the target digital facial model.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the digital facial model construction method as described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the digital facial model construction method as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Method and device for creating face model and electronic device
CN110111418A