A Structured 3D Facial Digital Template and Its Manufacturing Method

Abstract

The present invention relates to a structured three-dimensional digital face template and a manufacturing method, including the following features: based on the three-dimensional facial data of Chinese adult males collected in batches, size normalization and overlapping alignment are performed based on the Procrustes analysis algorithm to construct an average shape three-dimensional digital face model; through parametric processing, a structured three-dimensional face template is constructed, which has 19,534 triangular patches and 9,856 vertices, including 216 mid-face line points and 4,820 bilateral points on each side of the left and right. The bilateral points are symmetric based on the mid-face line and have a one-to-one correspondence. A process for constructing a three-dimensional face template is provided, and the effect of repairing facial defect data can be achieved by applying the three-dimensional face template.

Description

Technical Field

[0001] The present invention relates to a three-dimensional face template; specifically, it relates to a method for making a structured three-dimensional face digital template. Background Art

[0002] 1 Background

[0003] The three-dimensional morphological characteristics of the facial region are key issues of concern in clinical diagnosis and treatment in stomatology. Departments such as oral orthognathic surgery, orthodontics, and prosthodontics will make corresponding disease diagnoses, treatment plan designs, and treatment effect evaluations based on the facial morphology of patients before and after surgery. Traditionally, oral clinicians mainly diagnose and analyze two-dimensional image data of patients. However, two-dimensional image data cannot comprehensively and truly reflect the three-dimensional morphological characteristics of the patients' facial regions. With the development of digital technology, the acquisition of three-dimensional facial data containing three-dimensional information of the facial region has become increasingly convenient and has also received more and more attention from oral clinicians. Diagnosing and analyzing three-dimensional facial data is the current development trend of stomatology.

[0004] When obtaining three-dimensional facial data, the obtained original point cloud data is messy, and there is no corresponding relationship between the three-dimensional facial data obtained from the same patient multiple times or the three-sided facial data of different patients. When processing and analyzing three-dimensional facial data, the effective information of the three-dimensional facial data cannot be fully utilized. Currently, the main methods for processing and analyzing three-dimensional facial data are the landmark method and the surface registration method. However, according to the corresponding research results, the limitations of the former are as follows: Only a few manually selected facial anatomical landmarks cannot represent the continuous changes on the surface of the facial soft tissues, and there are also problems with the effectiveness and repeatability of manually selecting facial anatomical landmarks; the limitations of the latter are as follows: Although the surface registration method can register the messy point clouds of two different three-dimensional facial data together and calculate a certain corresponding relationship, this corresponding relationship does not have anatomical significance. In view of the deficiencies of previous studies, if a dense corresponding point relationship can be established between different three-dimensional facial point cloud data, the effectiveness and efficiency of diagnosing and analyzing three-dimensional facial data can be effectively improved, and statistical analysis results with anatomical significance can be obtained, which will have good application prospects. The present invention constructs a "face template" with the three-dimensional facial morphological characteristics of Chinese people for the analysis of three-dimensional facial data of Chinese people in response to this need.

[0005] 2 Domestic and Foreign Research Development Trends and Current Situations

[0006] In the past few decades, the three-dimensional measurement and analysis of the facial region have gone through many development stages. The main analysis methods include the landmark method and the surface registration method. The relevant research is as follows:

[0007] Landmark Method:

[0008] In 2001, Virgilio F. Ferrario [1] et al. analyzed the three-dimensional facial data of healthy subjects based on the landmark method to study the influence of gender and age on facial asymmetry. By calculating the distances from the facial anatomical landmarks on the left and right sides to the median sagittal plane, the facial asymmetry of the subjects was analyzed and evaluated.

[0009] In 2007, Hyoung-Seon Baik and Soo-Yeon Kim [2] measured the three-dimensional coordinate changes of the corresponding landmarks, as well as the three-dimensional line segment, angle, and ratio changes of the three-dimensional facial data of patients before and after surgery using the landmark method when analyzing the facial soft tissue changes of patients undergoing orthognathic surgery for skeletal class III. They compared these changes and pointed out that the landmark method can be used to evaluate the facial soft tissue changes of patients undergoing orthognathic surgery for skeletal class III.

[0010] In 2010, Hyoung-Seon Baik [3] et al. analyzed the three-dimensional facial data of patients undergoing orthognathic surgery with different surgical methods before and after surgery using the landmark method, calculated the distances and angles between the landmarks. According to the research results, the landmark method can be used to estimate the facial soft tissue changes of patients after orthognathic surgery.

[0011] Surface registration method:

[0012] In 2007, Lucy Miller [4] et al. registered and superimposed the three-dimensional facial data of orthognathic patients before and after surgery based on the surface registration method in the visualization study of three-dimensional changes in facial soft tissue after orthognathic surgery, and calculated and analyzed the three-dimensional changes in facial soft tissue of patients before and after surgery.

[0013] In 2012, Primozic J. [5] et al. registered and superimposed the three-dimensional facial data of subjects and their mirror image data based on the surface registration method when evaluating facial asymmetry in growing populations, and analyzed and evaluated the asymmetric regions of the subjects' faces.

[0014] In 2013, Tim J. Verhoeven [6] et al. used the surface registration method to analyze the three-dimensional deviation between the three-dimensional facial data of patients with unilateral condylar hyperplasia and their mirror image data, so as to quantify the facial soft tissue asymmetry of the patients.

[0015] Although the current analysis methods of three-dimensional facial data are mainly the landmark method and the surface registration method, both have their limitations [7, 8]. The complex features of the facial surface cannot be represented by a few facial anatomical landmarks, while the dense and chaotic point cloud can express the continuous changes of the facial surface, but there is a lack of corresponding relationships with anatomical significance between different data.

[0016] 3 Review and summary

[0017] Through the above review of previous studies, it can be seen that the main analysis methods of current three-dimensional facial data have obvious limitations. How to establish dense point cloud correspondence relationships between different three-dimensional facial data is of great significance for the processing and analysis of three-dimensional facial data. The present invention constructs an average three-dimensional human face model of Chinese people based on the Procrustes analysis (PA) algorithm by collecting and screening three-dimensional facial data of Chinese people, and further constructs a "three-dimensional face template" with the average facial features of Chinese people. When used in conjunction with a non-rigid registration algorithm, this template can achieve personalized deformation matching of the template for the comparison and analysis of three-dimensional facial data.

[0018] References

[0019] [1] Ferrario VF, Sforza C, Ciusa V, et al. The effect of sex and age on facial asymmetry in healthy subjects: A cross-sectional study from adolescence to mid-adulthood [J]. Journal of Oral and Maxillofacial Surgery, 2001, 59(4): 382-388.

[0020] [2] Baik H, Jeon J, Lee H. Facial soft-tissue analysis of Korean adults with normal occlusion using a 3-dimensional laser scanner [J]. American Journal of Orthodontics and Dentofacial Orthopedics, 2007, 131(6): 759-766.

[0021] [3] Baik HS, Kim SY. Facial soft-tissue changes in skeletal Class III orthognathic surgery patients analyzed with 3-dimensional laser scanning [J]. Am J Orthod Dentofacial Orthop, 2010, 138(2): 167-178.

[0022] [4] Miller L, Morris DO, Berry E. Visualizing three-dimensional facial soft tissue changes following orthognathic surgery[J]. Eur J Orthod, 2007, 29(1): 14-20.

[0023] [5] Primozic J, Perinetti G, Zhurov A, et al. Assessment of facial asymmetry in growing subjects with a three-dimensional laser scanning system[J]. Orthodontics & Craniofacial Research, 2012, 15(4): 237-244.

[0024] [6] Verhoeven TJ, Nolte JW, Maal TJ, et al. Unilateral condylar hyperplasia: a 3-dimensional quantification of asymmetry[J]. PLoS One, 2013, 8(3): e59391.

[0025] [7] Alqattan M, Djordjevic J, Zhurov AI, et al. Comparison between landmark and surface-based three-dimensional analyses of facial asymmetry in adults[J]. The European Journal of Orthodontics, 2015, 37(1): 1-12.

[0026] [8] Verhoeven T, Xi T, Schreurs R, et al. Quantification of facial asymmetry: A comparative study of landmark-based and surface-based registrations[J]. J Craniomaxillofac Surg, 2016, 44(9): 1131-1136. Summary of the Invention

[0027] (I) Technical problems to be solved

[0028] The object of the present invention is to provide a method for making a structured three-dimensional human face digital template, and to propose an average human face model with the morphological characteristics of Chinese people based on the PA normalization algorithm; a structured three-dimensional human face template with the morphological characteristics of Chinese people is constructed based on the average Chinese human face model, which has 19,534 triangular patches, 9,856 total points, 216 midline points, and 9,640 bilateral points; and a process for constructing a three-dimensional human face template is provided, and the effect of repairing facial defect data can be achieved by applying the three-dimensional human face template.

[0029] (II) Technical solution

[0030] A structured three-dimensional human face digital template of the present invention includes: ① an average three-dimensional human face digital model of the three-dimensional facial data of Chinese adult males; ② having 19,534 triangular patches and 9,856 vertices; ③ having 216 midline facial points and 4,820 bilateral points on each of the left and right sides, and the bilateral points are symmetric based on the midline of the face and have a one-to-one correspondence.

[0031] A method for making a structured three-dimensional human face digital template of the present invention includes the following steps: ① Based on the Procrustes analysis algorithm, perform size normalization and overlapping alignment on the three-dimensional facial data of Chinese adult males collected in batches; ② Calculate the average morphological three-dimensional human face model for the size-normalized and overlapping-aligned three-dimensional facial data of Chinese adult males; ③ Perform parametric processing on the average human face model to construct a structured three-dimensional human face template, which has 19,534 triangular patches and 9,856 vertices, including 216 midline facial points and 4,820 bilateral points on each of the left and right sides.

[0032] (III) Beneficial effects

[0033] The advantages of the present invention are as follows:

[0034] An average human face model with the morphological characteristics of Chinese people is proposed based on the PA normalization algorithm; a structured three-dimensional human face template with the morphological characteristics of Chinese people is constructed based on the average Chinese human face model, which has 19,534 triangular patches, 9,856 total points, 216 midline points, and 9,640 bilateral points; and a process for constructing a three-dimensional human face template is provided, and the effect of repairing facial defect data can be achieved by applying the three-dimensional human face template. Description of the drawings

[0035] Figure 1 is a flowchart for establishing a three-dimensional human face template of the present invention;

[0036] Figure 1Among them, A: Partial sample data for constructing a three-dimensional face template and 32 facial anatomical landmark points required for the PA algorithm used in the present invention; B: Three-dimensional facial data of the subject; C: Three-dimensional facial anatomical landmark points automatically determined from the deformed template data;

[0037] Figure 2 It is a schematic diagram of the three-dimensional face template of the present invention;

[0038] Figure 2 Among them, D: Right view; E: Front view; F: Left view. Specific implementation manner

[0039] The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0040] A structured three-dimensional face digital template of the present invention includes the following steps:

[0041] By collecting three-dimensional facial data of 30 Chinese adult males with good facial symmetry, performing size normalization and overlap alignment based on the Procrustes analysis algorithm, an average three-dimensional face digital model of Chinese adult males is constructed; and then through parametric processing, a structured three-dimensional face template with 19,534 triangular patches and 9,856 vertices is constructed.

[0042] A method for making a structured three-dimensional face digital template of the present invention includes the following steps:

[0043] (1) Collection and screening of three-dimensional facial data:

[0044] The three-dimensional scanning subject follows the instructions of the photographer, keeps a natural head position during the scanning process, looks straight ahead with both eyes, has a natural expression, and ensures that there is no occlusion in the facial contour area; the data collection range is from the hairline to the neck and from the left and right to the auricles, and the data is saved and exported in the OBJ file format;

[0045] Import the above OBJ file into Geomagic Studio 2013 software, adjust the spatial posture of the three-dimensional facial data, and make the median sagittal plane of the three-dimensional facial data of the subject in the natural head position coincide with the YZ plane in the software; referring to the facial asymmetry index of healthy Chinese adults, use formula ① to calculate the asymmetry index of bilateral anatomical landmark points of each three-dimensional facial data in the subject group. The AI value of the midline anatomical landmark point is the distance from the midline anatomical landmark point to the YZ plane, that is, the X coordinate value of the midline anatomical landmark point; obtain 4 AI values of bilateral anatomical landmark points of the inner canthus point and the outer canthus point, and 7 AI values of midline anatomical landmark points of the glabella point and the tip of the nose. Calculate the average value of the AI values of the above 11 anatomical landmark points as the overall AI value of the three-dimensional facial data of the subject; according to the data results of the facial asymmetry index of healthy Chinese adults, the average value of the AI values of the above 11 anatomical landmark points is 2.15. Screen out the three-dimensional facial data with an overall AI value less than 2.15, and define it as the three-dimensional facial data with good facial symmetry, which is used as the sample data for constructing the three-dimensional human face template of young men; appropriately trim the screened three-dimensional facial data, and keep the data range uniformly: up to the hairline, both left and right sides to the tragus and the mandibular angle, and down to the lower part of the chin.

[0046]

[0047] Among them, R represents the anatomical landmark point on the right side of the patient's face, L represents the anatomical landmark point on the left side of the patient's face, and dx, dy, and dz respectively represent the distances from the corresponding anatomical landmark point to the YZ plane, the XZ plane, and the XY plane; Asymmetry index, AI: asymmetry index.

[0048] Asymmetry index: used to describe the degree of asymmetry of three-dimensional facial anatomical landmark points. For bilateral homologous three-dimensional facial anatomical landmark points, calculate their three-dimensional asymmetry degree based on their three-dimensional coordinates; for a single three-dimensional facial anatomical landmark point on the facial midline, evaluate its asymmetry degree based on its distance from the median sagittal plane.

[0049] (2) Establishment of the three-dimensional human face template:

[0050] A. Size normalization and alignment of three-dimensional facial data:

[0051] Use the "point feature" function in Geomagic Studio 2013 software to mark 32 facial anatomical landmark points required for size normalization of the three-dimensional facial data trimmed and screened in step (1).

[0052] Specifically, as shown in Table 1:

[0053] Table 1 32 facial anatomical landmark points required for size normalization

[0054]

[0055] According to GB T 2428-1998 Chinese adult head and face size data: the average value of the morphological face length of more than 11,150 adult males: the distance between the root of the nose and the submental point is 119mm, and the average value of the face width: the distance between the left and right tragus points is 143mm; the above statistical data is used as a reference for the normalization of the three-dimensional facial data size, and one sample data with the face length and face width feature dimensions closest to the national standard average value in the above three-dimensional facial data is selected: the morphological face length is 116mm, and the face width is 145mm, as the reference data for the size normalization of other sample data; the 32 anatomical landmark point set files of the reference data in CSV format and the anatomical landmark point coordinate set files of the remaining sample data are imported into MATLAB R2019b software in pairs, and the Protsky analysis algorithm is used to calculate the size scaling coefficients and rotation matrices of the remaining sample data relative to the reference data based on the three-dimensional coordinates of the 32 anatomical landmark points of the reference data and the remaining sample data; thereby obtaining the size scaling coefficients and rotation transformation matrices of the three-dimensional facial sample data relative to the reference data, and realizing the size normalization and overlapping alignment of the three-dimensional facial data of the above subjects;

[0056] The PA algorithm in MATLAB R2019b is an algorithm that can optimally match and overlap point set data with a one-to-one correspondence. With the help of the PA algorithm, this method achieves the optimal matching of the sample data landmark point set and the reference data landmark point set, thereby obtaining the size scaling coefficient and rotation transformation matrix of the three-dimensional facial sample data relative to the reference data, and achieving the size normalization and overlapping alignment of the above-mentioned volunteer three-dimensional facial data.

[0057] B. Construction of “3D face template”:

[0058] In Geomagic Studio 2013 software, the "average" function of multiple 3D facial data was used to calculate the average shape model of the 3D facial data of young Chinese men after the above-mentioned size normalization and overlap alignment, that is, to obtain the "average 3D face model", with 11699 data points;

[0059] In order to construct a face template with completely symmetrical points on the left and right sides and with midline landmarks that completely coincide with the midline of the face, the following processing steps need to be performed on the above average 3D face model in Geomagic Studio 2013 software:

[0060] ① Use the "Feature Plane" function to construct the midsagittal plane of the average face model and delete half of the face data;

[0061] ② Downsample the retained half-face point cloud data based on curvature, and adjust the coordinates of the "near-medial line points" closest to the midsagittal plane in the half-face data to coincide exactly with the midsagittal plane, which are defined as "medial line points", with a quantity of 216. The remaining data points are defined as "unilateral points", with a quantity of 4820;

[0062] ③ Use the "mirror" function based on the midsagittal plane to create mirror-side face data, "combine" the left and right face point cloud data and re-perform triangular mesh triangulation to finally obtain a "3D face template" with 9856 points;

[0063] (3) Application of the 3D face template:

[0064] ① Obtain structured 3D facial data: Use the above 3D face template, combine with a non-rigid registration program to deform and register the 3D face template onto the 3D facial data to obtain the deformed 3D face template, that is, structured 3D facial data with the same morphology as the 3D facial data, a total of 19534 triangular patches, a total of 9856 points, and a one-to-one correspondence between the left and right side points can be obtained;

[0065] ② Realize the automatic determination of facial anatomical landmark points in the 3D facial data: In the point cloud of the 3D face template, mark the vertices that can represent the facial anatomical landmark points, determine their serial number information in the point cloud of the 3D face template, combine with the non-rigid registration algorithm to realize the deformed matching of the 3D face template and the patient's 3D facial data, and then quickly and batch determine the 3D coordinates of the facial anatomical landmark points of the patient's 3D facial data based on the serial number information of the anatomical landmark points on the 3D face template.

[0066] ③ Repair facial defect data: Use the above 3D face template, combine with a non-rigid registration program to deform and register the 3D face template onto the 3D facial data with morphological defects. The deformed 3D face template has a complete facial morphology and can achieve the effect of repairing facial defect data.

[0067] Software for running the Procrustes analysis algorithm: MATLAB R2019b, but not limited to this software;

[0068] Procrustes analysis algorithm: Procrustes analysis, author: Ross A. In 2004, course report, Department of Computer Science and Engineering, University of South Carolina.

[0069] Facial asymmetry index of Chinese healthy adults: It is based on the data results of Huang C.S. et al.'s research on the facial asymmetry index of 60 Chinese healthy adults;

[0070] GB T 2428-1998: Adult head and face dimensions. This standard provides the basic data of adult head and face dimensions and the two-dimensional distribution table of the main dimensions.

[0071] As described above, the present invention can be more fully realized. The above description is only a relatively reasonable implementation example of the present invention. The protection scope of the present invention includes but is not limited to this. Any non-substantive variant changes based on the technical solution of the present invention by those skilled in the art are included within the scope of the present invention.

Claims

1. A method for fabricating a structured three-dimensional digital human face template, characterized in that It includes the following steps: (1) Collect and screen three-dimensional facial data; (2) Establish a three-dimensional face template: A. Size normalization and alignment of three-dimensional facial data: For the three-dimensional facial data after screening and trimming in step (1), 32 facial anatomical landmark points required for size normalization are respectively marked; Select 1 example of sample data with the facial length and width characteristic dimensions closest to the national standard average value in the above three-dimensional facial data as the reference data for size normalization of other sample data; use the Procrustes analysis algorithm to calculate the size scaling coefficients and rotation matrices of the remaining sample data relative to the reference data based on the three-dimensional coordinates of the 32 facial anatomical landmark points of the reference data and the remaining sample data respectively, so as to obtain the size scaling coefficients and rotation transformation matrices of the sample data relative to the reference data, and realize the size normalization and overlapping alignment of the three-dimensional facial data; B. Construction of the "three-dimensional face template": Calculate the average shape model of the three-dimensional facial data of Chinese young men after the above size normalization and overlapping alignment, that is, obtain the "average three-dimensional face model", and the number of data points is 11,699; Perform the following processing steps on the above average three-dimensional face model: ①Construct the median sagittal plane of the average face model and delete the half-side face data; ②Perform curvature-based downsampling on the retained half-side face point cloud data, and adjust the coordinates of the "near-median line points" closest to the median sagittal plane in the half-side face data to completely coincide with the median sagittal plane, and define them as "median line points", with a quantity of 216, and the remaining data points are defined as "unilateral points", with a quantity of 4,820; ③Based on the median sagittal plane, create mirror-side face data, "combine" the left and right face point cloud data and re-perform triangular mesh triangulation to finally obtain the "three-dimensional face template" with 9,856 points; (3) Application of the three-dimensional face template: ①Obtain structured three-dimensional facial data: Use the above three-dimensional face template and combine it with a non-rigid registration program to deform and register the three-dimensional face template onto the three-dimensional facial data to obtain the deformed three-dimensional face template; ②Realize the automatic determination of facial anatomical landmark points of three-dimensional facial data: In the point cloud of the three-dimensional face template, mark the vertices that can represent the facial anatomical landmark points, determine the serial number information of them in the point cloud of the three-dimensional face template, combine with the non-rigid registration algorithm to realize the deformed matching between the three-dimensional face template and the patient's three-dimensional facial data, and then quickly and batch determine the three-dimensional coordinates of the facial anatomical landmark points of the patient's three-dimensional facial data based on the serial number information of the anatomical landmark points on the three-dimensional face template; ③Repair facial defect data: Use the above three-dimensional face template and combine it with a non-rigid registration program to deform and register the three-dimensional face template onto the three-dimensional facial data with morphological defects. The deformed three-dimensional face template has a complete facial morphology and can achieve the effect of repairing facial defect data.

Images