Method for generating three-dimensional face scanning data

By taking pictures of face and tooth parts of structured light patterns, extracting and listing the contour data of the tooth area, the problem that the existing technology cannot accurately obtain tooth data is solved, and high-resolution three-dimensional face scanning data generation is achieved, suitable for dentistry demand in the field.

CN118973466APending Publication Date: 2024-11-15RAY CO LTD

Patent Information

Application Number
CN202380029126.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-22
Filing Date
2023-03-21
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing three-dimensional face scanning technology cannot accurately obtain data on the teeth, resulting in the inability to meet the needs of the dental field, especially the need for high-resolution facial expression changes required in orthodontic correction and simulated dental surgery.

Method used

By taking images of face and tooth parts that project structured light patterns, the facial feature points are extracted to separate and obtain the three-dimensional data of the tooth area contour, and insert the entire column into the three-dimensional data of the face to generate a three-dimensional face containing high-resolution tooth data. Scan the data.

Benefits of technology

It realizes the use of low-priced devices to generate high-resolution dental data, effectively assisting cosmetic dental treatment, and meeting the precision data needs of the dental field.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118973466A_ABST
    Figure CN118973466A_ABST
Patent Text Reader

Abstract

The method for generating the three-dimensional face scanning data comprises the following steps: acquiring a face image of the face of a patient; generating face three-dimensional (3D) data from the face image; acquiring a tooth part image of the tooth part of the patient; generating three-dimensional data of the tooth part from the image of the tooth part; extracting a tooth region contour related to the face image or the face three-dimensional data; separating and acquiring tooth region contour three-dimensional data corresponding to the tooth region contour from the tooth part three-dimensional data; and inserting the tooth region contour three-dimensional data into the face three-dimensional data array.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention generally relates to techniques for generating three-dimensional facial scan data of a human being.

[0002] In particular, the present invention relates to the following three-dimensional facial scanning data generation technology, which is to respectively photograph the face and teeth parts projected with structured light patterns to individually obtain three-dimensional (3D) data related to the face and teeth parts, and after extracting the tooth area contour through facial feature points, the tooth area contour three-dimensional data separated from the tooth part three-dimensional data is inserted into the facial three-dimensional data as a whole. Background Art

[0003] In the past, it was necessary to 3D scan the patient's face to obtain 3D facial scan data in order to perform cosmetic dental treatments, etc. For example, when performing orthodontic treatment, 3D facial scan data related to the patient's face is required in order to simulate the changes in facial expressions in advance during the process of orthodontic treatment.

[0004] For this purpose, special devices such as Bellus3D are used, which can capture the entire face of the patient in three dimensions at one time. Such special devices are expensive, and because they capture the entire face at one time, they cannot obtain accurate three-dimensional data related to the teeth, making them unsuitable for use in the dental field.

[0005] On the other hand, you can also try using a device that scans the patient's face, such as Face Hunter, but since this device also takes a photo of the face in a one-time scan, there is a problem that it cannot accurately obtain data on the dental area.

[0006] Patent document 0001: Korean Patent Publication No. 10-2020-0014645 (February 11, 2020) "Method for manufacturing dental implant prosthesis using intraoral scanning"

[0007] Patent document 0002: Korean Patent Publication No. 10-2022-0023914 (March 3, 2022) "Dental Scanner Body"

[0008] Patent Document 0003: Korean Patent No. 10-1176770 (August 17, 2012) "Dental 3D Scanner and Scanning Method Using the Same"

[0009] Patent document 0004: Korean patent No. 10-2334519 (November 30, 2021) "Simplified automatic detection method of facial feature points from three-dimensional scanning data for dental use"

[0010] Patent Document 0005: Korean Patent No. 10-1744079 (May 31, 2017) "Facial Model Generation Method for Simulating Dental Surgery" Summary of the invention

[0011] Technical issues

[0012] The object of the present invention is to generally provide a technology for generating three-dimensional facial scan data of a human being.

[0013] In particular, the purpose of the present invention is to provide the following three-dimensional facial scanning data generation technology, which respectively photographs the face and teeth area projected with a structured light pattern to obtain three-dimensional data related to the face and teeth area individually, and after extracting the tooth area contour through facial feature points, the three-dimensional data of the tooth area contour obtained by separating from the tooth area three-dimensional data is inserted into the facial three-dimensional data as a whole.

[0014] On the other hand, the technical problems of the present invention are not limited to these matters, and other technical problems can be understood from the description of this specification.

[0015] Technical Solution

[0016] In order to achieve the above-mentioned purpose, the method for generating three-dimensional facial scanning data of the present invention includes the following steps: obtaining a facial image of a patient's face; generating facial three-dimensional data from the above-mentioned facial image; obtaining a dental part image of the patient's teeth; generating dental part three-dimensional data from the above-mentioned dental part image; extracting a dental area contour related to the above-mentioned facial image or the above-mentioned facial three-dimensional data; separating and obtaining a dental area contour three-dimensional data corresponding to the above-mentioned dental area contour from the above-mentioned dental part three-dimensional data; and inserting the above-mentioned dental area contour three-dimensional data into the above-mentioned facial three-dimensional data.

[0017] In the above-mentioned method for generating three-dimensional facial scanning data, the above-mentioned step of obtaining a dental part image of the dental part of the above-mentioned patient also includes the step of obtaining a plurality of partial images of the dental part by photographing the dental part in multiple directions, and the above-mentioned step of generating three-dimensional dental part data from the above-mentioned dental part images includes the following steps: generating a plurality of partial three-dimensional dental part data from the plurality of the above-mentioned partial images of the dental part; and arranging the plurality of the above-mentioned partial three-dimensional dental part data to generate three-dimensional dental part data related to the entire dental part.

[0018] In the method for generating the above-mentioned three-dimensional facial scanning data, the above-mentioned step of inserting the above-mentioned tooth area contour three-dimensional data into the above-mentioned facial three-dimensional data includes the following steps: removing the data of the above-mentioned tooth area contour part from the above-mentioned facial three-dimensional data; and inserting the above-mentioned tooth area contour three-dimensional data into the above-mentioned tooth area contour part of the above-mentioned facial three-dimensional data.

[0019] In the above method for generating three-dimensional facial scan data, the method further includes the following steps: generating auxiliary three-dimensional part data related to a preset part of the face; and inserting the auxiliary three-dimensional part data into the entire column of the above three-dimensional facial data.

[0020] In the method for generating the above-mentioned three-dimensional facial scan data, the above-mentioned facial image, the above-mentioned facial three-dimensional data, the above-mentioned tooth part image and the above-mentioned tooth part three-dimensional data are obtained when a structured light pattern is projected onto the face or tooth part of the above-mentioned patient.

[0021] In the above-mentioned method for generating three-dimensional facial scanning data, the above-mentioned step of extracting the tooth area contour related to the above-mentioned facial image or the above-mentioned facial three-dimensional data includes the following steps: identifying facial feature points from the above-mentioned facial image or the above-mentioned facial three-dimensional data; and extracting the tooth area contour related to the above-mentioned facial image or the above-mentioned facial three-dimensional data based on the above-mentioned facial feature points.

[0022] On the other hand, the computer program of the present invention is stored in a non-transitory computer-readable storage medium in order to execute the above-mentioned method for generating three-dimensional face scan data on a computer.

[0023] Beneficial Effects

[0024] According to the present invention, the present invention has the following advantages: three-dimensional face scan data including high-resolution tooth data can be generated by a relatively low-cost device, thereby effectively assisting multiple fields (for example, cosmetic dental treatment). BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 FIG. 2 is a diagram showing a three-dimensional face scanning system used in the present invention.

[0026] Figure 2 FIG. 1 is a flow chart showing a method for generating three-dimensional face scan data according to the present invention.

[0027] Figure 3 This is a diagram showing an example of an image captured by a camera after a structured light pattern is projected onto the face and teeth in the present invention.

[0028] Figure 4 This is a diagram conceptually showing three-dimensional data of the face and teeth parts in the present invention.

[0029] Figure 5 This is a diagram showing the concept of aligning three-dimensional data of the face and teeth parts in the present invention.

[0030] Figure 6 The figure shows the concept of extracting facial feature points and tooth area contours from an image in the present invention.

[0031] Figure 7 The figure is a diagram showing the concept of separating and acquiring tooth region contour three-dimensional data from tooth part three-dimensional data in the present invention.

[0032] Figure 8 This figure shows the concept of removing the tooth region contour portion from the face three-dimensional data and inserting the tooth region contour three-dimensional data in a row in the present invention.

[0033] Fig. 9 FIG. 2 is a diagram showing another embodiment of a three-dimensional face scanning system used in the present invention.

[0034] Fig.10 FIG. 4 is a flow chart showing another embodiment of a method for generating three-dimensional face scan data according to the present invention. DETAILED DESCRIPTION

[0035] Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

[0036] Figure 1 FIG. 3 is a diagram showing a 3D face scanning system (3D Face Scan System) used in the present invention.

[0037] Reference Figure 1 The three-dimensional face scanning system includes a structured light projector 101, a face shooting camera 102, teeth shooting cameras 103, 104, and a data processing device 200. These structural elements can be embodied as individual devices, or all or part of them can be formed into one.

[0038] First, the structured light projector 101 generates a structured light pattern, such as Figure 3 As shown, it is used to project the structured light pattern onto the patient's face A. However, in addition to the structured light, other methods such as a stereo camera or a time of flight method can be used.

[0039] The facial camera 102 obtains the patient's face A by photographing the patient's face. Figure 3 In this case, the photographing of the patient's face A can be performed while the structured light pattern is projected onto the patient's face A. Furthermore, the photographing of the patient's face A can also be performed at at least one of the front surface, left side, lower side, and upper side of the patient's face, but is not limited thereto.

[0040] The tooth-shooting cameras 103 and 104 obtain the following information by shooting the teeth of the patient's face A: Figure 3In this case, the teeth of the patient's face can be photographed while the structured light pattern is projected onto the patient's face A.

[0041] In the present invention, the face shooting camera 102 for shooting the face and the tooth shooting cameras 103 and 104 for shooting the teeth are separately configured. In this case, there may be one tooth shooting camera 103 and 104, and preferably, there may be multiple tooth shooting cameras 103 and 104 to shoot the patient's teeth in multiple directions to obtain multiple images of the teeth. Figure 1 The embodiment is shown with two tooth imaging cameras 103 and 104. In this case, there are parts that can be observed by multiple cameras 103 and 104 and parts that can be observed only by one camera (103 or 104), and the accuracy of the three-dimensional data can be improved by integrating and synthesizing these parts.

[0042] The data processing device 200 receives the facial image acquired by the facial camera 102 and the tooth images acquired by the tooth cameras 103 and 104, and processes the facial image by Figure 2 or Fig.10 The data processing shown generates three-dimensional facial scan data.

[0043] Figure 2 FIG. 1 is a flow chart showing a method for generating three-dimensional face scan data according to the present invention.

[0044] Step S110, Step S120: First, a preset structured light pattern is projected onto the patient's face A by the structured light projector 101, and a facial camera 102 is used to obtain the following information: Figure 3 Part (a) shows a facial image of a patient's face.

[0045] Then, a facial image with a structured light pattern is generated as follows: Figure 4 The three-dimensional face data shown in part (a) of FIG. 3D data can be generated by using the structured light pattern to grasp the three-dimensional shape of the face. The technology for acquiring three-dimensional data from the image of an object projecting a structured light pattern is a well-known technology, and therefore, a detailed description thereof will be omitted.

[0046] Step S130, step S140: In addition, when the structured light projector 101 projects the preset structured light pattern onto the patient's face A, the tooth-shooting cameras 103 and 104 acquire the following information: Figure 3Part (b) and part (c) of the present invention are teeth images of the same person. In this case, the structured light pattern used for the face image and the structured light pattern used for the teeth image may be the same or different.

[0047] Then, the tooth part image forming the structured light pattern is generated as follows Figure 4 The three-dimensional data of the tooth area shown in parts (b) and (c) of FIG. Figure 3 Compared with the tooth area contained in the facial image of part (a), Figure 3 The resolution of the tooth images in parts (b) and (c) is significantly higher. Figure 4 Compared with the three-dimensional data of the teeth area included in the three-dimensional facial data of part (a), Figure 4 The three-dimensional data of the tooth parts in parts (b) and (c) are significantly more accurate.

[0048] like Figure 1 As shown, the tooth shooting cameras 103 and 104 are formed with multiple teeth. In step S130, preferably, the tooth parts are photographed in multiple directions to obtain multiple partial images of the tooth parts. In this case, in step S140, from the tooth parts formed with the structured light pattern, Figure 3 The partial images of the multiple tooth parts shown in parts (b) and (c) are generated as shown in FIG. Figure 4 The plurality of tooth part three-dimensional data shown in parts (b) and (c) of the present invention are arranged to generate tooth part three-dimensional data related to the entire patient's tooth part.

[0049] There are parts that are commonly presented in multiple tooth-photographing cameras 103 and 104 and parts that are only presented in individual tooth-photographing cameras (103 or 104). Therefore, by synthesizing these entire columns, extremely accurate and precise high-resolution three-dimensional data of the patient's dental parts can be obtained.

[0050] Figure 5 This is a diagram showing the concept of aligning three-dimensional data of the face and teeth parts in the present invention. Figure 5 Part (a) shows the entire column of 2 Figure 4 An example of generating tooth part three-dimensional data by using the tooth part three-dimensional data of part (b) and part (c) of Figure 5 Part (b) shows an example in which the facial three-dimensional data is aligned with the tooth part three-dimensional data.

[0051] Steps S150 and S160: Next, face landmarks are identified and teeth zone contours are extracted based on the face landmarks. Figure 3 The facial image shown in part (a) of FIG. 1 can also be obtained from Figure 4 The three-dimensional facial data shown in part (a) is performed.

[0052] Figure 6 To show that Figure 3 FIG. 1 is a diagram showing the concept of extracting facial landmarks and tooth area contours from a facial image shown in part (a). The concept of facial landmarks and the technology of extracting facial landmarks from a facial image are well-known technologies, so a detailed description thereof will be omitted. For example, when the keyword "face landmark detection" is entered in Google search, a plurality of well-known technologies can be obtained. Moreover, the tooth area contour corresponds to the boundary line of the tooth area in the face, Figure 6 In the image, the lip boundary line is used as the tooth area outline.

[0053] On the other hand, the use of facial feature points is exemplified for extracting the tooth region contour, and other methods such as using an object recognition algorithm based on artificial intelligence may also be used, but the present invention is not limited thereto.

[0054] Step S170: From the three-dimensional data of the tooth part generated in the above step S140 (for example, Figure 5 The tooth area contour three-dimensional data corresponding to the tooth area contour extracted in step S160 is separated from the tooth area three-dimensional data of part (a). Figure 7 The figure is a diagram showing the concept of separating and acquiring tooth region contour three-dimensional data from tooth part three-dimensional data along the tooth region contour in the present invention.

[0055] Step S180: Next, the three-dimensional data of the tooth area contour separated and obtained in step S170 is inserted into the three-dimensional face data obtained in step S120 to obtain three-dimensional face scan data. Figure 8 This is a diagram showing the concept of inserting the three-dimensional data of the tooth area contour into the three-dimensional data of the face in the present invention. Figure 8 In part (b), the tooth region contour three-dimensional data related to the tooth region contour is significantly high-resolution three-dimensional data compared to the facial three-dimensional data processing the entire area of ​​the patient's face.

[0056] In this case, the 3D data of the tooth area contour can be directly enhanced on the 3D facial data, such as Figure 8As shown in part (a) of FIG. 1 , the data of the tooth area contour is removed from the facial three-dimensional data, as shown in FIG. Figure 8 As shown in part (b) of the embodiment, this can be achieved by inserting the entire tooth region contour three-dimensional data into the tooth region contour part of the facial three-dimensional data.

[0057] Fig. 9 FIG. 2 is a diagram showing another embodiment of a three-dimensional face scanning system for use in the present invention. Fig.10 FIG. 4 is a flow chart showing another embodiment of a method for generating three-dimensional face scan data according to the present invention.

[0058] and Figure 1 In comparison, Fig. 9 The second embodiment of the present invention shown in the figure is further provided with an auxiliary 3D camera 105. When the face shooting camera 102 is used to shoot the front surface of the face, the auxiliary 3D camera 105 is used to obtain 3D data related to other parts of the face except the front surface, for example, the side face or the chin, etc. In this case, preferably, the shooting object of the auxiliary 3D camera 105 can be set by the operation of the administrator (for example, a doctor).

[0059] Therefore, with Figure 2 In comparison, Fig.10 The second embodiment of the present invention shown has the following differences: in step S250, auxiliary part three-dimensional data related to one or more preset parts of the patient's face (for example, the side of the face, the chin part) is generated by the auxiliary three-dimensional camera 105; in step S280, a process of inserting the tooth area contour three-dimensional data and the auxiliary part three-dimensional data into the three-dimensional data of the front surface of the face is executed.

[0060] On the other hand, the present invention can be embodied in a computer-readable non-temporary recording medium, for example, in a non-volatile recording medium in the form of a computer-readable code. Such non-volatile recording media exist in various forms of storage devices, for example, hard disks, SSDs, CDROMs, NAS, tapes, network disks, cloud disks, etc., and can also embody a form in which the code is stored and executed in multiple storage devices connected via a network. Furthermore, the present invention can be embodied in the form of a computer program stored in a medium in order to perform specific steps in combination with hardware.

Claims

1. A method for generating three-dimensional facial scanning data, It is characterized in that The steps include: Acquire a facial image capturing the patient's face; Generating facial three-dimensional data from the facial image; Acquire a dental region image of the patient's dental region; generating three-dimensional data of the tooth part from the tooth part image; Extracting a tooth area contour related to the facial image or the three-dimensional facial data; Separating and acquiring tooth region contour three-dimensional data corresponding to the tooth region contour from the tooth region three-dimensional data; and The above-mentioned tooth area contour three-dimensional data is inserted into the above-mentioned facial three-dimensional data array.

2. The method for generating three-dimensional facial scan data according to claim 1, It is characterized in that The step of acquiring the tooth part image of the patient's tooth part further includes the step of acquiring multiple tooth part partial images by photographing the tooth part in multiple directions. The step of generating three-dimensional data of the tooth part from the tooth part image comprises the following steps: generating a plurality of tooth part three-dimensional data from the plurality of tooth part images; and The plurality of tooth part three-dimensional data are aligned to generate tooth part three-dimensional data related to the entire tooth part.

3. The method for generating three-dimensional facial scan data according to claim 2, It is characterized in that The step of inserting the above-mentioned tooth area contour three-dimensional data into the above-mentioned facial three-dimensional data comprises the following steps: removing the data of the tooth area contour portion from the three-dimensional facial data; and The tooth region contour three-dimensional data is inserted into the tooth region contour portion of the facial three-dimensional data.

4. The method for generating three-dimensional facial scan data according to claim 3, It is characterized in that The above method further comprises the following steps: generating auxiliary part three-dimensional data related to a preset part of the face; and The auxiliary part three-dimensional data is inserted into the entire facial three-dimensional data.

5. The method for generating three-dimensional facial scan data according to claim 1, It is characterized in that The facial image, the facial three-dimensional data, the tooth part image, and the tooth part three-dimensional data are acquired by projecting a structured light pattern onto the face or tooth part of the patient.

6. The method for generating three-dimensional facial scan data according to claim 1, It is characterized in that The step of extracting the tooth area contour related to the facial image or the three-dimensional facial data comprises the following steps: Identifying facial feature points from the facial image or the three-dimensional facial data; and A tooth area contour related to the facial image or the three-dimensional facial data is extracted based on the facial feature points.

7. A computer program, It is characterized in that The method for generating three-dimensional face scan data according to any one of claims 1 to 6 is stored in a storage medium in order to be executed by a computer.

Citation Information

Patent Citations

  • Dental 3-dimensional scanner and method of scanning by using the same

    KR101176770B1

  • The face model generation method for the Dental procedure simulation

    KR101744079B1

  • Method for fabricating customized dental implant using oral cavity scanning

    KR1020200014645A

  • Dental scan body

    KR1020220023914A

  • Automated and simplified method for extracting landmarks of three dimensional dental scan data and computer readable medium having program for performing the method

    KR102334519B1

Cited By

  • Scanning processing method and related device

    WO2026144936A1