Method for generating three-dimensional digital model of teeth under corresponding tooth layout on the basis of tooth photo

Abstract

An aspect of the present application provides a computer-executed method for generating, on the basis of a tooth photo of a patient, a three-dimensional digital model of teeth of the patient under a tooth layout shown in the tooth photo, comprising: obtaining a tooth photo of a patient; obtaining a first three-dimensional digital model representing the teeth of the patient in a first tooth layout; obtaining a rough photographing parameter; on the basis of the rough photographing parameter, rendering the first three-dimensional digital model, to obtain a two-dimensional image of the teeth; on the basis of the tooth photo and the two-dimensional image of the teeth, calculating an optical flow; and on the basis of the tooth photo, the first three-dimensional digital model, and the optical flow, obtaining an updated photographing parameter and an updated three-dimensional digital model by means of calculation, wherein the updated photographing parameter is closer to an actual photographing parameter of the tooth photo than the rough photographing parameter, and a tooth layout represented by the updated three-dimensional digital model is closer to the tooth layout shown in the tooth photo than the first tooth layout.

Description

Method for generating three-dimensional digital tooth models with corresponding tooth layout based on tooth photos Technical Field

[0001] The present application generally relates to a method for generating a three-dimensional digital model of teeth representing a corresponding tooth arrangement based on a dental photograph. Background Art

[0002] With the rapid development of computer technology, dental diagnosis and treatment are increasingly relying on computer technology. When formulating an orthodontic treatment plan, a 3D digital model representing the initial tooth layout of the dentition is usually used to generate a 3D digital model representing the target tooth layout of the dentition at each treatment stage.

[0003] During the orthodontic treatment process, dental professionals need to regularly check the patient's orthodontic treatment progress to determine whether the current orthodontic treatment plan is reasonable and whether it needs to be modified.

[0004] Traditional orthodontic examinations require patients to regularly return to clinics or other specialized institutions. Intraoral scanning equipment is used to obtain a 3D digital model of the patient's current dental configuration. This model is then compared with the 3D digital model of the dentition at the target dental configuration for the corresponding treatment phase to determine the success of orthodontic treatment. Throughout the orthodontic treatment process, patients need to return to specialized institutions multiple times, which not only increases the burden on patients but also takes up a significant amount of the professionals' time.

[0005] In view of the above, it is necessary to provide a new method to make it more convenient to check the progress of orthodontic treatment for patients.

[0006] Summary of the Invention

[0007] One aspect of the present application provides a computer-implemented method for generating a three-dimensional digital model of a patient's teeth in a tooth layout shown in a patient's tooth photograph based on the patient's tooth photograph, comprising: obtaining a patient's tooth photograph; obtaining a first three-dimensional digital model representing the patient's teeth in a first tooth layout; obtaining coarse shooting parameters; rendering the first three-dimensional digital model based on the coarse shooting parameters to obtain a two-dimensional tooth image; calculating optical flow based on the tooth photograph and the two-dimensional tooth image; and calculating updated shooting parameters and an updated three-dimensional digital model based on the tooth photograph, the first three-dimensional digital model, and the optical flow, wherein the updated shooting parameters are closer to the actual shooting parameters of the tooth photograph than the coarse shooting parameters, and the tooth layout represented by the updated three-dimensional digital model is closer to the tooth layout shown in the tooth photograph than the first tooth layout.

[0008] In some embodiments, the method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the dental photograph based on the patient's dental photograph further includes: replacing the rough shooting parameters with the updated shooting parameters, replacing the first three-dimensional digital model with the updated three-dimensional digital model, repeating the rendering, optical flow calculation, and calculation of the updated shooting parameters and the updated three-dimensional digital model at least once to obtain final shooting parameters and a final three-dimensional digital model.

[0009] In some implementations, the optical flow may be a dense optical flow.

[0010] In some embodiments, the optical flow is generated by a trained optical flow neural network based on the tooth photograph and the tooth two-dimensional image.

[0011] In some embodiments, the optical flow neural network also generates an occlusion mask based on the dental photograph and the two-dimensional dental image, wherein the updated shooting parameters and the updated three-dimensional digital model are calculated based on the optical flow filtered by the occlusion mask, and the occlusion mask represents the parts of the teeth that are invisible to each other in the dental photograph and the two-dimensional dental image.

[0012] In some embodiments, the optical flow neural network also generates confidence levels for the optical flows of each point pair based on the dental photograph and the two-dimensional dental image. In the calculation of the updated shooting parameters and the updated three-dimensional digital model, each optical flow is assigned a weight based on its confidence level, and the higher the confidence level, the higher the weight.

[0013] In some embodiments, the method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the dental photograph based on the patient's dental photograph also includes: using a trained first deep neural network to detect each tooth in the dental photograph; matching the teeth detected in the dental photograph with the corresponding teeth in the first three-dimensional digital model; and calculating the coarse shooting parameters based on the matching result.

[0014] In some embodiments, the first deep neural network generates a bounding box or mask of each tooth in the dental photo based on the dental photo, and the coarse shooting parameters are calculated based on the center of the bounding box or the centroid of the mask and the centroid of each tooth in the first three-dimensional digital model.

[0015] In some embodiments, the optical flow neural network is trained using simulated data in a supervised learning manner.

[0016] In some embodiments, after the optical flow neural network is trained using the simulated data in a supervised learning manner, the optical flow neural network is self-supervised trained based on real data using transformation consistency loss as a loss function, or the optical flow neural network is trained again based on real data in a reinforcement learning manner.

[0017] In some embodiments, the first three-dimensional digital model is obtained before taking the dental photograph.

[0018] In some embodiments, the dental photograph and the first three-dimensional digital model together include at least three teeth.

[0019] In some implementations, the updated shooting parameters and the updated three-dimensional digital model are calculated using a GPnP algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features of the present application will be further described below in conjunction with the accompanying drawings and detailed description thereof. It should be understood that these drawings only illustrate several exemplary embodiments of the present application and should not be considered to limit the scope of protection of the present application. Unless otherwise specified, the drawings are not necessarily to scale, and similar reference numerals represent similar components.

[0021] FIG1 is a schematic flowchart of a method executed by a computer for generating a three-dimensional digital model of a patient's teeth in a tooth arrangement shown in a photograph based on a photograph of the patient's teeth in one embodiment of the present application;

[0022] Figure 2 shows an example of a dental photograph taken by the patient himself;

[0023] FIG3 is a schematic flowchart of a method for obtaining coarse shooting parameters in one embodiment of the present application; and

[0024] FIG. 4 is a two-dimensional image of teeth obtained by rendering the first three-dimensional digital model in an example. DETAILED DESCRIPTION

[0025] The following detailed description refers to the drawings that form a part of this specification. The illustrative embodiments mentioned in the specification and drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. In light of this application, those skilled in the art will understand that many other embodiments can be adopted and various changes can be made to the described embodiments without departing from the subject matter and scope of protection of this application. It should be understood that the various aspects of the present application described and illustrated herein can be arranged, replaced, combined, separated and designed according to many different configurations, all of which are within the scope of protection of this application.

[0026] One aspect of the present application provides a computer-implemented method for generating, based on a photograph of a patient's teeth, a three-dimensional digital model representing the patient's teeth in the arrangement of the teeth shown in the photograph.

[0027] In one embodiment, the patient's teeth can be photographed by a non-dental professional, such as the patient himself, his family, or his friends. The patient provides the photographed teeth to a professional institution, which uses a computer to generate a three-dimensional digital model of the patient's teeth in the dental configuration at the time the photographs were taken, based on the photographs and a previously obtained three-dimensional digital model representing the patient's teeth in the first dental configuration.

[0028] One application scenario of the method is to check the progress of a patient's orthodontic treatment during orthodontic treatment. Another application scenario of the method is to monitor whether the patient's tooth layout deviates from the target tooth layout after a period of time after orthodontic treatment. It is understood that the application scenarios of the method are not limited to the above.

[0029] Please refer to FIG1 , which is a schematic flowchart of a method 100 executed by a computer in one embodiment of the present application for generating a three-dimensional digital model of a patient's teeth in a tooth arrangement shown in a photograph of the patient's teeth based on the photograph.

[0030] In 101 , a dental photograph of a patient is obtained.

[0031] In one embodiment, the patient's dental photograph can be taken by the patient himself at any place.

[0032] Currently, there are devices that can be used by non-dental professionals to take dental photos, for example, the dental photography devices disclosed in Chinese Patent No. ZL202220617972.5 or Chinese Patent No. ZL202223452359.1 of the present applicant.

[0033] In one embodiment, the patient's dental photograph may only include the teeth that need to be observed. In another embodiment, the patient's dental photograph may also include other teeth in addition to the teeth that need to be observed.

[0034] In one embodiment, multiple photographs of a patient's teeth taken from different angles may be obtained, and based on the multiple photographs, a three-dimensional digital model of the patient's teeth in the arrangement shown in the photographs may be generated, wherein the arrangement of the patient's teeth was consistent when the multiple photographs were taken. In another embodiment, only a single photograph of the patient's teeth may be obtained, and based on the single photograph, a three-dimensional digital model of the patient's teeth in the arrangement shown in the photograph may be generated. The method of this application will be described in detail below using a single photograph as an example.

[0035] Each photograph of the patient's teeth includes at least three teeth, more preferably at least four teeth, and even more preferably at least five teeth.

[0036] In one embodiment, each dental photograph of the patient may include only teeth of a single jaw (upper jaw or lower jaw teeth), or may include both upper jaw and lower jaw teeth.

[0037] Please refer to FIG2 , which is an example of a dental photo taken by a patient himself.

[0038] In 103, a first three-dimensional digital model representing the patient's teeth in a first tooth configuration is obtained.

[0039] When designing an orthodontic treatment plan using a shell-shaped dental appliance, a three-dimensional digital model representing the patient's teeth in their current tooth configuration is usually first obtained. This tooth configuration is hereinafter referred to as an initial tooth configuration.

[0040] In one embodiment, a three-dimensional digital model representing the patient's dentition in the initial dental configuration can be obtained by directly scanning the patient's jaw. In another embodiment, a physical model of the patient's jaw, such as a plaster model, can be scanned to obtain a three-dimensional digital model representing the patient's dentition in the initial dental configuration. In yet another embodiment, an impression of the patient's jaw can be scanned to obtain a three-dimensional digital model representing the patient's dentition in the initial dental configuration.

[0041] Next, the three-dimensional digital model representing the patient's dentition under the initial tooth layout is segmented so that each tooth is independent of each other, so that each tooth can be operated individually.

[0042] Then, based on the three-dimensional digital model representing the patient's dentition under the initial tooth layout, a three-dimensional digital model representing the patient's dentition under a series of successive tooth layouts is generated, and this series of successive tooth layouts respectively represent the target tooth layouts expected to be achieved in a series of successive correction steps.

[0043] In one embodiment, the first tooth arrangement may be any one of these tooth arrangements.

[0044] The first three-dimensional digital model can be a three-dimensional digital model representing the patient's single-jaw dentition (maxillary or mandibular dentition), or a three-dimensional digital model representing the patient's maxillary and mandibular dentition, or a three-dimensional digital model representing the patient's partial single-jaw dentition, or a three-dimensional digital model representing the patient's partial maxillary dentition and partial mandibular dentition.

[0045] In 105 , coarse shooting parameters are acquired.

[0046] The coarse shooting parameters include external parameters of the camera, or external parameters and internal parameters. The perspective difference between the two-dimensional tooth image obtained by rendering the first three-dimensional digital model based on the coarse shooting parameters and the tooth photo is small enough to enable subsequent optical flow network processing.

[0047] Please refer to FIG3 , which is a schematic flowchart of a method 105 for obtaining the coarse shooting parameters in one embodiment of the present application.

[0048] In 1051 , each tooth in the dental photograph is detected.

[0049] In one embodiment, a trained object detection neural network can be used to detect the teeth in the dental photo to obtain a bounding box (or bounding box) for each tooth. The bounding box for each tooth is marked in the dental photo shown in FIG2 . The object detection neural network can be any applicable deep neural network, including but not limited to a YOLO (You Only Look Once Network), an R-CNN (Region-Based Convolutional Neural Network), and an SSD (Single Shot Detector) network.

[0050] In another embodiment, a trained segmentation neural network can be used to segment the teeth in the dental photo to obtain the boundary of each tooth, and then obtain a mask for each tooth. The boundary of each tooth is also marked in the dental photo shown in Figure 2. The segmentation neural network can be any applicable deep neural network, including but not limited to Mask-RCNN network, PolorMask network, SOLO network, and PointBlend network.

[0051] In 1053 , the teeth detected in the dental photograph are matched with the teeth in the first three-dimensional digital model.

[0052] Matching the teeth detected in the dental photograph with the teeth in the first three-dimensional digital model is to establish a one-to-one correspondence between the teeth.

[0053] In one embodiment, the matching can be performed based on the tooth number. In another embodiment, the matching can also be performed based on the order of the teeth in the mesiodistal direction.

[0054] In 1055 , the rough shooting parameters are calculated based on the tooth photograph, the first three-dimensional digital model, and the matching result.

[0055] In one embodiment, the coarse shooting parameters can be calculated using a PnP algorithm (Perspective-n-Point) based on the center point of the bounding box of each tooth in the dental photograph or the centroid of the mask, and the centroid of each tooth in the first three-dimensional digital model.

[0056] It is understood that, in addition to the above method, any other applicable method may also be used to generate the coarse shooting parameters based on the tooth photograph and the first three-dimensional digital model.

[0057] In another embodiment, dental photos are taken using the dental photography device provided by the applicant, and the software will generate attribute information for each set of dental photos taken, which includes approximate shooting information (including camera internal parameters, approximate shooting distance and angle), and this approximate shooting information can be used as the coarse shooting parameters.

[0058] In yet another embodiment, after obtaining the dental photograph, a rough shooting angle may be manually given as the coarse shooting parameter.

[0059] In 107 , the first three-dimensional digital model is rendered based on the coarse shooting parameters to obtain a two-dimensional image of the teeth and a mask of each tooth.

[0060] In one embodiment, the first three-dimensional digital model can be rendered using PyRender to obtain a two-dimensional image of the teeth and a mask of each tooth. It is understood that in addition to PyRender, any other suitable method can also be used to render the first three-dimensional digital model.

[0061] Please refer to FIG4 , which is a two-dimensional image of teeth obtained by rendering the first three-dimensional digital model in an example.

[0062] In 109 , an optical flow between the teeth photograph and the two-dimensional teeth image is calculated based on the two-dimensional teeth image.

[0063] In a preferred embodiment, the optical flow may be a dense optical flow.

[0064] In one embodiment, the optical flow can be generated using a trained optical flow neural network, which can be any applicable deep neural network, including but not limited to FlowNet, LiteFlowNet, and PWC-Net.

[0065] In one embodiment, the dental photo and the two-dimensional dental image are input into the optical flow neural network, which outputs the optical flow (i.e., the correspondence between each pixel in the dental photo and the two-dimensional dental image), the optical flow confidence (i.e., the confidence of each correspondence), and the occlusion mask (including the tooth parts that are visible in the dental photo but not in the two-dimensional dental image, and the tooth parts that are visible in the two-dimensional dental image but not in the dental photo).

[0066] In one embodiment, the optical flow neural network may be trained in a supervised learning manner using simulation data.

[0067] For details on how to generate simulated data for training optical flow neural networks and how to train optical flow neural networks in a supervised learning manner, please refer to the paper "FlowNet: Learning Optical Flow with Convolutional Networks" published by Philipp Fischer et al. on arXiv in 2015.

[0068] In another embodiment, the optical flow neural network can be self-supervised trained based on real data using transformation consistency loss as a loss function. The loss function can be defined by the following equation (1):

[0069] Loss = D(warp(I0, F), I1) Equation (1)

[0070] Among them, I0 and I1 represent two images input into the optical flow neural network, F represents the optical flow predicted by the optical flow neural network based on the input I0 and I1, the warp function transforms the image I0 based on the predicted optical flow F to obtain the image I1', and D measures the similarity between I1 and I1', which can include pixel-level error, deep feature-level error and key point error. Among them, the key point error is used to solve the mismatching problem caused by the lack of tooth texture, which can make the optical flow learning more accurate.

[0071] In another embodiment, the optical flow neural network may be trained in a reinforcement learning manner based on real data.

[0072] Specifically, a second three-dimensional digital model can be calculated based on the optical flow predicted by the optical flow neural network, the dental photo and the first three-dimensional digital model, and the similarity between the second three-dimensional digital model and the three-dimensional digital model representing the patient's dental layout when the dental photo was taken is used as a reward for reinforcement learning.

[0073] In one embodiment, the optical flow neural network can be trained using a combination of the above training methods. For example, the optical flow neural network is first trained in a supervised learning manner using simulated data, and then the optical flow neural network is self-supervised trained based on real data with transformation consistency loss as the loss function, or the optical flow neural network is first trained in a supervised learning manner using simulated data, and then the optical flow neural network is trained in a reinforcement learning manner based on real data.

[0074] In another embodiment, in addition to the optical flow neural network, other methods may also be used to calculate the optical flow, for example, the Lucas-Kanade algorithm and the Farneback algorithm.

[0075] In 111 , based on the optical flow, the tooth photograph, and the first three-dimensional digital model, updated shooting parameters and an updated three-dimensional digital model are calculated.

[0076] After obtaining the optical flow, the correspondence between the first three-dimensional digital model and the points on the dental photograph is actually obtained. Then, based on this correspondence, the dental photograph, the first three-dimensional digital model, and the mask of each tooth in the two-dimensional dental image, the updated shooting parameters and the updated three-dimensional digital model can be calculated. This calculation process is actually a process of searching for the actual shooting parameters of the dental photograph and the posture of each tooth shown in the dental photograph. Compared with the rough shooting parameters, the updated shooting parameters are closer to the actual shooting parameters of the dental photograph. Compared with the first three-dimensional digital model, the posture of each tooth in the updated three-dimensional digital model is closer to the actual posture of the corresponding tooth in the photograph.

[0077] In one embodiment, the updated shooting parameters and the updated three-dimensional digital model may be calculated using a GPnP (Generalized Perspective-n-Point) method.

[0078] In one embodiment, the occlusion mask may be used to remove the corresponding portion of the optical flow, and calculation may be performed based only on the unoccluded portion, so as to solve the occlusion problem in the motion of the three-dimensional object.

[0079] In one embodiment, in the calculation of the shooting parameters and tooth posture, a weight can be assigned to the optical flow of each point pair according to the confidence of the optical flow of the point pair. A high weight is correspondingly assigned to the optical flow of the point pair with high confidence, and a low weight is correspondingly assigned to the optical flow of the point pair with low confidence. This helps to improve the robustness of the calculation.

[0080] In one embodiment, the confidence level may be a variance, and the larger the variance, the smaller the confidence level.

[0081] In one embodiment, for the calculation of shooting parameters, constraints may be constructed using all teeth, and for the calculation of tooth poses, constraints may be constructed using a single tooth.

[0082] It is understandable that, in addition to the GPnP method, any other applicable method may be used to calculate the updated shooting parameters and the second three-dimensional digital model, for example, the PPC method (Point-to-Plane Correspondence).

[0083] At this point, the posture of each tooth in the updated three-dimensional digital model may not be close enough to the actual posture of the corresponding tooth in the dental photograph. The updated shooting parameters can be used to replace the rough shooting parameters, and the updated three-dimensional digital model can be used to replace the first three-dimensional digital model. Repeat steps 107 to 111 at least once to obtain the final shooting parameters and the final three-dimensional digital model.

[0084] In one embodiment, an iteration number N may be preset, and operations from 107 to 111 may be repeated N times to obtain final shooting parameters and a final three-dimensional digital model.

[0085] In another embodiment, a shooting parameter change threshold and a tooth posture change threshold may be pre-set. If the difference between the updated shooting parameters currently calculated and the updated shooting parameters previously calculated is less than the shooting parameter change threshold, and the posture difference between the teeth of the updated three-dimensional digital model currently calculated and the updated three-dimensional digital model previously calculated is less than the tooth posture change threshold, then the iteration is stopped and the updated shooting parameters and the updated three-dimensional digital model currently calculated are used as the final results.

[0086] Although various aspects and embodiments of the present application are disclosed herein, other aspects and embodiments of the present application will be readily apparent to those skilled in the art in light of this disclosure. The various aspects and embodiments disclosed herein are for illustrative purposes only and are not intended to be limiting. The scope and subject matter of this application are determined solely by the appended claims.

[0087] Similarly, various diagrams may illustrate exemplary architectures or other configurations of the disclosed methods and systems that aid in understanding the features and functionality that may be included in the disclosed methods and systems. The claimed content is not limited to the exemplary architectures or configurations shown, and the desired features may be implemented using a variety of alternative architectures and configurations. Furthermore, for flow charts, functional descriptions, and method claims, the order of blocks presented herein should not limit various embodiments to being implemented in the same order to perform the described functionality, unless the context clearly dictates otherwise.

[0088] Unless otherwise expressly stated, the terms and phrases used herein and their variations should be interpreted as open ended rather than restrictive. In some instances, the appearance of broad words and phrases such as "one or more," "at least," "but not limited to," or other similar terms should not be understood as intending or requiring a narrowing of the context in which such broad terms may not be used.

Claims

1. A computer-implemented method for generating a three-dimensional digital model of a patient's teeth in a tooth arrangement shown in a patient's tooth photograph based on the patient's tooth photograph, comprising: Obtain photos of the patient's teeth; Acquire a first three-dimensional digital model representing the patient's teeth in a first tooth configuration; Get rough shooting parameters; Based on the rough shooting parameters, the first three-dimensional digital model is rendered to obtain a two-dimensional image of the teeth; Calculating optical flow based on the tooth photo and the tooth two-dimensional image; as well as Based on the dental photograph, the first three-dimensional digital model and the optical flow, updated shooting parameters and an updated three-dimensional digital model are calculated, wherein the updated shooting parameters are closer to the actual shooting parameters of the dental photograph than the rough shooting parameters, and the dental layout represented by the updated three-dimensional digital model is closer to the dental layout shown in the dental photograph than the first dental layout.

2. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 1, characterized in that: It also includes: replacing the rough shooting parameters with the updated shooting parameters, replacing the first three-dimensional digital model with the updated three-dimensional digital model, repeating the rendering, optical flow calculation and calculation of the updated shooting parameters and the updated three-dimensional digital model at least once, and obtaining final shooting parameters and a final three-dimensional digital model.

3. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 1, characterized in that: The optical flow is a dense optical flow.

4. The method of claim 1, wherein the method comprises: generating a three-dimensional digital model of the patient's teeth in the tooth arrangement shown in the tooth photo based on the patient's tooth photo; The optical flow is generated by a trained optical flow neural network based on the tooth photo and the tooth two-dimensional image.

5. The method for generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 4, characterized in that: The optical flow neural network also generates an occlusion mask based on the dental photograph and the two-dimensional dental image, wherein the updated shooting parameters and the updated three-dimensional digital model are calculated based on the optical flow filtered by the occlusion mask, and the occlusion mask represents the tooth parts that are invisible to each other in the dental photograph and the two-dimensional dental image.

6. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 4, characterized in that: The optical flow neural network also generates the confidence of the optical flow of each point pair of the dental photograph and the dental two-dimensional image based on the two. In the calculation of the updated shooting parameters and the updated three-dimensional digital model, each optical flow is assigned a weight according to its confidence. The higher the confidence, the higher the weight.

7. The method of claim 1, wherein the method comprises: It also includes: Detecting each tooth in the tooth photo using a trained first deep neural network; matching the teeth detected in the dental photograph with the corresponding teeth in the first three-dimensional digital model; and The rough shooting parameters are obtained by calculation based on the matching result.

8. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 7, characterized in that: The first deep neural network generates a bounding box or a mask of each tooth in the dental photo based on the dental photo, and the coarse shooting parameters are calculated based on the center of the bounding box or the centroid of the mask and the centroid of each tooth in the first three-dimensional digital model.

9. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 4, characterized in that: The optical flow neural network is trained using simulated data in a supervised learning manner.

10. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 9, characterized in that: After the optical flow neural network is trained by supervised learning using the simulation data, the optical flow neural network is self-supervised trained based on real data using transformation consistency loss as a loss function, or the optical flow neural network is trained by reinforcement learning based on real data.

11. The method of claim 1 for generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo, characterized in that: The first three-dimensional digital model is obtained before taking the tooth photograph.

12. The method of claim 1, wherein the method comprises: The tooth photograph and the first three-dimensional digital model together include at least three teeth.

13. The method of claim 1, wherein the method comprises: The updated shooting parameters and the updated three-dimensional digital model are calculated using the GPnP algorithm.

Images