Method for generating tooth three-dimensional digital model under corresponding tooth layout based on tooth picture

Abstract

One aspect of the invention provides a computer-implemented method for generating a three-dimensional digital model of teeth of a patient under a tooth layout shown by a tooth photo based on the tooth photo of the patient. The method comprises the following steps: acquiring the tooth photo of the patient; obtaining a first three-dimensional digital model representing the teeth of the patient under a first tooth layout; acquiring coarse shooting parameters; based on the coarse shooting parameters, rendering the first three-dimensional digital model to obtain a tooth two-dimensional image; calculating an optical flow based on the tooth picture and the tooth two-dimensional image; on the basis of the tooth photo, the first three-dimensional digital model and the optical flow, an updated shooting parameter and an updated three-dimensional digital model are obtained through calculation, the updated shooting parameter is closer to the actual shooting parameter of the tooth photo than the coarse shooting parameter, and the updated three-dimensional digital model is closer to the actual shooting parameter of the tooth photo than the coarse shooting parameter. The tooth layout represented by the updated three-dimensional digital model is closer to the tooth layout represented by the tooth picture than the first tooth layout.

Description

Technical Field

[0001] The present application generally relates to a method for generating a three-dimensional digital model of teeth representing a corresponding tooth arrangement based on a tooth photograph. Background Art

[0002] With the rapid development of computer technology, dental diagnosis and treatment are increasingly relying on computer technology. When formulating a dental orthodontic treatment plan, a three-dimensional digital model representing the dentition under the target tooth layout at each treatment stage is usually generated based on the three-dimensional digital model representing the dentition under the initial tooth layout.

[0003] During the orthodontic treatment process, dental professionals need to regularly check the patient's orthodontic treatment progress to determine whether the current orthodontic treatment plan is reasonable and whether it needs to be modified.

[0004] The traditional examination method is to require patients to regularly visit clinics and other professional institutions, use intraoral scanning equipment to scan the patient's current tooth layout and lower dentition to obtain a three-dimensional digital model, and then compare it with the three-dimensional digital model of the target tooth layout and lower dentition in the corresponding treatment stage to determine the achievement of orthodontic treatment. During the entire orthodontic treatment process, patients need to visit professional institutions many times, which not only increases the burden on patients, but also takes up a lot of time for professionals.

[0005] In view of the above, it is necessary to provide a new method to make it more convenient to check the achievement of orthodontic treatment for patients. Summary of the invention

[0006] One aspect of the present application provides a computer-implemented method for generating a three-dimensional digital model of a patient's teeth in a tooth layout shown in a tooth photograph based on a patient's teeth photograph, comprising: obtaining a tooth photograph of the patient; obtaining a first three-dimensional digital model representing the patient's teeth in a first tooth layout; obtaining coarse shooting parameters; rendering the first three-dimensional digital model based on the coarse shooting parameters to obtain a two-dimensional tooth image; calculating optical flow based on the tooth photograph and the two-dimensional tooth image; and calculating updated shooting parameters and an updated three-dimensional digital model based on the tooth photograph, the first three-dimensional digital model and the optical flow, wherein the updated shooting parameters are closer to the actual shooting parameters of the tooth photograph than the coarse shooting parameters, and the tooth layout represented by the updated three-dimensional digital model is closer to the tooth layout shown in the tooth photograph than the first tooth layout.

[0007] In some embodiments, the method of generating a three-dimensional digital model of the patient's teeth under the tooth layout shown in the tooth photograph based on the patient's tooth photograph also includes: replacing the rough shooting parameters with the updated shooting parameters, replacing the first three-dimensional digital model with the updated three-dimensional digital model, repeating the rendering, optical flow calculation, and calculation of the updated shooting parameters and the updated three-dimensional digital model at least once, to obtain final shooting parameters and a final three-dimensional digital model.

[0008] In some implementations, the optical flow may be a dense optical flow.

[0009] In some embodiments, the optical flow is generated by a trained optical flow neural network based on the tooth photograph and the tooth two-dimensional image.

[0010] In some embodiments, the optical flow neural network also generates an occlusion mask based on the dental photograph and the two-dimensional dental image, wherein the updated shooting parameters and the updated three-dimensional digital model are calculated based on the optical flow filtered by the occlusion mask, and the occlusion mask represents the parts of the teeth that are invisible to each other in the dental photograph and the two-dimensional dental image.

[0011] In some embodiments, the optical flow neural network also generates confidence of the optical flow of each point pair of the dental photograph and the dental two-dimensional image based on the two. In the calculation of the updated shooting parameters and the updated three-dimensional digital model, each optical flow is assigned a weight according to its confidence, and the higher the confidence, the higher the weight.

[0012] In some embodiments, the method of generating a three-dimensional digital model of the patient's teeth under the tooth layout shown in the tooth photograph based on the patient's tooth photograph also includes: using a trained first deep neural network to detect each tooth in the tooth photograph; matching the teeth detected in the tooth photograph with the corresponding teeth in the first three-dimensional digital model; and calculating the coarse shooting parameters based on the matching result.

[0013] In some embodiments, the first deep neural network generates a bounding box or mask of each tooth in the dental photo based on the dental photo, and the coarse shooting parameters are calculated based on the center of the bounding box or the centroid of the mask and the centroid of each tooth in the first three-dimensional digital model.

[0014] In some embodiments, the optical flow neural network is trained using simulated data in a supervised learning manner.

[0015] In some embodiments, after the optical flow neural network is trained using the simulated data in a supervised learning manner, the optical flow neural network is self-supervised trained based on real data using transformation consistency loss as a loss function, or the optical flow neural network is trained based on real data in a reinforcement learning manner.

[0016] In some embodiments, the first three-dimensional digital model is obtained before taking the tooth photograph.

[0017] In some embodiments, the dental photograph and the first three-dimensional digital model together include at least three teeth.

[0018] In some implementations, the updated shooting parameters and the updated three-dimensional digital model are calculated using a GPnP algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features of the present application will be further described below in conjunction with the accompanying drawings and detailed descriptions thereof. It should be understood that these drawings only illustrate several exemplary embodiments according to the present application and should not be considered as limiting the scope of protection of the present application. Unless otherwise specified, the drawings are not necessarily to scale and similar reference numerals represent similar components.

[0020] Figure 1 A schematic flow chart of a method executed by a computer in one embodiment of the present application for generating a three-dimensional digital model of the patient's teeth in a tooth layout shown in the photo based on a photo of the patient's teeth;

[0021] Figure 2 For example, a dental photo taken by the patient himself;

[0022] Figure 3 A schematic flowchart of a method for obtaining coarse shooting parameters in one embodiment of the present application; and

[0023] Figure 4 The figure is a two-dimensional image of teeth obtained by rendering the first three-dimensional digital model in an example. DETAILED DESCRIPTION

[0024] The following detailed description refers to the drawings that form part of this specification. The illustrative embodiments mentioned in the specification and drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present application. Under the guidance of the present application, those skilled in the art will understand that many other embodiments can be adopted and various changes can be made to the described embodiments without departing from the subject matter and scope of protection of the present application. It should be understood that the various aspects of the present application described and illustrated herein can be arranged, replaced, combined, separated and designed according to many different configurations, and these different configurations are within the scope of protection of the present application.

[0025] One aspect of the present application provides a computer-implemented method for generating, based on a photograph of a patient's teeth, a three-dimensional digital model representing the patient's teeth in a configuration of the teeth shown in the photograph.

[0026] In one embodiment, the patient's teeth photos can be taken by a non-dental professional, such as the patient himself, his family, or his friends, etc. The patient provides the taken teeth photos to a professional institution, which uses a computer to generate a three-dimensional digital model of the patient's teeth under the dental layout of the patient when the photos were taken based on the photos and the three-dimensional digital model of the patient's teeth under the first dental layout obtained previously.

[0027] One application scenario of the method is to check the achievement of orthodontic treatment of a patient during orthodontic treatment. Another application scenario of the method is to monitor whether the patient's tooth layout deviates from the target tooth layout after a period of time after orthodontic treatment. It can be understood that the application scenarios of the method are not limited to the above.

[0028] Please refer to Figure 1 , is a schematic flowchart of a method 100 executed by a computer in one embodiment of the present application for generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the photo based on a photo of the patient's teeth.

[0029] In 101, a dental photograph of a patient is obtained.

[0030] In one embodiment, the patient's dental photograph can be taken by the patient himself at any place.

[0031] Currently, there are devices that can allow non-dental professionals to take dental photos, for example, the dental photography device disclosed in Chinese Patent No. ZL202220617972.5 or Chinese Patent No. ZL202223452359.1 of the applicant.

[0032] In one embodiment, the patient's dental photograph may only include the teeth that need to be observed. In another embodiment, the patient's dental photograph may include other teeth in addition to the teeth that need to be observed.

[0033] In one embodiment, multiple photos of the patient's teeth taken at different angles may be obtained, and a three-dimensional digital model of the patient's teeth under the tooth layout shown in the photos may be generated based on the multiple photos, wherein the patient's tooth layout is consistent when the multiple photos are taken. In another embodiment, only a single photo of the patient's teeth may be obtained, and a three-dimensional digital model of the patient's teeth under the tooth layout shown in the photo may be generated based on the single photo. The method of the present application is described in detail below using a single photo as an example.

[0034] Each of the patient's dental photographs includes at least three teeth, more preferably, at least four teeth, and more preferably, at least five teeth.

[0035] In one embodiment, each patient's dental photograph may include only single-jaw teeth (upper or lower jaw teeth), or may include both upper and lower jaw teeth.

[0036] Please refer to Figure 2 , is an example of a dental photo taken by the patient himself.

[0037] In 103, a first three-dimensional digital model representing the patient's teeth in a first tooth configuration is obtained.

[0038] When designing an orthodontic treatment plan using a shell-shaped dental appliance, a three-dimensional digital model representing the patient's teeth in the current tooth layout is usually obtained first. This tooth layout is hereinafter referred to as the initial tooth layout.

[0039] In one embodiment, the three-dimensional digital model representing the dentition of the patient in the initial tooth layout can be obtained by directly scanning the patient's jaw. In another embodiment, the three-dimensional digital model representing the dentition of the patient in the initial tooth layout can be obtained by scanning a physical model of the patient's jaw, such as a plaster model. In another embodiment, the three-dimensional digital model representing the dentition of the patient in the initial tooth layout can be obtained by scanning an impression of the patient's jaw.

[0040] Next, the three-dimensional digital model representing the dentition of the patient under the initial tooth layout is segmented so that each tooth is independent of each other, so that each tooth can be operated separately.

[0041] Then, based on the three-dimensional digital model representing the dentition of the patient under the initial tooth layout, a three-dimensional digital model representing the dentition of the patient under a series of successive tooth layouts is generated, and the series of successive tooth layouts respectively represent the target tooth layouts expected to be achieved in a series of successive correction steps.

[0042] In one embodiment, the first tooth arrangement may be any one of these tooth arrangements.

[0043] The first three-dimensional digital model can be a three-dimensional digital model representing the patient's single jaw dentition (maxillary or mandibular dentition), or a three-dimensional digital model representing the patient's maxillary and mandibular dentition, or a three-dimensional digital model representing part of the patient's single jaw dentition, or a three-dimensional digital model representing part of the patient's maxillary dentition and part of the mandibular dentition.

[0044] In 105 , coarse photographing parameters are acquired.

[0045] The coarse shooting parameters include external parameters, or external parameters and internal parameters of the camera. The perspective difference between the two-dimensional tooth image obtained by rendering the first three-dimensional digital model based on the coarse shooting parameters and the tooth photo is small enough to enable the subsequent optical flow network to process.

[0046] Please refer to Figure 3 , which is a schematic flowchart of a method 105 for obtaining the coarse shooting parameters in an embodiment of the present application.

[0047] In 1051, each tooth in the dental photograph is detected.

[0048] In one embodiment, the trained target detection neural network can be used to detect the teeth in the tooth photo to obtain a bounding box (or outer rectangular box, Bounding Box) of each tooth. Figure 2 The bounding box of each tooth is marked in the tooth photo shown. The target detection neural network can be any applicable deep neural network, including but not limited to YOLO network (You Only Look Once Network), R-CNN network (Region-Based Convolutional Neural Network) and SSD network (Single Shot Detector).

[0049] In another embodiment, the teeth in the tooth photo may be segmented using a trained segmentation neural network to obtain the boundary of each tooth, and then obtain the mask of each tooth. Figure 2The teeth photo also marks the boundary of each tooth. The segmentation neural network can be any applicable deep neural network, including but not limited to Mask-RCNN network, PolorMask network, SOLO network and PointBlend network.

[0050] In 1053, the teeth detected in the teeth photo are matched with the teeth in the first three-dimensional digital model.

[0051] Matching the teeth detected in the teeth photo with the teeth in the first three-dimensional digital model is to establish a one-to-one correspondence between the teeth.

[0052] In one embodiment, the matching can be performed according to the tooth number. In another embodiment, the matching can also be performed according to the order of the teeth in the mesiodistal direction.

[0053] In 1055, the rough shooting parameters are calculated based on the tooth photograph, the first three-dimensional digital model, and the matching result.

[0054] In one embodiment, the coarse shooting parameters may be calculated using a PnP algorithm (Perspective-n-Point) based on the center point of the bounding box of each tooth in the dental photograph or the centroid of the mask and the centroid of each tooth in the first three-dimensional digital model.

[0055] It can be understood that, in addition to the above method, any other applicable method can also be used to generate the rough shooting parameters based on the tooth photo and the first three-dimensional digital model.

[0056] In another embodiment, dental photographs are taken using the dental photography device provided by the applicant, and the software will generate attribute information for each set of dental photographs taken, which includes approximate shooting information (including camera internal parameters, approximate shooting distance and angle), and the approximate shooting information can be used as the coarse shooting parameters.

[0057] In yet another embodiment, after obtaining the tooth photograph, a rough shooting angle may be manually given as the rough shooting parameter.

[0058] In 107 , based on the coarse shooting parameters, the first three-dimensional digital model is rendered to obtain a two-dimensional image of the teeth and a mask of each tooth.

[0059] In one embodiment, the first three-dimensional digital model may be rendered using PyRender to obtain the two-dimensional image of the teeth and the mask of each tooth. It is understood that in addition to PyRender, any other applicable method may be used to render the first three-dimensional digital model.

[0060] Please refer to Figure 4 , is a two-dimensional image of teeth obtained by rendering the first three-dimensional digital model in an example.

[0061] In 109 , an optical flow between the teeth photograph and the two-dimensional teeth image is calculated based on the teeth photograph and the two-dimensional teeth image.

[0062] In a preferred embodiment, the optical flow may be a dense optical flow.

[0063] In one embodiment, the optical flow may be generated using a trained optical flow neural network, which may be any applicable deep neural network, including but not limited to a FlowNet network, a LiteFlowNet network, and a PWC-Net network.

[0064] In one embodiment, the dental photo and the two-dimensional dental image are input into the optical flow neural network, which outputs the optical flow (i.e., the correspondence between each pixel in the dental photo and the two-dimensional dental image), the optical flow confidence (i.e., the confidence of each correspondence), and the occlusion mask (including the tooth parts visible in the dental photo but invisible in the two-dimensional dental image, and the tooth parts visible in the two-dimensional dental image but invisible in the dental photo).

[0065] In one embodiment, the optical flow neural network may be trained in a supervised learning manner using simulation data.

[0066] For details on how to generate simulation data for training optical flow neural networks and how to train optical flow neural networks in a supervised learning manner, please refer to the paper "FlowNet: Leaming Optical Flow with Convolutional Networks" published by Philipp Fischer et al. on arXiv in 2015.

[0067] In another embodiment, the optical flow neural network can also be self-supervised trained based on real data with transformation consistency loss as the loss function. The loss function can be defined by the following equation (1):

[0068] Loss = D(warp(I0, F), I1) Equation (1)

[0069] Wherein, I0 and I1 represent two images input into the optical flow neural network, F represents the optical flow predicted by the optical flow neural network based on the input I0 and I1, the warp function transforms the image I0 based on the predicted optical flow F to obtain the image I1′, and D measures the similarity between I1 and I1′, which can include pixel-level error, deep feature-level error and key point error, wherein the key point error is used to solve the mismatch problem caused by the lack of tooth texture, which can make the optical flow learning more accurate.

[0070] In yet another embodiment, the optical flow neural network may be trained in a reinforcement learning manner based on real data.

[0071] Specifically, the second three-dimensional digital model can be calculated based on the optical flow predicted by the optical flow neural network, the dental photo and the first three-dimensional digital model, and the similarity between the second three-dimensional digital model and the three-dimensional digital model representing the patient's dental layout when the dental photo was taken is used as a reward for reinforcement learning.

[0072] In one embodiment, the optical flow neural network can be trained using a combination of the above training methods. For example, the optical flow neural network is first trained using simulated data in a supervised learning manner, and then the optical flow neural network is self-supervised trained based on real data with transformation consistency loss as the loss function, or the optical flow neural network is first trained using simulated data in a supervised learning manner, and then the optical flow neural network is trained in a reinforcement learning manner based on real data.

[0073] In another embodiment, in addition to the optical flow neural network, other methods may also be used to calculate the optical flow, for example, the Lucas-Kanade algorithm and the Fameback algorithm.

[0074] In 111 , based on the optical flow, the tooth photo, and the first three-dimensional digital model, updated shooting parameters and an updated three-dimensional digital model are calculated.

[0075] After obtaining the optical flow, the correspondence between the first three-dimensional digital model and the points on the tooth photo is actually obtained. Then, based on this correspondence, the tooth photo, the first three-dimensional digital model and the mask of each tooth in the two-dimensional tooth image, the updated shooting parameters and the updated three-dimensional digital model can be calculated. This calculation process is actually a process of searching for the real shooting parameters of the tooth photo and the posture of each tooth shown in the tooth photo. Compared with the rough shooting parameters, the updated shooting parameters are closer to the actual shooting parameters of the tooth photo. Compared with the first three-dimensional digital model, the posture of each tooth in the updated three-dimensional digital model is closer to the actual posture of the corresponding tooth in the photo.

[0076] In one embodiment, the updated shooting parameters and the updated three-dimensional digital model may be calculated using a GPnP (Generalized Perspective-n-Point) method.

[0077] In one embodiment, the occlusion mask may be used to remove the corresponding portion in the optical flow, and calculation may be performed based only on the unoccluded portion to solve the occlusion problem in the motion of the three-dimensional object.

[0078] In one embodiment, in the calculation of the shooting parameters and tooth posture, a weight can be assigned to the optical flow of each point pair according to the confidence of the optical flow of the point pair. A high weight is correspondingly assigned to the optical flow of the point pair with high confidence, and a low weight is correspondingly assigned to the optical flow of the point pair with low confidence. This helps to improve the robustness of the calculation.

[0079] In one embodiment, the confidence level may be a variance, and the larger the variance, the smaller the confidence level.

[0080] In one embodiment, for the calculation of the shooting parameters, constraints may be constructed using all teeth, and for the calculation of the tooth pose, constraints may be constructed using a single tooth.

[0081] It can be understood that, in addition to the GPnP method, any other applicable method may be used to calculate the updated shooting parameters and the second three-dimensional digital model, for example, the PPC method (Point-to-Plane Correspondence) and the like.

[0082] At this point, the posture of each tooth in the updated three-dimensional digital model may not be close enough to the actual posture of the corresponding tooth in the tooth photograph, and the updated shooting parameters can be used to replace the rough shooting parameters, and the updated three-dimensional digital model can be used to replace the first three-dimensional digital model. Repeat 107 to 111 at least once to obtain the final shooting parameters and the final three-dimensional digital model.

[0083] In one embodiment, an iteration number N may be preset, and the operations from 107 to 111 may be repeated N times to obtain final shooting parameters and a final three-dimensional digital model.

[0084] In another embodiment, a shooting parameter change threshold and a tooth posture change threshold may be pre-set. If the difference between the updated shooting parameters currently calculated and the updated shooting parameters previously calculated is less than the shooting parameter change threshold, and the posture difference between each tooth between the updated three-dimensional digital model currently calculated and the updated three-dimensional digital model previously calculated is less than the tooth posture change threshold, then the iteration is stopped and the updated shooting parameters and the updated three-dimensional digital model currently calculated are used as the final results.

[0085] Although various aspects and embodiments of the present application are disclosed herein, other aspects and embodiments of the present application will be apparent to those skilled in the art in light of the present application. The various aspects and embodiments disclosed herein are for illustrative purposes only and not for limiting purposes. The scope and subject matter of the present application are determined solely by the appended claims.

[0086] Likewise, various diagrams may illustrate exemplary architectures or other configurations of the disclosed methods and systems that aid in understanding the features and functions that may be included in the disclosed methods and systems. The claimed content is not limited to the exemplary architectures or configurations shown, and the desired features may be implemented with various alternative architectures and configurations. In addition, for flow charts, functional descriptions, and method claims, the order of blocks presented herein should not be limited to various embodiments that are implemented in the same order to perform the described functions, unless otherwise clearly indicated in the context.

[0087] Unless otherwise expressly noted, the terms and phrases used herein and their variations should be interpreted as open ended, not restrictive. In some instances, the appearance of broad words and phrases such as "one or more", "at least", "but not limited to", or other similar terms should not be understood as an intent or need to indicate a narrowing of the example where such broad terms may not be present.

Claims

1. A computer-implemented method for generating a three-dimensional digital model of a patient's teeth in a tooth arrangement shown in a patient's tooth photograph based on the patient's tooth photograph, comprising: Obtain photos of the patient's teeth; Acquire a first three-dimensional digital model representing the patient's teeth in a first tooth configuration; Get rough shooting parameters; Based on the rough shooting parameters, the first three-dimensional digital model is rendered to obtain a two-dimensional image of the teeth; Calculating optical flow based on the tooth photo and the tooth two-dimensional image; as well as Based on the dental photograph, the first three-dimensional digital model and the optical flow, updated shooting parameters and an updated three-dimensional digital model are calculated, wherein the updated shooting parameters are closer to the actual shooting parameters of the dental photograph than the rough shooting parameters, and the dental layout represented by the updated three-dimensional digital model is closer to the dental layout shown in the dental photograph than the first dental layout.

2. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 1, characterized in that: It also includes: replacing the rough shooting parameters with the updated shooting parameters, replacing the first three-dimensional digital model with the updated three-dimensional digital model, repeating the rendering, optical flow calculation and calculation of the updated shooting parameters and the updated three-dimensional digital model at least once, and obtaining final shooting parameters and a final three-dimensional digital model.

3. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 1, characterized in that: The optical flow is a dense optical flow.

4. The method of claim 1, wherein the method comprises: generating a three-dimensional digital model of the patient's teeth in the tooth arrangement shown in the tooth photo based on the patient's tooth photo; The optical flow is generated by a trained optical flow neural network based on the tooth photo and the tooth two-dimensional image.

5. The method for generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 4, characterized in that: The optical flow neural network also generates an occlusion mask based on the dental photograph and the two-dimensional dental image, wherein the updated shooting parameters and the updated three-dimensional digital model are calculated based on the optical flow filtered by the occlusion mask, and the occlusion mask represents the tooth parts that are invisible to each other in the dental photograph and the two-dimensional dental image.

6. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 4, characterized in that: The optical flow neural network also generates the confidence of the optical flow of each point pair of the dental photograph and the dental two-dimensional image based on the two. In the calculation of the updated shooting parameters and the updated three-dimensional digital model, each optical flow is assigned a weight according to its confidence. The higher the confidence, the higher the weight.

7. The method of claim 1, wherein the method comprises: It also includes: Detecting each tooth in the tooth photo using a trained first deep neural network; matching the teeth detected in the dental photograph with the corresponding teeth in the first three-dimensional digital model; and The rough shooting parameters are obtained by calculation based on the matching result.

8. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 7, characterized in that: The first deep neural network generates a bounding box or a mask of each tooth in the dental photo based on the dental photo, and the coarse shooting parameters are calculated based on the center of the bounding box or the centroid of the mask and the centroid of each tooth in the first three-dimensional digital model.

9. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 4, characterized in that: The optical flow neural network is trained using simulated data in a supervised learning manner.

10. The method of generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo according to claim 9, characterized in that: After the optical flow neural network is trained by supervised learning using the simulation data, the optical flow neural network is self-supervised trained based on real data using transformation consistency loss as a loss function, or the optical flow neural network is trained by reinforcement learning based on real data.

11. The method of claim 1 for generating a three-dimensional digital model of the patient's teeth in the tooth layout shown in the tooth photo based on the patient's tooth photo, characterized in that: The first three-dimensional digital model is obtained before taking the tooth photograph.

12. The method of claim 1, wherein the method comprises: The tooth photograph and the first three-dimensional digital model together include at least three teeth.

13. The method of claim 1, wherein the method comprises: The updated shooting parameters and the updated three-dimensional digital model are calculated using the GPnP algorithm.

Images