Method for reconstructing three-dimensional digital model of dental arch
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2026-01-21
- Publication Date
- 2026-08-13
Smart Images
Figure CN2026073835_13082026_PF_FP_ABST
Abstract
Description
Methods for reconstructing three-dimensional digital models of dental arches Technical Field
[0001] This application generally relates to a method for reconstructing a three-dimensional digital model of a dentition, and more specifically, to reconstructing a three-dimensional digital model of the patient's dentition representing the dental layout shown in the dental photographs, based on at least two patient dental photographs and a three-dimensional digital model representing the patient's dentition in a first dental layout. Background Technology
[0002] With the continuous development of computer science, dental professionals are increasingly relying on computer technology in dental treatment. For example, in orthodontic treatment using shell-shaped braces, patients need to visit the dental clinic periodically for intraoral scans to obtain a three-dimensional digital model of the dentition, representing the current tooth layout. Dental professionals can then use this 3D digital model to assess the progress of orthodontic treatment. However, this process is complex and cumbersome, representing an additional burden for both patients and dental professionals.
[0003] Currently, a method has emerged that uses a trained neural network to reconstruct a 3D digital model of the patient's dentition based on photographs of the patient's teeth and a 3D digital model representing the initial tooth layout. However, the accuracy of the reconstructed 3D digital model is still not high enough; that is, the tooth poses in the reconstructed 3D digital model still differ significantly from the actual tooth poses when the photographs were taken. This can affect dental professionals' analysis of orthodontic treatment.
[0004] In view of the above, it is necessary to provide a method that can more accurately reconstruct a three-dimensional digital model of the dental arch. Summary of the Invention
[0005] One aspect of this application provides a computer-executed method for reconstructing a three-dimensional digital model of a dental arch, comprising: acquiring N photographs of a patient's teeth taken from different angles, wherein N is a natural number greater than or equal to 2; acquiring a first three-dimensional digital model representing a dental arch of the patient in a first dental layout; predicting a second dental layout and a first set of camera poses based on multi-view tooth pose prediction using a first deep neural network, based on the intrinsic parameters of the camera used to take the N dental photographs, the N dental photographs, and the first three-dimensional digital model, wherein the first set of camera poses includes N camera poses corresponding one-to-one with the N dental photographs; adjusting the tooth pose based on a single view, using each of the N dental photographs as an independent reference, based on the second dental layout, to obtain N dental layouts corresponding one-to-one with the N dental photographs; and optimizing the tooth pose and camera pose by using a solution to the Object Level Bundle. An optimization algorithm for the adjustment problem calculates a third tooth layout and a second set of camera poses based on the N tooth layouts and the first set of camera poses. The second set of camera poses includes N camera poses that correspond one-to-one with the N tooth images.
[0006] In some implementations, the single-view tooth pose adjustment includes: for each of the N tooth photographs, using a second deep neural network, adjusting the pose of the tooth based on a local image of each tooth in the tooth photograph and a two-dimensional rendering of the tooth, to obtain one of the N tooth layouts corresponding to the tooth photograph, wherein the two-dimensional rendering of the tooth is obtained based on a camera pose corresponding to the first group of camera poses.
[0007] In some implementations, the pose of the teeth is adjusted iteratively using the second deep neural network a predetermined number of times.
[0008] In some implementations, the single-view tooth pose adjustment further includes: based on the N tooth layouts, for each of the N tooth photographs, superimposing a 2D rendering of each tooth in the photograph onto the photograph for display; and adjusting the pose of at least one tooth in at least one of the N tooth layouts according to user instructions to obtain updated N tooth layouts, wherein the user instructions are input by the user based on a comparison between the 2D rendering of the tooth and the corresponding part in the tooth photograph, and the optimization of the camera pose and tooth pose is based on the updated N tooth layouts.
[0009] In some implementations, the first three-dimensional digital model is one of a series of successive three-dimensional digital models of the dentition representing the patient’s initial tooth layout to the target tooth layout in an orthodontic treatment plan using a shell-shaped dental appliance.
[0010] In some implementations, the first three-dimensional digital model is a three-dimensional digital model of the dental arch representing the patient's initial tooth layout.
[0011] In some embodiments, the first three-dimensional digital model is a three-dimensional digital model representing the patient's maxillary or mandibular dentition.
[0012] In some implementations, the second tooth layout is closer to the current tooth layout than the first tooth layout, and the third tooth layout is closer to the current tooth layout than the second tooth layout.
[0013] In some embodiments, the computer-executed method for reconstructing a three-dimensional digital model of the dental arch further includes: iterating the single-view-based tooth pose adjustment and the tooth pose and camera pose optimization at least once.
[0014] Another aspect of this application provides a computer system for reconstructing a three-dimensional digital model of a dental arch, comprising a storage device and a processor, wherein the storage device stores a computer program for reconstructing the three-dimensional digital model of a dental arch, and when the program is run, the processor executes the method for reconstructing the three-dimensional digital model of a dental arch. Attached Figure Description
[0015] The above and other features of this application will be further described below with reference to the accompanying drawings and their detailed description. It should be understood that these drawings only illustrate several exemplary embodiments according to this application and should not be considered as limiting the scope of protection of this application. Unless otherwise specified, the drawings are not necessarily to scale, and similar reference numerals denote similar parts.
[0016] Figure 1 is a schematic flowchart of a method for reconstructing a three-dimensional digital model of a dental arch according to an embodiment of this application;
[0017] Figure 2 shows a first three-dimensional digital model representing the patient's mandibular dentition under a first tooth layout in an example of this application;
[0018] Figures 3A-3D show four photographs of the patient's teeth taken from different angles in one example;
[0019] Figure 4 shows a partial photograph of tooth 31 in Figure 3A-3D and a two-dimensional rendering based on the pose of the first set of cameras and the layout of the second tooth, displayed in an interface of a computer program used to reconstruct a three-dimensional digital model of the dental arch in one embodiment of this application.
[0020] Figure 5 shows a partial photograph of tooth 31 in Figure 3A-3D and a two-dimensional rendering of the tooth after the pose was adjusted using the second neural network, as displayed in an interface of the computer program used to reconstruct the three-dimensional digital model of the dental arch.
[0021] Figure 6 shows a partial photograph of tooth 31 in Figure 3A-3D and a two-dimensional rendering after further manual adjustment of its pose, displayed in an interface of the computer program used to reconstruct the three-dimensional digital model of the dental arch.
[0022] Figure 7 shows a partial photograph of tooth 31 in Figure 3A-3D and a two-dimensional rendering based on the pose of the second set of cameras and the layout of the third tooth, displayed on an interface of the computer program used to reconstruct the three-dimensional digital model of the dental arch; and
[0023] Figure 8 shows a partial photograph of tooth 31 in Figures 3A-3D and a two-dimensional rendering of the updated second set of camera poses and third tooth layout obtained after repeating operations 105-109, as displayed in an interface of the computer program used to reconstruct the three-dimensional digital model of the dental arch. Detailed Implementation
[0024] The following detailed description incorporates the accompanying drawings, which form part of this specification. The illustrative embodiments mentioned in the specification and drawings are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art will understand, based on the teachings of this application, that many other embodiments can be employed and various changes can be made to the described embodiments without departing from the spirit and scope of this application. It should be understood that the various aspects of this application illustrated herein can be arranged, substituted, combined, separated, and designed in many different configurations, all of which are within the scope of this application.
[0025] One aspect of this application provides a computer-executed method for reconstructing a three-dimensional digital model of a dental arch. First, using a first neural network, based on N photographs of a patient's teeth taken from different angles, the intrinsic parameters of the cameras that took these photographs, and a first three-dimensional digital model representing the patient's teeth under a first dental layout, a second dental layout and a first set of camera poses are predicted. For each of the dental photographs, the first set of camera poses includes a corresponding camera pose. Next, based on the second dental layout, using each dental photograph as an independent reference, the poses of each tooth are adjusted to obtain N dental layouts. Then, based on the first set of camera poses and the N dental layouts, an optimization algorithm for solving the Object Level Bundle Adjustment problem is used to calculate a second set of camera poses and a third dental layout.
[0026] The second tooth layout predicted by the first neural network is likely not accurate enough, meaning it does not closely approximate the patient's actual tooth layout when the dental photographs were taken. Adjusting the pose of each tooth using each dental photograph as an independent reference can significantly improve the accuracy of the obtained tooth layout.
[0027] The first set of camera poses predicted by the first neural network is likely not accurate enough, meaning it is not close enough to the actual pose of the camera that took the dental photograph. Furthermore, obtaining a more accurate camera pose allows for the calculation of a more accurate tooth layout. The second set of camera poses calculated using an optimization algorithm based on the first set of camera poses and the N tooth layouts is closer to the actual pose of the camera that took the dental photograph than the first set of camera poses.
[0028] Another aspect of this application provides a computer system for reconstructing a three-dimensional digital model of a dental arch, comprising a storage device and a processor, wherein the storage device stores a computer program for reconstructing the three-dimensional digital model of a dental arch, and when the program is run, the processor executes the method for reconstructing the three-dimensional digital model of a dental arch.
[0029] In orthodontic treatment, patients only need to take their own dental photographs. The dental clinic can then use the method described in this application to generate a dental layout based on these photographs and a previously obtained three-dimensional digital model of the patient's dentition. This layout closely approximates the actual dental layout of the patient at the time the photographs were taken. This allows dental professionals to treat patients based on this obtained dental layout, reducing the number of follow-up visits required during orthodontic treatment, significantly simplifying treatment monitoring, and alleviating the burden on both patients and dental professionals.
[0030] Please refer to Figure 1, which is a schematic flowchart of a computer-executed method 100 for reconstructing a three-dimensional digital model of a dental arch according to an embodiment of this application.
[0031] In 101, a first three-dimensional digital model representing a row of teeth in a patient with a first tooth layout is obtained.
[0032] The dentition can be the maxillary dentition or the mandibular dentition. The first three-dimensional digital model can be a complete three-dimensional digital model of a single dentition (maxillary or mandibular dentition), or it can be a three-dimensional digital model of a portion of a single dentition (including multiple consecutive teeth).
[0033] As is known to those skilled in the art, orthodontic treatment of a dentition (maxillary or mandibular) using shell-shaped orthodontic appliances typically requires dozens of successive shell-shaped orthodontic appliances, each corresponding to an orthodontic step, used to reposition the dentition from the tooth layout achieved in the previous orthodontic step to the target tooth layout of the current orthodontic step.
[0034] These successive shell-shaped orthodontic appliances are fabricated based on three-dimensional digital models representing the target tooth layout for each treatment step. A common method for generating these three-dimensional digital models representing the series of tooth layouts is to first obtain a three-dimensional digital model representing the patient's initial tooth layout (the patient's tooth layout before orthodontic treatment) through intraoral scanning or scanning a solid model (e.g., a plaster model) or impression of the patient's jaw. Then, based on this, a three-dimensional digital model representing the patient's target tooth layout (the tooth layout expected to be achieved through orthodontic treatment) is generated. Next, based on the three-dimensional digital models representing the initial and target tooth layouts, a three-dimensional digital model representing the target tooth layout (hereinafter referred to as the "intermediate tooth layout") for the series of treatment steps is generated.
[0035] In one embodiment, after obtaining the three-dimensional digital model representing the initial tooth layout, it can be segmented using deep learning methods to make each tooth independent of the others and the teeth and gums, thereby enabling individual manipulation of each tooth (e.g., translation and rotation).
[0036] In one embodiment, the first three-dimensional digital model can be any one of a series of successive three-dimensional digital models from the initial three-dimensional digital model to the target three-dimensional digital model. It is understood that, in addition to this series of successive three-dimensional digital models, the first three-dimensional digital model can also be a three-dimensional digital model representing a patient's teeth in any dental layout, for example, a three-dimensional digital model of a patient's teeth obtained by scanning at any time, or a three-dimensional digital model representing a patient's teeth in a new dental layout obtained by manipulating a scanned three-dimensional digital model of the patient's teeth.
[0037] Please refer to Figure 2, which shows a first three-dimensional digital model of a patient's mandibular dentition in a first tooth layout, as an example of this application.
[0038] In step 103, N photographs of the patient's teeth taken from different angles are obtained.
[0039] Where N is a natural number greater than or equal to 2.
[0040] In one embodiment, the photographs of the patient's teeth can be taken by the patient themselves. Currently, various oral imaging devices are available on the market that allow patients to take their own dental photographs, such as the mooeli developed by the applicant of this application. TM product.
[0041] Please refer to Figures 3A-3D, which show four photographs of the patient's teeth taken from different angles using the aforementioned oral imaging device in one example.
[0042] The first tooth layout is different from the patient's actual tooth layout when the N dental photographs were taken.
[0043] In step 105, using a first neural network, based on the N dental photographs, camera intra-parameters, and the first three-dimensional digital model, a first set of camera poses and a second tooth layout are predicted.
[0044] Using neural networks to predict the patient's tooth layout and camera pose (i.e., extrinsic parameters) when the dental photographs are taken, based on multiple dental photographs from different angles, camera intra-parameters, and a three-dimensional digital model of the dentition, is a prior art in this field and will not be described in detail here.
[0045] In one embodiment, the pre-trained neural network described in the paper "Megapose: 6d pose estimation of novel objects via render & compare" by Yann Labbe et al., published on arXiv preprint arXiv:2212.06870 (2022), can be used to predict the first set of camera poses and the second tooth layout. It is understood that any other suitable neural network besides the one described above can be employed, such as the neural network in the paper "FoundPose: Unseen Object Pose Estimation with Foundation Features" by Evin Pinar Ornek et al., published on arXiv:2311.18809v2 (19 Jul 2024), or the neural network in the paper "GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence" by Van Nguyen Nguyen et al., published on arXiv:2311.14155v2 (15 Mar 2024).
[0046] Those skilled in the art will understand that camera intrinsic parameters are parameters related to the camera's own characteristics, such as focal length and pixel size. Camera extrinsic parameters are parameters in a world coordinate system, such as the camera's position and rotation direction. Unlike the invariant intrinsic parameters, extrinsic parameters change with camera movement.
[0047] In one embodiment, the N dental photographs are taken using a known camera, and the camera's intrinsic parameters can be obtained through calibration.
[0048] Those skilled in the art will understand that a tooth layout includes the pose (including position and orientation) of each tooth.
[0049] In step 107, based on the second tooth layout, each of the N tooth photographs is used as an independent reference to adjust the pose of each tooth, thereby obtaining N tooth layouts.
[0050] In predicting the second tooth layout, a linked mode is used. Using one of the N dental photographs as a reference, the pose of one tooth is adjusted. The 2D rendering of that tooth under other rendering parameters (the rendering parameters corresponding to other dental photographs) changes accordingly with the adjustment of its pose. In this mode, for a patient's dentition, there is only one 3D digital model, which uses N dental photographs as references and adjusts the poses of each tooth in that 3D digital model.
[0051] In 107, an independent mode is used, which means that there is a three-dimensional digital model of the dental arch for each of the N dental photographs. Using each of the N dental photographs as an independent reference, the pose of each tooth in the corresponding three-dimensional digital model of the dental arch is adjusted to obtain N three-dimensional digital models of the dental arch, that is, N tooth layouts.
[0052] In one embodiment, for each of the N dental photographs, a second neural network can be used to predict the updated pose of the tooth based on a local photograph of each tooth in the photograph and a two-dimensional rendering of the tooth in its current pose. Finally, for the N dental photographs, N tooth layouts are predicted accordingly.
[0053] The above operation can be iterated multiple times to improve the overlap between the corresponding 2D rendering of each tooth and the corresponding portion of that tooth in the corresponding dental photograph. In one embodiment, the number of iterations of the above operation can be preset. In another embodiment, a difference threshold can be set, and the above operation stops iterating only when the difference between the corresponding 2D rendering of a tooth and the corresponding portion of that tooth in the corresponding dental photograph is less than the threshold.
[0054] It can be understood that for a tooth, a trained deep neural network can be used to intercept a local photo corresponding to the tooth from the N tooth photos. This is the prior art, so it will not be described in detail here.
[0055] In the method of the paper "Megapose: 6d pose estimation of novel objects via render&compare" published by Yann Labbe et al. in arXiv preprint arXiv:2212.06870 (2022), it includes a neural network and an optimization algorithm. Among them, the neural network is used to adjust the tooth pose based on the local photo and the 2D rendering of a single tooth, and the optimization algorithm is used to achieve the linkage of multiple views. Therefore, in one embodiment, the second neural network can still adopt the neural network in this paper. It can be understood that in addition to the neural network in this paper, other applicable neural networks can also be adopted.
[0056] In one embodiment, if after the above adjustment, the coincidence degree between the 2D rendering of a tooth and the corresponding part of the tooth in the corresponding tooth photo is still not high enough, manual adjustment can be continued on this basis. Specifically, the computer system can superimpose and display the local photo and the 2D rendering of the tooth, and the user inputs an instruction to adjust the tooth pose according to the difference between the corresponding part of the tooth in the local photo of the tooth and the 2D rendering. The computer system updates the pose of the tooth in a tooth layout corresponding to the current tooth photo according to the user instruction, and re-renders based on the updated pose of the tooth to obtain the updated 2D rendering of the tooth until the 2D rendering of the tooth coincides with the corresponding part of the tooth in the tooth photo.
[0057] In one embodiment, during the above manual adjustment process, real-time rendering can be adopted, that is, as the pose of a tooth is adjusted each time, the 2D rendering of the tooth is updated.
[0058] Inspired by the present application, it can be understood that in 107, in addition to adjusting the tooth pose by combining the neural network and manual methods as above, the tooth pose can also be adjusted only by using the neural network, or only by manual means.
[0059] In 107, the adjustment of the camera pose is not involved, and the rendering of the tooth three-dimensional digital model is based on the first set of camera poses.
[0060] Please refer to Figure 4, which shows a partial photograph of tooth 31 in Figures 3A-3D and a two-dimensional rendering based on the pose of the first set of cameras and the layout of the second tooth, as displayed in an interface of a computer program used to reconstruct a three-dimensional digital model of the dental arch in one embodiment of this application.
[0061] Please refer to Figure 5, which shows a partial photograph of tooth 31 in Figure 3A-3D and a two-dimensional rendering of the tooth after its pose has been adjusted using the second neural network, as displayed in an interface of the computer program used to reconstruct the three-dimensional digital model of the dental arch.
[0062] Please refer to Figure 6, which shows a partial photograph of tooth 31 in Figure 3A-3D and a two-dimensional rendering after further manual adjustment of its pose, as displayed in an interface of the computer program used to reconstruct the three-dimensional digital model of the dental arch.
[0063] After step 107, a tooth layout is obtained for each of the N dental photographs, resulting in a total of N tooth layouts, which may differ from each other. However, what is ultimately needed is a single tooth layout that is sufficiently close to the patient's actual tooth layout when the dental photographs were taken. Furthermore, the first set of camera poses may not be accurate enough, i.e., not close enough to the actual camera pose when the dental photographs were taken. A more accurate tooth layout can be calculated based on a more accurate camera pose. Therefore, the final tooth layout and camera pose can be calculated based on the N tooth layouts and the first set of camera poses.
[0064] In step 109, an optimization algorithm is used to calculate the third tooth layout and the second set of camera poses based on the N tooth layouts and the first set of camera poses.
[0065] In one embodiment, an optimization algorithm for solving the Object Level Bundle Adjustment problem can be used to calculate the third tooth layout and the second set of camera poses based on the N tooth layouts and the first set of camera poses, which are then used as the final tooth layout and camera poses.
[0066] The general process of the above optimization algorithm is as follows:
[0067] 1) For each tooth photograph, calculate the joint pose of each tooth.
[0068] Let M c-a M is the camera pose corresponding to the tooth image a in the first set of camera poses. a-t Let M be the pose of tooth t in the tooth layout corresponding to tooth photograph a among the N tooth layouts. Then, the joint pose M of tooth t corresponding to tooth photograph a is... a-t-com M can be calculated according to the following equation (1): a-t-com =M c-a *Ma-t Equation (1)
[0069] If a dental photograph contains O teeth, then there are O joint poses.
[0070] 2) Calculate the relative camera poses of each tooth between each pair of dental photographs and filter out candidate relative camera poses.
[0071] If dental photographs a and b contain the same O teeth, then there are O relative camera poses. For dental photographs a and b, the relative camera pose M of tooth t is... ab-t-r It can be calculated according to the following equation (2):
[0072] Then, for each calculated relative camera pose, the average error of all teeth is calculated. If the average error is less than a preset threshold, the relative camera pose is used as a candidate relative camera pose.
[0073] Let tooth t be in M a-t Let P be the set of points under the pose. a-t In M b-t Let P be the set of points under the pose. b-t , where P a-t and P b-t Let be the homogeneous coordinates of the point t. Then, the error e of the tooth t is... ab-t It can be obtained by calculating according to the following equation (3): e ab-t =‖P a-t -M ab-t-r *P b-t Equation (3)
[0074] For a given relative camera pose, calculate the error of all teeth and average it to obtain the average error of all teeth under that relative camera pose.
[0075] 3) Based on the obtained candidate relative camera poses, optimize to obtain a set of relative camera poses.
[0076] For tooth images a and b, there may be multiple candidate relative camera poses.
[0077] For the N dental photographs, as long as N-1 relative camera poses are calculated, the pairwise relationship between the camera poses corresponding to these N dental photographs can be determined. Therefore, the set of relative camera poses includes N-1 relative camera poses.
[0078] Calculate the error under all possible parameter combinations and select the set of relative camera poses with the smallest error.
[0079] 4) Based on the selected set of relative camera poses, calculate the absolute camera poses corresponding to the N dental photographs.
[0080] Using one of the N dental photographs as the corresponding unit matrix in the first set of camera poses, the absolute camera pose corresponding to each dental photograph can be obtained based on the selected set of relative camera poses, which is the second set of camera poses.
[0081] 5) Optimize the pose of each tooth based on the absolute camera pose.
[0082] In one embodiment, the pose of each tooth can be optimized based on the absolute camera pose corresponding to each tooth photograph to obtain a set of tooth poses with the smallest average error of the joint pose corresponding to all tooth photographs, which is the third tooth layout.
[0083] In one embodiment, the above optimization can employ nonlinear least squares optimization methods, such as the Levenberg-Marquardt or Gauss-Newton algorithms, to minimize the error function.
[0084] Please refer to Figure 7, which shows a partial photograph of tooth 31 in Figure 3A-3D and a two-dimensional rendering based on the second set of camera poses and the third tooth layout, as displayed in an interface of the computer program used to reconstruct the three-dimensional digital model of the dental arch.
[0085] In one embodiment, operations 105-109 can be repeated based on the third tooth layout and the second set of camera poses to obtain a more accurate tooth layout and camera pose.
[0086] Please refer to Figure 8, which shows a partial photograph of tooth 31 in Figures 3A-3D and a two-dimensional rendering of the updated second set of camera poses and third tooth layout obtained after repeating operations 105-109, as displayed in an interface of the computer program used to reconstruct the three-dimensional digital model of the dental arch.
[0087] The overlap between the two-dimensional rendering of tooth 31 in Figure 8 and the corresponding part of tooth 31 in Figures 3A-3D is significantly higher than that in Figure 4. In other words, the accuracy of the third tooth layout and the second set of camera poses calculated using the method of this application is significantly higher than that of the second tooth layout and the first set of camera poses calculated using traditional methods. Or, the third tooth layout is closer to the patient's actual tooth layout when the N dental photographs were taken than the second tooth layout.
[0088] Based on the teachings of this application, it is understood that the method of this application can be used in a variety of different scenarios, including but not limited to monitoring the treatment effect during orthodontic treatment, and monitoring whether the condition relapses after orthodontic treatment is completed.
[0089] Although various aspects and embodiments of this application have been disclosed herein, other aspects and embodiments of this application will be apparent to those skilled in the art upon inspiration from this application. The various aspects and embodiments disclosed herein are for illustrative purposes only and not for limiting purposes. The scope and spirit of this application are determined solely by the appended claims.
[0090] Similarly, the diagrams may illustrate exemplary architectures or other configurations of the disclosed methods and systems, which aid in understanding the features and functions that may be included in the disclosed methods and systems. The claims are not limited to the exemplary architectures or configurations shown, and the desired features may be implemented with various alternative architectures and configurations. Furthermore, the order of the blocks given herein with respect to flowcharts, functional descriptions, and method claims should not be limited to various embodiments implemented in the same order to perform the said functions, unless explicitly indicated in the context.
[0091] Unless otherwise expressly stated, the terms and phrases used herein, and their variations thereof, should be interpreted as open-ended rather than restrictive. In some instances, the appearance of extended words and phrases such as “one or more,” “at least,” “but not limited to,” or other similar expressions should not be construed as an intention or necessity to indicate a narrower scope in examples where such extended expressions might not exist.
Claims
1. A computer-executed method for reconstructing a three-dimensional digital model of a dental arch, comprising: Obtain N photographs of the patient's teeth taken from different angles, where N is a natural number greater than or equal to 2; Obtain a first three-dimensional digital model of the patient's dentition representing the first dental layout; Based on multi-view tooth pose prediction, using a first deep neural network, based on the intrinsic parameters of the camera used to capture the N tooth photos, the N tooth photos, and the first three-dimensional digital model, a second tooth layout and a first set of camera poses are predicted, wherein the first set of camera poses includes N camera poses that correspond one-to-one with the N tooth photos. Based on single-view tooth pose adjustment, and using each of the N dental photographs as an independent reference, the pose of each tooth is adjusted to obtain N tooth layouts that correspond one-to-one with the N dental photographs; and Tooth pose and camera pose optimization: Using an optimization algorithm to solve the Object Level Bundle Adjustment problem, a third tooth layout and a second set of camera poses are calculated based on the N tooth layouts and the first set of camera poses. The second set of camera poses includes N camera poses that correspond one-to-one with the N tooth images.
2. The computer-implemented method of reconstructing a three-dimensional digital model of a dentition of claim 1, wherein, The single-view tooth pose adjustment includes: for each of the N tooth photographs, using a second deep neural network, adjusting the pose of the tooth based on the local image of each tooth in the tooth photograph and the two-dimensional rendering of the tooth, to obtain one of the N tooth layouts corresponding to the tooth photograph, wherein the two-dimensional rendering of the tooth is obtained based on the camera pose corresponding to the first group of camera poses.
3. The computer-implemented method of reconstructing a three-dimensional digital model of a dentition of claim 2, wherein, The pose of the teeth is adjusted iteratively a predetermined number of times using the second deep neural network.
4. The computer-implemented method of reconstructing a three-dimensional digital model of a dentition of claim 2, wherein, The single-view tooth pose adjustment also includes: Based on the N tooth layouts, for each of the N tooth photographs, a 2D rendering of each tooth in that photograph is overlaid on the photograph for display; and According to user instructions, the pose of at least one tooth in at least one of the N tooth layouts is adjusted to obtain an updated N tooth layout. The user instructions are input by the user based on the comparison between the two-dimensional rendering of the tooth and the corresponding part of the tooth in the tooth photograph. The optimization of the camera pose and the tooth pose is based on the updated N tooth layout.
5. The computer-implemented method of reconstructing a three-dimensional digital model of a dentition of claim 1, wherein, The first three-dimensional digital model is one of a series of successive three-dimensional digital models of the dentition, representing the patient's initial tooth layout to the target tooth layout, in an orthodontic treatment plan using a shell-shaped dental appliance.
6. The computer-implemented method of reconstructing a three-dimensional digital model of a dentition of claim 5, wherein, The first three-dimensional digital model is a three-dimensional digital model of the dental arch representing the patient's initial tooth layout.
7. The computer-implemented method of reconstruction of a three-dimensional digital model of a dentition of claim 1, wherein, The first three-dimensional digital model is a three-dimensional digital model representing the patient's maxillary or mandibular dentition.
8. The computer-implemented method of reconstruction of a three-dimensional digital model of a dentition of claim 1, wherein, The second tooth layout is closer to the current tooth layout than the first tooth layout, and the third tooth layout is closer to the current tooth layout than the second tooth layout.
9. The computer-implemented method of reconstruction of a three-dimensional digital model of a dentition of claim 1, wherein, It also includes iterating the single-view-based tooth pose adjustment and the tooth pose and camera pose optimization at least once.
10. A computer system for reconstructing a three-dimensional digital model of a dentition, comprising a storage device storing a computer program for reconstructing a three-dimensional digital model of a dentition, and a processor which, when the computer program is run, will perform the method of claim 1 for reconstructing a three-dimensional digital model of a dentition.