Methods for automatically creating 3D models for dental and orthodontic use

Abstract

A system and method are web-based applications for creating and managing 3D models of orthodontic laboratory prescriptions for use in dental clinics or laboratories. The method includes automatically detecting and removing any appliances included within a 3D image file. After the appliance has been removed, the system automatically replaces the removed image data with new image data to create a clean second 3D image, which is calculated to simulate the tooth surface positioned beneath the removed appliance. The method can also automatically refine the second 3D image of the patient's teeth or remove any artifacts remaining after the creation of the second 3D image. The second 3D image can then be used as the basis for modifying the patient's orthodontic or dental prescription.

Description

Technical Field

[0001] This invention relates to the field of manufacturing custom orthodontic and dental appliances, and more particularly to the automatic editing of 3D images for creating orthodontic or dental prescriptions for patients. Background Technology

[0002] In orthodontic clinics and laboratories, prescriptions that were once prepared by hand using resin impressions or molds are increasingly being executed in digital workspaces. Instead of creating physical molds of the patient's teeth, patients simply have their mouths and teeth scanned via intraoral digital scanning. This process is uncomfortable and time-consuming for patients.

[0003] Once a patient has had their teeth digitally scanned, the orthodontist can use the data from that scan to create a 3D image corresponding to the patient's teeth. Since each prescription is unique to each patient and can consist of multiple components or appliances with very specific designs, the 3D image is manipulated accordingly to add or remove any appliances that may be necessary to perform a new or updated prescription by the orthodontist. Once completed, the specifications corresponding to the appliances added to the 3D image, as well as the 3D image itself, are sent to the laboratory for fabrication. The use of 3D models allows orthodontists to virtually apply or reapply different appliances to a patient's teeth without having to use physical castings of the patient's teeth, thus significantly reducing the time and cost required to prepare orthodontic prescriptions for patients.

[0004] However, problems arise for patients who already wear orthodontic appliances (including braces) and subsequently undergo intraoral scans that necessitate modifications or changes to their prescriptions. Therefore, the final 3D image of a patient's teeth inherently includes these pre-existing appliances, making it difficult (if not impossible) for orthodontists to apply new orthodontic appliances or adjust pre-existing ones within that 3D image.

[0005] Furthermore, while 3D image editing tools exist that can remove brackets from initial 3D images, these tools are quite labor-intensive and can take several hours to edit for a single patient. Specifically, many 3D image tools require the user to manually select the image target to be removed, remove that target, and then complete any additional image editing. Moreover, many of these same 3D image editing tools lack the ability to reconstruct the tooth surface after bracket removal, resulting in “holes” or blank spots where no image data exists.

[0006] What is needed is a method for modifying 3D images (e.g., removing orthodontic appliances including brackets from 3D images) to aid in the design and / or manufacture of dental prostheses. This method should be simple and easy to use, and largely automated with minimal user input required for rapid implementation. Summary of the Invention

[0007] The illustrated embodiments of the present invention include, within their scope, a method for creating and manipulating a 3D model that can subsequently be used in the creation of orthodontic or dental prescriptions. The method includes obtaining a 3D image corresponding to a patient's teeth, automatically removing brackets from the 3D image, and / or restoring and refining the 3D image after bracket removal. The method also includes automatically trimming or basing the 3D image on the desired 3D model. The modified 3D image can then be used in the design of custom orthodontic or dental appliances, thereby helping the prescribing user update the patient's prescription.

[0008] This invention provides a method for automatically creating a 3D tooth model. The method includes uploading a first 3D image to an upload database, detecting image data corresponding to at least one appliance within the first 3D image, and then deleting the detected image data corresponding to the at least one appliance from the first 3D image. Next, 3D image data is inferred by calculating an approximate surface of the tooth positioned below the deleted image data to create a second 3D image. The second 3D image is then saved to a processed database. The method also specifies that the steps of detecting and deleting image data corresponding to at least one appliance from the first 3D image and the step of inferring the calculated 3D image data approximating the tooth surface to create the second 3D image are executed sequentially and automatically.

[0009] In one embodiment, the method further includes refining the inferred 3D image data of the second 3D image, the inferred 3D image data corresponding to a tooth surface positioned below the deleted image data corresponding to at least one appliance. Specifically, in a related embodiment, refining the inferred 3D image data includes trimming at least a portion of the inferred 3D image data of the second 3D image, the inferred 3D image data corresponding to a tooth surface positioned below the deleted image data corresponding to at least one appliance. In another related embodiment, refining the inferred 3D image data includes adding to at least a portion of the inferred 3D image data, the inferred 3D image data corresponding to a tooth surface positioned below the deleted image data corresponding to at least one appliance. In yet another related embodiment, refining the inferred 3D image data includes defining the intersection between the inferred 3D image data and image data corresponding to the patient's gingiva within the second 3D image, the inferred 3D image data corresponding to a tooth surface positioned below the deleted image data corresponding to at least one appliance. In another embodiment, refining the inferred 3D image data includes smoothing at least a portion of the inferred 3D image data, which corresponds to a tooth surface positioned below the deleted image data corresponding to at least one appliance. Regardless of how the inferred 3D image data is refined, any residual artifacts can also be removed from the inferred 3D image data of the second 3D image.

[0010] In another embodiment, after calculating inferred 3D image data of the tooth surface below the image data to be deleted, the saving of a second 3D image to the processed database is performed automatically in sequence.

[0011] In another embodiment, detecting image data corresponding to at least one device within the first 3D image includes detecting image data corresponding to orthodontic brackets.

[0012] In another embodiment, the method further includes transmitting metadata associated with the first 3D image to a server communicating with an upload database and a processed database. In this case, the first 3D image is sent from the server to a queue before image data corresponding to at least one device within the first 3D image is detected.

[0013] In another embodiment, uploading the first 3D image to the upload database specifically includes automatically saving the first 3D image to a file storage device within the upload database.

[0014] In a related embodiment, uploading the first 3D image to the upload database includes uploading a first STL file containing the first 3D image.

[0015] In one embodiment, saving the second 3D image to the processed database further includes saving the second 3D image to a file storage device within the processed database.

[0016] The present invention also provides a method for automatically creating a 3D tooth model. The method includes uploading a first 3D image to an upload database, detecting image data corresponding to the gingiva within the first 3D image, and then trimming the detected image data corresponding to the gingiva. Next, the image data corresponding to the gingiva is further smoothed to create a second 3D image, and then the second 3D image is saved to a processed database. The method also specifies that the steps of detecting, trimming, and smoothing the image data corresponding to the gingiva within the first 3D image are performed automatically and sequentially.

[0017] In one embodiment, the method further includes removing any artifacts from the image data of the second 3D image.

[0018] In another embodiment, the method further includes establishing a basis for image data corresponding to the gingiva within the first 3D image.

[0019] In a particular embodiment, trimming the image data of the detection corresponding to the gingiva includes generating a trimming line.

[0020] In another embodiment, the method includes transmitting metadata associated with a first 3D image to a server communicating with an upload database and a processed database. Then, the first 3D image is sent from the server to a queue before image data corresponding to gingiva within the first 3D image is detected.

[0021] Although the apparatus and method have been or will be described for the sake of grammatical fluency with a functional interpretation, it should be clearly understood that, unless expressly stated pursuant to Section 112 of Title 35 of the United States Code, the claims should not be construed as necessarily limited to any manner of construction limited by “means” or “steps,” but should conform to the meaning and full scope of equivalents provided by the definition of claims under the principle of judicial equivalence, and where claims are expressly stated pursuant to Section 112 of Title 35 of the United States Code, the claims should conform to full legal equivalents pursuant to Section 112 of Title 35 of the United States Code. This disclosure can now be better visualized by turning to the following figures, in which the same numerals denote the same elements. Attached Figure Description

[0022] Figure 1 This is a flowchart of a method for automatically creating 3D dental or orthodontic models provided by the present invention.

[0023] Figure 2 It is shown that it is used from in Figure 1The flowchart shown is a sub-method for automatically removing orthodontic appliances from a 3D image provided in a method for automatically creating a 3D dental or orthodontic model.

[0024] Figure 3 This shows the use of automatic trimming in Figure 1 The flowchart shows a sub-method based on a 3D image provided in the method for automatically creating a 3D dental or orthodontic model.

[0025] This disclosure and its various embodiments can now be better understood by turning to the following detailed description of preferred embodiments, which are illustrated examples of the embodiments defined in the claims. It should be clearly understood that the embodiments defined by the claims may be more extensive than the illustrated embodiments described below. Detailed Implementation

[0026] The illustrated system and accompanying method involve a web-based application that receives Standard Trigonometric Language (STL) image files from intraoral digital scans or cone-beam CT scans of a patient's mouth and teeth, or from digital research model services directly from a laboratory, physician, or another web-based application used to manage prescriptions within a dental clinic, office, or laboratory. The method allows orthodontists to manipulate their prescriptions or change prescriptions for patients already wearing braces or other orthodontic appliances including brackets or other similar orthodontic components. The system and method provide an algorithm or a set of algorithms to automatically detect the presence of any brackets, straps, appendages, wires, or any other orthodontic or dental appliance and remove their corresponding image data from the 3D image 12. The system and method also provide an algorithm or a set of algorithms to automatically perform additional functions for the design and manufacture of assistive dental appliances. These additional functions include automatically cleaning the STL file, closing the model, and adding a base in a printable or further detailed file format that can be described using CAD software. Image data corresponding to the tooth surfaces beneath the removed appliance is then reconstructed to provide a complete 3D image of the patient's teeth without any appliances.

[0027] The current systems and methods disclosed herein can be used independently or as standalone applications. For example, the current systems and methods can be integrated into the platforms disclosed in U.S. Patent 10,299,891, filed March 16, 2016, entitled "System and Method for Ordering and Manufacturing Customized Orthodontic Appliances and Product," and U.S. Application 16 / 712,362, filed December 12, 2019, entitled "System and Method for Ordering and Manufacturing Customized Dental Appliances and the Tracking of Orthodontic Products," the entire contents of which are incorporated herein by reference.

[0028] By turning Figures 1-3The illustrated system and method 10 can be understood. First, a user 12, possibly a dentist, orthodontist, or lab technician, performs an intraoral scan of a patient's teeth to generate a Standard Triangulated Language (STL) file or other 3D image file. User 12 begins by transmitting metadata to server 201, including the file name and other parameters. User 12 receives a response including an upload point to upload database 14, a web-based application that includes a file storage 203 and a relational database 202 for storing and saving uploaded STL image files. Upon receiving the STL image file from database 14, server 201 sends the uploaded STL image file to queue 204, where it awaits further editing in file processing step 205. File processing step 205 uses a domain-specific language (DSL) or native computer language to detect any orthodontic or dental appliances that may be included in the 3D image, and then deletes image data corresponding to any detected appliances. Next, for those teeth from which the appliance has been removed, data processing step 205 includes calculating the curvature of each tooth surface and using the result to approximate the appearance of the tooth surface beneath the now-removed appliance. Once the original STL image file is received from user 12 without any further input from user 12, an Automatic Bracket Removal (ABR) process occurs. The ABR process is defined by the detection of image data corresponding to the appliance, the deletion of image data corresponding to the detected appliance, and the reconstruction of the tooth surface beneath the removed appliance. Once the STL file has been automatically edited through data processing step 205, it is sent to a processed database 16 accessible to user 12. The processed database 16 itself includes its own corresponding file storage 207 and relational database 206, which are used to store or save the edited STL file. The relational database 202 in the upload database 14 and the relational database 206 in the processed database 16 both include metadata such as processing status and upload time that can be quickly forwarded to user 12, while the file storage 203 in the upload database 14 and the file storage 207 in the processed database 16 respectively include large STL image files before and after processing.

[0029] Turn now Figure 2 Flowchart Figure 2The flowchart below outlines the ABR method 100 of the present invention that occurs in data processing step 205. First, an STL file comprising one or more 3D meshes 110 is obtained via an intraoral scan of the patient's mouth and teeth, as is known in the art. The 3D meshes 110 include not only a display of the patient's teeth and gingival line, but also a display of any brackets or other orthodontic appliances currently used for the patient's orthodontic treatment.

[0030] First, the obtained 3D mesh 110 is as follows Figure 2 The bracket segmentation and detection module 112, as shown, operates by automatically prompting a 3D segmentation step 101, followed by a detection step 102. Detection step 102 is performed after orthogonal image detection of image data associated with any bracket or orthodontic appliance. The bracket segmentation and detection module 112 divides the original 3D mesh 110 into a segmented mesh and multiple separate bracket meshes. Specifically, unique separate bracket meshes are determined by identifying vertex clusters and labeling them as brackets. The size, shape, and structure of each separate bracket mesh are then compared to a set of reference brackets to determine whether the bracket mesh should be retained or ignored for detection. Conversely, if the bracket segmentation and detection module 112 fails to detect any image data associated with any bracket, a threshold criterion is not met, and the ABR method 100 is terminated. The segmented mesh then contains image data corresponding to any brackets that were subtracted or removed from the detection in step 104. Simultaneously, 3D inference is performed on the separated bracket mesh in step 103, where 3D image data is calculated to approximate the tooth surface positioned below the deleted image data to create a complete 3D image of the patient's teeth. Specifically, 3D inference step 103 uses machine learning techniques trained on a dataset from intraoral scans without brackets to create new image data corresponding to the tooth surface previously positioned below the now-deleted image data associated with brackets. At step 105, new image data corresponding to the 3D mesh without brackets is formed, where the resulting image from 3D inference step 103 is added, combined, or integrated into the new image data corresponding to the segmented mesh without brackets in step 104. The final result is a second, updated, or corrected 3D mesh 114 with image data associated with any brackets or other orthodontic appliances that have been effectively and efficiently removed, thus providing a clean image of the patient's teeth and offering the user a convenient means to adjust or modify their prescription accordingly. Furthermore, in step 105, any remaining artifacts or miscellaneous image data are removed from the corrected 3D mesh 114, thereby generating a clean image free from errors that could interfere with any subsequent operations by the user.

[0031] In the relevant embodiments, we now turn to Figure 3Flowchart Figure 3 The flowchart outlines the trimming and foundational method 200 of the present invention that occurs in data processing step 205. First, an STL image file comprising one or more 3D meshes 110 is obtained via an intraoral scan of the patient's mouth and teeth, as known in the art. The 3D meshes 110 include not only a display of the patient's teeth and gingival line, but also a display of any brackets or other orthodontic appliances currently used for the patient's orthodontic treatment.

[0032] First, the obtained 3D mesh 110 is as follows Figure 3 The tooth segmentation and gingival line detection module 116, as shown, operates by automatically prompting a 3D segmentation step 201, followed by a trimming line generation step 202. Specifically, trimming lines in step 202 are generated by selecting all vertices of the segmented teeth and vertices within a user-defined distance (typically around 2 or 3 mm) from the segmented teeth. The trimming lines are then defined as vertices at the selected boundaries. Any unselected image data is then trimmed or subtracted from the 3D mesh 110, and these unselected image data are further smoothed or trimmed in step 204. At this point, depending on the user's final goal, the trimmed and smoothed 3D mesh may now undergo further foundation or additive processes in foundation step 204. After foundation step 204, any remaining artifacts or miscellaneous image data are removed from the 3D mesh 114 in cleaning step 205, resulting in a clean image free from errors that could potentially interfere with any subsequent user operations. Alternatively, the trimmed and smoothed 3D mesh can be cleaned or corrected directly via cleaning step 205 as described above, instead of performing foundation step 204. In any case, the end result of the trimming and basing method 200 is an updated or corrected 3D mesh 118, which contains image data related to any detected gingiva that has been effectively and efficiently trimmed, smoothed, and based on, thus providing a clean image of the patient's teeth and offering the user a convenient means to adjust or modify their prescription accordingly. It is noteworthy that the trimming and basing method 200 can be performed independently of the ABR method 100 discussed above, thus allowing the user to trim and base the 3D mesh 110 without first digitally removing any image data related to any brackets, and vice versa.

[0033] Once all automated image processing is complete, the user can use the revised 3D meshes 114, 118 to create new or different prescriptions for the patient by applying new brackets, straps, or appendages. Once applied, the user can send the revised 3D image to a laboratory where the corresponding appendage can be fabricated and returned to the user, who can then apply it to the patient.

[0034] The illustrated embodiments of the system and method can now be understood as an orthodontic or dental appliance removal system and method designed to efficiently and automatically remove orthodontic or dental appliances from intraoral scan images, thereby allowing physicians and other users to access and review and update their patients' prescriptions. The system allows for flexibility based on customization specific to the laboratory using the system. As it is a web-based system, it can be dynamically updated, with physician data stored via the cloud. Furthermore, the system and method can further refine the 3D model by removing any remaining image artifacts and by adding or trimming elements from the 3D model, which will then be used in the design of the appliance.

[0035] Many changes and modifications can be made by those skilled in the art without departing from the spirit and scope of the embodiments. Therefore, it must be understood that the illustrated embodiments are illustrative for purposes of illustration only and should not be construed as limiting the embodiments defined by the following embodiments and their various embodiments.

[0036] Therefore, it must be understood that the illustrated embodiments are illustrative for purposes of illustration only and should not be construed as limiting the embodiments defined by the following claims. For example, although the elements of a claim are set forth below in some combination, it must be clearly understood that the embodiments include combinations of other fewer, more, or different elements disclosed above, even if not initially claimed in such combinations. The teaching of combining two elements in a claimed combination is further understood to also allow claimed combinations in which two elements are not combined with each other, but such claimed combination may be used alone or in combination with other combinations. The omission of any disclosed elements in the embodiments is expressly considered within the scope of the embodiments.

[0037] The terms used in this specification to describe various embodiments are to be understood not only in the sense of their common meaning, but also to include structures, materials, or actions beyond the scope of their common meaning by the specific definitions in this specification. Therefore, if an element can be understood to include more than one meaning in the context of this specification, its use in the claims must be understood to apply universally to all possible meanings supported by this specification and by the terms themselves.

[0038] Therefore, the terms or elements of the following claims are defined in this specification not only to include combinations of elements literally stated, but also to include all equivalent structures, materials, or actions used to perform substantially the same function in substantially the same manner to obtain substantially the same result. Thus, in this sense, it is possible to consider equivalent substitutions of two or more elements for any element in the following claims, or to substitute a single element for two or more elements in the claims. Although elements may be described above as acting in certain combinations and even combinations as originally claimed, it should be clearly understood that in some cases, one or more elements from the claimed combination may be removed from the combination, and the claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0039] Non-substantial variations to the claimed subject matter that are now known or subsequently designed and observed by a person skilled in the art are expressly considered equivalent to those within the scope of the claims. Therefore, obvious substitutes that a person skilled in the art now or later knows are defined as being within the scope of the defined elements.

[0040] Therefore, the claims should be understood to include the content specifically shown and described above, the content that is conceptually equivalent, the content that can be obviously substituted, and the content that essentially contains the basic ideas of the embodiments.

Claims

1. A method for automatically creating 3D tooth models, comprising: Upload the first 3D image to the upload database; The first 3D image is divided into a segmentation grid and multiple slot grids; Image data corresponding to at least one appliance in the first 3D image is detected by comparing each of the plurality of bracket grids with a set of reference brackets; Delete the detected image data corresponding to the at least one device from the segmented grid; A second 3D image is created by inferring the 3D image data calculated to approximate the tooth surface set below the deleted image data by calculating the curvature of the tooth surface below the deleted image data. as well as Save the second 3D image to the processed database. The steps include, in sequence, automatically executing the steps of detecting and deleting image data corresponding to the at least one appliance from the first 3D image and inferring 3D image data calculated to approximate the tooth surface to create a second 3D image.

2. The method of claim 1, further comprising refining the inferred 3D image data of the second 3D image corresponding to the tooth surface disposed below the deleted image data corresponding to the at least one appliance.

3. The method according to claim 1, wherein, After inferring the 3D image data calculated to approximate the tooth surface set below the deleted image data, the second 3D image is automatically saved to the processed database in sequence.

4. The method according to claim 2, wherein, Refining the inferred 3D image data of the second 3D image corresponding to the tooth surface positioned below the deleted image data corresponding to the at least one appliance includes trimming at least a portion of the inferred 3D image data of the second 3D image, the inferred 3D image data corresponding to the tooth surface positioned below the deleted image data corresponding to the at least one appliance.

5. The method according to claim 2, wherein, The refined 3D image data corresponding to the tooth surface positioned below the deleted image data corresponding to the at least one appliance includes adding at least a portion of the inferred 3D image data corresponding to the tooth surface positioned below the deleted image data corresponding to the at least one appliance.

6. The method according to claim 2, wherein, Refining the inferred 3D image data of the second 3D image corresponding to the tooth surface positioned below the deleted image data corresponding to the at least one appliance includes defining the intersection between the inferred 3D image data and image data corresponding to the patient's gingiva within the second 3D image, the inferred 3D image data corresponding to the tooth surface positioned below the deleted image data corresponding to the at least one appliance.

7. The method according to claim 2, wherein, Refining the inferred 3D image data of the second 3D image corresponding to the tooth surface disposed below the deleted image data corresponding to the at least one appliance includes smoothing at least a portion of the inferred 3D image data corresponding to the tooth surface disposed below the deleted image data corresponding to the at least one appliance.

8. The method according to claim 1, wherein, Detecting the image data corresponding to the at least one device in the first 3D image includes detecting image data corresponding to orthodontic brackets.

9. The method according to claim 2, wherein, Refining the inferred 3D image data of the second 3D image corresponding to the tooth surface positioned below the deleted image data corresponding to the at least one appliance includes removing any artifacts from the inferred 3D image data of the second 3D image.

10. The method of claim 1, further comprising transmitting metadata related to the first 3D image to a server communicating with the uploaded database and the processed database.

11. The method of claim 10, further comprising sending the first 3D image from the server to a queue before detecting image data corresponding to at least one device within the first 3D image.

12. The method according to claim 1, wherein, Uploading the first 3D image to the upload database includes automatically storing the first 3D image in a file storage device within the upload database.

13. The method according to claim 1, wherein, Uploading the first 3D image to the upload database includes uploading a first STL file containing the first 3D image.

14. The method according to claim 1, wherein, Saving the second 3D image to the processed database includes saving the second 3D image to a file storage device within the processed database.

Images