Dental restoration automation

Abstract

A computer-implemented method and system of automation of a virtual dental restoration design, comprising: receiving a 3D virtual dental model of at least a portion of a dentition of a patient, the 3D virtual dental model comprising at least one virtual prepared tooth, the virtual prepared tooth comprising a digital characterization of a physical prepared tooth prepared by a dentist; performing an automated virtual prosthesis design using the 3D virtual dental model; virtually displaying to the dentist one or more physical preliminary tooth problems detected while performing the automated virtual prosthesis design; and virtually displaying the generated virtual prosthesis to the dentist for one or more adjustments if no physical preliminary tooth problem is detected while performing the automated virtual prosthesis design.

Description

[0001] This application claims priority to and the benefit of co-pending U.S. Provisional Patent Application Serial No. 63 / 369,151, filed on July 22, 2022, entitled “Integrated Dental Restoration Design Process and System,” co-pending U.S. Provisional Patent Application Serial No. 63 / 380,374, filed on October 20, 2022, entitled “Dental Restoration Automation,” and U.S. Utility Patent Application Serial No. 18 / 353,947, filed on July 18, 2023, entitled “Dental Restoration Automation,” all of which are incorporated herein by reference in their entirety. Background Art

[0002] Traditional dental practice typically involves a dentist receiving a patient in his or her dental office and performing an examination of the patient's dentition. The examination / treatment in a dental office may be referred to as "chairside" because the patient is in a dental chair. During a typical chairside visit, the patient may require a dental restoration for one or more teeth. The dentist typically anesthetizes the patient or a portion of the patient's dentition to physically prepare the one or more teeth to receive the dental restoration. During this preparatory chairside visit, the dentist may modify one or more existing teeth in a manner that allows the restoration to properly fit onto the particular prepared tooth or teeth. Examples of restorations may include, but are not limited to, crowns, bridges, inlays, and the like.

[0003] In some cases, once one or more physical teeth are prepared, the dentist can directly scan the patient's dentition including the physical prepared tooth / teeth to generate a 3D virtual model of the patient's dentition, the 3D virtual model including the virtual prepared teeth representing the physical prepared teeth. In some cases, such scanning can be performed, for example, using an optical scanner (such as an intraoral scanner). This can generate a 3D virtual model of at least a portion of the patient's dentition including the physical prepared tooth / teeth.

[0004] In some cases, the dentist can have the patient create an impression by having the patient bite down into an impression material disposed in an impression tray to create a physical impression. This physical impression can be mailed to a dental laboratory where a plaster mold of the patient's dentition is made from the physical impression.

[0005] In some cases, a 3D physical plaster model of the patient's dentition can be made from the physical impression and then optically scanned in a dental laboratory. Alternatively, the physical impression itself can be scanned using an optical scanner (such as an intraoral scanner or a computed tomography ("CT") scanner) to generate a 3D virtual model of the patient's dentition including the prepared tooth / teeth in a dental laboratory.

[0006] More recently, CAD / CAM dentistry (computer-aided design and computer-aided manufacturing in dentistry) has provided a wide range of dental restorations, including crowns, veneers, inlays and onlays, fixed bridges, dental implant restorations, and orthodontic appliances. In a typical CAD / CAM-based dental procedure, the treating dentist may prepare the tooth to be restored as a crown, inlay, onlay, or veneer. The prepared tooth and its surroundings are then scanned by a three-dimensional (3D) imaging camera and uploaded to a computer for design. Alternatively, the dentist may obtain an impression of the tooth to be restored, and the impression may be scanned directly, or formed into a model to be scanned and uploaded to a computer for design.

[0007] Current dental CAD is often tedious, time consuming, and can result in design inconsistencies and errors. In some cases, due to problems with the physically prepared tooth, problems may arise that may prevent or hinder the design of a restoration for the prepared tooth. These problems may include, but are not limited to, for example, one or more undercut areas on the physically prepared tooth, margin line problems, one or more gap problem areas, and / or restoration insertion problems. If not resolved, these problems may recur over time. Detecting and resolving these problems with the dentist may be desirable because errors can be minimized. Summary of the invention

[0008] A computer-implemented method for automating virtual dental restoration design may include: receiving a 3D virtual dental model of at least a portion of a patient's dentition, the 3D virtual dental model including at least one virtual prepared tooth, the virtual prepared tooth including a digital representation of a physical prepared tooth prepared by a dentist; performing automated virtual restoration design using the 3D virtual dental model; virtually displaying to the dentist one or more physical prepared tooth problems detected when performing the automated virtual restoration design; and, if no physical prepared tooth problems are detected when performing the automated virtual restoration design, virtually displaying the generated virtual restoration to the dentist for one or more adjustments.

[0009] A non-transitory computer-readable medium storing executable computer program instructions for providing automation of virtual dental restoration design, the computer program instructions may include instructions for: receiving a 3D virtual dental model of at least a portion of a patient's dentition, the 3D virtual dental model including at least one virtual prepared tooth, the virtual prepared tooth including a digital representation of a physical prepared tooth prepared by a dentist; performing automated virtual restoration design using the 3D virtual dental model; virtually displaying to the dentist one or more physical prepared tooth problems detected when performing the automated virtual restoration design; and, if no physical prepared tooth problems are detected when performing the automated virtual restoration design, virtually displaying the generated virtual restoration to the dentist for one or more adjustments.

[0010] A system for automating virtual dental restoration design may include: a processor; and a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium including instructions executable by the processor to perform steps including the following: receiving a 3D virtual dental model of at least a portion of a patient's dentition, the 3D virtual dental model including at least one virtual prepared tooth, the virtual prepared tooth including a digital representation of a physical prepared tooth prepared by a dentist; performing automated virtual restoration design using the 3D virtual dental model; virtually displaying to the dentist one or more physical prepared tooth problems detected when performing the automated virtual restoration design; and, if no physical prepared tooth problems are detected when performing the automated virtual restoration design, virtually displaying the generated virtual restoration to the dentist for one or more adjustments. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A diagram of a cloud computing environment for a dental restoration automation system in some embodiments is shown.

[0012] Figure 2 A diagram illustrating automation of dental restorations in some embodiments.

[0013] Figure 3 A diagram illustrating automation of dental restorations in some embodiments.

[0014] Figure 4 A diagram illustrating one or more steps that may be used in some embodiments for automated design of a virtual dental restoration.

[0015] Figure 5(a) to Figure 5(c) Decimation performed on a 3D virtual model mesh in some embodiments is shown.

[0016] Figure 6 A diagram of a convolutional neural network, such as in some embodiments, is shown.

[0017] Figure 7A top perspective view of an example of a 2D depth map of a digital model, such as in some embodiments, is shown.

[0018] Figure 8 A top perspective view of the 3D virtual model with occlusal orientation, virtual preparation mold area, and buccal orientation is shown.

[0019] 9(a) and 9(b) illustrate diagrams of hierarchical neural networks, such as in some embodiments.

[0020] Fig.10 A diagram of a deep neural network, such as in some embodiments, is shown.

[0021] Fig.11 A diagram illustrating a computer-implemented method of automatic edge line proposal, such as in some embodiments.

[0022] Fig.12 A perspective view of an example of a 3D digital model showing, for example, proposed edge lines from a reference edge line in some embodiments is shown.

[0023] 13( a ) and 13 ( b ) show perspective views of a 3D digital model with prepared teeth and proposed margin lines, for example in some embodiments.

[0024] Fig.14 A 3D virtual model of a virtually prepared tooth with margin lines in some embodiments is shown.

[0025] Fig.15 is a flow chart of a process for generating a 3D denture model using a deep neural network according to some embodiments of the present disclosure.

[0026] Fig.16 is a graphical representation of the input and output of a deep neural network according to some embodiments of the present disclosure.

[0027] 17( a ) and 17 ( b ) are flowcharts of a method for training a deep neural network to generate 3D dentures according to some embodiments of the present disclosure.

[0028] Fig.18 The 3D virtual model of the virtual crown is shown lifted above the virtual prepared tooth along the insertion direction in some embodiments.

[0029] Fig.19 A diagram illustrating a computer-implemented method of automatic edge line proposal in some embodiments.

[0030] Fig. 20 A 3D virtual model of a virtually prepared tooth with an open margin line in some embodiments is shown.

[0031] Fig.21 A perspective view of a 3D virtual model with virtually prepared teeth and marked open edges in some embodiments is shown.

[0032] Fig. 22 A perspective view of a 3D virtual model with virtually prepared teeth and marked spacing issues in some embodiments is shown.

[0033] Figure 23(a) to Figure 23(g) One or more adjustments made to the virtual restoration in the 3D virtual dental model are shown.

[0034] Fig.24 is a flow chart of some embodiments.

[0035] Fig.25 is a diagram illustrating a computing system in some embodiments. DETAILED DESCRIPTION

[0036] For the purpose of this description, certain aspects, advantages and novel features of the embodiments of the present disclosure are described herein. The disclosed methods, devices and systems should not be interpreted as limiting in any way. On the contrary, the present disclosure is directed to all novel and non-obvious features and aspects of the various disclosed embodiments (individually and in various combinations and sub-combinations with each other). The methods, devices and systems are not limited to any particular aspect or feature or combination thereof, nor do the disclosed embodiments require the presence of any one or more specific advantages or solutions to problems.

[0037] Although the operations of some disclosed embodiments are described in a particular order of arrangement for ease of presentation, it should be understood that unless a specific statement set forth below requires a specific order, this description covers reordering. For example, operations described in sequence may be reordered or performed simultaneously in some cases. In addition, for simplicity, the accompanying drawings may not show the various ways in which the disclosed methods can be used in conjunction with other methods. In addition, the specification sometimes uses terms such as "providing" or "implementing" to describe the disclosed methods. The actual operations corresponding to these terms may vary depending on the specific implementation, and can be easily discerned by a person of ordinary skill in the art.

[0038] As used in this application and the claims, the singular forms "a," "an," and "the" include the plural forms unless the context clearly dictates otherwise. Furthermore, the term "includes" means "comprises." Furthermore, the terms "coupled" and "associated" generally indicate electrical, electromagnetic, and / or physical (e.g., mechanical or chemical) coupling or linking, and do not exclude the presence of intervening elements between coupled or associated items unless specifically stated to the contrary.

[0039] In some examples, values, processes, or devices may be referred to as "lowest," "optimal," "minimum," etc. It should be appreciated that such descriptions are intended to indicate that choices may be made among many alternatives, and that such choices are not necessarily better, lesser, or otherwise preferred over other choices.

[0040] In the following description, certain terms such as "above", "below", "upper", "lower", "horizontal", "vertical", "left", "right", etc. may be used. Where applicable, these terms are used to provide some clarity when dealing with relative relationships. However, these terms are not intended to imply absolute relationships, positions and / or orientations. For example, for an object, the "upper" surface can become the "lower" surface simply by turning the object over. Nevertheless, it is still the same object.

[0041] All references to any other U.S. applications and patents, and any other publications, are incorporated herein by reference in their entirety.

[0042] As used herein, the term "dental restoration" may refer to any dental restoration (prosthesis) including, but not limited to, crowns, bridges, dentures, partial dentures, dental implants, onlays, inlays, or veneers.

[0043] Some embodiments disclose computer-implemented methods for automating virtual dental restoration design. Some embodiments of the present disclosure may include workflows that can automatically perform one or more steps in an integrated virtual restoration design automation process and system. As part of the integrated workflow, some embodiments of the present disclosure may include a computer-implemented method and / or system for designing a dental restoration associated with a dental model of a dentition. In some embodiments, the method and / or system may provide a simplified automated workflow to automatically virtually design a dental restoration and provide feedback to a dentist about preparing prepared teeth. In some embodiments, the integrated workflow, together with any corresponding computer-implemented method and / or system, may be implemented in a cloud computing environment. As known in the art, a cloud computing environment may include, but is not limited to, one or more devices, such as, for example, one or more computing units (such as servers), such as networks, storage devices, and / or applications enabled over the Internet and accessible by one or more licensed client devices. For example, an example of a cloud computing environment may include Amazon Web Services. Other cloud computing environment types may be used, including, but not limited to, for example, private cloud computing environments that are available to a limited set of clients (such as clients within a company). In some embodiments, the system may be implemented in a cloud computing environment.

[0044] Figure 1An example of a cloud computing environment or system 102 for supporting an integrated digital workflow for providing dental restoration design and / or manufacturing according to some embodiments is shown. In some embodiments, the cloud computing environment 102 may include a dental restoration cloud server 104, an automated design feature 103, a storage device 107 (stored directly in a file system of a storage device, in a database, or in any other storage device known in the art), and other components for cloud computing generally known in the art. In some embodiments, each of the components in the cloud computing environment 102 may be interconnected, for example, through a dental restoration cloud server 104 or directly interconnected with each other or through one or more other servers or computers within the cloud computing environment or system 102. In some embodiments, one or more client devices 108 may be connected to the dental restoration cloud server 104 directly or through one or more networks 105. For example, one or more client devices 108 may each be connected to one or more scanners 109 known in the art to scan a patient's dentition or a dental impression and provide a 3D virtual dental model of at least a portion of the patient's dentition. In some embodiments, one or more client devices 108 may be located, for example, in a dental clinic. The dental restoration cloud server 104 may be connected to one or more manufacturing suppliers 110 directly and / or via one or more networks 105. For simplicity and clarity of description, Figure 1 Only one dental restoration cloud server 104, one automated design feature 103, one client device 108, one scanner 109, one third-party manufacturing supplier 110, and one storage device 107 are shown. Embodiments of the cloud computing environment 102 may have multiple dental restoration cloud servers 104, automated design features 103, client devices 108, scanners 109, manufacturing suppliers 110, and storage devices 107. Similarly, Figure 1 The features and arrangements / connections made by the various entities of may be different in different embodiments. In some embodiments, the scanner 109 may be located in a dentist's office. In some embodiments, the client device 108 may be located in a dentist's office. In some embodiments, one or more manufacturing suppliers 110 may include, for example, one or more dental laboratories. Figure 1 One or more of the illustrated features may be implemented with and / or in one or more computing environments.

[0045] As part of the automation of dental restorations, in some embodiments, the dental restoration cloud server 104 may receive dental restoration cases from client devices 108 operated by clients, manage dental restoration cases between different clients, and then provide clients with completed dental restoration designs and / or milled dental restorations. In some embodiments, the dental restoration case may include a design-only case that only requests the dental restoration cloud server 104 to provide a virtual design of a dental restoration. In some embodiments, the dental restoration case may request the dental restoration cloud server 104 to not only provide a design, but also manufacture a dental restoration. In some embodiments, the dental restoration case may only request manufacturing.

[0046] Some embodiments of a computer-implemented method for automating the design of a virtual dental restoration may include receiving a 3D virtual dental model of at least a portion of a patient's dentition. The 3D virtual dental model may include at least one virtual prepared tooth. The virtual prepared tooth may be a virtual representation of a physical prepared tooth prepared by a dentist. In some embodiments, the virtual dental restoration design automation may also receive an opposing jaw 3D virtual dental model. The opposing jaw 3D virtual dental model may include at least a portion of the patient's dentition opposite the physical prepared tooth. The opposing jaw 3D virtual dental model may include at least one virtual opposing jaw tooth corresponding to at least one virtual prepared tooth, but on an opposing jaw. In some embodiments, the dentist may be a clinical dentist sitting in a dental clinic. In some embodiments, receiving the 3D virtual dental model may be performed in a cloud computing environment.

[0047] In some embodiments, the 3D virtual dental model and the antagonist 3D virtual dental model may be generated by any process of scanning the patient's dentition or a physical impression of the patient's dentition and generating a virtual 3D dental model of the patient's dentition. In some embodiments, the 3D virtual dental model and the antagonist 3D virtual dental model may be generated from an intraoral scan of the patient's dentition. In some embodiments, for example, an intraoral scan may be performed to produce two virtual dental models - a 3D virtual dental model with virtual prepared teeth and an antagonist 3D virtual dental model with virtual antagonist teeth. For example, in some embodiments, an intraoral scanning device may be handheld and may be used by a dentist, technician, or user to scan the patient's dentition. A standard intraoral scanning device and associated hardware and software may then generate the virtual 3D dental model as a standard STL file or other suitable standard format. An example of an intraoral scanner may be provided by Align Technologies. Intraoral Scanner. Another example of an intraoral scanner may be the i700 intraoral scanner provided by Medit. Other intraoral scanners or other scanners and / or scanning techniques for generating a 3D virtual dental model of at least a portion of a patient's dentition may also be used. In some embodiments, the scanning process may generate an STL, PLY, or CTM file, which may be suitable for use with dental design software such as FastDesign provided by Glidewell Laboratories of Newport Beach, California. TM Dental design software. In some embodiments, the intraoral scan can be performed outside of a dental laboratory. For example, in some embodiments, the intraoral scan can be performed in a dental office. In some embodiments, the dentist / dental office can be part of a dental support organization ("DSO"). A DSO can include one or more dentists practicing together.

[0048] In some embodiments, one or more steps disclosed herein may occur in near real time. For example, in some embodiments, one or more steps in the present disclosure may be performed while a patient is visiting a dental office for treatment. In many cases, the patient may be under general or local anesthesia and in a treatment chair while the dentist prepares one or more physical prepared teeth and then scans at least a portion of the patient's dentition to capture the prepared physical teeth and a portion of the patient's dentition to generate a 3D virtual dental model. In some embodiments, while the patient is in the treatment chair, the dentist may also scan at least a portion of the patient's opposing teeth to generate an opposing 3D virtual dental model of the opposing teeth / opposing jaw. While the patient is still in the chair receiving treatment, the scan may be performed using an intraoral scanner or other scanner in the dentist's office.

[0049] Once a 3D virtual dental model including one or more virtual prepared teeth and an opposing jaw 3D virtual dental model is generated, a dentist, staff, or any other authorized user in a dental clinic can create a virtual case for the patient as part of the virtual dental restoration design automation and upload a virtual dental model (also referred to as a "scan") that can be received for virtual restoration automated design. In some embodiments, the automated design can be performed in a cloud computing environment ("cloud"). For example, the virtual case can be stored on a storage device or database or associated with information about a particular patient and treatment on a storage device or database. In some embodiments, the information can include, for example, information about the patient and the preparation, such as (but not limited to) the patient's name, a user-selectable prepared tooth number, a user-selectable material, a user-selectable prepared tooth color, and the 3D virtual dental model itself.

[0050] In some embodiments, the 3D virtual dental model and the antagonist 3D virtual dental model may be provided to the automated dental restoration design process and system via a graphical user interface ("GUI"), which may be displayed on a client device by a cloud computing environment. In some embodiments, for example, the GUI may provide an interface that allows a customer to log in to a dental restoration design server and upload a scanned virtual 3D dental model. In some embodiments, for example, the customer may be a dentist, a dental technician, or any other user. In some embodiments, the customer may be located in a dental clinic.

[0051] In some embodiments, the cloud computing environment can receive a 3D virtual dental model from a client device, which can, for example, include a computing device at a dental clinic. The generation and uploading of the case can be performed through an interface, such as a graphical user interface ("GUI") displayed on a client device display to allow entry of case information and uploading of the 3D virtual dental model. In some embodiments, the interface can, for example, be connected to one or more clouds, or to one or more computer servers or other systems operated by a dental laboratory and connected to the dental clinic via the Internet to store case information. In some embodiments, the case can be created manually and the information provided by a dentist or other person at the dental clinic, or in some embodiments, the case can be created automatically and the information provided by scanning software used by the scanner. For example, in some embodiments, when a patient's dentition is scanned at a dental clinic, The intraoral scanner can automatically open the case and fill in the case information, and upload the 3D virtual dental model.

[0052] Some embodiments of a computer-implemented method of automating virtual dental restoration design may include performing automated virtual restoration design using a 3D virtual dental model. In some embodiments, the automated virtual restoration design may be performed while the patient is in the chair during a patient visit to a dental office. In some embodiments, the automated virtual restoration design may be performed in a cloud computing environment.

[0053] Figure 2208. A dentist or dental technician or other user located in a dental clinic 202 can open a new case 208 and use an intraoral scanner 206 to scan at least a portion of the patient's dentition, which in some embodiments associated with the case 208 includes at least one solid prepared tooth. In some embodiments, another intraoral scan can also be performed on at least a portion of the patient's opposing dentition (i.e., the other jaw), and the intraoral scan is also associated with the same case 208. The intraoral scanner can generate a virtual dental model for each scan. For example, the intraoral scanner can generate a 3D virtual dental model for the dentition with at least one solid prepared tooth, and generate an opposing 3D virtual dental model for the opposing dentition. In some embodiments, an intraoral scan 206 can be performed first and an order 208 can be automatically generated. In some embodiments, once the intraoral scan is completed, the 3D virtual dental model and the opposing 3D virtual dental model can be sent to the automated design 210.

[0054] In some embodiments, the automated design 210 may be performed in a cloud computing environment. In some embodiments, the automated design 210 may determine the quality of the physical prepared tooth prepared by the dentist. In some embodiments, if the physical prepared tooth is of acceptable quality and no problems are found at 211, the automated design 210 may generate a virtual restoration and display the generated virtual restoration to the dentist, dental technician, or other user in the dental clinic 202 on a computing device display for design review ("DC") 214. In some embodiments, the design review 214 may allow the dentist, etc. to make adjustments to the generated virtual restoration and / or margin lines.

[0055] In some embodiments, the dentist, other user, etc. may make minor adjustments that may be applied to the virtual restoration. In some embodiments, the dentist, other user, etc. may make major adjustments. In some embodiments, minor and major changes may be applied when they are made. For example, major adjustments may include, but are not limited to, adjusting margin lines. In some embodiments, when major adjustments 222 are made, the major adjustments 222 trigger the automated design 210 to regenerate a new virtual restoration based on the major adjustments made. Once all adjustments are completed and the design is finalized by the dentist or user, the final virtual restoration design is forwarded 224 to the design manufacturability check 220. In some embodiments, the dentist, other user, etc. may simply accept the initially proposed virtual restoration without making any adjustments, and then forward 224 the initially proposed virtual restoration to the design manufacturability check 220.

[0056] The design machinability check 220 can determine whether the generated virtual restoration can be automatically milled. The design machinability check 220 can also be located in a cloud computing environment. For both the automated design 210 and the design machinability check 220, the cloud computing environment can be the same, or a separate cloud computing environment can be used. If the virtual restoration can be automatically milled, the virtual restoration can be sent to milling 216 for physical manufacturing of the virtual restoration. In some embodiments, if the design machinability check 220 is passed, milling 216 can be automatic milling. An example of automated milling can be found in US Patent No. US10470853B2 of Leeson et al., the entire contents of which are incorporated herein by reference. In some embodiments, if the design machinability check 220 indicates that automatic milling cannot be performed, milling 216 can be manual milling. In some embodiments, milling 216 can occur at the dental laboratory 204.

[0057] In some embodiments, if the automated design 210 identifies one or more issues 209 with the physical prepared teeth, the computer-implemented method may provide virtual guidance / feedback to a dentist and / or other user, etc., in the dental clinic 202 at 212. In some embodiments, the dentist may then adjust the physical prepared teeth, the physical opposing teeth, and / or a portion of the patient's dentition based on the virtual guidance / feedback, and rescan the patient's adjusted physical dentition, which may include one or more physical prepared teeth and / or one / more opposing teeth and / or dentition 215. In some embodiments, the dentist or user may then re-upload the rescanned 3D virtual dental model of the rescanned 3D virtual dental model, which may include at least one rescanned virtual prepared tooth and / or at least one corresponding rescanned opposing tooth. In some embodiments, the rescan may be added to the same order 208, and may be processed as described above. Figure 2 The processing continues.

[0058] In some embodiments, the dentist / user may simply accept and submit 219 the 3D virtual dental model and any opposing 3D virtual dental models to the design lab technician 232 or other user typically in the dental lab 204 upon receiving virtual guidance feedback 212 indicating one or more questions. The design lab technician or other user may then determine one or more grinding areas on the virtual prepared tooth based on the questions raised by the automated design to design a virtual grinding crown as described in U.S. Pat. No. US11351015B2 to Leeson et al. (hereinafter referred to as '015). The design lab technician may then manufacture a physical grinding / guiding crown as described in '015 and send the physical grinding / guiding crown to the dentist to provide physical feedback to the dentist or user of the dental clinic regarding which physical prepared tooth, opposing tooth, and / or surrounding dentition area to be physically ground. In some embodiments, the design technician may also design a virtual restoration for the virtual ground prepared tooth and send the designed virtual restoration to the design machinability check 220 for subsequent manufacture of the physical restoration. In some embodiments, the design laboratory technician can use the automated design 210 to generate a virtual restoration from the virtual ground prepared teeth, the virtual ground antagonist teeth, and / or the virtual ground patient dentition. In some embodiments, the dental technician can manually design the virtual restoration.

[0059] Figure 3 302. It is a diagram showing an overview of an example of dental restoration automation in some embodiments. A dentist or dental technician or other user located in a dental clinic 302 can open a new case 308 and use an intraoral scanner 306 to scan at least a portion of a patient's dentition, which in some embodiments associated with the case 308 includes at least one solid prepared tooth. In some embodiments, another intraoral scan can also be performed on at least a portion of the patient's opposing dentition (i.e., the other jaw), and the intraoral scan is also associated with the same case 308. The intraoral scanner can generate a virtual dental model for each scan. For example, the intraoral scanner can generate a 3D virtual dental model for a dentition with at least one solid prepared tooth, and generate an opposing 3D virtual dental model for the opposing dentition. In some embodiments, an intraoral scan 306 can be performed first and an order 308 can be automatically generated. In some embodiments, once the intraoral scan is completed, the 3D virtual dental model and the opposing 3D virtual dental model can be sent to the automated design 310.

[0060] In some embodiments, the automated design 310 may be performed in a cloud computing environment. In some embodiments, the automated design 310 may determine the quality of the physical prepared tooth prepared by the dentist. In some embodiments, if the physical prepared tooth is of acceptable quality and no problems are found at 311, the automated design 310 may generate a virtual restoration and display the generated virtual restoration to the dentist, dental technician, or other user in the dental clinic 302 on a computing device display for design review ("DC") 314. In some embodiments, as previously described with respect to Figure 2 As discussed, the design review 314 may allow a dentist or the like to make adjustments to the generated virtual restoration and / or margin lines.

[0061] In some embodiments, the dentist, other user, etc. may make minor adjustments that may be applied to the virtual restoration. In some embodiments, the dentist, other user, etc. may make major adjustments. For example, major adjustments may include, but are not limited to, adjusting the margin lines. In some embodiments, when major adjustments 322 are made, the major adjustments 222 trigger the automated design 310 to regenerate a new virtual restoration based on the major adjustments made. In some embodiments, minor and major changes may be applied as they are made. Once all adjustments are completed and the design is finalized by the dentist or user, the final virtual restoration design is forwarded 324 to the design manufacturability check 320. In some embodiments, the dentist, other user, etc. may simply accept the initially proposed virtual restoration without making any adjustments, and then forward 324 the initially proposed virtual restoration to the design manufacturability check 320.

[0062] In some embodiments, if the automated design 310 identifies one or more issues 309 with the physical prepared teeth, the computer-implemented method may forward the uploaded 3D virtual dental model with one or more virtual prepared teeth and the antagonistic teeth / dentition 3D virtual dental model 309 to the design technician 316. In some embodiments, for example, the design technician 316 may belong to the dental laboratory 304. In some embodiments, upon completion of the manual design, the dental technician 316 may forward the manually generated virtual restoration to the design processability check 320 and the manufacturing process as previously discussed. In some embodiments, the dental laboratory 304 may also provide feedback to the dentist and / or user regarding one or more issues with the physical prepared teeth determined by the automated design 310. In some embodiments, the feedback may include one or more images or 3D models with any issues marked. In some embodiments, the design technician 316 may provide the virtual restoration design at 318 for the design processability check 320 upon completion of the virtual restoration design.

[0063] The design machinability check 320 can determine whether the generated virtual restoration can be automatically milled. The design machinability check 320 can also be located in a cloud computing environment. The cloud computing environment can be the same for both the automated design 310 and the design machinability check 320, or a separate cloud computing environment can be used. If the virtual restoration can be automatically milled, the virtual restoration can be sent to automatic milling 332 for physical manufacturing of the virtual restoration. In some embodiments, if the design machinability check 320 is passed, the milling 316 can be automatic milling. An example of automated milling can be found in U.S. Patent No. US10470853B2 of Leeson et al., the entire contents of which are incorporated herein by reference. In some embodiments, if the design machinability check 320 indicates that automatic milling cannot be performed, the milling 316 can be manual milling 336. In some embodiments, the virtual restoration can be sent 334 to manual milling 336. In some embodiments, milling can occur at the dental laboratory 304.

[0064] In some embodiments, performing automated virtual restoration design may include, for example, one or more of the following steps: decimation of a 3D virtual dental model, meshing, segmentation, determining occlusal direction, determining occlusal alignment, positioning a preparation mold, determining buccal direction, determining the edge of at least one virtual prepared tooth, determining the insertion direction onto the virtual prepared tooth, determining the cement space for the virtual prepared tooth, generating a virtual restoration, and pulling the virtual restoration to the edge. In some embodiments, one or more of the automated virtual restoration design steps may be performed on a 3D virtual model including one or more virtual prepared teeth. In some embodiments, one or more of the automated virtual restoration design steps may be performed on an opposing 3D virtual model including one or more opposing virtual teeth that are opposite to the corresponding one or more virtual prepared teeth. In some embodiments, one or more steps may be performed sequentially or in any other suitable order. In some embodiments, one or more steps may be excluded.

[0065] Figure 4An example of one or more steps in the automated virtual restoration design step 400 is shown. In some embodiments, one or more automated virtual restoration design steps can be performed in a cloud computing environment. In some embodiments, performing the automated virtual restoration design may include, for example, performing one or more of the following steps in sequence or in any other suitable order: extracting a 3D virtual dental model 402, mesh restoration 404, segmentation 406, determining an occlusal direction 408, determining an occlusal alignment 410, positioning a preparation mold 412, determining a buccal direction 414, determining an edge 416 of at least one virtual prepared tooth, determining an insertion direction 418 onto the virtual prepared tooth, determining an adhesive space 420 for the virtual prepared tooth, generating a virtual restoration 422 or a dental bridge 424, and pulling the virtual restoration to the edge 426. In some embodiments, one or more steps may be performed in any order. In some embodiments, one or more steps may be performed, for example, in a sequential order. Figure 4 The order shown is performed sequentially. In some embodiments, some steps can be excluded.

[0066] In some embodiments, automated virtual restoration design may include optionally performing extraction. Extraction may include reducing the file size of the 3D virtual dental model. In some embodiments, this may be achieved by reducing the number of polygons in the 3D virtual dental model. In some embodiments, the amount of extraction performed may be a user-configurable value that may, for example, sample a subset of polygons or points in the 3D virtual dental model. In some embodiments, extraction may be performed by selecting a subset of points that define the 3D virtual dental model. For example, in some embodiments, the computer-implemented method may select every other point or every other point in the 3D virtual dental model. In some embodiments, extraction may be performed while the patient is in a dental clinic. In some embodiments, extraction may be triggered when the number of polygons (in some embodiments, virtual triangles) in the virtual mesh exceeds a user-configurable threshold. In some embodiments, extraction may include combining two or more virtual triangles in the virtual mesh to reduce the number of virtual triangles in the mesh, adjusting the mesh to increase uniformity, and reducing the size of virtual triangles in areas of greater curvature. FIG. 5( a) shows an example of an initial virtual mesh having more virtual triangles than a user-configurable threshold for extraction. FIG. 5( b) shows an example of an extracted virtual mesh having fewer virtual triangles than a user-configurable threshold for extraction. Figure 5(c) shows an example of a portion of a virtual mesh 500 where one or more regions of increased curvature have smaller virtual triangles 501 than other less curved regions. Other highly curved regions with smaller virtual triangles are also circled in Figure 5(c).

[0067] In some embodiments, automated virtual restoration design may optionally include performing mesh repair. In some embodiments, mesh repair may include, for example, removing triangles, hole filling, and smoothing the virtual dental model. In some embodiments, mesh repair may be performed while the patient is in a dental office. In some embodiments, mesh repair may include correcting triangulation of the mesh (grid), removing any self-intersections of the mesh, removing spikes and elongated virtual triangles, removing ducts, filling holes, removing empty points (not belonging to any virtual triangle), and removing separate small mesh areas. In some embodiments, mesh repair may be performed while the patient is being treated in a dental office.

[0068] In some embodiments, automated virtual restoration design may include automatically performing segmentation on the 3D virtual dental model. Segmentation may identify one or more virtual teeth, gums, and / or other features within the 3D virtual dental model. One or more examples of automatically performing segmentation on the 3D virtual dental model may be found in U.S. Application No. 17 / 140,739 to Azemikov et al., the entire contents of which are incorporated herein by reference. As described in the application, segmentation may include, for example, receiving a 3D virtual model of patient scan data of at least a portion of the patient's dentition; generating a panoramic image from the 3D virtual model; marking one or more regions of the panoramic image using a first trained neural network to provide a marked panoramic image; mapping one or more regions of the marked panoramic image to one or more corresponding rough virtual surface triangle markers in the 3D virtual model to provide a marked 3D virtual model; and segmenting the marked 3D virtual model to provide a segmented 3D virtual model. Another example of automatically performing segmentation on the 3D virtual dental model may also be found in U.S. Application No. 16 / 451,968 to Nikolskiy et al., the entire contents of which are incorporated herein by reference. In some embodiments, segmentation may be performed during treatment of a patient while the patient is in a dental office.

[0069] In some embodiments, a trained 3D deep neural network ("DNN") may be used on a volume (voxel) representation to automatically determine one or more features. In some embodiments, the DNN may be a convolutional neural network ("CNN"), which is a network that uses convolutions instead of general matrix multiplications in at least one of the hidden layers of the deep neural network. A convolutional layer may calculate the convolutional layer output value by applying a kernel function to a subset of the values ​​of the previous layer. A computer-implemented method may train a CNN by adjusting the weights of the kernel function based on training data. The same kernel function may be used to calculate each value in a particular convolutional layer.

[0070] Figure 6An example of a CNN in some embodiments is shown. For illustration purposes, a 2D CNN is shown. A 3D CNN may have a similar architecture, but uses a three-dimensional kernel (xyz axis) to provide a three-dimensional output after each convolution. A CNN may include one or more convolutional layers, such as a first convolutional layer 502. The first convolutional layer 502 may apply a kernel (also referred to as a filter) such as kernel 504 across an input image such as input image 503, and optionally apply an activation function to generate one or more convolutional outputs such as a first kernel output 508. The first convolutional layer 502 may include one or more feature channels. The application of a kernel such as kernel 504 and an optional activation function may produce a first convolutional output such as convolutional output 506. The kernel may then be advanced to the next set of pixels in the input image 503 based on the stride length, and the kernel 504 and the optional activation function may be applied to produce a second kernel output. The kernel may be advanced in this manner until the kernel has been applied to all pixels in the input image 503. In this manner, a CNN may generate a first convolutional image 506 that may include one or more feature channels. In some embodiments, the first convolution image 506 may include one or more feature channels, such as 507. In some cases, the activation function may be, for example, a RELU activation function. Other types of activation functions may also be used.

[0071] The CNN may also include one or more pooling layers, such as a first pooling layer 512. The first pooling layer may apply a filter such as a pooling filter 514 to the first convolution image 506. Any type of filter may be used. For example, the filter may be a maximum filter (outputting the maximum value of the pixel on which the filter is applied) or an average filter (outputting the average value of the pixel on which the filter is applied). One or more pooling layers may downsample and reduce the size of the input matrix. For example, the first pooling layer 512 may reduce / downsample the first convolution image 506 by applying the first pooling filter 514 to provide a first pooled image 516. The first pooled image 516 may include one or more feature channels 517. The CNN may optionally apply one or more additional convolution layers (and activation functions) and pooling layers. For example, the CNN may apply a second convolution layer 518 and an optional activation function to output a second convolution image 520 that may include one or more feature channels 519. The second pooling layer 522 may apply a pooling filter to the second convolution image 520 to generate a second pooled image 524 that may include one or more feature channels. The CNN may include one or more convolutional layers (and activation functions) and one or more corresponding pooling layers. The output of the CNN may optionally be sent to a fully connected layer, which may be part of one or more fully connected layers 530. One or more fully connected layers may provide output predictions, such as output predictions 524. In some embodiments, the output predictions 524 may include, for example, markings of teeth and surrounding tissue. In some embodiments, the output predictions 524 may include identification of one or more features in a 3D digital dental model.

[0072] In some embodiments, automated virtual restoration design may include automatically determining the bite direction. An example of automatically determining the bite direction can be found in U.S. Application No. 17 / 245,944 to Azemikov et al., the entire contents of which are incorporated herein by reference. In some embodiments, determining the bite direction can be performed while the patient is in the dental office.

[0073] In some embodiments, the bite direction can be automatically determined using a neural network trained on the bite direction. In some embodiments, the CNN trained on the bite direction can be a 3D CNN trained using one or more 3D voxel representations, each 3D voxel representation representing the patient's dentition, optionally with enhanced data, such as the surface normal of each voxel. The 3D CNN can perform 3D convolutions, which use 3D kernels instead of 2D kernels and operate on 3D inputs. In some embodiments, the trained 3D CNN receives a 3D voxel representation with voxel normals. In some embodiments, an NxNxNx3 floating point tensor can be used. In some embodiments, N can be, for example, 100. Other suitable N values ​​can be used. In some embodiments, the trained 3D CNN can include 4 levels of 3D convolutions and can include 2 linear layers. In some embodiments, the 3D CNN can operate in a regression regime, in which the 3D CNN regresses the voxels representing the patient's dentition and their corresponding normals to three digital X, Y, and Z coordinates of the unit bite vector. In some embodiments, the training set of the 3D CNN may include one or more 3D voxel representations, each 3D voxel representation representing the patient's dentition. In some embodiments, each 3D voxel representation in the training set may include a bite direction manually marked by a user or marked by other techniques known in the art. In some embodiments, the training set may include tens of thousands of three-dimensional voxel representations, each of which has a marked bite direction. In some embodiments, the training data set may include a 3D point cloud model with a marked bite direction in each 3D point cloud model. Therefore, the technician marks a bite direction for each image / model (3D voxel representation) of the patient's dentition in the training data set, and the training data set may include tens of thousands of images / models (3D voxel representations) of the corresponding patient's dentition. In the training data, for example, in some embodiments, the coordinates of the unit bite vector may be such that X^2+Y^2+Z^2=1.

[0074] In some embodiments, automated virtual restoration design may include automatically determining an occlusal setting. In some embodiments, an occlusal setting may be determined between a 3D virtual dental model including one or more virtual prepared teeth and an opposing 3D virtual dental model including one or more corresponding virtual opposing teeth. In some embodiments, automatically determining an occlusal setting on a 3D virtual dental model may be found in U.S. Application No. 17 / 007,922 to Chelnokov et al., the entire contents of which are incorporated herein by reference. As described in the application, in some embodiments, automatically determining an occlusal setting may include receiving a first and second virtual jaw model, such as a 3D virtual dental model with one / more virtual prepared teeth and an opposing 3D virtual dental model with corresponding one / more virtual opposing teeth, determining a rough occlusal approximation of the first and second virtual jaw models, determining one or more initial occlusal positions of the first and second virtual jaw models from the rough approximation, determining one or more iterative occlusal positions of the first and second virtual jaw models for each of the one or more initial occlusal positions, determining a score for each iterative occlusal position, and outputting an occlusal setting based on the score. In some embodiments, determining an occlusal setting may be performed while the patient is in a dental clinic.

[0075] In some embodiments, automated virtual restoration design may include automatically performing preparation mold positioning. An example of automatically performing mold positioning can be found in U.S. Application No. 17 / 245,944 to Azemikov et al., previously incorporated by reference. As described in that application, automatically performing mold positioning may include using a neural network on an occlusally aligned 3D point cloud to determine a 3D center of the virtual preparation. In some embodiments, preparation mold positioning may be performed while the patient is in the dental office.

[0076] In some embodiments, the 3D center of the digital preparation mold can be determined automatically. For example, in some embodiments, a neural network can be used on an occlusally aligned 3D point cloud to determine the 3D center of the digital preparation. In some embodiments, the trained neural network can provide the 3D coordinates of the center of the digital preparation bounding box. In some embodiments, the neural network can be any neural network that can perform segmentation on a 3D point cloud. For example, in some embodiments, the neural network can be a PointNet++ neural network segmentation as described in the present disclosure. In some embodiments, the digital preparation mold can be determined by a sphere with a fixed radius around the 3D center of the digital preparation. In some embodiments, for example, for molars and premolars, the fixed radius can be 0.8 cm. For example, in some embodiments, other suitable values ​​can be determined and used for the fixed radius. In some embodiments, training the neural network can include using a sampled point cloud (unenhanced) of the digital jaw centered on the center of mass of the jaw. In some embodiments, the digital jaw point cloud can be oriented in a manner that vertically positions the occlusal direction. In some embodiments, a computer-implemented method may train a neural network using a training data set to determine a digital preparation site / mold in a 3D digital dental model, the training data set may include a 3D digital model of a patient's dentition point cloud, such as a digital jaw that may include a preparation site, wherein one or more points within the margin line of the preparation site are marked by a user using an input device or any technique known in the art. In some embodiments, the training set may be tens of thousands. In some embodiments, the neural network may utilize segmentation in operation to return a bounding box containing the selected points. In some embodiments, the segmentation used may be, for example, PointNet++ segmentation.

[0077] In some embodiments, the 3D center of the digital preparation mold can be automatically determined based on a planar depth map image of the jaw. In the training data set, the location of the mold center can be determined as the geometric center of the edge marked by the technician. In some embodiments, for example, the final edge point from a completed case can be used. In some embodiments, the network can receive a depth map image of the jaw from an occlusal view and return the location (X, Y) of the mold center in the pixel coordinates of the image. For training, a data set containing depth map images and corresponding correct answers (floating point X and Y values) can be used. In some embodiments, the training set can be tens of thousands.

[0078] In some embodiments, the 3D center of the digital preparation mold may optionally be set manually by a user.In some embodiments, the 3D center of the digital preparation mold may be set using any technique known in the art.

[0079] In some embodiments, automated virtual restoration design may include automatically determining buccal orientation. In some embodiments, the 3D digital model may include buccal orientation. In some embodiments, the buccal orientation may be manually set by a user. In some embodiments, the buccal orientation may be determined using any technique known in the art. An example of automatically determining buccal orientation may be found in U.S. Application No. 17 / 245,944, previously incorporated by reference. In some embodiments, automatically determining buccal orientation may include, for example, providing a 2D depth map image of the 3D virtual model mesh to a trained 2D convolutional neural network ("CNN"), such as GoogleNet Inception v3. In some embodiments, the trained 2D CNN operates on an image representation. In some embodiments, determining buccal orientation may be performed while the patient is in a dental office.

[0080] In some embodiments, the trained 2D CNN operates on the image representation. In some embodiments, the buccal orientation can be determined by providing a 2D depth map image of the 3D digital model mesh to the trained 2D CNN. In some embodiments, the method may optionally include generating a 2D image from the 3D digital model. In some embodiments, the 2D image may be a 2D depth map. The 2D depth map may include a 2D image that contains in each pixel the distance from an orthogonal camera to an object along a line passing through the pixel. For example, in some embodiments, the object may be, for example, a digital jaw model surface. In some embodiments, the input may include, for example, an object, such as a 3D digital model of a patient's dentition ("digital model"), such as a jaw, and a camera orientation. In some embodiments, the camera orientation may be determined based on an occlusal direction. The occlusal direction is the normal to the occlusal plane, and the occlusal plane may be determined for the digital model using any technique known in the art. Alternatively, in some embodiments, for example, as described herein, the occlusal direction may be specified by a user using an input device such as a mouse or a touch screen for manipulating a digital model on a display. In some embodiments, the occlusion direction can be determined using, for example, the occlusion axis technique described in U.S. Patent Application No. 16 / 451,968 to Nikolsky et al. (U.S. Patent Publication No. US20200405464A1), the entire contents of which are incorporated herein by reference.

[0081] A 2D depth map may be generated using any technique known in the art, including, for example, a depth buffer (z-buffer) or ray tracing. For example, in some embodiments, a computer-implemented method may, for example, initialize the depth of each pixel (j, k) to a maximum length and initialize the pixel color to a background color. A computer-implemented method may determine the depth z of a polygon at (x, y) corresponding to pixel (j, k) for each pixel in a projection of the polygon on a digital surface (such as a 3D digital model). If z<pixel depth (j, k), the depth of the pixel is set to depth z. "Z" may refer to the convention of the central visual axis of the camera in the direction of the camera's Z axis, and is not necessarily the absolute Z axis of the scene. In some embodiments, the computer-implemented method may also set the pixel color to, for example, a color other than the background color. In some embodiments, for example, the polygon may be a digital triangle. In some embodiments, the depth in the map may be per pixel. Figure 7 An example of a 2D depth map of a digital model in some embodiments is shown.

[0082] In some embodiments, the 2D depth map image may include a Von Mises average of 16 rotated versions of the 2D depth map. In some embodiments, the buccal direction may be determined after determining the occlusal direction and the 3D center of the digital preparation mold. In some embodiments, the 2D depth map image may be a portion of a digital jaw surrounding the digital preparation mold. In some embodiments, regression may be used to determine the buccal direction. In some embodiments, the 2D CNN may include, for example, GoogleNet Inception v3 known in the art. In some embodiments, a computer-implemented method may use a training data set to train a neural network trained on the buccal surface. In some embodiments, the training data set may include, for example, buccal directions marked in a 3D point cloud model. In some embodiments, the training data set may include tens of thousands to hundreds of thousands of images. In some embodiments, the computer-implemented method preprocesses the training data set, for example, by converting each training image into a 2D depth map as described above, and trains the 2D CNN using the 2D depth map.

[0083] In some embodiments, for example, the occlusal direction, the buccal direction, and / or the prepared mold area may provide a normalized orientation of the 3D virtual model. For example, the occlusal direction may be a normal to the occlusal plane, the virtual prepared mold may be an area around the virtually prepared teeth, and the buccal direction may be a direction toward the cheek in the oral cavity. Figure 8An example of a 3D virtual model 600 of at least a portion of a patient's dentition is shown, for example, the 3D virtual model 600 may include a virtual jaw 602 including virtual prepared teeth 604. The 3D virtual model 600 may include an occlusal direction 606, a virtual prepared mold area 608, and a buccal direction 610.

[0084] In some embodiments, automated virtual restoration design may include automatically determining adjustable margin lines of virtual prepared teeth. An example of automatically determining adjustable margin lines of virtual prepared teeth can be found in U.S. Application No. 17 / 245,944 to Azemikov et al., the entire contents of which are incorporated herein by reference.

[0085] In some embodiments, a computer-implemented method may determine a margin line proposal by receiving a 3D digital model having a digital preparation site and determining an internal representation of the 3D digital model using a neural network trained with the internal representation.

[0086] Some embodiments of the computer-implemented method may include determining the internal representation of the 3D digital model using a neural network trained with an internal representation. In some embodiments, the neural network trained with the internal representation may include an encoder neural network. In some embodiments, the neural network trained with the internal representation may include a neural network for 3D point cloud analysis. In some embodiments, the neural network trained with the internal representation may include a trained hierarchical neural network ("HNN"). In some embodiments, the HNN may include a PointNet++ neural network. In some embodiments, the HNN may be any message passing neural network that operates on geometric structures. In some embodiments, the geometric structures may include graphics, meshes, and / or point clouds.

[0087] In some embodiments, the computer-implemented method may be encoded using an HNN such as PointNet++. PointNet++ is described in "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space" published by Charles R.Qi, Li Yi, Hao Su, and Leonidas J.Guibas in June 2017, published by Stanford University, the entire contents of which are incorporated herein by reference. For example, a hierarchical neural network can process a sampled point set in a metric space in a hierarchical manner. In some embodiments, an HNN such as PointNet++ or other HNNs can be implemented by determining a local structure induced by a metric. In some embodiments, an HNN such as PointNet++ or other HNNs can be implemented by first dividing a point set into two or more overlapping local regions based on a distance metric. The distance metric can be based on the underlying space. In some embodiments, local features can be extracted. For example, in some embodiments, a granular geometry from a small local neighborhood can be determined. In some embodiments, small local neighborhood features can be grouped into larger units. In some embodiments, larger units can be processed to provide higher-level features. In some embodiments, the process is repeated until all features of the entire point set are obtained. Unlike volumetric CNNs that scan space with a fixed step, the local receptive field in HNNs such as PointNet++ or other HNNs depends on both the input data and the metric. In addition, in contrast to CNNs that scan vector spaces that are independent of the data distribution, the sampling strategy in HNNs such as PointNet++ or other HNNs generates receptive fields in a data-dependent manner.

[0088] In some embodiments, an HNN such as PointNet++ or other HNNs may, for example, determine how to partition a point set and an abstract set of points or local features using a local feature learner. In some embodiments, for example, the local feature learner may be PointNet or any other suitable feature learner known in the art. In some embodiments, for example, a local feature learner may process an unordered set of points to perform semantic feature extraction. A local feature learner may abstract one or more local point / feature sets into a higher level representation. In some embodiments, an HNN may recursively apply a local feature learner. For example, in some embodiments, PointNet++ may recursively apply PointNet on nested partitions of an input set.

[0089] In some embodiments, the HNN can define partitions of overlapping point sets by defining each partition as a neighborhood sphere in Euclidean space, where the parameters of the neighborhood sphere can include, for example, the centroid location and scale. For example, the centroid can be selected from the input set by farthest point sampling as known in the art. Because the local receptive field can depend on the input data and the metric, an advantage of using the HNN can include, for example, efficiency and effectiveness. In some embodiments, the HNN can exploit neighborhoods at multiple scales. For example, this can allow robustness and detail capture.

[0090] In some embodiments, the HNN may include hierarchical point set feature learning. For example, in some embodiments, the HNN may construct a hierarchical grouping of points and abstract larger and larger local areas along the hierarchical structure. In some embodiments, the HNN may include multiple set abstraction levels. In some embodiments, the point set is processed and abstracted at each level to produce a new set with fewer elements. In some embodiments, the set abstraction level may include three layers in some embodiments: a sampling layer, a grouping layer, and a local feature learner layer. In some embodiments, the local feature learner layer may be, for example, PointNet. For example, in some embodiments, the set abstraction level may receive an N×(d+C) matrix from N points with d-dimensional coordinates and C-dimensional point features as input, and output an N′×(d+C′) matrix of N′ downsampled points with d-dimensional coordinates and a new C′-dimensional feature vector, and the new C′-dimensional feature vector may summarize the local context.

[0091] In some embodiments, the sampling layer can select or sample a set of points from the input points. For example, in some embodiments, the HNN can define these selected / sampled points as the centroid of the local region. For example, in some embodiments, for the input point {x 1 ,x 2 ,...,x n}, we can use iterative farthest point sampling (FPS) to select a subset of points {x i1 ,x i2 ,...,x im}, so that x ij is the distance between the rest of the points and the set {x i1 ,x i2 ,...,x ij-1} points that are farthest apart. Compared to random sampling, given the same number of centroids, this can advantageously provide, for example, better coverage of the entire point set. Compared to convolutional neural networks (CNNs) that scan a vector space that is independent of the data distribution, this can also advantageously generate, for example, receptive fields in a data-dependent manner.

[0092] For example, in some embodiments, the grouping layer may determine one or more local area sets by determining neighboring points around each centroid. In some embodiments, the input to this layer may be a point set of size N×(d+C) and the coordinates of the centroid of size N'×d. In some embodiments, the output of the grouping layer may include, for example, groups of point sets of size N'×K×(d+C). For example, in some embodiments, each group may correspond to a local area, and K may be the number of points within the neighborhood of the centroid point. In some embodiments, K may vary from group to group. However, for example, the next layer - the PointNet layer - may convert a flexible number of points into a local area feature vector of fixed length. For example, in some embodiments, the neighborhood may be defined by a metric distance. For example, in some embodiments, a ball query may determine all points within a radius of the query point. An upper limit on K may be set. In an alternative embodiment, a K nearest neighbor (kNN) search may be used. kNN may determine a fixed number of neighboring points. However, for example, in some embodiments, the local neighborhood of the ball query may guarantee a fixed area scale, thereby making one or more local area features more generalizable in space. For example, in some embodiments this may be preferred for semantic point labeling or other tasks requiring local pattern recognition.

[0093] In some embodiments, the local feature learner layer can encode the local region pattern into a feature vector. For example, suppose X = (M, d)) is a discrete metric space, and the metric of the discrete metric space inherits from the Euclidean space X n ,in Where X is a set of points and d is a distance metric, the local feature learner layer can determine a function f that takes X as input and outputs semantically interesting information about X. The function f can be a classification function that assigns a label to X or a segmentation function that assigns each point label to each member of M.

[0094] Some embodiments may use PointNet as a local feature learner layer. i ∈R d The unordered point set {x 1 ,x 2 ,…,x n}, the local feature learner layer can define a set function f:X→R that maps a point set to a vector, such as:

[0095]

[0096] In some embodiments, γ and h can be, for example, a multilayer perceptron (MLP) network or other suitable alternatives known in the art. For example, in some embodiments, the function f can be invariant to the input point arrangement and can approximate any continuous set function. In some embodiments, the response of h can be interpreted as a spatial encoding of the point. PointNet is described in "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation" published by RQ Charles, H. Su, M. Kaichun, and LJ Guibas in 2017, published in the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 77-85, the entire contents of which are incorporated herein by reference.

[0097] In some embodiments, the local feature learner layer may receive N' local regions of points. The data size may be, for example, N'×K×(d+C). In some embodiments, for example, each local region is abstracted in the output by the centroid of each local region and local features encoding the neighborhood of the centroid. The output data size may be, for example, N'×(d+C). In some embodiments, the coordinates of the points in the local region may be transformed into a local coordinate system relative to the centroid point: for i=1,2,...K and j=l,2,...,d, x i (j) =x i (j) -x^ (j) , where in some embodiments, for example, x^ is the centroid coordinate. In some embodiments, for example, using relative coordinates with point features can capture point-to-point relationships in local areas. In some embodiments, PointNet can be used for local pattern learning.

[0098] In some embodiments, for example, the local feature learner can address non-uniform density in the input point set through a density adaptive layer. When the input sampling density changes, the density adaptive layer can learn to combine features of regions of different scales. In some embodiments, for example, the density adaptive hierarchical network is a PointNet++ network. For example, in some embodiments, the density adaptive layer can include multi-scale grouping ("MSG") or multi-resolution grouping ("MRG").

[0099] In some embodiments, in MSG, multi-scale patterns can be captured by applying a grouping layer with different scales and then extracting features for each scale. For example, in some embodiments, extracting features for each scale can be performed by utilizing PointNet. In some embodiments, for example, features of different scales can be cascaded to provide multi-scale features. In some embodiments, HNN can learn an optimized multi-scale feature combination through training. For example, random input dropout can be used, where a random input point is an input point dropped with a random probability. As an example, for example, a dropout rate θ uniformly sampled from [0, p] can be used in some embodiments, where p is less than or equal to 1. As an example, in some cases, p can be set to 0.95 so that no empty point set is generated. In some embodiments, for example, other suitable values ​​can be used.

[0100] In MRG, at level L i The feature of a region at can be, for example, a concatenation of two vectors, in some embodiments, for example, where the first vector is obtained by i-1 Obtained by summarizing the features at each sub-region. This can be done using a collection abstraction hierarchy. In some embodiments, the second vector may be a feature obtained by directly processing the local region raw points using, for example, a single PointNet. In the case where the local region density is low, in some embodiments, the second vector may be given a greater weight because the first vector contains fewer points and includes sampling defects. In the case where the local region density is high, for example, in some embodiments, the first vector may be given a greater weight because the first vector can provide finer details due to recursive inspection at a higher resolution at a lower level.

[0101] In some embodiments, point features may be propagated for set segmentation. For example, in some embodiments, a hierarchical propagation strategy may be used. In some embodiments, feature propagation may include propagating point features from N l ×(d+C) points propagate to N l-1 points, where N l-1 and N l (N l Less than or equal to N l-1 ) is the point set size for setting the input and output of abstraction level l. In some embodiments, feature propagation can be performed by l-1 The coordinates of the points are l In some embodiments, for example, an inverse distance weighted average based on k nearest neighbors may be used (in the following equation, p=2, k=3; other suitable values ​​may be used). For example, in some embodiments, N l-1The interpolated features on can be cascaded with the point features of the skip link from the set abstraction level. In some embodiments, the cascaded features can be passed through a unit PointNet, for example, the unit PointNet can be similar to the convolution in a convolutional neural network. For example, in some embodiments, a shared fully connected layer and a ReLU layer can be applied to update the feature vector of each point. In some embodiments, the process can be repeated until the features propagated to the original point set are determined.

[0102] in

[0103] In some embodiments, the computer-implemented method may implement one or more neural networks as disclosed or known in the art. Any specific structures and values ​​for one or more neural networks and any other features as disclosed herein are provided only as examples, and any suitable variants or equivalents may be used. In some embodiments, as an example, one or more neural network models may be implemented based on the PyTorch-geometric package.

[0104] 9(a) and 9(b) show examples of HNNs in some embodiments. The HNN may include a hierarchical point set feature learner 702, the output of which may be used to perform segmentation 704 and / or classification 706. The hierarchical point set feature learner 702 example uses points in 2D Euclidean space as an example, but may operate on an input 3D image in three dimensions. For example, as shown in the example of FIG. 9(a), the HNN may receive an input image 708 having (N, d+C), and perform a first sampling and grouping operation 710 to generate an image having (N 1 ,K,d+C). The HNN can then provide the first sampled and grouped image 712 to the PointNet at 714 to provide a first sampled and grouped image with (N 1 ,d+C 1 ) of the first abstract image 716. The first abstract image 716 may undergo sampling and grouping 718 to provide a first abstract image having (N 2 ,K,d+C 1 ). The second sampled and grouped image 720 may be provided to a PointNet neural network 722 to output a second sampled and grouped image having (N 2 ,d+C2) of the second abstract image 724.

[0105] In some embodiments, the second abstract image 724 may be segmented by HNN segmentation 704. In some embodiments, HNN segmentation 704 may take the second abstract image 724 and perform a first interpolation 730, the output of which may be concatenated with the first abstract image 716 to provide a second abstract image having (N 1 ,d+C 2 +C 1 ). The first interpolated image 732 can be provided to the unit PointNet at 734 to provide a first interpolated image with (N 1 ,d+C 3 ) of the first segmented image 736. The first segmented image 736 may be interpolated at 738, and the output at 738 may be concatenated with the input image 708 to provide a first segmented image having (N 1 ,d+C 3 +C). The second interpolated image 740 can be provided to the unit PointNet 742 to provide a segmented image 744 with (N, k). For example, the segmented image 744 can provide a score for each point.

[0106] As shown in the example of FIG9( b ), in some embodiments, the second abstract image 724 may be classified by the HNN classifier 706. In some embodiments, the HNN classifier may take the second abstract image 724 and provide it to a PointNet network 760, and the output 762 of the PointNet network 760 may be provided to one or more fully connected layers, such as a connection layer 764, and the output of the one or more fully connected layers may provide a class score 766.

[0107] Some embodiments of the computer-implemented method may include determining edge line proposals from reference edge lines and an internal representation of the 3D digital model using a neural network trained with the displacement values.

[0108] In some embodiments, a baseline edge line can be pre-calculated once for each network type. In some embodiments, the network types can include molars and premolars. In some embodiments, other suitable network types can be used, for example. In some embodiments, the network types can include other types. In some embodiments, the same baseline edge line can be used as the initial edge line for each scan. In some embodiments, the baseline edge line is three-dimensional. In some embodiments, the baseline edge line can be determined based on edge lines from a training data set, which is used to train a neural network trained with internal representations and a neural network trained with displacement values. In some embodiments, the baseline edge line can be a pre-calculated mean or average of the edge lines of the training data set. In some embodiments, any type of mean or average can be used.

[0109] In some embodiments, the edge line proposal can be a free-form edge line proposal. In some embodiments, the neural network trained by the displacement value can include a decoder neural network. In some embodiments, the decoder neural network can cascade the internal representation with the specific point coordinates to achieve guided decoding. In some embodiments, guided decoding can generate closed surfaces, such as described in "A Zhangzi Method for Learning 3D Surface Generation (APapier-Mache Approach to Learning 3D Surface Generation)" published by T.Groueix, M.Fisher, VGKim, BCRussell and M.Aubry in 2018, published in the 2018IEEE / CVF Computer Vision and Pattern Recognition Conference, pages 216-224, the entire contents of which are incorporated herein by reference.

[0110] In some embodiments, the decoder neural network may include a deep neural network ("DNN"). Fig.10 , which is a high-level block diagram showing the structure of a deep neural network (DNN) 800 according to some embodiments of the present disclosure. The DNN 800 includes a plurality of layers N i ,N h,1 ,N h,l-1 ,N h,l ,N o Etc. The first layer N i is an input layer that can ingest one or more dental scan datasets. The last layer N o is the output layer. The deep neural network used in the present disclosure may output probabilities and / or complete 3D margin line proposals. For example, the output may be a probability vector including one or more probability values ​​for each feature or aspect of the dental model belonging to certain categories. Furthermore, the output may be a margin line proposal.

[0111] Each layer N may include multiple nodes that connect to each node in the next layer N+1. h,l-1 Each computing node in layer N is connected to h,l Each computing node in the input layer N i and the output layer N o Layer N between h,1 ,N h,l-1 ,N h,l is a hidden layer. Fig.10 The nodes in the hidden layer represented as "h" in can be hidden variables. In some embodiments, DNN800 can include multiple hidden layers, such as 24, 30, 50, etc.

[0112] In some embodiments, DNN 800 may be a deep feedforward network. DNN 800 may also be a convolutional neural network, which is a network that uses convolution instead of general matrix multiplication in at least one of the hidden layers of the deep neural network. DNN 800 may also be a generative neural network or a generative adversarial network. In some embodiments, training may use a labeled training data set to supervise the learning process of the deep neural network. The label is used to map the feature to the probability value of the probability vector. Alternatively, training may use an unstructured and unlabeled training data set to train a generative deep neural network in an unsupervised manner that does not necessarily require a labeled training data set.

[0113] In some embodiments, the DNN may be a multilayer perceptron ("MLP"). In some embodiments, the MLP may include 4 layers. In some embodiments, the MLP may include a fully connected MLP. In some embodiments, the MLP is normalized using BatchNorm.

[0114] Fig.11 A diagram of a computer-implemented method for automatic margin line proposal in some embodiments is shown as an example. In some embodiments, for example, before initiating margin line proposal for any 3D digital model, the computer-implemented method may pre-compute 901 a reference margin line 903 in three dimensions, where each point of the reference margin line 903 has a 3D coordinate, such as coordinate 905. The computer-implemented method may receive a 3D digital model 902 of at least a portion of a jaw. In some embodiments, the 3D digital model may be in the form of a 3D point cloud. For example, the 3D digital model may include prepared teeth 904. The computer-implemented method may determine an internal representation 908 of the 3D digital model using a neural network 906 trained with internal representations. In some embodiments, the neural network 906 trained with internal representations may be, for example, a neural network that performs grouping and sampling 907 and other operations on a 3D digital model (such as an HNN) in some embodiments. In some embodiments, the computer-implemented method may determine a margin line proposal from the reference margin line 903 and the internal representation 908 of the 3D digital model using a neural network 910 trained with displacement values. In some embodiments, a neural network trained with displacement values ​​may provide one or more three-dimensional displacement values ​​912 for digital surface points, such as reference edge lines 903 .

[0115] In some embodiments, the neural network trained with the displacement values ​​can determine edge line displacement values ​​in three dimensions from the reference edge line. In some embodiments, the neural network trained with the displacement values ​​uses BilateralChamferDistance as a loss function. In some embodiments, the computer-implemented method can move one or more points of the reference edge line by the displacement values ​​to provide an edge line proposal. Fig.12 A diagram is shown of an example of adjusting a reference edge line 1002 of a 3D digital model 1000 in some embodiments. In the example, one or more reference edge line points, such as reference edge line point 1004, may be displaced by a displacement value and direction 1006. For example, other reference edge line points may be similarly adjusted by their corresponding displacement values ​​and directions to form an edge line proposal 1008.

[0116] FIG. 13( a) shows one example of a proposed digital margin line 1104 for a digitally prepared tooth 1102 of a 3D digital model 1105. As can be seen from the figure, a margin line proposal can be made even in the case where the margin line is partially or completely covered by gums, blood, saliva or other elements. FIG. 13( b) shows another example of a proposed digital margin line 1106 for a digitally prepared tooth 1108 of a 3D digital dental model 1110. In some embodiments, the proposed margin line is displayed on the 3D digital model and can be manipulated by a user (such as a dental technician or doctor) using an input device to make adjustments to the margin line proposal.

[0117] In some embodiments, the neural network trained with internal representations and the neural network trained with displacement values ​​can be trained using the same training data set. In some embodiments, the training data set may include one or more training samples. In some embodiments, the training data set may include 70,000 training samples. In some embodiments, one or more training samples may each include an occlusal direction, a preparation mold center, and a buccal direction as normalized positioning and orientation of each sample. In some embodiments, the occlusal direction, the preparation mold center, and the buccal direction may be set manually. In some embodiments, the training data set may include an untrimmed digital surface of the jaw and a target margin line on the surface of the corresponding trimmed digital surface. In some embodiments, the target margin line may be prepared by a technician. In some embodiments, training may use regression. In some embodiments, training may include using a loss function to compare margin line proposals with target margin lines. In some embodiments, the loss function may be a Chamfer-loss function. In some embodiments, the chamfer loss function may include:

[0118]

[0119] In some embodiments, training can be performed on a computing system that can include at least one graphics processing unit ("GPU"). In some embodiments, for example, the GPU can include two 2080-Ti NVIDIA GPUs. Other suitable GPU types, quantities, and equivalents can be used.

[0120] In some embodiments, the computer-implemented method may be performed automatically. Some embodiments may further include displaying the free-form edge line on the 3D digital model. In some embodiments, the free-form edge line may be adjusted by a user using an input device.

[0121] In some embodiments, the adjustable margin line can be displayed to the dentist or other user for initial adjustment. In some embodiments, the adjustment can include moving one or more portions of the margin line based on the input of the dentist or other user. In some embodiments, the adjustment can include discarding the adjustable margin line and allowing the dentist to manually provide the margin line. In some embodiments, the computer-implemented method can display at least a portion of the 3D virtual dental model of the patient's dentition and the automatically determined margin line proposal, and allow the dentist or other user to modify the determined margin line. Fig.14 An example of a determined margin line proposal 772 for a virtual prepared tooth 775 in a 3D virtual dental model 776 in a GUI 774 is shown. In some embodiments, determining the adjustable margin line occurs before generating a 3D virtual dental restoration. Therefore, during the initial adjustment of the margin line, no 3D virtual dental restoration appears, so that the proposed (determined) margin line is visible. In some embodiments, the GUI 774 may provide a virtual control 778 to allow a user to adjust the automatically determined margin line. This may advantageously allow, for example, correction of the automatically determined margin line. In some embodiments, the automatically determined margin line may be performed while the patient is in a dental clinic.

[0122] In some embodiments, automated virtual restoration design may include determining the insertion direction of the restoration onto the prepared tooth based on an adjustable margin line. Examples of automatic determination of the insertion direction based on an adjustable margin line can be found in U.S. Application No. 16 / 918,586 to Leeson et al., the entire contents of which are incorporated herein by reference. In some embodiments, determining the insertion direction can be performed while the patient is in a dental office.

[0123] In some embodiments, automated virtual restoration design may include determining the inner surface of the virtual restoration based on the cement space. One or more examples of automatically determining the inner surface of the virtual restoration to take into account the cement space on the 3D virtual dental model can be found in U.S. Application No. 16 / 918,586 of Leeson et al., the entire contents of which are incorporated herein by reference. In some embodiments, determining the inner surface of the virtual restoration based on the cement space can be performed when the patient is in a dental clinic. In some embodiments, the cement space may include the space between the solid prepared teeth and the solid restoration. In some embodiments, the cement space may include determining the inner surface of the virtual restoration. In some embodiments, the inner surface may include an offset from the outer surface of the virtual restoration, the offset including the cement gap. In some embodiments, considering the cement space may include providing space for cement around the virtual edge and along one or more horizontal and vertical surfaces of the virtual restoration. In some embodiments, automated virtual restoration design may include tool radius compensation for a milling tool. In some embodiments, determining the inner surface of the virtual restoration based on the cement space can be performed when the patient is in a dental clinic.

[0124] In some embodiments, automated virtual restoration design can include automatically generating a 3D virtual restoration based on an adjustable margin line. Examples of automatically generating a 3D virtual restoration can be found in U.S. Patent No. US11291532B2 of Azemikov et al. and U.S. Patent No. US11007040B2 of Azemikov et al., the entire contents of each of which are incorporated herein by reference. In some embodiments, automatically generating a 3D virtual restoration can be performed when the patient is in a dental clinic. In some embodiments, the 3D virtual restoration can include a virtual crown. In some embodiments, the 3D virtual restoration can include a virtual bridge.

[0125] Some embodiments may include using a trained deep neural network to generate a virtual 3D denture model based on a virtual 3D dental model. Some embodiments may include using a trained generative deep neural network to automatically generate a 3D digital denture model (virtual 3D denture model) in a 3D digital dental model. An example of using a deep neural network to generate a denture is described in U.S. Patent Application No. 15 / 925,078 (now U.S. Patent No. 11,007,040), the entire contents of which are incorporated herein by reference. Another example of using a deep neural network to generate a denture is described in U.S. Patent Application No. 15 / 660,073, the entire contents of which are incorporated herein by reference.

[0126] Example embodiments of methods and computer-implemented systems for generating a 3D model of a denture using a deep neural network are described herein. Certain embodiments of the method may include: training a deep neural network by one or more computing devices to generate a first 3D denture model using a training data set; receiving, by one or more computing devices, patient scan data representing at least a portion of a patient's dentition; and generating a first 3D denture model based on the received patient scan data using the trained deep neural network.

[0127] The training data set may include a dentition scan data set with preparation site data and a denture data set. The preparation site on the gum line may be defined by a preparation margin or margin line on the gum. The denture data set may include scanned denture data associated with each preparation site in the dentition scan data set.

[0128] The scanned prosthesis may be a scan of a real patient's crown created based on a dental template library, which may have 32 or more dental templates. The dentition scan data set with preparation site data may include scan data of real preparation sites from a patient's scanned dentition.

[0129] In some embodiments, the training data set may include a natural dentition scan data set with digitally manufactured preparation site data and a natural denture data set, and the natural denture data set may include segmented tooth data associated with each digitally manufactured preparation site in the dentition scan data set. The natural dentition scan data set may have two main parts. The first part is a data set including scanned dentition data of the patient's natural teeth. The data in the first part includes all of the patient's teeth in a natural and unmodified digital state. The second part of the natural dentition scan data is a missing teeth data set in which one or more teeth are removed from the scan data. In place of the missing teeth, the preparation site manufactured by the deep neural network can be placed at the site of the removed teeth. This process generates two dentition data sets: dentition scan data of the patient's natural teeth that are complete and unmodified; and a missing teeth data set (natural denture data set), in which one or more teeth are digitally removed from the dentition scan data.

[0130] In some embodiments, the method further includes generating a digital model of the entire dental arch and segmenting each tooth in the entire dental arch to generate natural crown data used as training data. The method may also include: training another deep neural network to generate a second 3D denture model using a natural dentition scan dataset with digitally manufactured preparation site data and a natural denture dataset; using another deep neural network to generate a second 3D denture model based on received patient scan data; and mixing the features of the first 3D denture model and the second 3D denture model together to generate a hybrid 3D denture model.

[0131] Fig.15 A denture generation process 1200 using a deep neural network (DNN) is shown. The process 1200 begins at 1205, where a dentition scan dataset is received or ingested into a database. The dentition scan dataset may include one or more scan datasets of a real patient's dentition having tooth preparation sites and technician-generated (non-DNN-generated) dentures created for those preparation sites. A tooth preparation site (also referred to as a tooth preparation or prepared tooth) is a tooth, multiple teeth, or an area on a tooth that has been prepared to receive a denture (e.g., a crown, bridge, inlay, etc.). A technician or non-DNN-generated denture is a denture designed primarily by a technician. In addition, a technician-generated denture can be designed based on a tooth template library having multiple dental restoration templates. Each tooth in an adult's mouth can have one or more dental restoration templates in the tooth template library.

[0132] In some embodiments, the received dentition scan data set with tooth preparation parts can include scan data of the dentition of a real patient with one or more tooth preparation parts. The preparation parts can be defined by preparation edges. The received dentition scan data set can also include scan data of the denture after the denture is installed in its corresponding tooth preparation part. Such data can be referred to as a denture data set. In some embodiments, the denture data set can include scan data of the prosthesis generated by a technician before the denture is installed.

[0133] In some embodiments, each dentition scan data set received may be optionally preprocessed before using the data set as input to a deep neural network. Denture scan data is typically a 3D digital image or file representing one or more portions of a patient's dentition. A 3D digital image (3D scan data) of a patient's dentition may be acquired by performing an intraoral scan of the patient's mouth. Alternatively, an impression or physical model of the patient's teeth may be scanned to generate 3D scan data of the patient's dentition. In some embodiments, the 3D scan data may be converted to a 2D data format using, for example, a 2D depth map and / or a snapshot.

[0134] At 1210, a deep neural network may be trained using a dentition scan dataset (e.g., via a computer-implemented method or another process) having scan data of real tooth preparations and their corresponding technician-generated dentures - after and / or before installation. The above combination of a dataset of real tooth preparations and their corresponding technician-generated dentures may be referred to herein as a technician-generated dentition scan dataset. In some embodiments, the deep neural network may be trained using only the technician-generated dentition scan dataset. In other words, the training data only contains technician-generated dentures that were created based on one or more dental restoration library templates.

[0135] The tooth template of the dental restoration library can be considered as the optimal restoration model because it is designed for the specific features of a specific tooth (e.g., tooth #3). In general, there are 32 teeth in the mouth of a typical adult. Therefore, the dental restoration library can have at least 32 templates. In some embodiments, each tooth template can have one or more specific features (e.g., side wall size and shape, buccal and lingual cusps, occlusal surfaces, buccal and lingual arches, etc.), which can be unique to one of the 32 teeth. For example, each tooth in the restoration library is designed to include features, landmarks, and directions that best match the adjacent teeth, surrounding gums, and tooth positions and positions within the dental arch form. In this way, deep neural networks can be trained to recognize certain features that may be prominent for a specific tooth (e.g., side wall size and shape, cusps, grooves, pits, etc.) and their relationships (e.g., the distance between the cusps).

[0136] In some embodiments, a computer-implemented method or any other process may train a deep neural network to identify one or more dentition categories present or identified in a training data set based on an output probability vector. For example, assume that the training data set contains a large number of depth maps representing the patient's upper jaw and / or depth maps representing the patient's lower jaw. The computer-implemented method or another process may use the training data set to train a deep neural network to identify each individual tooth in the dental arch form. Similarly, the deep neural network may be trained to map a depth map of the lower jaw to a probability vector that includes probabilities that the depth map belongs to the upper jaw and the lower jaw, where the probability that the depth map belongs to the lower jaw is the highest in the vector, or is significantly higher than the probability that the depth map belongs to the upper jaw.

[0137] In some embodiments, a computer-implemented method or another process may train a deep neural network using a dentition scan dataset having one or more scan datasets of real tooth preparations and corresponding technician-generated dentures to generate a complete 3D dental restoration model. In this manner, the 3D dental restoration model generated by the DNN inherently incorporates one or more features of one or more tooth templates of a dental restoration library, which may be part of the database 150.

[0138] A computer-implemented method or another process may train a deep neural network (such as Fig.10 , Fig.16 , Fig.14 (a) or other neural network to generate a 3D model of a dental restoration using only a technician-designed dentition scan data set. In this way, the 3D denture generated by the DNN will inherently include one or more features of a denture designed by a human technician using a library template. In some embodiments, a computer-implemented method or another process can train a deep neural network to output a probability vector that includes the probability that the occlusal surface of the denture generated by the technician represents the occlusal surface of the missing tooth at the preparation site or margin. In addition, the computer-implemented method or another process can train the deep neural network by mapping the occlusal surface with the highest probability and the margin line data from the scanned dentition data to the preparation site to generate a complete 3D dental restoration model. In addition, the computer-implemented method or another process can train a deep neural network to generate the side walls of the 3D dental restoration model by mapping the side wall data of the denture generated by the technician to a probability vector that includes the probability that one of the side walls matches the occlusal surface and margin line data from the preparation site.

[0139] Reference again Fig.15 In order to generate a new 3D model of the denture for the new patient, dentition scan data (e.g., a scanned dental impression, a physical model, or an intraoral scan) of the new patient is received and ingested at 1215. In some embodiments, the dentition scan data of the new patient can be pre-processed to convert the 3D image data into 2D image data, which can make the dentition scan data more easily ingested by certain neural network algorithms. At 1220, one or more dental features in the dentition scan data of the new patient are identified using the previously trained deep neural network. The identified features can be, for example, preparation sites, corresponding margin lines, adjacent teeth and corresponding features, and surrounding gingiva.

[0140] At 1225, using the trained deep neural network, a full 3D dental restoration model may be generated based on the features identified at 1220. In some embodiments, the trained deep neural network may be tasked to generate a full 3D dental restoration model by: generating an occlusal portion of the denture for the preparation site; obtaining margin line data from a generated margin proposal as previously described or from a patient's dentition scan data; optionally optimizing the margin line; and generating side walls between the generated occlusal portion and the margin line. Generating the occlusal portion may include generating an occlusal surface having one or more of a mesio-buccal cusp, a buccal groove, a distal buccal cusp, a distal cusp, a distal buccal groove, a distal fossa, a lingual groove, a mesio-lingual cusp, and the like.

[0141] In some embodiments, the trained deep neural network can obtain margin line data from edge proposals generated as previously described or from a patient's dentition scan data. In some embodiments, the trained deep neural network can optionally modify the contour of the obtained margin line by comparing and mapping the obtained margin line with thousands of other similar margin lines (e.g., margin lines of the same tooth preparation site) with similar adjacent teeth, surrounding gingiva, etc.

[0142] To generate a full 3D model, a trained deep neural network can generate sidewalls to fit between the generated occlusal plane and the margin line. This can be done by mapping thousands of sidewalls of a technician-generated denture to the generated occlusal portion and margin line. In some embodiments, the sidewall with the highest probability value (in the probability vector) can be selected as the base model in which the final sidewall between the occlusal plane and the margin line will be generated.

[0143] Fig.16 Example inputs and outputs of a trained deep neural network 1300 (e.g., a GAN) according to some embodiments of the present disclosure are shown. As shown, the input data set 1305 can be a scan of the dentition of a new patient with a preparation site 1310. The dental prosthesis server can generate a (DNN-generated) 3D model of a dental prosthesis 1315 using the trained one or more deep neural networks 1300. The DNN-generated denture 1315 includes an occlusal portion 1320, a margin line portion 1325, and a sidewall portion 1330. In some embodiments, the deep neural network can generate the sidewalls of the prosthesis 1315 by analyzing thousands of technician-generated dentures (which are generated based on one or more library templates) and mapping them to the preparation site 1310. Finally, the sidewall with the highest probability value can be selected as the model to generate the sidewall 1330.

[0144] FIG. 17( a) is a high-level block diagram showing the structure of a generative adversarial network (GAN network) that can be used to identify and model tooth anatomical features and restorations according to some embodiments of the present disclosure. At a high level, the GAN network uses two independent neural networks to generate an output model that is substantially indistinguishable from a true model. In other words, the GAN network uses a minimax optimization problem to obtain convergence between two competing neural networks. The GAN network includes a generator neural network 1410 and a discriminator neural network 1420. In some embodiments, both the neural network 1410 and the discriminator neural network 1420 are deep neural networks constructed to perform unstructured and unsupervised learning. In the GAN network, both the generator network 1410 and the discriminator network (discriminative deep neural network) 1420 are trained simultaneously. The generator network 1410 is trained to generate samples 1415 from data input 1405. The discriminator network 1420 is trained to provide a probability that the sample 1415 belongs to the training data sample 1430 (which comes from the real sample, the real data 1425) and does not belong to one of the data samples of the input 1405. The generator network 1410 is recursively trained to maximize the probability that the discriminator network 1420 fails to distinguish (at 1435) the training data set and the output samples generated by the generator 1410.

[0145] At each iteration, the discriminator network 1420 can output a loss function 1440, which is used to quantify whether the generated sample 1415 is a real natural image or an image generated by the generator 1410. The loss function 1440 can be used to provide feedback required by the generator 1410 to improve each subsequent sample generated in a subsequent cycle. In some embodiments, in response to the loss function, the generator 1410 can change one or more of the weights and / or bias variables and generate another output.

[0146] In some embodiments, a computer-implemented method or another process may train two adversarial networks, a generator 1410 and a discriminator 1420, simultaneously. The computer-implemented method or another process may use one or more of a patient's dentition scan datasets to train the generator 1410 to generate sample models of one or more dental features and / or restorations. For example, the patient's dentition scan data may be a 3D scan of a mandible including a prepared tooth / site and its adjacent teeth. At the same time, the computer-implemented method or another process may train the discriminator 1420 to distinguish between the generated 3D model of a crown for a prepared tooth (generated by the generator 1410) and crown samples from a real dataset (a collection of multi-scan datasets with crown images). In some embodiments, the GAN network is designed for unsupervised learning, so the input 1405 and the real data 1425 (e.g., a dentition training dataset) may be unlabeled.

[0147] Figure 17(b) is a flowchart of a method 1450 for generating a 3D model of a dental restoration according to some embodiments of the present disclosure. Method 1450 may be performed by a computer-implemented method or another process on a dental restoration server or by one or more other computers in a cloud computing environment. The instructions, processes, and algorithms of method 1450 may be stored in a memory of a computing device, and when executed by a processor, the instructions, processes, and algorithms enable the computing device to perform training of one or more deep neural networks for generating 3D dentures. Some or all of the processes and flows described in method 1450 may be performed by one or more other entities or processes within a dental restoration server or within another remote computing device. In addition, one or more boxes (processes) of method 1450 may be executed in parallel, in different orders, or even omitted.

[0148] At 1455, a computer-implemented method or another process may use the unlabeled dentition dataset to train a generative deep neural network (e.g., GAN generator 1410) to generate a 3D model of a dental prosthesis (such as a crown). In some embodiments, a labeled and classified dentition dataset may be used, but is not required. The generative deep neural network may be implemented by a computer-implemented method or another process or in a separate and independent neural network within or outside the dental prosthesis server.

[0149] At 1460 and substantially the same, the computer-implemented method or another process may also train a discriminative deep neural network (e.g., discriminator 1420) to identify whether a dental restoration generated by a generative deep neural network is a model and a digital model of a real dental restoration. During the identification process, the discriminative deep neural network may generate a loss function based on a comparison of the real dental restoration and the generated dental restoration model. The loss function provides a feedback mechanism for the generative deep neural network. The generative deep neural network may use the information from the output loss function to generate a better model that may better deceive the discriminative neural network into thinking that the generated model is the real model.

[0150] The generative deep neural network and the discriminative neural network can be considered to be opposed to each other. In other words, the goal of the generative deep neural network is to generate a model that cannot be distinguished by the discriminative deep neural network as a model belonging to the real sample distribution or the fake sample distribution (the generated model). At 1465, if the generated model has a probability value indicating that the model is likely to be false, the training of the two deep neural networks is repeated again at 1455 and 1460. Continue and repeat the process until the discriminative deep neural network cannot distinguish between the generated model and the real model. In other words, the probability that the generated model is false is very low, or the probability that the generated model belongs to the distribution of real samples is very high.

[0151] Once the deep neural network is trained, the method 1450 is ready to generate a model of the dental restoration based on the patient's dentition dataset received at 1470. At 1475, a model of the patient's dentition dataset is generated using the received patient's dentition dataset.

[0152] In some embodiments, the automated virtual restoration design may include determining the insertion direction of the generated 3D virtual restoration. An example of automatically determining the insertion direction of the generated 3D virtual restoration can be found in U.S. Patent No. US20210304874A1 to Nikolsky et al., the entire contents of which are incorporated herein by reference. In some embodiments, the automated virtual restoration design may illustrate pulling the restoration to the edge along the determined insertion path. Fig.18 An example of a virtual crown 802 that in some embodiments is raised along an insertion path of a virtual prepared tooth 804 is shown. In some embodiments, the raised crown may be moved along the insertion path using a GUI element such as a slider 806 or other equivalent GUI element.

[0153] Some embodiments of a computer-implemented method of automating virtual dental restoration design may include detecting the presence or absence of a physical prepared tooth problem during automated virtual restoration design. In some embodiments, the virtual dental restoration design automation of detecting the presence or absence of a physical prepared tooth problem during automated virtual restoration design may be performed while the patient is in a dental office.

[0154] In some embodiments, the one or more physical prepared tooth problems may include automatically determining one or more undercut areas of the virtual prepared tooth. One or more examples of automatically determining one or more undercut areas of the virtual prepared tooth may be found in U.S. Application No. 16 / 918,586, previously incorporated by reference. As described in that application, in some embodiments, automatically determining one or more undercut areas of the virtual prepared tooth may occur in Figure 4During step 418 the insertion direction onto the virtual prepared tooth is automatically determined.

[0155] like Fig.19 As shown, in some embodiments, the computer-implemented method can set a default insertion path, which is an optimal insertion path that minimizes the undercut 911. Fig.19 A cross-sectional illustration 950 of a portion of a patient's dentition is shown in 2D. It will be appreciated that the patient's dentition is in 3D. In some embodiments, an undercut may represent one or more side wall regions extending from a virtual tooth surface. For example, an undercut 911 may cause one or more virtual prepared tooth side surface regions 958 of a virtual prepared tooth 963 to obstruct an edge when viewed from an insertion path 952 or 956. This may cause restoration seating issues because in some embodiments the restoration 954 is arranged to be, for example, maximally connected to the edge. If any portion of the virtual edge is obstructed by one or more virtual prepared tooth side surface regions along the insertion path, the computer-implemented method may determine that a virtual open edge exists. In some embodiments, an open edge may include one or more undercut regions that obstruct the edge along the insertion direction.

[0156] Fig. 20 An example of a virtual open edge is shown. Fig. 20 An insertion path 1022 with a virtual edge 1026 is shown. The virtual prepared tooth has an open virtual edge 1030 due to the virtual prepared tooth side surface area 1028. In some embodiments, the computer-implemented method can automatically determine the virtual preparation sidewall grinding value 1024 (also known as the undercut value). The computer-implemented method can determine the value, for example, by determining how much of the sidewall results in the open virtual edge 1030 when viewed along the insertion path 1022. Fig. 20 As shown, in some embodiments, the computer-implemented method determines one or more virtual prepared tooth grinding regions as virtual prepared tooth side surface regions 1028 that obstruct the virtual edge 1026 when viewed from the insertion point 1022. The computer-implemented method can determine the amount of virtual prepared tooth side surface grinding 1024 required to close the open virtual edge 1030 from the insertion point 1022. In some embodiments, the computer-implemented method can determine one or more virtual prepared tooth side surface regions to be ground to close the edge. In some embodiments, the virtual prepared tooth side surface regions to be ground are virtual prepared tooth grinding regions.

[0157] In some embodiments, one or more undercut areas may indicate a poor quality physical tooth preparation. In some embodiments, one or more undercut areas that block the insertion path may indicate a poor quality physical tooth preparation, thus requiring a change in the physical tooth preparation. In some embodiments, one or more undercut areas that create an open edge may indicate a poor quality physical tooth preparation, thus requiring a change in the physical tooth preparation.

[0158] In some embodiments, the one or more physical preparation tooth problems may include automatically determining a clearance problem of a virtual restoration with virtual opposing teeth. One or more examples of automatically determining a clearance of a virtual restoration with virtual opposing teeth may be found in U.S. Patent Application No. 16 / 918,586, the entire contents of which were previously incorporated by reference. In some embodiments, determining a clearance problem may occur at Figure 4 In the cement space step 420 of the automated design shown. When the restoration is installed to the solid prepared tooth, a minimum gap is required to provide appropriate contact with the opposing teeth on the other jaw. In some embodiments, the minimum gap can include the cement space plus the minimum restoration thickness value.

[0159] In some embodiments, a lack of clearance with the virtual antagonist teeth may mean a lower quality physical preparation of the teeth, thus requiring changes to the physical preparation of the teeth.

[0160] In some embodiments, one or more solid prepared tooth problems may include determining edge lines that cannot be automatically generated. In some embodiments, determining edge lines that cannot be automatically generated may occur in the edge AI step 416 in the automated design. In some embodiments, determining that the edge lines that cannot be automatically generated may be unclear scans of solid prepared teeth or unclear solid edge lines. In some embodiments, determining that the edge lines that cannot be automatically generated may be solid edge lines from over-smoothing. In some embodiments, determining that the edge lines that cannot be automatically generated may be from the lack of solid prepared teeth or the curvature of the solid edge lines. In some embodiments, the probability of successfully generating a solid restoration is related to the amount of edge lines that can be determined. The more edge lines that can be determined, the greater the probability of successfully generating a solid restoration. In some embodiments, when the probability of successfully generating a solid restoration exceeds a user-configurable threshold, the probability of successfully generating a solid restoration is high. In some embodiments, edge line problems may also occur when the edge extends to adjacent teeth.

[0161] In some embodiments, the margin line determined to be unable to be automatically generated may include a poor quality solid prepared tooth, thereby requiring changes to the solid margin and / or solid prepared tooth to increase visibility of the solid margin and / or correct margin line extension into adjacent teeth.

[0162] Some embodiments of a computer-implemented method for automating virtual dental restoration design may include virtually displaying to a dentist or other user one or more physical prepared tooth problems detected when performing automated virtual restoration design. In some embodiments, the virtual dental restoration design automation that virtually displays to a dentist or other user one or more physical prepared tooth problems detected when performing automated virtual restoration design may be performed while the patient is in a dental clinic, such as during a patient visit and / or while the patient is receiving dental treatment in a dental chair. Virtually displaying the one or more physical prepared tooth problems to a dentist or other user may provide guidance and / or feedback to the dentist and / or user regarding the quality of the physically prepared teeth. In some embodiments, virtually displaying the one or more physical prepared tooth problems to a dentist or other user may be performed in near real time. In some embodiments, virtually displaying the one or more virtual prepared tooth problems to a dentist may include highlighting one or more areas on a 3D virtual dental model that require grinding.

[0163] In some embodiments, the computer-implemented method may display the problem on a computer display having a graphical user interface ("GUI"). In some embodiments, virtually displaying one or more physically prepared tooth problems to the dentist on the display may include providing guidance to the dentist on the virtual 3D dental model to correct one or more areas of the physically prepared tooth. In some embodiments, this may be done while the patient is still in the dental office and in the dental chair. In some embodiments, the guidance provides the dentist with insights into problems caused by their preparation of the physically prepared tooth. In some embodiments, the guidance allows the dentist to learn to reduce future situations where problems are caused by their preparation of the physically prepared tooth.

[0164] In some embodiments, virtually displaying one or more physical prepared tooth problems to the dentist may include displaying one or more marked sidewall regions on the display that result in undercut regions on the virtual prepared tooth in the virtual dental model. Fig.21As shown, in some embodiments, the computer-implemented method may determine one or more virtual prepared tooth grinding areas as virtual prepared tooth side surface areas 1130 that obscure the virtual edge 1126 when viewed from the insertion point 1122. The virtual prepared tooth side surface areas 1130 may indicate problems with the insertion and correct seating of the physical restoration on the physical prepared tooth. The computer-implemented method may determine the amount of virtual prepared tooth side surface grinding 1124 required to close the open virtual edge 1128 from the insertion point 1122. In some embodiments, the computer-implemented method may determine and display one or more virtual prepared tooth side surface areas to be ground to close the edge, and the amount of grinding required to close the edge. In some embodiments, the computer-implemented method may display one or more physical prepared tooth side wall areas to be ground, and the amount of grinding, to the dentist or other user. The virtual prepared tooth side surface areas to be ground correspond to the physical prepared tooth grinding areas. Displaying the virtual prepared tooth side surface areas to the dentist or other user while the patient is being treated may allow the dentist to grind the physical prepared tooth while the patient is still in the chair and in the same visit. This may reduce turnaround time by allowing the dentist to resolve physical preparation issues in one visit, so that the patient does not have to return to the clinic for further physical preparation reduction to resolve issues with the physical preparation.

[0165] In some embodiments, virtually displaying one or more physical prepared tooth problems to the dentist may include displaying one or more marked gap problems on the virtual prepared teeth on the display. In some embodiments, the one or more marked gap problems may be detected during the cement space step of the automated design. In some embodiments, the gap problem may be related to the thickness of the restoration and the gap between the physical prepared teeth and the physical opposing teeth.

[0166] like Fig. 22 As shown, in some embodiments, the computer-implemented method can detect an insufficient virtual bite gap 1250 by determining that a virtual bite gap 1221 between one or more virtual preparation tooth bite surfaces and one or more virtual antagonist tooth bite surfaces is less than a required minimum bite gap. In some embodiments, the computer-implemented method determines the virtual bite gap when the virtual jaw is closed or clenched, or when the virtual preparation teeth 1230 and the virtual antagonist teeth 1234 are closest to each other. In this way, the computer-implemented method can determine that the height of the virtual preparation teeth 1230 and the height of the virtual antagonist teeth 1234 are too large to accommodate the minimum restoration thickness of the restoration between them when the jaw is closed or clenched.

[0167] In order to accommodate the restoration between the virtual prepared tooth 1230 and the virtual antagonist tooth 1234, the computer-implemented method can determine the total virtual grinding required to meet the required minimum occlusal gap. The computer-implemented method can display the total virtual grinding on the GUI to illustrate the total grinding required. In some embodiments, the total virtual grinding is, for example, the difference between the virtual occlusal gap and the required minimum occlusal gap. In some embodiments, the computer-implemented method can determine a default virtual grinding value. For example, in some embodiments, the insufficient gap can be an insufficient occlusal gap 1250 between the occlusal surface of the virtual prepared tooth 1230 and the occlusal surface of the virtual antagonist tooth 1234.

[0168] In some embodiments, the dentist can modify the physical prepared teeth to solve one or more physical prepared tooth problems. Some embodiments may include allowing the dentist to modify the physical prepared teeth (and / or the opposing or surrounding teeth / dentition), rescan at least a portion of the patient's dentition including the modified physical prepared teeth and / or the opposing teeth and / or the surrounding dentition to generate a modified 3D virtual dental model including the modified virtual prepared teeth and / or the opposing teeth and / or the surrounding dentition, and upload the modified 3D virtual dental model. In some embodiments, the modified 3D virtual dental model is used for the same case without generating a new case.

[0169] Once the dentist has made any required modifications to the physical prepared teeth / antagonist teeth / surrounding dentition, the computer-implemented method may further include receiving a modified 3D virtual dental model, the modified 3D virtual dental model including at least one modified virtual prepared tooth and / or at least one modified virtual antagonist and / or modified dentition, in some embodiments, the modified virtual prepared teeth include a virtual representation of the modified physical prepared teeth modified by the dentist. In some embodiments, the modifications, rescanning, and re-uploading by the dentist are performed while the patient is being treated in the dental chair.

[0170] Some embodiments of a computer-implemented method of automating virtual dental restoration design may include, while performing the automated virtual restoration design, virtually displaying the generated virtual restoration to a dentist or other user for one or more adjustments if no virtual prepared tooth problems are detected. In some embodiments, while performing the automated virtual restoration design while the patient is in a dental office, while performing the automated virtual restoration design, virtually displaying the generated virtual restoration to a dentist or other user for one or more adjustments if no physical prepared tooth problems are detected.

[0171] In some embodiments, one or more adjustments may include adjusting a virtual margin line. In some embodiments, one or more adjustments may include adjusting a virtual contact with opposing teeth. In some embodiments, one or more adjustments may include adjusting a virtual contact with one or more teeth adjacent to the prepared teeth. In some embodiments, the adjustments may include adjustments to a margin line or adjustments to a virtual restoration. The adjustments may also be referred to as a "design check" or "DC."

[0172] Some embodiments may include performing adjustments or DCs on the generated 3D virtual dental restoration model. In some embodiments, a computer-implemented method may display at least a portion of a 3D virtual model of a patient's dentition including the generated 3D virtual dental restoration model on a display such as a computer screen in a graphical user interface ("GUI"), which may include interactive controls that may allow a dental technician, dentist, or other user to manipulate one or more features of the generated 3D virtual dental restoration model.

[0173] FIG. 23 (a) shows an example of a GUI 1350 that can be used as part of a DC process in some embodiments. In the example, a generated 3D virtual restoration model for DC is loaded. The DC process can display at least a portion of a 3D virtual dental model 1352 of the patient's dentition and a generated 3D virtual dental restoration model 1354 and information about the 3D virtual dental restoration model 1354 and its position and orientation relative to the surrounding dentition in the GUI 1350. For example, in some embodiments, a computer-implemented method can display and indicate virtual tooth numbers 1356 and their adjacent virtual teeth in a representation of the maxillary or mandibular. In some embodiments, the DC process can display information about virtual teeth 1354 and their adjacent virtual teeth in a panel such as a panel 1358 or other suitable GUI display elements known in the art. For example, the panel 1358 can provide information about the occlusal, mesial and distal relationships of the generated 3D virtual dental restoration model relative to the surrounding dentition. In some embodiments, the GUI 1350 can provide one or more control features to adjust the 3D virtual dental restoration model 1354. For example, in some embodiments, the DC process may provide a control key to adjust the contact point of the automatically generated 3D virtual dental restoration with the adjacent virtual teeth in the 3D virtual dental model.

[0174] In some embodiments, the DC process may provide GUI controls to adjust contact points, such as, for example, a mesial contact point, a distal contact point, and / or an occlusal contact point. The mesial contact point and the distal contact point may be between the generated 3D virtual dental restoration model and the adjacent virtual teeth in the 3D virtual dental model. The occlusal contact point may be between the occlusal surface of the generated 3D virtual dental restoration model and the opposing virtual teeth on the opposite virtual jaw. FIG. 23 (b) shows an example of adjusting the mesial contact point in some embodiments. As shown, GUI 1360 may display at least a portion of the 3D virtual dental model and the mesial side of the generated 3D virtual dental restoration model 1362. GUI 1360 may display a mesial contact surface area 1364 between the 3D virtual dental restoration model 1362 and its mesial adjacent teeth (not shown) in the 3D virtual dental model. For example, GUI 1360 may provide an adjustment tool that may allow a user to use an input device to select a mesial adjustment surface area 1366 that the computer-implemented method may remove. In some embodiments, the adjustment can reduce the size of the adjustment surface area (such as the adjustment surface area 1366). In some embodiments, the adjustment can increase the size of the adjustment surface area 1366. Figure 23 (c) shows an example of adjusting the distal contact point in some embodiments. As shown, the GUI 1370 can display at least a portion of the 3D virtual dental model and the distal side of the generated 3D virtual dental restoration model 1372. The GUI 1370 can display the distal contact surface area 1374 between the 3D virtual dental restoration model 1372 and its distal adjacent teeth (not shown) in the 3D virtual dental model. For example, the GUI 1370 can provide an adjustment tool that allows a user to use an input device to select a distal adjustment surface area 1376 that the computer-implemented method can adjust. In some embodiments, the adjustment can reduce the size of the adjustment surface area (such as the adjustment surface area 1376). In some embodiments, the adjustment can increase the size of the adjustment surface area 1376.

[0175] In some embodiments, the DC process may provide a GUI control that allows the user to adjust the occlusal contact points. For example, FIG. 23( d ) shows a generated 3D virtual dental restoration model 1380 showing an occlusal surface (such as occlusal surface 1382) viewed from the occlusal direction. In some embodiments, the computer-implemented method may provide one or more virtual tools to adjust the occlusal contact area, such as occlusal contact area 1384. In some embodiments, the adjustment may reduce the size of the adjustment surface area (such as occlusal contact area 1384). In some embodiments, the adjustment may increase the size of the adjustment surface area 1384.

[0176] In some embodiments, the DC process may provide, for example, a GUI control that allows the user to adjust the shape or outline of the automatically generated 3D virtual dental restoration, as shown in Figures 23 (e) and 23 (f). In the example of Figure 23 (e), the user may select a contour area (such as contour area 1385) for adjustment. In the example of Figure 23 (f), the user may select a contour area (such as contour area 1387). In some embodiments, the contour control may include a visual indicator that may define a contour area. An example of such a visual indicator may include an "X" or a pound sign whose size may be adjusted by the user. An example of such a visual indicator is visual indicator 1386. In some embodiments, when the pointer is hovered over the generated 3D virtual dental restoration, the visual indicator may appear on at least a portion of the generated 3D virtual dental restoration model. In some embodiments, the user may adjust the shape or outline of the contour area, for example, by pressing a button on an input device such as a mouse and dragging the contour area in the desired direction. In some embodiments, other GUI controls known in the art may be used to allow the user to adjust the shape or outline of the automatically generated 3D virtual dental restoration.

[0177] In some embodiments, the DC process can display at least a portion of the 3D virtual dental model 1388 of the patient's dentition and the automatically determined margin line proposal, and allow the user to modify the determined margin line. Figure 23 (g) shows an example of a DC process displaying a determined margin line proposal 1389 in the GUI. In some embodiments, the 3D virtual dental restoration is hidden or not displayed so that the determined margin line is visible. In some embodiments, the GUI can provide a virtual handle 1390 to allow the user to adjust the automatically determined margin line. This can advantageously allow, for example, correction of the automatically determined margin line as part of the DC process.

[0178] In some embodiments, once the changes to the generated 3D virtual dental restoration model as part of the DC process are completed, the computer-implemented method can apply the changes to the 3D virtual dental restoration model to provide a modified 3D virtual dental restoration model. In some embodiments, the changes are applied to provide visual feedback when the changes are made. In some embodiments, in the case where the changes to the model are major or fundamental, the computer-implemented method can use a generative neural network to regenerate the 3D virtual dental restoration. In some embodiments, the major and / or fundamental changes can include, but are not limited to, for example, changes to the proposed margin line. In some embodiments, for example, the major and / or fundamental changes can be based on changes to user-configurable values ​​measured geometrically in the model. In some embodiments, for the regenerated 3D virtual dental restoration model, the computer-implemented method can perform one or more features described herein and provide the regenerated 3D virtual dental restoration model for the DC process. In some embodiments, in the case where the DC changes are not major or fundamental, the computer-implemented method can apply the changes to the virtual margin line and the generated 3D virtual dental restoration model without regeneration. For example, in some embodiments, when receiving a minor adjustment to the margin line, the computer-implemented method can include adjusting the virtual margin line and / or virtual restoration as needed. In some embodiments, upon receiving minor adjustments to the virtual restoration, the computer-implemented method may include applying the minor adjustments to the virtual restoration without regenerating the virtual restoration. In some embodiments, the computer-implemented method may include determining one or more physical preparation tooth issues for regenerating the virtual restoration. In some embodiments, the dentist in the dental office may make any adjustments while the patient is being treated in the chair.

[0179] Some embodiments may include tracking the performance of dentists based on the quality of one or more physically prepared teeth. Some embodiments may include tracking the performance of dentists based on the quality of one or more physically prepared teeth. Some embodiments may include tracking the performance of a dental clinic including one or more dentists based on the quality of one or more physically prepared teeth prepared by one or more dentists in the dental clinic. In some embodiments, tracking includes analysis of dentists and dental clinics and performance over a given time period. In some embodiments, the analysis includes one or more of the following: order ID, tooth ID, patient ID, patient name, doctor name, notes, factory / manufacturing location, restoration type, material, scan file source (scanner, computer or saved scan) and guidance information. In some embodiments, the guidance information includes the number of scanning sessions (re-scans) performed, the duration of the scan, the type of problems found for each scan, and the solution taken for each scan (how to solve). In some embodiments, the scan data is recorded and is not overwritten during re-scanning. Some embodiments may include developing a baseline level of performance. In some embodiments, the baseline level of performance includes the number of physical preparations that cause problems. Some embodiments may include curating educational content based on the types of problems commonly encountered when preparing prepared teeth.

[0180] In some embodiments, the captured analysis and guidance information can be provided as summary feedback to dentists, clinicians, and practices. In some embodiments, analysis, guidance, and other information can be provided to show trends over time. In some embodiments, providing this very short closed feedback loop can advantageously improve the prepared tooth / teeth over time. The analysis, guidance, and information, as well as any summary feedback, can provide progress reports, which can help the DSO work with dentists and physicians who need to improve to meet internal performance metrics.

[0181] Some embodiments may include providing feedback to individual dentists about their specific preparation techniques to minimize future preparation problems. Some embodiments may include incorporating educational content into a dentist's continuing education program. In some embodiments, performance includes a scorecard. In some embodiments, a scorecard including guidance information is displayed after each treatment session. In some embodiments, a dashboard shows application usage and overall performance. In some embodiments, the scorecard may indicate success / failure rates over a period of time. In some embodiments, the scorecard may provide success / failure rates for one or more other dentists. In some embodiments, one or more other dentists are in the same dental office. In some embodiments, the dental office may include a DSO. Some embodiments may include providing a scorecard summary to the DSO. This can help the DSO identify areas where the dentist may need more training or guidance, for example, and can also be used to determine the performance of an individual dentist.

[0182] Some embodiments may include milling a solid 3D virtual restoration from a successfully generated 3D virtual restoration. In some embodiments, the manufacturing facility may be a dental laboratory. In some embodiments, the manufacturing facility performs a manufacturing inspection. In some embodiments, a dental laboratory or other manufacturing facility is located outside a dental clinic.

[0183] In some embodiments, milling may be a computer-aided manufacturing ("CAM") process. Some embodiments may include preparing one or more physical restorations. In some embodiments, preparation may include grouping one or more physical restorations by patient for transport. Some embodiments may include bundling one or more physical restorations. In some embodiments, bundling may include grouping one or more restorations for a dental office together for transport to the dental office together.

[0184] Some embodiments may include routing the virtual 3D virtual restoration model to a computer-aided manufacturing ("CAM") process. In some embodiments, the CAM process may perform a design machinability check. In some embodiments, the design machinability check may include the following steps, not necessarily limited to this order: (1) Generate a virtual milled restoration (milling strategy) from the NC file. In some embodiments, the result of this stage may be a milled surface that should coincide with the actual milled crown. In some embodiments, the virtual milled restoration does not include one or more undercuts. (2) Compare the virtual milled restoration to the virtual restoration design. In some embodiments, the comparison may be a comparison of the outer and inner surfaces of the virtual milled restoration and the occlusal table. In some embodiments, if there are differences between the virtual milled restoration and the virtual restoration design, the design machinability check fails. In some embodiments, when the design machinability check fails, the virtual restoration is routed to manual milling. In some embodiments, if there are no differences between the virtual milled restoration and the virtual restoration design, the automatic milling check passes. In some embodiments, when the automatic milling check succeeds, the virtual restoration is routed to automatic milling. In some embodiments, automated milling includes: (a) nesting; (b) determining blocks based on magnification factor and shading; (c) milling a restoration (e.g., such as a crown); (d) determining staining / shading—gingival and occlusal; (e) sintering the restoration.

[0185] In some embodiments, the machinability failure may be an unmillable undercut, a restoration size that is too large for the block, or a shadow that is unavailable. In some embodiments, upon a failed machinability check, the virtual restoration design is routed to manual machining. In some embodiments, upon a successful machinability check, the virtual restoration design is routed to automatic machining. In some embodiments, upon passing the design machinability check, the CAM process routes the virtual 3D virtual restoration model to an automatic milling machine to perform automatic milling. In some embodiments, upon completion of the automatic milling, a virtual milling DC is performed. In some embodiments, upon passing the virtual milling DC, preparation is performed.

[0186] One or more advantages of one or more features may include, for example, saving overall turnaround time, reducing chairside time, improving patient experience by avoiding patients returning for multiple treatments. One or more advantages of one or more features may include, for example, grinding away or eliminating the scrapping of physical restorations due to one or more problems such as gaps, insertions and / or undercuts. One or more advantages of one or more features may include, for example, improved quality of physical restorations. One or more advantages of one or more features may include, for example, reducing the remake of physical restorations. One or more advantages of one or more features may include, for example, making the patient go to the dentist's or other user's office less often, while reducing the number of times the patient is placed under anesthesia. One or more advantages of one or more features may include, for example, providing near real-time feedback to the dentist about the physical preparation of the teeth to allow correction while treating the patient at the chairside. One or more advantages of one or more features may include, for example, allowing the dentist to prepare a tooth and generate a virtual restoration for the tooth in a single visit. One or more advantages of one or more features may include, for example, improved (faster) turnaround time for obtaining a physical restoration. One or more advantages of one or more features may include, for example, reducing the patient's chair time. One or more advantages may include, for example, reducing the number of rejected restorations due to one or more problems (optionally including but not limited to undercut problems, clearance problems, and / or opening edge problems).

[0187] Some embodiments include a processing system for automating virtual dental restoration design, which may include: a processor; a computer-readable storage medium including instructions executable by the processor to perform steps including the following: receiving a 3D virtual dental model of at least a portion of a patient's dentition, the 3D virtual dental model including at least one virtual prepared tooth, the virtual prepared tooth including a digital representation of a physical prepared tooth prepared by a dentist; performing automated virtual restoration design using the 3D virtual dental model; virtually displaying to the dentist one or more physical prepared tooth problems detected when performing the automated virtual restoration design; and in a case where no physical prepared tooth problems are detected when performing the automated virtual restoration design, virtually displaying the generated virtual restoration to the dentist for one or more adjustments.

[0188] Fig.24 1406 , virtually displaying to the dentist one or more problems with the prepared teeth detected when performing the automated virtual restoration design; and, if no problems with the prepared teeth are detected when performing the automated virtual restoration design, virtually displaying to the dentist the generated virtual restoration for one or more adjustments at 1408 .

[0189] Fig.25 A processing system 14000 in some embodiments is shown. The system 14000 may include a processor 14030, a computer-readable storage medium 14034 having instructions executable by the processor to perform one or more steps described in the present disclosure. In some embodiments, one or more features may be performed, for example, by a user using an input device while viewing a virtual model on a display. In some embodiments, the computer-implemented method may allow the input device to manipulate the virtual model displayed on the display. For example, in some embodiments, the computer-implemented method may rotate, scale, move, and / or otherwise manipulate the virtual model in any manner known in the art.

[0190] In some embodiments, one or more virtual surfaces of the 3D virtual model can be selected, for example, on a virtual tooth or other area using an input device whose pointer is shown on a display. For example, a pointer can be used to select an area of ​​one point by clicking on an input device such as a mouse or tapping on a touch screen. For example, in some embodiments, a virtual surface of multiple points can be selected by dragging a pointer on the virtual surface. Other techniques known in the art can be used to select points or virtual surfaces.

[0191] In some embodiments, the computer-implemented method may, for example, display the virtual model on a display and receive input from an input device such as a mouse or a touch screen on the display. In some embodiments, upon receiving a manipulation command, the computer-implemented method may rotate, scale, move, and / or otherwise manipulate the virtual model in any manner known in the art.

[0192] One or more of the features disclosed herein can be automatically executed and / or obtained without manual or user intervention. One or more of the features disclosed herein can be executed by a computer-implemented method. The disclosed features (including but not limited to any method and system) can be implemented in a computing system. For example, the computing environment 14042 for performing these functions can be any of various computing devices (e.g., desktop computers, laptop computers, server computers, tablet computers, gaming systems, mobile devices, programmable automation controllers, video cards, etc.) that can be incorporated into a computing system including one or more computing devices. In some embodiments, the computing system can be a cloud-based computing system.

[0193] For example, computing environment 14042 may include one or more processing units 14030 and memory 14032. The processing unit executes computer executable instructions. Processing unit 14030 may be a central processing unit (CPU), a processor in an application specific integrated circuit (ASIC), or any other type of processor. In some embodiments, one or more processing units 14030 may, for example, execute multiple computer executable instructions in parallel. In a multi-processing system, multiple processing units execute computer executable instructions to increase processing power. For example, a typical computing environment may include a central processing unit and a graphics processing unit or a synergistic processing unit. Tangible memory 14032 may be a volatile memory (e.g., registers, caches, RAM), a non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.) accessible by (multiple) processing units, or some combination of the two. The memory stores software that implements one or more innovations described herein in the form of computer executable instructions suitable for execution by (multiple) processing units.

[0194] The computing system may have additional features. For example, in some embodiments, the computing environment includes storage 14034, one or more input devices 14036, one or more output devices 14038, and one or more communication connections 14037. An interconnection mechanism such as a bus, controller, or network interconnects the components of the computing environment. Typically, operating system software provides an operating environment for other software executed in the computing environment and coordinates the activities of the components of the computing environment.

[0195] Tangible storage 14034 may be removable or non-removable and include magnetic or optical media such as disks, tapes or cassettes, CD-ROMs, DVDs, or any other media that can be used to store information in a non-transitory manner and that can be accessed within a computing environment. Storage 14034 stores instructions for software that implement one or more innovations described herein.

[0196] The input device(s) may be, for example: a touch input device such as a keyboard, mouse, pen, or trackball; a voice input device; a scanning device; any of a variety of sensors; another device that provides input to the computing environment; or a combination of the above. For video encoding, the input device(s) may be a camera, video card, TV tuner card, or similar device that accepts video input in analog or digital form, or a CD-ROM or CD-RW that reads video samples into the computing environment. The output device(s) may be a display, printer, speaker, CD writer, or another device that provides output from the computing environment.

[0197] (Multiple) communication connections enable communication with another computing entity via a communication medium. The communication medium transmits information, such as computer executable instructions, audio or video input or output, or other data in a modulated data signal. A modulated data signal is a signal whose characteristics, one or more of which are set or changed in some manner to encode information in the signal. By way of example and not limitation, the communication medium may use electrical, optical, RF or other carriers.

[0198] Any of the disclosed methods may be implemented as computer-executable instructions (e.g., computer-executable instructions causing one or more processors of a computer system to perform the method) stored on one or more computer-readable storage media 14034 (e.g., one or more optical media disks, volatile memory components (such as DRAM or SRAM), or non-volatile memory components (such as flash memory or a hard drive)) and executed on a computer (e.g., any commercially available computer, including a smart phone, other mobile device including computing hardware, or a programmable automation controller). The term "computer-readable storage medium" does not include communication connections, such as signals and carrier waves. Any of the computer-executable instructions for implementing the disclosed techniques and any data created and used during the implementation of the disclosed embodiments may be stored on one or more computer-readable storage media 14034. The computer-executable instructions may be, for example, a dedicated software application or part of a software application accessed or downloaded via a web browser or other software application (such as a remote computing application). Such software may be executed, for example, on a single local computer (e.g., any commercially available computer) or in a network environment (e.g., via the Internet, a wide area network, a local area network, a client-server network (such as a cloud computing network), or other such network) using one or more network computers.

[0199] For the sake of clarity, only certain selected aspects based on software implementation are described. Other details well known in the art are omitted. For example, it should be understood that the disclosed technology is not limited to any particular computer language or program. For example, the disclosed technology can be implemented by software written in C++, Java, Perl, Python, JavaScript, Adobe Flash, or any other suitable programming language. Similarly, the disclosed technology is not limited to any particular computer or hardware type. Certain details of suitable computers and hardware are well known and need not be elaborated in this disclosure.

[0200] It should also be well understood that any of the functions described herein may be performed, at least in part, by one or more hardware logic components rather than software. For example, but not limited to, illustrative types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0201] In addition, any of the software-based embodiments (including, for example, computer-executable instructions for causing a computer to perform any of the disclosed methods) may be uploaded, downloaded, or remotely accessed via suitable communication means, including, for example, the Internet, the World Wide Web, an intranet, a software application, cables (including fiber optic cables), magnetic communications, electromagnetic communications (including RF, microwave, and infrared communications), electronic communications, or other such communications means.

[0202] In view of the many possible embodiments to which the principles of the present disclosure may be applied, it should be recognized that the illustrated embodiments are examples only and should not be taken as limiting the scope of the disclosure.

Claims

1. A computer-implemented method for automating virtual dental restoration design, comprising: receiving a 3D virtual dental model of at least a portion of a patient's dentition, the 3D virtual dental model comprising at least one virtual prepared tooth, the virtual prepared tooth comprising a digital representation of a physical prepared tooth prepared by a dentist; performing automated virtual restoration design using the 3D virtual dental model; virtually displaying to the dentist one or more physical prepared tooth problems detected while performing the automated virtual restoration design; as well as In the event that no physical prepared tooth problems are detected while performing the automated virtual restoration design, the generated virtual restoration is virtually displayed to the dentist for one or more adjustments.

2. The method according to claim 1, characterized in that The automated virtual restoration design includes one or more selected from the group consisting of: extracting the 3D virtual dental model, meshing, segmenting, determining occlusal direction, determining occlusal alignment, positioning a preparation mold, determining buccal direction, determining an edge of at least one virtual prepared tooth, determining an insertion direction onto the virtual prepared tooth, determining a cement space for the virtual prepared tooth, generating the virtual restoration, and pulling the virtual restoration to the edge.

3. The method according to claim 1, characterized in that The one or more solid prepared dental problems include one or more undercut areas.

4. The method according to claim 1, characterized in that The one or more physical preparation tooth issues include a lack of space.

5. The method according to claim 1, characterized in that The one or more physical preparation tooth problems include lack of insertion direction.

6. The method according to claim 1, characterized in that The one or more solid prepared tooth problems include margin lines that cannot be automatically generated.

7. The method according to claim 1, characterized in that Displaying the one or more physical preparation tooth problems includes highlighting one or more areas on the 3D virtual dental model that require removal.

8. The method of claim 1, further comprising showing the dentist an insertion direction when the automated design is successful.

9. The method according to claim 1, characterized in that Perform one or more steps during a single patient visit.

10. The method according to claim 9, characterized in that The patient receives dental treatment in the dental chair.

11. The method of claim 1 , further comprising allowing a dentist to modify the physical prepared teeth when one or more problems are detected during automated design, rescanning at least a portion of the patient's dentition including the modified physical prepared teeth to generate a modified 3D virtual dental model including the modified virtual prepared teeth, and uploading the modified 3D virtual dental model.

12. The method according to claim 1, characterized in that The generated virtual restoration can be adjusted by the dentist.

13. The method of claim 1, further comprising determining a quality of the one or more physically prepared teeth based on the automated design process.

14. The method according to claim 13, characterized in that Automated design errors indicate lower quality solid tooth preparations.

15. The method of claim 1, further comprising tracking the dentist's performance based on the quality of one or more physically prepared teeth.

16. The method of claim 1, further comprising routing the 3D virtual restoration to a computer-aided manufacturing ("CAM") process, wherein: The CAM process includes performing a design machinability check for automated milling.

17. The method of claim 16, further comprising milling the virtual restoration.

18. A non-transitory computer readable medium storing executable computer program instructions for providing virtual dental restoration design automation, the computer program instructions comprising instructions for: receiving a 3D virtual dental model of at least a portion of a patient's dentition, the 3D virtual dental model comprising at least one virtual prepared tooth, the virtual prepared tooth comprising a digital representation of a physical prepared tooth prepared by a dentist; performing automated virtual restoration design using the 3D virtual dental model; virtually displaying to the dentist one or more physical prepared tooth problems detected while performing the automated virtual restoration design; as well as In the event that no physical prepared tooth problems are detected while performing the automated virtual restoration design, the generated virtual restoration is virtually displayed to the dentist for one or more adjustments.

19. The medium according to claim 18, wherein The automated virtual restoration design includes one or more selected from the group consisting of: extracting the 3D virtual dental model, meshing, segmenting, determining occlusal direction, determining occlusal alignment, positioning a preparation mold, determining buccal direction, determining an edge of at least one virtual prepared tooth, determining an insertion direction onto the virtual prepared tooth, determining a cement space for the virtual prepared tooth, generating the virtual restoration, and pulling the virtual restoration to the edge.

20. The medium according to claim 18, wherein The one or more solid prepared dental problems include one or more undercut areas.

21. The medium according to claim 18, wherein The one or more physical preparation tooth problems include lack of space.

22. The medium of claim 18, wherein: The one or more physical preparation tooth problems include lack of insertion direction.

23. The medium of claim 18, wherein: The one or more solid prepared tooth problems include margin lines that cannot be automatically generated.

24. The medium of claim 18, wherein: Displaying the one or more physical preparation tooth problems includes highlighting one or more areas on the 3D virtual dental model that require removal.

25. The medium of claim 18, further comprising showing the dentist an insertion direction when the automated design is successful.

26. The medium of claim 18, wherein: Perform one or more steps during a single patient visit.

27. The medium according to claim 26, wherein The patient receives dental treatment in the dental chair.

28. The medium of claim 18, further comprising allowing a dentist to modify the physical prepared teeth when one or more problems are detected during automated design, rescanning at least a portion of the patient's dentition including the modified physical prepared teeth to generate a modified 3D virtual dental model including the modified virtual prepared teeth, and uploading the modified 3D virtual dental model.

29. The medium of claim 18, wherein: The generated virtual restoration can be adjusted by the dentist.

30. The medium of claim 18, further comprising determining a quality of the one or more physically prepared teeth based on the automated design process.

31. The medium of claim 30, wherein: Automatic design errors indicate lower quality solid tooth preparations.

32. The medium of claim 18, further comprising tracking the dentist's performance based on the quality of one or more physically prepared teeth.

33. The medium of claim 18, further comprising routing the 3D virtual restoration to a computer-aided manufacturing ("CAM") process, wherein the CAM process includes performing a design machinability check for automated milling.

34. The medium of claim 33, further comprising milling the virtual restoration.

35. A system for automating virtual dental restoration design, the system comprising: processor; as well as A non-transitory computer-readable storage medium comprising instructions executable by the processor to perform steps comprising: receiving a 3D virtual dental model of at least a portion of a patient's dentition, the 3D virtual dental model comprising at least one virtual prepared tooth, the virtual prepared tooth comprising a digital representation of a physical prepared tooth prepared by a dentist; performing automated virtual restoration design using the 3D virtual dental model; virtually displaying to the dentist one or more physical prepared tooth problems detected while performing the automated virtual restoration design; as well as In the event that no physical prepared tooth problems are detected while performing the automated virtual restoration design, the generated virtual restoration is virtually displayed to the dentist for one or more adjustments.

36. The system of claim 35, wherein: The automated virtual restoration design includes one or more selected from the group consisting of: extracting the 3D virtual dental model, meshing, segmenting, determining occlusal direction, determining occlusal alignment, positioning a preparation mold, determining buccal direction, determining an edge of at least one virtual prepared tooth, determining an insertion direction onto the virtual prepared tooth, determining a cement space for the virtual prepared tooth, generating the virtual restoration, and pulling the virtual restoration to the edge.

37. The system of claim 35, wherein: The one or more solid prepared dental problems include one or more undercut areas.

38. The system of claim 35, wherein: The one or more physical preparation tooth problems include lack of space.

39. The system of claim 35, wherein: The one or more physical preparation tooth problems include lack of insertion direction.

40. The system of claim 35, wherein: The one or more solid prepared tooth problems include margin lines that cannot be automatically generated.

41. The system of claim 35, wherein: Displaying the one or more physical preparation tooth problems includes highlighting one or more areas on the 3D virtual dental model that require removal.

42. The system of claim 35, further comprising showing the dentist an insertion direction when the automated planning is successful.

43. The system of claim 35, wherein: Perform one or more steps during a single patient visit.

44. The medium of claim 43, wherein: The patient receives dental treatment in the dental chair.

45. The system of claim 35, further comprising allowing a dentist to modify the physical prepared teeth when one or more problems are detected during automated design, rescanning at least a portion of the patient's dentition including the modified physical prepared teeth to generate a modified 3D virtual dental model including the modified virtual prepared teeth, and uploading the modified 3D virtual dental model.

46. ​​The system of claim 35, wherein: The generated virtual restoration can be adjusted by the dentist.

47. The system of claim 35, further comprising determining a quality of the one or more physically prepared teeth based on the automated design process.

48. The medium of claim 47, wherein: Automatic design errors indicate lower quality solid tooth preparations.

49. The system of claim 35, further comprising tracking the dentist's performance based on the quality of one or more physically prepared teeth.

50. The system of claim 35, further comprising routing the 3D virtual restoration to a computer-aided manufacturing ("CAM") process, wherein the CAM process includes performing a design machinability check for automated milling.

51. The medium of claim 50, further comprising milling the virtual restoration.

Images