Method, system and apparatus for instant automated design of custom dental objects

Abstract

Embodiments of the present invention provide methods, systems, devices, and software for providing customized, clinically relevant designs, such as treatment plans / solutions for at least a single tooth replacement therapy, on demand and in real time, enabling immediate confirmation of the effectiveness and implementation of the patient-specific treatment solution. A machine learning algorithm is used in a computer-implemented method or system for automated restoration design. The restoration design includes the design of restorative elements such as crowns and abutments. Software for performing the method when executed on a digital processor is provided. The method also includes fabrication of the restorative elements.

Description

Technical Field

[0001] The present invention relates to computer-implemented methods or systems for automated restoration design. The present invention may employ machine learning algorithms in computer-implemented automated restoration design methods or systems. Restoration design includes the design of restorative dental objects, such as abutments or crowns for implants or crowns for tooth roots. Software for performing these methods when executed on a digital processor is provided. The fabrication of restorative dental objects is also within the scope of the present invention. Background Art

[0002] Design proposals for restorative and surgical planning for implant-based single-tooth replacement treatments are typically generated in a dental laboratory or dentist's office using desktop-based dental CAD / CAM software. Plans are often manually generated by a dental technician and / or clinician using this software. In some cases, the dental technician and / or clinician collaborate on the plans to achieve a clinically relevant design that meets the clinician's and patient's treatment solution expectations. This process of generating, reviewing, and accepting a custom single-tooth replacement plan can take anywhere from a few hours to several days or more of clock time.

[0003] Alternatively, an analog impression (a cast model derived from a scanned patient oral impression by a laboratory or centralized manufacturer) or a digital impression (an intraoral scan) is taken, and the scan data is transferred to a centralized design and manufacturing center as part of the order. The design proposal is initiated using traditional methods and is typically reviewed and adjusted as necessary by an abutment design technician (ADT). The design can then be shared with the ordering party using a variety of 2D and 3D viewing and editing desktop or web-based software applications. The client is given the opportunity to review and alter the design before approving the plan. As with desktop-based CAD / CAM software, the clock time from receiving the model or scan data to receiving the design can range from a few hours to several days. Furthermore, current workflows with centralized manufacturing organizations require treatment solution providers to utilize a variety of disparate software and systems to achieve a patient-specific design. Some of these electronic environments are not fully integrated or connected, making the use of the complete workflow challenging from a usability perspective.

[0004] For any of the scenarios listed above, current tooth replacement treatment solution providers always experience a significant delay between the input data they collect (i.e., scans of digital impressions or models) and the output they require (i.e., a patient-specific treatment solution proposal).

[0005] The delay between input and output is a major source of frustration and lost value for treatment solution providers. In some cases, the need for additional analog or digital impressions is determined hours or days after the patient has left the office. This "rework" requires additional patient visits, which treatment solution providers cannot bill for.

[0006] Currently, treatment solution providers must specify treatment solution details before generating or receiving patient-specific design recommendations from their internal systems or centralized manufacturers. As such, alternative treatment solutions are only offered in a serial or iterative manner. In some cases, current workflows can take hours or even days to arrive at a patient-specific treatment solution that meets all clinical, cost, and other requirements.

[0007] During this delay period, patients seeking tooth replacement typically experience at least some discomfort or inconvenience.

[0008] Current “tooth in a day” systems do not provide a high-quality, highly esthetic, patient-specific tooth replacement solution.

[0009] Even with the advancements in digital dental technology, creating custom dental restoration objects (e.g., abutments and crowns) that fit a specific patient's teeth in a fully automated manner remains a very challenging problem. Most dental design software provides the user with a starting point for designing the dental object of interest and a set of tools that allow the provided initial design to be modified or adjusted. In order to minimize the amount of manual adjustments required by the user, it is important to be able to provide an initial design that fits the given patient's dentition as closely as possible. This may require some clinical knowledge of dentistry. Designing and implementing a computerized algorithm to automatically create such dental objects would be a very complex and challenging process, but its effectiveness would vary depending on the amount of effort invested in its development.

[0010] Traditional approaches to design automation rely on intermediate detection of dental features (DFD: Tooth Feature Detection) on 3D scan data. Each software component or module typically relies on the output of the previous one, making its output quality dependent on the quality of its inputs. Errors in each component can easily propagate or accumulate throughout the entire process. Furthermore, the functionality of each component is explicitly designed through the programming of specific algorithms based on heuristics or known rules and constraints, which leads to the complexity and difficulty of creating such software modules. The desired output cannot be guaranteed, as certain edge cases may not be included in the algorithm. Another drawback of this chain of software components or modules is the long execution time due to the computationally intensive nature of the complex algorithms. For example, the entire automated process typically takes 5-10 minutes to obtain a final restoration design starting from the raw input.

[0011] References:

[0012] - “Probabilistic record linkage model derived from training data”, as described in US6523019B1, Choicemaker Technologies Inc.

[0013] - “Computer-implemented dental restoration design,” as described in US2015056576A, Glidewell Dental Ceramics.

[0014] - “Dental CAD Automation Using Deep Learning”, as described in US2018 / 0028294A1, and “Method for creating flexible arch models of teeth for use in restorative dentistry”, as described in US9814549B2.

[0015] -US2017 / 0095319, US 9504541, US2007 / 0154868,

[0016] US7551760, US7362890, US2008 / 0002869 and US

[0017] 2006 / 0072810 are also incorporated by reference.

[0018] - "System and Method for Adding Surface Detail to Digital Crown ModelsCreated Using Statistical Techniques" as described in US20160224690A1.

[0019] -Ronald S. Scharlack, Alexander Yarmarkovich, Bethany Grant, Method and system for designing custom restorations for dental implants, international patent, published as WO2007081557A1.

[0020] -Charles R.Qi, Hao Su, Kaichun Mo, Leonidas J.Guibas, PointNet: DeepLearning on Point Sets for 3D Classification and Segmentation, CVPR 2017. Summary of the Invention

[0021] It is an object of the present invention to provide a computer based automated design of custom denture designs.Preferably, embodiments of the present invention represent an improvement over conventional methods.

[0022] Embodiments of the present invention may utilize computer-implemented methods or systems for automated restoration design. For example, embodiments of the present invention preferably utilize machine learning algorithms for computer-implemented methods or systems for automated restoration design. The restoration design may include the design of a restorative object, such as an abutment for an implant or a crown for an implant or tooth root.

[0023] Computer-based methods or systems for the automated design of custom abutments and crowns can be used for single-tooth restorations on implants or crowns for single tooth roots. Software for performing these methods when executed on a digital processor is provided. The fabrication of the corresponding custom restoration objects (custom abutments and crowns) is also included.

[0024] Embodiments of the present invention are preferably based on solving the dental restoration design problem by connecting end-to-end inputs and outputs in the problem domain via an artificial neural network (ANN) and training the system with a known dataset to find direct patterns between inputs and outputs without relying on any intermediate form of data, i.e., operating in an end-to-end manner.

[0025] This approach may utilize a specific instantiation of a parameter-based underlying restoration model.Embodiments of the present invention may be applied to any specific parametric restoration model.

[0026] In one aspect, embodiments of the present invention relate to a computer-based method for automated design of a custom abutment and crown for a single tooth restoration on an implant or a crown for a single tooth root, the method comprising the steps of:

[0027] 1) obtaining a 3D image of the patient's oral cavity, for example, by scanning or retrieving it from archives, and

[0028] 2) Apply end-to-end computer-based machine learning models.

[0029] The output can be an automated output, such as a representation of a shape, such as a set of parameters defining an abutment or crown. These parameters are part of parametric modeling. Software is provided for performing these methods when executed on a digital processor using parametric modeling techniques. The output representing the shape of the abutment or crown can be obtained within five minutes of receiving the patient's scan data, alternatively within one minute of receiving the scan data, and more preferably within less than 30 seconds of receiving the patient's scan data. For example, the output can be provided via an I / O port or interface.

[0030] The fabrication of corresponding custom restorative dental objects (custom abutments and crowns) is also included.

[0031] Preferred embodiments of the present application include automated design of custom abutments and crowns for single tooth restorations on implants or crowns for single tooth roots based on ML (machine learning). The following are not necessarily part of, and preferably are not part of, embodiments of the present invention:

[0032] -Automated surgical planning

[0033] - Scan data registration (CBCT and surface scan data) and tooth segmentation

[0034] -Dental feature detection

[0035] -Surgical guide design

[0036] Aspects of the present invention include:

[0037] - Computer-based design of a custom abutment or crown for an implant or a crown for a single tooth root based on scanned data or measurements of the patient's anatomy (including the implant position in the jaw, soft tissue, and tooth anatomy, all in relation to each other). Embodiments of the invention may include scanning a model of the patient's mouth after implant surgery using a fiducial device called a FLO (Feature Locating Object) to obtain the required measurements.

[0038] - The computer-based design of a custom abutment or crown can be fully parametric, and the geometry can be uniquely defined by the values ​​of multiple parameters, such as tens of parameters, twenty parameters, fifty or sixty parameters.

[0039] -Embodiments of the present invention create and output parameter sets for the underlying target model. For example,

[0040] Output may be provided via an I / O port or interface.

[0041] Embodiments of the present invention can be integrated into computer-based technology platforms. The use of artificial intelligence and machine learning in embodiments of the present invention has been shown to be very effective in solving certain problems that are very difficult to solve with traditional methods. Embodiments of the present invention may relate to the formulation of a specific problem of automated restoration design and / or treatment planning based on implants, and to solving such problems. Restorative dental objects include abutments and crowns. Embodiments of the present invention may utilize computer-implemented artificial neural networks (ANNs). Embodiments of the present invention may include effectively training the ANN and predicting a desired set of design parameters based on input data, whereby the output includes parameter values ​​or representations, from which the parameter values ​​and 3D shape of the dental object in question can be derived. These parameter values ​​or their representations can be output with high accuracy. These parameter values ​​or their representations can be output very quickly (e.g., in less than five minutes, less than one minute, or less than 30 seconds after receiving the data scan) and can produce quality comparable to that created by a professional human designer. Embodiments of the present invention include training the ANN in an end-to-end manner. They may also include running a trained computer-based ANN in an operational mode, including inputting 3D images of a patient's mouth (e.g., via an I / O port or interface), processing those images in the computer-based ANN, and outputting parameter values ​​or representations (e.g., via an I / O port or interface), from which parameter values ​​and 3D shapes of relevant dental objects can be derived in an end-to-end manner.

[0042] Embodiments of the present invention include methods for creating computer-implemented ANNs based on 3D convolutional neural networks (CNNs). Embodiments of the present invention include efficient methods for converting regression problems into classification problems. Embodiments of the present invention may also include specific methods for how to generate input data in a good or optimal manner to reflect the different coordinate systems in which each design parameter is represented. In addition, a hierarchical structure can be provided for training neural networks, whereby individual parameters or groups of parameters can be used for training. Parameters that have a greater impact on the final design can be used to train or predict less important parameters. Thus, a hierarchy refers to a ranking of the importance of one or more parameters (used to define the overall shape of a structure such as a restorative dental object) relative to other parameters of the same parametric model. Regardless of the level in the hierarchy, all parameters are relevant to the entire abutment, the entire crown, etc.

[0043] In addition, or alternatively, a hierarchical predictive structure can be provided to predict a single parameter value or a set of multiple parameters, for example, taking into account the dependencies between them. According to the present invention, dental feature detection (DFD) or 2D contour imaging of the crown (i.e., not a complete 3D restoration) are optional intermediate inputs or outputs, but they are not only less preferred but also unnecessary compared to inputting a direct 3D image of the patient's jaw (e.g., a point cloud). Therefore, the output according to embodiments of the present invention does not need to rely on any intermediate results, such as DFD results or 2D contour imaging.

[0044] Embodiments of the present invention provide a "model-based" predictive system and method. The output of the system or method is a representation of the model parameters of the 3D shape of a dental object, for example, a numerical parameter value that defines a restorative dental object (such as an abutment and crown required by a patient). Once predicted by a computer-based ML system or method according to an embodiment of the present invention, these parameters can be used to uniquely define the complete 3D shape, position, and orientation of a restorative dental object (such as an abutment and crown). To calculate this shape, the parameter values ​​can be fed into the underlying 3D parametric model.

[0045] Embodiments of the present invention can utilize "probability vectors" associated with partitioning the range of values ​​that a model parameter can represent, for example, dividing the range of a model parameter into multiple intervals (bins), each interval having a smaller range. This converts the regression problem into a classification problem, with each interval being a class. Embodiments of the present invention do not require the input data to be pre-"segmented," for example, segmenting volumetric image data or segmenting a 3D surface into different regions.

[0046] Embodiments of the present invention include systems and methods for providing a computer-based workflow that revolutionizes how dental care providers interact with centralized model organizations. Embodiments of the present invention may provide software that, when executed on a processing engine such as a microprocessor, performs any of the methods of the present invention. Embodiments of the present invention may include any or all of the following:

[0047] 1) A method comprising or taking as input an optical scan of the surface of a tooth or jaw of a patient's mouth to obtain 3D surface data and sharing this data, including preferably automated detection of one or more implant positions. Existing scans of the tooth or jaw can be used. The scanned image is preferably a 3D image rather than a 2D image, such as in a point cloud or triangular mesh. The 3D scan can be performed by an optical scanning device. Scanning can be performed by medical image acquisition methods such as X-ray, MRI, CT scan, etc., but this is less preferred.

[0048] 2) ML-based automated design of restorative dental objects (such as abutments and crowns) based on the results of existing historical cases that have been successfully completed, for example, by training using machine learning methods and systems with raw data (such as 3D input images) and output results of existing historical cases in an end-to-end manner; and

[0049] 3) A method for presenting a design to a patient or client, for example, in the form of a 2D or 3D graphic, preferably via an easy-to-use graphic editor, so that the patient or client can observe or control the details of the treatment they will undergo. These methods can be completed in seconds, which opens up the possibility of sharing treatment solutions with patients who may be "still in the dentist's chair." The output of a representation of the shape of an abutment or crown that can be used to present the shape to the patient can be obtained within five minutes of receiving the patient's scan data, alternatively within one minute of receiving the scan data, and preferably within less than 30 seconds of receiving the patient's scan data.

[0050] The workflow method as an embodiment of the present invention can occur before any work order is placed. There is essentially zero elapsed clock time delay between the input (e.g., raw scan data, which is preferably 3D scan data in the form of a point cloud) and the output (i.e., the patient-specific restoration design).

[0051] A digital workflow according to an embodiment of the present invention comprises a series of connected, preferably cloud-based software services that are completely decoupled from existing ordering channels and downstream "back-office" software elements.

[0052] Embodiments of the present invention may include some or all of the following elements and functions:

[0053] 1) Scan data (eg in the form of 3D scan data which may be a point cloud) and upload (either measured or restored from a storage device), including a 3D image of the patient's mouth.

[0054] 2) Automated FLO detection and scan quality checking, whereby some assistive products require a certain minimum scan quality.

[0055] 3) Design parameter estimation based on a computer-implemented ML algorithm. The ML algorithm is trained based on existing case studies completed by human experts. The ML algorithm can be adapted to capture user-made changes to the design. These changes can be incorporated into the training, allowing the ML algorithm to evolve into a "dynamic" and customer-specific model.

[0056] 4) Designs that can be shared with users such as patients. This can be done by presenting the graphics to the patient, for example, using a web-based enhanced UX viewer / editor.

[0057] 5) Optionally, a software application that uses augmented reality to assist the clinician in presenting treatment plan proposals to the patient.

[0058] 6) Streamlined (eg, single-click) sorting buttons or icons are preferably built directly into the viewer / editor user interface.

[0059] 7) Optionally, a connected suite of software in the "back office" takes the client-approved patient-specific design as input to the centralized manufacturing process. The design is made available to the centralized manufacturing organization.

[0060] 8) The output of the centralized manufacturing process can include a variety of customized restorative products, both digital and manufactured, which are actually produced and shipped (optionally with a sales invoice based on a fee charged to the customer). These products include digital files of cores, abutments, or crowns in STL or other appropriate formats, which are published to the customer's web-based account page for downloading and manufacturing of the final restoration, for example, through subtractive processing such as milling or layered manufacturing, such as 3D printing of metal or other manufacturing methods for use in the customer's laboratory or chairside. In addition, manufactured custom dental objects are also output options, including abutments, crowns, and ancillary custom products. This workflow enables "next day" permanent high-quality, highly aesthetic patient-specific tooth replacement solutions as well as "same day" patient-specific solutions where the TiBase stock solution and customized mesostructure (or complete restoration) are deemed to be a sufficient clinical solution.

[0061] In summary, embodiments of the present invention provide methods, systems, devices, and software that address the above-referenced problems by providing customized, clinically relevant designs (e.g., treatment plans / solutions for at least single tooth replacement therapy) in real time, as needed, so that the effectiveness and performance of a patient-specific treatment solution can be immediately confirmed. The treatment solution provider does not necessarily experience rework or delays because it is known which treatment solution meets all requirements. For the patient, discomfort and inconvenience are minimized while a permanent aesthetic solution can be provided. Embodiments of the present invention can connect computational algorithms and software services into an overall high-value new digital workflow. The combination of these elements to achieve improved results is not entirely obvious to experts in this field. The development and demonstration of successful methods requires trial and error and controlled experiments, where certain results are not known or anticipated in advance.

[0062] Advantages of embodiments of the present invention are any, some or all of the following:

[0063] - Instant or rapid design. Output of a representation of the shape of the abutment or crown may be obtained within five minutes of receiving the patient's scan data, alternatively within one minute of receiving the scan data, and more preferably within less than 30 seconds of receiving the patient's scan data.

[0064] - No need to write explicit, hand-crafted, complex algorithms to determine the shape of dental objects based on features of the patient-specific environment as in conventional design automation approaches.

[0065] - No black-box approach to the design of dental objects: Shape parameters can be controlled, i.e., defined to have clinical significance (e.g., the width of the abutment edge, the height of the abutment core, etc.). This means that certain features of the design can be easily modified after the automated design, if desired.

[0066] Embodiments of the present invention can be described as follows:

[0067] Item 1. A computer-implemented method for training a machine learning system, the machine learning system being installed on one or more computing devices, the method comprising training the machine learning system with a plurality of pre-existing treatment 3D datasets, the 3D datasets comprising 3D images of a patient's dentition as input on one end of the machine learning system and 3D shapes of representations of the patient's restorative dental objects as output on the other end.

[0068] Training of a machine learning system using multiple pre-existing therapeutic 3D datasets can be performed in an end-to-end manner.

[0069] Item 2. A computer-implemented method for providing a patient with a representation of the 3D shape of a restorative dental object, the method comprising

[0070] inputting a 3D scanned representation of at least a portion of the patient's dentition into a trained machine learning system (e.g., via an I / O device or interface), the 3D scanned representation defining at least one implant location, the machine learning system being installed on one or more computing devices, and

[0071] A trained machine learning system is used to identify a representation of a 3D shape of a restorative dental object for an implant, wherein the output of the representation can be obtained within five minutes of receiving patient scan data, alternatively within one minute of receiving the scan data, and more preferably within less than 30 seconds of receiving the patient scan data.

[0072] Recognition can be performed in an end-to-end manner.

[0073] Item 3. A computer-implemented method for providing a patient with a representation of a 3D shape of a restorative dental object, the method comprising:

[0074] The machine learning system is trained by one or more computing devices using a plurality of pre-existing therapeutic 3D datasets,

[0075] receiving, by one or more computing devices, 3D scan data of a patient representing at least a portion of the patient's dentition defining a location for at least one implant, and

[0076] A method for identifying a 3D shape of a restorative dental object for an implant using a trained machine learning device. The identification of the representation can be performed within five minutes of receiving the patient's scan data, alternatively within one minute of receiving the scan data, and more preferably within less than 30 seconds of receiving the patient's scan data.

[0077] Training of a machine learning system using multiple pre-existing therapeutic 3D datasets can be performed in an end-to-end manner.

[0078] Item 4. A computer-implemented method as described in Item 3, wherein the recognition is performed in an end-to-end manner.

[0079] Item 5. A computer-implemented method as described in any of Items 1 to 4, comprising applying a computer-based end-to-end machine learning model.

[0080] Item 6. A computer-implemented method as described in any of Items 1 to 4, wherein the machine learning system is a neural network.

[0081] Item 7. The computer-implemented method of item 7, wherein the neural network comprises a CNN,

[0082] Item 8. A computer-implemented method as described in any of Items 2 to 5, wherein receiving includes receiving a scanned 3D image of the patient's mouth or a 3D image of the patient's mouth retrieved from an archive.

[0083] Item 9. The computer-implemented method of any one of Items 2 to 8, wherein the identifying step comprises generating a 3D shape of the restorative dental object.

[0084] The restorative dental object may be intended for attachment to an implant directly or via one or more intermediates.

[0085] Item 10. The computer-implemented method of Item 9, wherein the 3D shape is not a free-form 3D shape, but rather a parametric model defined by a set of parameters.

[0086] Item 11. A computer-implemented method as described in Item 10, wherein a first range of values ​​for each parameter is divided among a set of intervals having a smaller second range, and the machine learning device is adapted to estimate the correct interval to which a particular parameter belongs.

[0087] Item 12. A computer-implemented method as described in Item 11, wherein the parameter relates to or represents a characteristic of the tooth surface anatomy, tooth dentition, or restoration type.

[0088] Item 13. A computer-implemented method as described in Item 10, 11 or 12, wherein the output of the trained machine learning system is a representation of a 3D shape expressed as a set of parameters defining an abutment or crown.

[0089] Item 14. The computer-implemented method of Item 13, wherein the parameter set is part of parametric modeling.

[0090] Item 15. A computer-implemented method as described in any of Items 1 to 14, wherein the machine learning system is adapted to use a discriminative ML algorithm.

[0091] Item 16. A computer-implemented method as described in any of Items 1 to 15, wherein the patient's dentition includes the upper and / or lower jaw, prepared and opposing jaws, missing teeth, implants, and number of teeth.

[0092] Item 17. The computer-implemented method of any one of Items 1 to 16, wherein the restorative dental object is an abutment or a crown for an implant.

[0093] Item 18. A computer-implemented method as described in any one of Items 1 to 17, wherein a point sample suitable for input to an artificial neural network (ANN) is extracted on the 3D surface of the jaw scan geometry for each clinical case.

[0094] Item 19. A computer-implemented method as described in any of Items 1 to 18, wherein the parameterized model is trained using an ANN by mapping input data to restorative dental object design parameters in an end-to-end manner, the restorative dental object design parameters being the output of the ANN for all pre-existing treatment 3D data sets.

[0095] Item 20. A computer-implemented method as described in Item 19, wherein the trained machine learning model is saved as a collection of computer files in a network location.

[0096] Item 21. The computer-implemented method of Item 19 or 20, further comprising determining parameterized values ​​for a new clinical case having an input of a 3D image using the parameterized values ​​classified in the interval.

[0097] Item 22. A computer-implemented method as described in any of Items 3 to 21, wherein for a new clinical case of implant-based restoration, the method includes receiving input of implant position and orientation from 3D scan data of a feature position object, user design preferences, and a 3D scan of a patient's jaw.

[0098] Item 23. A computer-implemented method as described in any of Items 18 to 22, wherein the samples on the 3D surface of the jaw scan geometry are extracted in the same format used for training.

[0099] Item 24. The computer-implemented method of Item 22 or 23, wherein the received input data is sent to one or more computing devices that process the request and return a set of predicted design parameters using the trained ANN model.

[0100] Item 25. The computer-implemented method of Item 24, wherein the 3D model of the dental restoration object is reconstructed from the predicted design parameters and rendered for presentation to the user.

[0101] Item 26. A computer-implemented method for manufacturing a dental object according to any one of Items 1 to 25, the method manufacturing the dental object according to the reconstructed shape of an abutment or a crown.

[0102] Item 27. A computer-implemented system for training a machine learning system, the machine learning system being installed on one or more computing devices, the system comprising components for training the machine learning system with a plurality of pre-existing treatment 3D datasets, the 3D datasets comprising 3D images of a patient's dentition as input on one end of the machine learning system and 3D shapes of representations of the patient's restorative dental objects as output on the other end.

[0103] Training of a machine learning system using multiple pre-existing therapeutic 3D datasets can be performed in an end-to-end manner.

[0104] Item 28. A computer-implemented system for providing a patient with a representation of a 3D shape of a restorative dental object, the system comprising:

[0105] means for inputting a 3D scanned representation of at least a portion of a patient's dentition (e.g., via an I / O port or interface), the 3D scanned representation defining at least one implant location, the input being used by a trained machine learning system installed on one or more computing devices, and

[0106] A component for using a trained machine learning system to identify a representation of the 3D shape of a restorative dental object for an implant, wherein the output of the representation can be obtained within five minutes of receiving patient scan data, alternatively within one minute of receiving the scan data, and more preferably within less than 30 seconds of receiving the patient scan data.

[0107] Recognition can be performed in an end-to-end manner.

[0108] Item 29. A computer-implemented system for providing a patient with a 3D shape representation of a restorative dental object, the system comprising:

[0109] means for training a machine learning system using a plurality of pre-existing therapeutic 3D datasets using one or more computing devices,

[0110] a receiver for receiving, by one or more computing devices, 3D scan data of a patient, the data representing at least a portion of the patient's dentition defining a location for at least one implant, and

[0111] A component for recognizing a representation of a 3D shape of a restorative dental object for an implant using a trained machine learning device.

[0112] Identification of the representation may be performed within five minutes of receiving the patient's scan data, alternatively within one minute of receiving the scan data, and more preferably within less than 30 seconds of receiving the patient's scan data.

[0113] Training of a machine learning system using multiple pre-existing therapeutic 3D datasets can be performed in an end-to-end manner.

[0114] Item 30. The computer-implemented system of Item 29, wherein the means for identifying is adapted to operate in an end-to-end manner.

[0115] Item 31. A computer-implemented system as described in any of Items 27 to 30, comprising components for applying a computer-based end-to-end machine learning model.

[0116] Item 32. A computer-implemented system as described in any of Items 27 to 31, wherein the machine learning system is a neural network.

[0117] Item 33. A computer-implemented system as described in Item 32, wherein the neural network is a CNN.

[0118] Item 34. A computer-implemented system as described in any of Items 29 to 33, wherein the receiver is adapted to receive a scanned 3D image of the patient's mouth or a 3D image of the patient's mouth retrieved from an archive.

[0119] Item 35. The computer-implemented system of any one of Items 27 to 34, wherein the means for identifying is adapted to generate a 3D shape of the restorative dental object.

[0120] The restorative dental object may be intended for attachment to an implant directly or via one or more intermediates.

[0121] Item 36. The computer-implemented system of Item 35, wherein the 3D shape is not a free-form 3D shape, but rather a parametric model defined by a set of parameters.

[0122] Item 37. A computer-implemented system as described in Item 36, wherein the first range of values ​​for each parameter is divided into a set of intervals having a smaller second range, and the machine learning device is adapted to estimate the correct interval to which a particular parameter belongs.

[0123] Item 38. A computer-implemented system as described in Item 37, wherein the parameter relates to or represents a characteristic of the tooth surface anatomy, tooth dentition, or restoration type.

[0124] Item 39. A computer-implemented system as described in Item 37 or 38, wherein the output of the trained machine learning system is a representation of a 3D shape expressed as a set of parameters defining an abutment or crown.

[0125] Item 40. The computer-implemented system of Item 39, wherein the parameter set is part of parametric modeling.

[0126] Item 41. A computer-implemented system as described in any of Items 27 to 40, wherein the machine learning system is adapted to use a discriminative ML algorithm.

[0127] Item 42. A computer-implemented system as described in any of Items 27 to 41, wherein the patient's dentition includes the maxillary and / or mandibular, prepared and opposing jaws, missing teeth, implants, and number of teeth.

[0128] Item 43. A computer-implemented system as described in any of Items 27 to 42, wherein the restorative dental object is an abutment or a crown for an implant.

[0129] Item 44. A computer-implemented system as described in any of Items 27 to 43, wherein a sample of points suitable for input to an artificial neural network (ANN) is extracted on the 3D surface of the jaw scan geometry for each clinical case.

[0130] Item 45. A computer-implemented system as described in any of Items 27 to 44, wherein the parameterized model is trained using an ANN by mapping input data to restorative dental object design parameters in an end-to-end manner, the restorative dental object design parameters being the output of the ANN for all pre-existing treatment 3D data sets.

[0131] Item 46. A computer-implemented system as described in Item 45, wherein the trained machine learning model is saved as a collection of computer files in a network location.

[0132] Item 47. A computer-implemented system as described in any of Items 45 or 46, further comprising using the parameterized values ​​classified in the interval to determine parameterized values ​​for a new clinical case having an input of a 3D image.

[0133] Item 48. A computer-implemented system as described in any of Items 28 to 47, wherein for a new clinical case of implant-based restoration, the method includes receiving input of implant position and orientation from 3D scan data of a feature position object, user design preferences, and a 3D scan of a patient's jaw.

[0134] Item 49. A computer-implemented system as described in any of Items 44 to 48, wherein the samples on the 3D surface of the jaw scan geometry are extracted in the same format used for training.

[0135] Item 50. The computer-implemented system of Item 48 or 49, wherein the received input data is sent to one or more computing devices that process the request and return a set of predicted design parameters using the trained ANN model.

[0136] Item 51. The computer-implemented system of Item 50, wherein the 3D model of the dental restoration object is reconstructed from the predicted design parameters and rendered for presentation to the user.

[0137] Item 52. The computer-implemented system of any one of Items 27 to 51, adapted to manufacture a dental object according to the reconstructed shape of an abutment or a crown.

[0138] Item 53. A storage medium storing a parameterized model as described in any one of items 1 to 26 or storing processor-implementable instructions for controlling a processor to implement the method as described in any one of items 1 to 26.

[0139] Item 54. A processor-executable instruction for controlling a processor to execute the method as described in any one of items 1 to 26.

[0140] Item 55. A computer product comprising software for performing the method of any one of items 1 to 26 when executed on a digital processor using parametric modeling techniques.

[0141] Item 56. Fabrication of a custom restorative element made from a design generated according to the method of any one of items 1 to 26, the design optionally being a custom abutment or crown for an implant. BRIEF DESCRIPTION OF THE DRAWINGS

[0142] Figure 1 : End-to-end input and output of automated abutment design according to an embodiment of the present invention. Left: Raw 3D optical scan of the patient's jaw. Right: Abutment design based on a given implant position and orientation.

[0143] Figure 2 : A network according to an embodiment of the present invention.

[0144] Figure 3 : A method and system for automated design of dental restoration objects using machine learning according to an embodiment of the present invention are shown.

[0145] Figure 4 : An example of transforming an input scan into an implant coordinate system, with associated rotation values ​​of a designed restoration object, according to an embodiment of the present invention.

[0146] Figure 5 : Surface points of an input scan are sampled in an implant coordinate system using a 3D occupancy grid according to an embodiment of the present invention.

[0147] Figure 6 : A neural network architecture using a 3D CNN and auxiliary input combined with a 3D occupancy grid according to an embodiment of the present invention.

[0148] Figure 7 : Softmax operation according to an embodiment of the present invention and its general placement in a neural network structure.

[0149] Figure 8 : The process of training an ANN model (left) and using the trained model to predict a design (right) according to an embodiment of the present invention.

[0150] Figure 9 : A client-server system according to an embodiment of the present invention, wherein a trained restoration design model can be provided directly from a cloud server upon request by an end user.

[0151] Figure 10Figure 2 shows the accuracy measured within x degrees of rotation parameter for a validation set of 776 people after training with 17,632 samples of tooth #5, compared to other methods. First column: Accuracy within x degrees; 0 = perfect match. Second column: Accuracy obtained using our new method (based on 3D CNN) and the number of cases within a given accuracy. Third column: Accuracy obtained using the method of the present invention based on nonlinear multidimensional numerical optimization. Fourth column: Accuracy measured on 25 different cases, each designed by 12 different human designers (300 samples in total).

[0152] Figure 11 The measurement accuracy of selected abutment shape parameters within the range of 0.5 mm (first column) and 1.0 mm (second column) respectively is shown. The results are based on 70,000 training samples and 7000 validation samples of the upper left posterior teeth (teeth #2, 3, 4, 5) according to an embodiment of the present invention.

[0153] Figure 12 Shown in the top row: abutment designs created by a human designer. Bottom row: abutment designs predicted by a machine learning model according to an embodiment of the present invention.

[0154] Figure 13 Shown in the top row: crown designs created by a human designer. The bottom row shows crown designs predicted by our machine learning model according to an embodiment of the present invention.

[0155] Figure 14 A comparison of the acceptance rates of abutment designs between an ML-based method according to an embodiment of the present invention and a conventional optimization method is shown. Five different expert human designers examined 100 cases created by the two different methods without knowing how each design was created.

[0156] definition

[0157] A probability vector is associated with a partition of the range of values ​​for a model parameter. For example, the possible range of values ​​for a single parameter can be divided into intervals, with each interval associated with a probability that the estimated value of the parameter falls within the subrange represented by the particular interval. Thus, the exact value of the parameter is not required; only the probability that the calculated parameter value lies within the range of an interval is required. Thus, parameter values ​​are classified rather than regressed.

[0158] "Prosthetic objects" include abutments for implants and crowns for single implants or single tooth roots.

[0159] “End-to-end model”: In machine learning, typically in a “deep learning” setting, an end-to-end model learns all the features that can appear between the original input (x) and the final output (y). In this case, x refers to a 3D image of the patient’s mouth, and y refers to the parameter values.

[0160] The term "end-to-end" as used in this description refers to methods and systems according to embodiments of the present invention having a machine learning model that, after being trained, can directly convert input data into output predictions while bypassing the intermediate steps that typically occur in the pipeline of traditional software components. Methods and systems according to embodiments of the present invention can handle the entire sequence of tasks. Additional steps (such as data collection or auxiliary processes) are not part of the end-to-end model unless the model can learn intermediate processes with the given data.

[0161] In this application, an artificial neural network (ANN) is trained using many examples, starting with a 3D image of the patient's mouth as input and outputting the final treatment plan. The ANN finds the relationship between the input (e.g., 3D raw scan data as a point cloud) on one end and the output (e.g., abutment or crown parameters) on the other end. The ANN is trained using a large number of such examples, for example, more than 500,000 different examples of such input and output.

[0162] "Training Neural Networks" The underlying elements of a neural network are called perceptrons, or artificial neurons. A perceptron accepts a series of inputs, performs a function on those inputs, and produces an output that can be passed to other neurons. For example, the function could be the sum of weighted inputs. A neural network is a collection of interconnected neurons. Neurons are grouped and connected in "layers." The output of one layer is typically connected to the next layer via an activation function. Sigmoid, TanH, or ReLU functions are common. Neural networks can be made more complex by adding additional layers, often called "hidden layers."

[0163] Forward and backpropagation are used to train the network and find better or optimal weights. After initializing the weights, input data with known outcomes is pushed "forward" through the network, producing output predictions. A cost, or loss function, is used to calculate how far the predictions compare to the expected outcomes.

[0164] The error or cost is then reduced to a low level or the lowest possible point, i.e. the global minimum. This can be achieved using, for example, the gradient descent method. The goal of the gradient descent algorithm is to find the partial derivatives of the cost function with respect to each weight. The direction (+ / -) and slope of the cost function are used to determine how much and in what direction to adjust the weights to achieve a low or zero cost. If the gradient is 0, then the minimum has been reached. Starting from the output layer, the gradient descent algorithm "backwards" to the next layer and further to the input. For each node of the neural network, it is calculated how much the weight needs to be changed to approach a zero or low cost.

[0165] Further training of the neural network includes forward and back propagation until the error has been minimized. This process can be used to adjust the weights of the neural network in an end-to-end manner for use in embodiments of the present invention.

[0166] "Regression" and "classification" parameter estimation are typically and often regression problems. For example, it involves estimating "numbers" rather than "classes." For example, in classification, a first range of values ​​(e.g., numbers) can be divided into smaller intervals, each of which has a second, smaller range (which can be called "classes"), and a machine learning system including an ANN estimates the correct interval to which a particular parameter belongs.

[0167] "Human designer-specific rules" These are the rules dental technicians typically use to design restorative dental objects. Rules can include how much pressure the abutment base can exert on the surrounding soft tissue, how high the abutment can be compared to its adjacent and opposing teeth, and so on.

[0168] The "underlying restoration model" (e.g., an abutment or a crown) is not a free-form 3D shape, but a parametric model that can be defined by a set of parameters (=numbers). For example, the shape of an abutment or crown model can be completely determined by a set of parameters (~60 numbers for an abutment), which is much more compact and controllable than a free-form model, whose shape should be defined by a large number of points and the triangles connecting them (i.e., a format that represents a general 3D object).

[0169] As used in this application, "design of a dental object" means:

[0170] a) a description of the 3D geometry of a dental object in a machine-readable format so that it can be manufactured or

[0171] b) A representation in a machine-readable format from which the 3D geometry of the dental object can be manufactured.

[0172] The design of the dental object is preferably based on the patient-specific characteristics of the existing intra-oral situation (ie remaining dentition, gingiva, etc.).

[0173] Automated design of customized dental objects by end-to-end connecting the following elements:

[0174] The artificial neural network is trained to find direct patterns between the input at one end of the process (i.e., for receiving raw 3D scan data, which may be a point cloud) and the output at the other end of the process, without relying on any intermediate form of data in between.

[0175] For example, an input on one end of the data entry device is adapted to receive input raw scan data. This data may be a digital representation of the (e.g., intraoral) environment, such as a 3D digital representation in the form of a point cloud, a triangulated surface representation, etc. It is preferably 3D scan data in the form of a point cloud.

[0176] The output of the artificial neural network is a machine-readable description of the dental object or a machine-readable representation of the dental object under consideration.

[0177] Principal Component Analysis (PCA) is a type of projection of a high-dimensional space of objects into a low-dimensional space. Preferably, as much information as possible is retained. Mathematically, this can be achieved by calculating the eigenvectors of the covariance matrix. Thus, the n eigenvectors with the largest eigenvalues ​​can be used to project into n-dimensional space. Using these n eigenvectors, it is possible to predict or reconstruct any object belonging to the class of the object under consideration.

[0178] "Approximating a shape" means that any difference between a proposed shape and the actual shape of the model, with reference to the parameterized model, involves finding a set of parameter values ​​that minimizes an error term quantifying the difference between the proposed shape and the actual shape. For each component of a representative training set comprising dental objects in its (intraoral) environment, the parameter values ​​that, when applied to the parameterized model, produce the shape (or approximate shape) of that dental object are determined.

[0179] An "algorithm" is understood to be a self-consistent sequence of steps leading to a desired result. These steps require physical manipulations of physical quantities, for example in the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, or otherwise manipulated. DETAILED DESCRIPTION

[0180] The accompanying drawings and the following description describe certain embodiments by way of illustration only. Those skilled in the art will readily appreciate from the following description that alternative embodiments of the structures and methods shown herein may be employed without departing from the principles described herein. Reference will now be made in detail to several embodiments, examples of which are illustrated in the accompanying drawings. It is noted that, whenever feasible, similar or like reference numerals may be used in the drawings to indicate similar or like functionality. It will be apparent to those skilled in the art that the present invention may be practiced in a variety of ways.

[0181] Embodiments of the present invention utilize machine learning algorithms for computer-implemented methods or systems for automated restoration design. The restoration design includes the design of a restorative dental object, such as an abutment or crown for an implant (e.g., a single implant), or a crown for a tooth root. Recognizing the shape of a restorative dental object, such as an abutment or crown for an implant, can be performed within five minutes of receiving the patient's scan data, or within one minute of receiving the scan data, and more preferably, within 30 seconds of receiving the patient's scan data.

[0182] Software for performing the method when executed on a digital processor is provided. The manufacture of restorative dental articles is also included.

[0183] Embodiments of the present invention use 3D scan data of a patient's dentition as input to a neural network. The scan data can be in the form of a point cloud of a 3D image. Methods and systems according to embodiments of the present invention utilize ML algorithms executed on a processing device, such as a computer microprocessor, that has access to computer memory, such as volatile and non-volatile memory.

[0184] Embodiments of the present invention also relate to computer-implemented systems suitable for automated restoration design using machine learning algorithms. The computer-implemented systems are preferably specially programmed or constructed for the desired purpose including computer programs. These can be stored on a computer-usable or computer-readable storage medium.

[0185] In addition, the present invention can take the form of a computer program product that can be stored on a non-transitory machine-readable storage medium. The non-transitory machine-readable storage medium can be an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system (or device or apparatus) or a propagation medium. Examples of non-transitory machine-readable storage media include, but are not limited to, semiconductor or solid-state memory, magnetic tape, removable computer floppy disk, random access memory (RAM), read-only memory (ROM), hard disk, optical disk, EPROM, EEPROM, magnetic card or optical card, or any type of computer-readable storage medium suitable for storing data, images or electronic instructions, and each coupled to a computer system bus. Examples of optical disks include compact disk - read only memory (CD-ROM), compact disk - read / write (CD-R / W) and digital video disk (DVD).

[0186] A processing device such as a microprocessor with memory is suitable for storing and / or executing program code and includes at least one microprocessor coupled to the memory, for example, directly or indirectly, via a system bus. The memory may include any local memory used during program code execution. A computer system may include input / output (I / O) devices, such as a keyboard, display, pointing device, or other device configured to receive data or present data, which are coupled to the system either directly or through an intervening I / O controller.

[0187] Network adapters may also be coupled to the processing device to enable coupling to other data processing systems or remote printers or storage devices through intervening private or public networks.Modems, cable modems, and network cards are examples of currently available types of network adapters.

[0188] Embodiments of the present invention include methods, systems and software for automated restoration design. The restoration design includes a restorative dental object derived from scan data of a patient's dentition using machine learning (e.g., utilizing a neural network). The patient's dentition dataset may include one or more of the patient's scan data, resulting from multiple one or more scans of all or part of the patient's mouth, with or without scans of other parts of the patient's body, such as one or more portions of the patient's jaw. The computer-implemented method of automated restoration design uses a 3D electronic image of at least a portion of the patient's mouth as a starting point for the design process. In some embodiments, the electronic image is obtained by directly scanning the patient's teeth. Embodiments of the present invention allow a dentist or dental technician to capture a 3D image, or may be obtained by scanning one or more impressions of the patient's teeth. Embodiments of the present invention are suitable for use in the dentist's chair.

[0189] Embodiments of the present invention are based on the observation that the relationship between the design of a customized dental object and the input data used to create that design can be learned by training a machine learning system or method using input (e.g., via an I / O port or interface) and relevant examples of many existing treatment plans and outcomes designed by human operators, such as dental technicians. This can be considered a typical supervised learning problem. This algorithm involves a problem domain where the input data is 1) known implant type, location, and orientation; 2) 3D scan data of a patient's jaw (e.g., as a point cloud); and 3) additional options (or customer preferences) for how to design a specific dental object. This information is complete and sufficient for a human designer to create the shape of a restorative dental object, such as a customized crown or abutment shape. The human designer applies their dental knowledge and specific rules to design an abutment or crown that will optimally fit the dentition surrounding the missing tooth based on the given input data. The output is a complete restoration design, which can be further defined by a set of parameters that uniquely define the shape, location, and orientation of the abutment or crown. The present invention relies on the underlying restoration model (eg, an abutment or crown) not being a free-form 3D shape, but rather a parametric model that can be defined by a set of parameters (eg, numbers).

[0190] Embodiments of the present invention use complete 3D data (e.g., occupancy grids, point clouds) as input to ML systems or methods. In embodiments of the present invention, the technical difficulty of using 3D data directly has been addressed by this and by 3D convolution operations (e.g., implemented on a convolutional neural network (CNN)) applied by the machine learning system or method.

[0191] For embodiments of the present invention, the input data is preferably in the form of a 3D optical scan of the patient's jaw (e.g. Figure 1 The position and orientation of the implant are typically known using a separate process that uses the detection of scanned volumes or Feature Location Objects (FLOs) placed on the implant. Raw 3D scan data and implant position and orientation are the minimum input information required for any type of implant-based prosthetic design, whether done automatically or manually.

[0192] Given an input, the desired output is a design of a target restoration object (e.g., an abutment or crown) from which the final physical product can be produced with no or minimal changes. The final restoration design (e.g., an abutment or crown) can be represented by either:

[0193] 1) a general 3D object format (e.g., a triangular mesh connecting dense points on the object's surface) or

[0194] 2) By defining a set of parameters that define the shape of the underlying model designed for a specific type of 3D object (e.g., an abutment model or a crown model). Figure 1 The right image is an example of an abutment model placed on an implant site.

[0195] Identifying the shape of a restorative dental object, such as an abutment for an implant or a crown, can be performed within five minutes of receiving the patient's scan data, or within one minute of receiving the scan data, and more preferably within less than 30 seconds of receiving the patient's scan data.

[0196] From patent US 9814549 (incorporated by reference), it is known that a flexible arch model (FAM) can be computed to capture the variation of multiple real dental arches in a training set and parameterized to reconstruct missing teeth in the patient's dental anatomy. Constructing the FAM involves obtaining multiple sets of digitized dental arches, where a pair of maxillary (upper) and mandibular (lower) are in correct relative positions and collecting a predefined set of landmark points on the occlusal surface of each dental arch, all in the same order and the same corresponding positions across multiple samples. The collected landmark point vectors are used to perform statistical modeling (e.g., principal component analysis) to create a linear subspace of feature points based on the principal components (when PCA is used) found during the process. Any set of landmark points on a pair of upper and lower arches can be reconstructed by a linear combination of the principal components within a reasonable range of variation captured from the training samples.

[0197] From US2017 / 0095319 (incorporated by reference), it is known that a computer-implemented method can be used to design a dental restoration component. The method comprises: receiving a set of design dimension constraints that must be satisfied for the dental restoration component; receiving a set of design parameters for the dental restoration component; receiving a definition of a penalty function that takes into account at least one of the design parameters and signals that a constraint has been met when a value of any one of the parameters violates a constraint; and assigning, using the penalty function, a value to each of the design parameters for the dental restoration component that is consistent with the constraint for the component.

[0198] From US 9504541 or US 2007 / 0154868 (both incorporated by reference), a method for designing a dental restoration component is described. A set of design dimension constraints that must be satisfied for the dental restoration component is defined. A set of design parameters for the dental restoration component is also defined. Each of the design parameters for the dental restoration component is assigned a value that is consistent with the constraints for the component, at least in part using a penalty function, the penalty function taking into account at least one of the design parameters and signaling that a constraint has been reached when the value of any one of the parameters violates a constraint.

[0199] 3D statistical analysis is known from US2016 / 0224690 (incorporated by reference) to generate crown models using statistical methods such as k-means clustering, principal component analysis (PCA), or similar statistical methods. This results in crown models that lack sharpness details below the threshold of the statistical technique. A method is provided that adds clarity back to the resulting model by combining a single fully characterized example into the algorithm that generates the statistical model. The end result is a relatively simple crown model that can be generated and manipulated in real time, yet still maintains the anatomical clarity of a natural tooth.

[0200] From US 7551760 or US 7362890 or US 2008 / 0002869 or US 2006 / 0072810 (all incorporated by reference), it is known that three-dimensional based modeling methods and systems can be used to make designs for dental and related medical (and appropriate non-medical) applications. A data capture component generates a point cloud representing the three-dimensional surface of an object (e.g., a dental arch). Three-dimensional identification of objects is provided, particularly in areas of low image clarity in the image field, and particularly in such areas appearing in overlapping portions of at least two images, to provide position, angle, and orientation information sufficient to enable highly accurate combination (or "stitching") of adjacent and overlapping images to the three-dimensional image processing software. Alignment and creation of aligned related objects or models thereof, such as the maxillary arch and the mandibular arch, are facilitated.

[0201] One known abutment model in the dental industry represents a variety of custom abutment shapes based on over 60 design parameters that can be tailored to a specific patient's dentition. Parametric models created through statistical analysis of a wide range of real-world tooth samples are also known. These methods are specific examples of parameter-based underlying restoration models.

[0202] Given a 3D scan of a patient's jaw and the position and orientation of an implant at a site with missing teeth, embodiments of the present invention enable the creation of a system or method, e.g., a computer-based system or method, that can present a complete restoration design proposal, e.g., a 3D model of a restorative dental object (e.g., an abutment or crown) to an end user in a CAD / CAM format very quickly (e.g., in a few seconds or less) and in a fully automated manner. Embodiments of the present invention leverage recent advances in machine learning (ML) and artificial intelligence (AI), and one aspect of embodiments of the present invention is to have the system learn the underlying patterns between inputs and outputs from many real-world examples, i.e., in an end-to-end manner. Embodiments of the present invention begin with many real-world clinical dental cases of restorative dental objects for which the starting position (e.g., missing teeth) is known and the final treatment plan, treatment, and outcome (i.e., the design of the restorative dental object) are known. The starting position (e.g., missing teeth) represents one end of the process, while the final treatment plan, treatment, and outcome (i.e., the design of the restorative dental object) represent the other end. Therefore, the method according to an embodiment of the present invention trains a neural network such as a CNN using only raw data and the final results, thereby performing end-to-end machine learning. Traditionally, end-to-end machine learning based on information available in existing clinical case documents has not been used. For example, dental CAD automation using deep learning, as described in US2018 / 0028294A1, primarily utilizes machine learning to allow the system to classify the correct tooth features around the dentition of a missing tooth, which is then used to estimate crown restorations using a generative machine learning algorithm.

[0203] In contrast to this prior art disclosure, the problem of restoration design according to some embodiments of the present invention is solved by having the system learn the pattern between input and output end-to-end, meaning that the input is the raw jaw scan and the output is the final restoration design, e.g. in the form of model parameters, without the need for any pre-processing or one or more intermediate feature detection processes, which makes the prediction process for the end user very fast and also very reliable.

[0204] In an embodiment of the present invention, one or more scans are taken of the patient's oral cavity. These scans may include some, any, or all of the occlusal, lingual, or buccal scans. The contact areas of adjacent teeth may also be scanned. The multiple scans may be combined into an underlying 3D digital model including the implant. The underlying 3D digital model is used to design a dental restoration (e.g., a crown or abutment). In an embodiment of the present invention, a restoration design program is provided by a trained machine learning system, and the patient and / or dentist and / or dental technician may view the restoration via a graphical interface. The interface may be adapted so that the dental technician or dentist may refine the proposed restoration.

[0205] In an embodiment of the present invention, a machine learning system, such as a neural network, of which CNN is an example, is trained using a number of training data sets comprising only raw image data and final treatment plans and / or restorative dental objects.

[0206] Embodiments of the present invention may employ a discriminative machine learning algorithm to determine a suitable dental restoration, such as an abutment or crown for an implant. The discriminative machine learning algorithm may be trained to output a 3D representation or model of a crown or abutment. The training set is preferably limited to cases having the same type of restoration, for example, the training data set may be those involving crowns or abutments. Outputting a representation of the shape of a restorative dental object, such as an abutment or crown for an implant, may be performed within five minutes of receiving the patient's scan data, or within one minute of receiving the scan data, and more preferably within less than 30 seconds of receiving the patient's scan data.

[0207] exist Figure 2 , is a block diagram of a computer-implemented system 10 suitable for a machine learning algorithm according to an embodiment of the present invention. The computer-implemented system is suitable for performing a method for automated restoration design according to an embodiment of the present invention. The restoration design includes the design of a restorative dental object (such as an abutment for an implant or a crown) according to an embodiment of the present invention. The system 10 may include an ML server 11 in a network 15, whereby the ML server 11 includes a web server 12, a model server 13, and a trained model 14. The scanner 19 is optional or may be located at a different location.

[0208] The network 15 enables communication with other devices on the network and can use standard telecommunication protocols. For example, the network 15 may include a network device 18 for receiving 3D images, for example from a scanner 19. The network 15 may be a conventional wired or wireless network, such as a local area network (LAN), a wide area network (WAN). The network 15 may include a cloud network using cloud computing technology. The model server 13 is suitable for executing an ML algorithm according to an embodiment of the present invention. The web server 12 may be suitable for receiving 3D images of a patient, for example, including scanned 3D images generated by a scanner 19, from the network device 18. After the model server 13 has prepared a restorative design based on these images, the web server 12 may be suitable for sending the complete restoration design to, for example, a dental technician or dentist who has access to the network device 18. The complete restoration design may be in the form of parameter values.

[0209] The model server 13 and model 14 can be trained using pre-existing 3D dental images and restorative dental object designs from these images by human experts for automated design of restorative dental objects. The model server 13 can be adapted to operate a neural network, such as a CNN. A number of training datasets from real dental patients with one or more crowns or abutments can be selected to form a set of training datasets specifically for crowns or abutments. The ML server 11 can communicate with a data server 17 that stores a database of pre-existing 3D dental images and restorative dental objects designed by humans.

[0210] In an embodiment of the present invention, the weights of the nodes of the CNN can be trained to minimize the error. The CNN network architecture according to an embodiment of the present invention includes a CNN consisting of or including multiple layers, including optionally a normalization layer, an additional convolutional layer, and a fully connected layer.

[0211] The first layer of a CNN can optionally perform image normalization. The normalizer can be hard-coded and does not need to be changed during the learning process. Performing normalization within a CNN allows the normalization scheme to be changed using the CNN architecture and accelerated by GPU processing.

[0212] Convolutional layers are not required to perform feature extraction. After the convolutional layers, there are fully connected layers that result in output parameter values. The CNN is trained end-to-end using existing treatment data.

[0213] Embodiments of the present invention are based on machine learning and can utilize an artificial neural network (ANN) framework for supervised learning. Methods or systems according to embodiments of the present invention are based on a known set of examples using a large number of inputs and outputs, with the output being the design of a restorative dental object. Training is accomplished using pre-existing 3D dental images and restorative dental objects designed by human experts based on these images. The neural network can be in either a training or an operational state.

[0214] refer to Figure 3The input for training is a large number of raw optical 3D scans 20 of past clinical cases of patients, and the output is the corresponding designs 22 of abutments and crowns generated by humans. The designs 22 are preferably defined by model parameters and can be generated by human restoration design technicians and stored in a database. This training step 24 can be computationally intensive and is preferably performed offline by a high-performance GPU (graphics processing unit). The training step 24 can be performed only once offline using all past existing data sets, so there is no time pressure for training. During this training step 24, the ANN will learn implicit patterns 26 and / or rules that associate all input training data with corresponding output training data. Once the model has been trained and created, the neural network can be placed in an operational state or online mode and the learned patterns 26 can be used immediately when new inputs 28 are given to the system. Moreover, the ANN can be used in an end-to-end online mode to instantly generate predictions of output designs 29 from new inputs 28. In addition to instant prediction time, other benefits of embodiments of the present invention may include one or more or all of the following:

[0215] 1) No need to write explicit, hand-crafted, complex algorithms to combine rules and patterns between input and output,

[0216] 2) The training step 24 will find rules and patterns 26 in a good or optimal way that would be difficult for humans or other traditional computational methods to find, and

[0217] 3) The output 29 is not sensitive to the quality of any intermediate detection process or other sub-components.

[0218] Layered training & prediction

[0219] Embodiments of the present invention include a hierarchical training approach that includes training important individual parameters and / or dividing the remaining design parameters into different groups based on their relative dependence on the coordinate system in which their values ​​are interpreted. Thus, a hierarchical training approach refers to a ranking of the importance of one or more parameters (used to define the overall shape of a structure such as a restorative dental object) relative to other parameters of the same parameterized model. Regardless of the level in the hierarchy, all parameters are related to the entire abutment, the entire crown, etc. Rotational parameters are the most important parameters and can be trained first using input scans in the implant coordinate system. Angle and position parameters can then be trained (e.g., each individually or in groups) based on input scans normalized by known rotation angles. Training of the remaining shape parameters (e.g., each individually or in groups) can then be completed based on input scans normalized by known positions and other angles. Based on experimental results, this hierarchical training approach can provide significantly better accuracy results than training all parameters using input scans in a single coordinate system.

[0220] Accordingly, the same hierarchical approach can be followed during the prediction step. Given a new input scan and FLO detection, the input scan is first transformed into the implant coordinate system (known from the FLO detection), and the parameter values ​​of the most important parameters are predicted using this input scan. The input scan is then further transformed based on this and other predicted parameter values, and these additional parameters are predicted using this input scan. Finally, the input scan is further transformed based on the previously predicted parameters, and the remaining parameters are predicted using this input scan. Therefore, embodiments of the present invention may optionally include a hierarchical parameter prediction method that includes predicting important individual parameters and / or grouping the remaining design parameters into different groups based on their relative dependency on the coordinate system used to interpret their values. Prediction then begins by performing a separate prediction for each of the most important parameters. Less important parameters can then be predicted in groups. Thus, hierarchical training and hierarchical prediction refer to the ranking of the importance of one or more parameters (used to define the overall shape of a structure, such as a restorative dental object) relative to other parameters of the same parameterized model. Regardless of the level in the hierarchy, all parameters are relevant to the entire abutment, the entire crown, and so on.

[0221] Thus, using a computer for training and / or prediction in a hierarchical manner involves training and / or predicting the more important parameters or groups of parameters first or early in the training or prediction process. These important parameters are related to the shape of the dental object defined by the underlying parametric model of the dental object (i.e., the entire abutment, the entire crown, etc.). Thus, hierarchical training and / or hierarchical prediction refers to the ranking of the importance of one or more parameters (for defining the overall shape of a structure such as a restorative dental object) relative to other parameters of the same parametric model. Regardless of the level within the hierarchy, all parameters are related to the entire abutment, the entire crown, etc.

[0222] Below is given an indication of certain parameters which have been shown to be important for an abutment or crown, but these are given as examples and for different applications the priorities may be different. These different priorities and different applications are included within the scope of the present invention.

[0223] Rotation parameter estimation

[0224] An example of an important design parameter to estimate correctly in the design of dental restorations is the rotation parameter. In particular, the rotation parameter that determines the correct facial or buccal aspect of the restoration object has been found to be the most important. For example, even a slight rotational misalignment is easily detectable by the human eye and can also affect the determination of other parameters such as the horizontal angle or other shape parameters. The rotation value can be determined from the orientation of the locking features of the implant ( Figure 5The FLO detection process will provide this horizontal implant orientation and this will be the input information necessary to determine the correct rotation value. Figure 5 f vector in ) and vertical implant orientation ( Figure 5 The u vector in is known, so if the raw scan is transformed into this implant coordinate system, this information can be embedded in the raw scan. In this way, no separate input of the implant orientation is required in addition to the transformed raw scan. Figure 4 Several examples of transformed input scans (e.g., resampled surface points) with different associated rotation values ​​are shown. Note that while the shape differences of the individual scans are small, transformed input scans with similar rotation values ​​will show similar global point distributions. This global point distribution is a strong indicator of predicting the correct rotation value, and the training step of a 3D convolutional neural network (CNN) has been found to be well suited to discovering such patterns.

[0225] Core angle parameter estimation

[0226] Buccal-lingual angle: This parameter determines how the restoration parts are angled buccal-lingually relative to the perpendicular implant axis.

[0227] Mesiodistal angulation: This parameter determines how much the restoration portion is angled buccal-lingually relative to the perpendicular implant axis.

[0228] These two restoration angle parameters (also called core angles) are the next most important parameters after the rotation angle and also have a global influence on the rest of the restoration shape parameters. However, unlike the rotation angle, these core angles are not affected by the rotational aspects of the implant ( Figure 5 ) and is only affected by the vertical orientation of the implant ( Figure 5 To remove unnecessary estimation degrees of freedom, we train these two parameters by transforming the input scans again based on the known rotation value of each data sample (i.e., the amount of rotation along the perpendicular implant axis), which has been shown to be more effective than training them from the implant coordinate system.

[0229] Restoration position parameters

[0230] Buccal-lingual offset: This parameter determines the position of the buccal-lingual positioning of the restoration parts relative to the center of the implant.

[0231] Mesiodistal offset: This parameter determines the mesiodistal positioning of the restoration part relative to the center of the implant.

[0232] These two parameters are equally important to the two core angle parameters in that they determine the position of the restoration part once the rotation value is set (i.e., how much it is displaced from the center of the implant). Therefore, it is important to train these two parameters using input samples transformed by known rotation values ​​(i.e., using normalized input samples after removing the rotational changes).

[0233] Abutment shape parameters

[0234] Beyond the five global angle and position parameters mentioned above, the remaining design parameters (of either the abutment or crown) primarily govern the local dimensions or shapes of the restoration components. Some examples of such abutment design parameters include facial cusp height, mesiodistal margin width, shoulder width, facial margin height, and so on, depending on the specific underlying model. Expertly designed abutment models can have over 50 such parameters, which define unique abutment shapes across a wide range of geometric variations. Figure 12 Some examples of abutments created with several different parameter sets are shown. These abutment shape parameters are all defined in a coordinate system that can be determined by five global parameters, which we call the "core coordinate system." This means that it is beneficial to transform the original input scan into this core coordinate system to train the remaining parameters by removing the variations in those global parameters.

[0235] Crown measurement parameters

[0236] Unlike abutments, the crown restoration portion needs to resemble a natural tooth with a biological shape, and its complete anatomical shape is difficult to describe with a small number of parameters associated with some direct geometric meaning. Understanding this limitation while still being able to train crown models with a certain overall size and shape (without complete anatomical details), we measured each sample of 3D anatomical crowns created by human designers based on the following five parameters: facial cusp height, lingual cusp height, mesiodistal width, buccal-lingual width, and cusp angle. We then trained these measurements in the same "core coordinate system" that we used for the abutment shape parameters.

[0237] Crown design can be built on top of a statistical method called principal component analysis (PCA), which is considered one of the unsupervised machine learning techniques. Using this method, the complete anatomical shape of the crown can be represented with a compact representation, such as using a small set of parameters called "PCA parameters" combined with the average shape of the crown, which provides a way to generate the complete anatomical shape of the crown over a wide range of variations. These PCA parameters can be trained in an end-to-end manner, and the system can learn the pattern between a given input scan and the PCA parameters to resolve the complete anatomical details using the above-mentioned machine learning framework. For this method, a large number of real-world clinical crown samples are required.

[0238] 1 preparation, including training

[0239] This embodiment has the following steps:

[0240] Step 1 : Create or obtain a parametric (mathematical) model of the dental object to be designed and capture its pattern of variation.

[0241] In a preferred embodiment, such a parameterized model is an active shape model (ASM) generated by applying principal component analysis to a training set of (example or reference) dental objects.The model can capture the true shape variations as well as position and scale variances.

[0242] According to another embodiment, a parametric model is developed for a specific type of component based on empirical or expert defined rules. The parametric model may or may not capture positional variations of the dental object relative to a defined coordinate system.

[0243] Step 2 : For each dental object in a representative training set comprising dental objects in its (intra-oral) environment, parameter values ​​are determined that, when applied to a parameterized model, produce the shape (or an approximate shape) of that dental object.

[0244] If the parameterized model is an ASM, then the representative training object set can be the training set used to generate the ASM. The parameter values ​​to be determined are weighting factors for each eigenvector, the linear combination of which determines the shape of the dental object. To determine the parameter values, an iterative search algorithm can be used to fit the statistical component model (ASM) to the dental objects in the training set.

[0245] In some cases, the parameter values ​​for a representative training set may already be known. This may be the case, for example, when the training set comprises dental objects that were originally designed using a parameterized model for a corresponding intra-oral environment.

[0246] Step 3 : Specifies the training hierarchy (i.e., which parameter or group of parameters to train first) for each parameter in a parameterized model.

[0247] According to one embodiment, for example, for a parametric model created, when the shape variation of a dental object does not depend on its position / scale variance, the hierarchy is exactly the same for each parameter.

[0248] According to another embodiment, first a training of the parameters of the model determining the position of the dental object relative to its environment is performed.

[0249] Step 4For each parameter in the parametric model (either individually, i.e., training each parameter individually or in parameter groups), an artificial neural network is trained to learn the underlying patterns between all training input data sets (i.e., all digital representations of the (intraoral) environment of the training set) and the corresponding output data (i.e., all parameter values ​​of the same parameter determined for each dental object of the training set). The input 3D scan data can be in the form of a point cloud. Training is performed according to the hierarchical structure specified in step 3. This means that each parameter or parameter group is trained separately from the other parameters or parameter groups. It has been found that attempting to train all parameters together does not lead to good results.

[0250] According to a preferred embodiment, the training of each parameter is done based on the input after normalization by the previous parameter(s).

[0251] Operation Mode (Design Automation)

[0252] Step 1 : A digital representation of the patient-specific (intraoral) environment is provided to an artificial neural network that has been previously trained to generate parameter values ​​for a parametric model of a dental object based on the learned patterns as described above. The input data may be 3D scan data, which may be in the form of a point cloud.

[0253] Step 2 : Parameter values ​​are applied to the parameterized model to create a geometric description of the desired patient-specific dental object or to create a representation of the geometric description of the desired patient-specific dental object from which the dental object can be manufactured. This can be done in a hierarchical manner by predicting, for example, individual parameters and then groups of parameters.

[0254] Step 3 : translating a geometric description of the dental object (if required) into a machine-readable description of the dental object, which description can be used to command a manufacturing device (e.g., a milling unit, a 3D printer, a spark erosion device, etc.) to manufacture the dental object. Alternatively, translating a representation of a geometric description of a desired patient-specific dental object (if required) into a machine-readable description of the dental object, which description can be used to derive commands to be provided to a manufacturing device (e.g., a milling unit, a 3D printer, a spark erosion device, etc.) to manufacture the dental object.

[0255] Input data generation

[0256] Embodiments of the present invention formulate input scan data for training and prediction. Due to technical difficulties encountered when using full 3D datasets as a source of training input, other known systems use 2D data or 2.5D data (e.g., 2D images with depth information about a specific direction) as their neural network input. Obviously, those 2D and 2.5D datasets lose some of the complete information that the 3D input scan data can represent.

[0257] 3D occupancy grid

[0258] Embodiments of the present invention use full 3D input scan data. One option for representation of 3D input data that can be used in embodiments of the present invention is a representation called a "3D occupancy grid" ( Figure 5 ). Given the implant position and orientation ( Figure 5 b, f, u in the image), the input scan is first transformed to this implant coordinate system. A NxNxN 3D grid is then defined that covers a specific geometric length on each axis. Each voxel on the grid can then be filled with 1 when the scanned surface overlaps with the voxel, and 0 otherwise. In this way, the presence of a surface on the input scan is represented by NxNxN 3D binary data. For example, if N=40 and covers 1 inch on each axis, this 40x40x40 binary data allows them to be easily trained within the limitations of 8GB GPU memory. Although Figure 5 Not shown, but preferably both the cast scan where the implant is present and its opposing cast scan (where such a scan exists) are included in the same occupancy grid.

[0259] The coverage of the occupancy grid is chosen starting from a given resolution of the occupancy grid, which is set by the limitations of the GPU memory. The occupancy grid can cover a wider range of the scan by sampling the surface data less densely, but at the expense of losing some surface details. Since the main goal is to find end-to-end patterns between the input scan data and the final restoration design of the implant, the most relevant information on the input scan is located near the implant location, including directly adjacent and opposing teeth. Based on multiple experiments, it has been found that a coverage of approximately 1 inch in each axis will provide the most accurate results at a given grid resolution.

[0260] Auxiliary input

[0261] In addition to the scan data and the input of implant position and orientation, other input information can be obtained that can provide further clues for the final restoration design during training. Preferably, such auxiliary data is only included in the training if the same information is also available at the time of prediction. Additional information can include the number of teeth, design preferences, and implant type and corresponding specifications. The number of teeth is useful to help improve the accuracy of training and prediction. Then there is some optional design preference information provided by the customer when placing an order, which can be useful to create some option-based designs (for example, how deep the abutment margin should be subgingival).

[0262] Neural Network Architecture

[0263] 3D Convolutional Neural Networks

[0264] Convolutional Neural Networks (CNNs) are the preferred network architecture for embodiments of the present invention. CNNs are used to find patterns between sampled input scan data (e.g., occupancy grids) and final design parameters in an end-to-end manner. CNN is a known general neural network architecture. In embodiments of the present invention, it can be used to find geometric features (2D or 3D) in the input data that are associated with distinguishing labels (i.e., known outputs). Since the input is 3D (e.g., a 3D occupancy grid), 3D convolutional filters can be conveniently used to improve or maximize performance. Although a technician may think that using 2D convolutional filters can make the training step faster, experimental results show that using 3D operations always provides better results. Figure 6 An embodiment of the present invention is shown that includes a dedicated neural network architecture 40, wherein the neural network 30 has multiple layers, such as multiple pooling layers 34 and multiple convolutional layers 35. The number of pooling layers 34 can be a maximum of three, and the number of convolutional layers 35 can be five. Alternatively, a pre-built network or pre-trained model can be used. Note that Figure 6 The specific architecture shown in is merely an example of a method for implementing machine learning formulations for dental restoration design according to embodiments of the present invention. The machine learning included in embodiments of the present invention can be implemented using other neural network architectures, as long as such networks can effectively find relationships or patterns between inputs and outputs in an end-to-end manner. For example, a neural network structure known as "PointNet" can be used in embodiments of the present invention—see, for example, Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas, PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, CVPR 2017.

[0265] Handle auxiliary input

[0266] like Figure 6 As shown in the block diagram in , the geometric input (e.g., occupancy grid) 31 is fed into the 3D CNN 30 and the auxiliary input 32 (e.g., number of teeth, design preferences, etc.) is fed into a separate densely connected layer 41, and finally merged with the dense layer 36 of the output of the 3D CNN 30 in layer 37. After merging in the merging layer 37, the output of this layer is connected to the final readout layer 38 (e.g., a softmax layer), which outputs one or more parameter values ​​39. These parameter values ​​are used to define the shape and position of the restorative dental object by inputting these parameters into the underlying parametric model. This structure of treating the auxiliary input 32 separately from the 3D scan data 31 is very effective in making the overall performance much better than using only the 3D input data 31 or treating them as combined inputs to the CNN.

[0267] Conversion from regression problem to classification problem

[0268] An optional feature that helps achieve high accuracy in training and prediction is to reformulate the regression problem (i.e., estimating the value of a parameter numerically) as a classification problem (i.e., estimating a class among a selection of multiple classes). The estimation of a parameter (i.e., a number) is essentially a regression problem. It is not difficult to build a network to solve a regression problem, and this is included in embodiments of the present invention. However, it has been found that estimating exact or highly precise parameter values ​​is not very relevant because there is no single parameterized design value that is absolutely correct in every design. Instead, there is some range of acceptable values ​​for each parameter. Therefore, it is sufficient if the predictive value is within some range of the known (target) parameter value. Based on this observation, in embodiments of the present invention, the range of numbers defines smaller intervals ("classes"). Figure 6 The neural network 40 in is used to estimate the correct interval that a particular parameter falls into. This is much more efficient than training the system to estimate the exact or highly accurate value of the parameter and provides better results.

[0269] One-Hot Encoded Softmax and Gaussian Softener

[0270] Softmax is the standard form of the output layer 38 of the neural network architecture 40, such as Figure 6 As shown in , it is used to solve classification problems (see Figure 7). This partitioning into ranges is often interpreted as a "probability vector" because after applying this softmax operation, each class (interval) is assigned a probability number (between 0 and 1). During the training process, this softmax output (or probability vector) is generated by the neural network 30 on each input sample 31 and compared with the known correct value (label) represented in a "one-hot encoding" format. For example, if there are five classes, the softmax output will be a vector of five probability values ​​[0.1, 0.2, 0.5, 0.1, 0.1] and the correct answer in "one-hot encoding" form (assuming it is the third class) will be [0, 0, 1, 0, 0]. The two vectors are then compared, and the training algorithm will penalize the answer estimated by the network based on their similarity. However, in an embodiment of the present invention, because the second and fourth classes are closer to the correct (third) class, it is considered inappropriate or unbalanced to penalize the second and fourth classes in the same way as the first and fifth classes. Based on this observation, in an embodiment of the present invention, an additional discrete Gaussian filter is applied to the one-hot encoded vector [0, 0, 1, 0, 0], resulting in the second and fourth classes being penalized less than the first and fifth classes compared to the softmax output [0.1, 0.2, 0.5, 0.1, 0.1] generated by the network. According to actual experiments, this feature of implementing a Gaussian softener for raw labels can significantly improve the accuracy of certain parameter sets.

[0271] Build System

[0272] Once the models have been trained for each design parameter, all trained models can be deployed on a machine learning computing device (such as a server), and they can be used to predict parameter values ​​for new input data. Figure 8Two complementary processes are described that are being used by such a system according to an embodiment of the present invention. One method 100 is used for an offline training process, while the other method 200 is used for an online prediction system for an end user. In method 100, for each past clinical case of an implant-based restoration, input data for training is collected in step 102, including some or all of the implant position, orientation, design preferences, and 3D jaw geometry scans (e.g., as a point cloud). In step 104, for each clinical case, point samples are extracted in a suitable form on the 3D surface of the jaw scan geometry for input to an artificial neural network (ANN). In step 106, the ANN is used to train the model by mapping the input data to the final restoration design parameters (output of the ANN) for all collected cases in an end-to-end manner. In step 108, the output model is trained and saved as a collection of computer files at a network location. The ANN is now trained in offline mode and is ready to be used in an operational mode (e.g., online mode) to determine parameterization values ​​for new input cases using the classified parameter values ​​in the interval.

[0273] In method 200 for a new clinical case of an implant-based restoration, design preferences, user input of a 3D scan of the patient's jaw are collected (e.g., as a point cloud), and in step 202, implant positions and orientations are calculated based on the 3D scan data of the feature position object. In step 204, point samples on the 3D surface of the jaw scan geometry are extracted in the same format as described above for training. In step 206, the input data collected in steps 1 and 2 is sent to a computing device such as a server, which processes the request and returns a set of predicted design parameters using the trained ANN model. In step 208, the restoration element of the 3D model (e.g., an abutment or crown) is reconstructed from the predicted design parameters and rendered for presentation to the user. A dental object can then be manufactured based on the shape of the abutment or crown.

[0274] Figure 9 An example client-server system 50 is depicted in which trained restoration design models are provided upon end-user request directly from a cloud-based server 52. System testing based on the presented setup has demonstrated response times from the cloud-based machine learning server 52 to the end-user as fast as 5 seconds for a complete restoration design including an abutment or crown on one implant site.

[0275] refer to Figure 9 The trained model 54 has been trained using a large number of raw optical 3D scans of past clinical cases received from patients and is stored as output of corresponding designs of abutments and crowns on the server 52. The server 52 may include a TensorFlow TMModel server 55 and web server 56. The design is preferably defined by model parameters and can be generated by a human prosthetic design technician and stored in a database. During training, the ANN 58 will learn implicit patterns and / or rules that associate all input data with corresponding output data in an end-to-end manner. Since the model has now been trained and created, the neural network 58 is in an operational state or online mode and can immediately use the learned patterns when new inputs 62 and any auxiliary data are given to the system. The ANN 58 can be used to quickly generate output design predictions from new inputs 62 and 64 in an end-to-end online mode. This can be transmitted to the user via the web server 56. In addition to instant prediction time, other benefits of embodiments of the present invention may include one or more or all of the following:

[0276] 1) No need to write explicit, hand-crafted, complex algorithms to combine rules and patterns between inputs and outputs, and

[0277] 2) The output design will not be sensitive to the quality of any intermediate testing process or other sub-components.

[0278] Experiments and results

[0279] exist Figure 10 In

[15] , experimental results comparing the accuracy of rotation parameter estimation using a machine learning system according to an embodiment of the present invention are presented. The results show that the accuracy achieved by the machine learning system and method according to an embodiment of the present invention is significantly better than that achieved by current systems based on multidimensional numerical optimization methods, and is similar to the accuracy achieved by human designers.

[0280] After training with 17,632 samples of tooth #5, the accuracy within x degrees of rotation parameters was measured for a validation set of 776 and compared with other methods. First column: Accuracy within x degrees; 0 = perfect match. Second column: Accuracy obtained using the method according to the present invention (based on 3D CNN) and the number of cases within a given accuracy. Third column: Accuracy obtained using our current method based on nonlinear multidimensional numerical optimization. Fourth column: Accuracy measured on 25 different cases, each designed by 12 different human designers (300 samples in total).

[0281] Figure 11 Accuracy results for some selected abutment parameters with a range of 0.5 mm and 1.0 mm are shown compared to the corresponding parameter values ​​designed by humans. Figure 12 and Figure 13 Some graphical results are shown for an entire abutment or crown design, respectively, created by an ML system according to an embodiment of the present invention compared to a human design.

[0282] Figure 11 The measurement accuracy of selected abutment shape parameters within 0.5 mm (first column) and 1.0 mm (second column) respectively is shown.The results are based on -70,000 training samples and -7000 validation samples of the upper left posterior teeth (teeth #2, 3, 4, 5).

[0283] Figure 12 Shown in the top row: abutment designs created by a human designer. The bottom row shows abutment designs predicted by a machine learning model according to an embodiment of the present invention.

[0284] Figure 13 The top row shows crown designs created by a human designer. The bottom row shows crown designs predicted by a machine learning model according to an embodiment of the present invention.

[0285] Design Verification

[0286] In addition to the comparison of the numerical accuracy of each design parameter, the perceived design quality was also compared with the participation of multiple professional human prosthetic designers. Figure 14 A comparison of the acceptance rates of abutment designs between an ML-based method according to an embodiment of the present invention and a current optimization method is shown. Five different expert human designers examined 100 cases created by the two different methods without knowing how each design was created. Figure 14 It is shown that the acceptance rate of abutment designs created by an ML system according to an embodiment of the present invention is much higher than the acceptance rate of abutment designs created by current optimization systems based on a blind test on the same set of 100 cases.

[0287] Application to automatic tooth number estimation

[0288] In a previous statement, we mentioned that tooth count is one of the most important auxiliary inputs, in addition to the 3D input scan and implant location and orientation. However, in separate experiments, we have demonstrated that the tooth count itself can be very reliably estimated using only the 3D input scan and implant location and orientation using the same problem formulation proposed in this invention. Experimental results show that we can estimate the exact tooth count with approximately 91% accuracy and estimate the ±1 tooth difference with 99% accuracy. This result suggests the possibility of a potential subsystem in which we can first automatically estimate the tooth count for the user based on the raw 3D input scan data and then provide a complete design estimate.

[0289] Implementation Method

[0290] The method according to the present invention can be performed by or as a stand-alone device or a processor or processing unit embedded in a subsystem or other device. The present invention can use a processing engine suitable for executing a function. The processing engine preferably has a digital processing capability such as provided by one or more microprocessors, FPGAs or central processing units (CPUs) and / or graphics processing units (GPUs), and it is suitable for performing the corresponding function by programming with software (i.e., one or more computer programs). Reference to software can encompass any type of program in any language, which can be directly or indirectly executable by a processor via a compiled or interpreted language. The embodiment of any method of the present invention can be performed by logic circuits, electronic hardware, processors or circuit systems, and circuit systems can encompass any type of logic or analog circuit systems, integrated to any degree, and are not limited to general-purpose processors, digital signal processors, ASICs, FPGAs, discrete components or transistor logic gates, etc.

[0291] The processing component or processor may have memory (such as non-transitory computer readable media, RAM and / or ROM), an operating system, optionally a display (such as a fixed format display), ports for data entry devices (such as a keyboard), a pointing device (such as a "mouse"), serial or parallel ports for communicating with other devices, a network card and connections to any network.

[0292] The software may be embodied in a computer program product adapted to perform the functions of any of the methods of the invention, for example, as described in detail below, when the software is loaded into a memory and executed on one or more processing engines such as a microprocessor, ASIC, FPGA, etc. Thus, the processing means or processor used with any embodiment of the invention may be combined with a computer system capable of running one or more computer applications in the form of computer software.

[0293] The method described above with respect to the embodiment of the present invention can be implemented by one or more computer applications running on a computer system by being loaded into a memory and executed in an operating system (such as Windows provided by Microsoft Corporation). TM, United States, Linux, Android, etc.) to execute. The computer system may include a main memory, preferably a random access memory (RAM), and may also include a non-transitory hard drive and / or a removable non-transitory memory, and / or a non-transitory solid-state memory. The non-transitory removable memory may be an optical disc, such as a compact disc (CD-ROM or DVD-ROM), a magnetic tape, which is read and written by a suitable reader. The removable non-transitory memory may be a computer-readable medium having computer software and / or data stored therein. Non-volatile storage memory can be used to store persistent information that should not be lost when the computer system is powered off. Applications can use and store information in non-volatile memory.

[0294] The software embodied in the computer program product is adapted to perform the following functions when the software is loaded onto one or more corresponding devices and executed on one or more processing engines (such as microprocessors, ASICs, FPGAs, etc.):

[0295] performing a computer-implemented method for training a machine learning system, the machine learning system being installed on one or more computing devices,

[0296] The machine learning system is trained in an end-to-end manner using a plurality of pre-existing treatment 3D datasets, the 3D datasets including 3D images of a patient's dentition as input on one end of the machine learning system and 3D shapes of representations of the patient's restorative dental objects as output on the other end.

[0297] inputting a 3D scanned representation of at least a portion of the patient's dentition into a trained machine learning system (e.g., via an I / O port or interface), the 3D scanned representation defining at least one implant location, the machine learning system being installed on one or more computing devices, and

[0298] A representation of the 3D shape of a restorative dental object for an implant is recognized in an end-to-end manner using a trained machine learning system.

[0299] The software embodied in the computer program product is adapted to perform the following functions when the software is loaded onto one or more corresponding devices and executed on one or more processing engines (such as microprocessors, ASICs, FPGAs, etc.):

[0300] The machine learning system is trained in an end-to-end manner by one or more computing devices using multiple pre-existing therapeutic 3D datasets,

[0301] receiving, by one or more computing devices, 3D scan data of a patient representing at least a portion of the patient's dentition defining a location for at least one implant, and

[0302] A representation of a 3D shape of a restorative dental object for an implant is recognized using a trained machine learning device.

[0303] Identification can be performed in an end-to-end manner,

[0304] The software embodied in the computer program product is adapted to perform the following functions when the software is loaded onto one or more corresponding devices and executed on one or more processing engines (such as microprocessors, ASICs, FPGAs, etc.):

[0305] Applying end-to-end computer-based machine learning models,

[0306] Optionally, the machine learning system is a neural network,

[0307] Thus the neural network can be a CNN,

[0308] Receiving includes receiving a scanned 3D image of the patient's mouth or a 3D image of the patient's mouth retrieved from an archive.

[0309] The software embodied in the computer program product is adapted to perform the following functions when the software is loaded onto one or more corresponding devices and executed on one or more processing engines (such as microprocessors, ASICs, FPGAs, etc.):

[0310] Generate 3D shapes of restorative dental objects,

[0311] The restorative dental object may be intended for attachment to an implant directly or via one or more intermediates.

[0312] The 3D shape is not a free-form 3D shape, but a parametric model defined by a set of parameters.

[0313] A first range of values ​​for each parameter is divided among a set of intervals having smaller second ranges, and the machine learning device is adapted to estimate the correct interval to which a particular parameter belongs.

[0314] Thus, parameters may characterize the tooth surface anatomy, the tooth dentition, or the type of restoration.

[0315] The software embodied in the computer program product is adapted to perform the following functions when the software is loaded onto one or more corresponding devices and executed on one or more processing engines (such as microprocessors, ASICs, FPGAs, etc.):

[0316] outputting from the trained machine learning system a representation of the shape expressed as a set of parameters defining an abutment or crown,

[0317] Thus parameter sets are part of parametric modeling.

[0318] Adapting machine learning systems to use discriminative ML algorithms,

[0319] Use the patient's dentition, including the maxillary and / or mandibular, prepared and opposing jaws, missing teeth, implants, and number of teeth.

[0320] The method is performed with a restorative dental object serving as an abutment for an implant or a crown,

[0321] Point samples suitable for input to an artificial neural network (ANN) were extracted on the 3D surface of the jaw scan geometry for each clinical case.

[0322] The software embodied in the computer program product is adapted to perform the following functions when the software is loaded onto one or more corresponding devices and executed on one or more processing engines (such as microprocessors, ASICs, FPGAs, etc.):

[0323] A parameterized model is trained using an ANN in an end-to-end manner by mapping input data to restoration dental object design parameters, which are the output of the ANN for all pre-existing treatment 3D datasets,

[0324] Save the trained parameter model as a collection of computer files in a network location,

[0325] The parameterization values ​​classified in the interval are used to determine the parameterization values ​​for the new clinical case using the 3D image as input.

[0326] The software embodied in the computer program product is adapted to perform the following functions when the software is loaded onto one or more corresponding devices and executed on one or more processing engines (such as microprocessors, ASICs, FPGAs, etc.):

[0327] Using a new clinical case of an implant-based restoration, the method includes receiving input of implant position and orientation, user design preferences, and a 3D scan of a patient's jaw from 3D scan data of a feature location object,

[0328] Extract samples on the 3D surface of the jaw scan geometry in the same format as used for training,

[0329] The received input data is sent to one or more computing devices, which process the request using the trained ANN model and return a set of predicted design parameters,

[0330] The dental restoration object of the 3D model is reconstructed according to the predicted design parameters and rendered to be presented to the user.

[0331] Any of the above software may be implemented as a computer program product that has been compiled for a processing engine in any server or node of the network. The computer program product may be stored on a non-transitory signal storage medium such as an optical disc (CD-ROM or DVD-ROM), a digital tape, a magnetic disk, a solid-state memory (such as a USB flash drive, ROM, etc.).

[0332] For purposes of illustration, specific examples of systems, methods, and apparatus have been described herein. These are merely examples. The techniques provided herein can be applied to systems other than the example systems described above. In the practice of the present invention, many changes, modifications, additions, omissions, and substitutions are possible. The present invention includes variations of the described embodiments that are obvious to those skilled in the art, including variations obtained by: replacing features, elements, and / or actions with equivalent features, elements, and / or actions; mixing and matching features, elements, and / or actions from different embodiments; combining features, elements, and / or actions from the embodiments described herein with features, elements, and / or actions of other technologies; and / or omitting combined features, elements, and / or actions from the described embodiments. It is therefore intended that the following appended claims and the claims introduced below be interpreted as including all such modifications, substitutions, additions, omissions, and sub-combinations that can be reasonably inferred. The scope of the claims should not be limited by the preferred embodiments set forth in the examples, but should be given the broadest interpretation consistent with the entire description.

[0333] Although the present invention has been described above with reference to specific embodiments, this is done to illustrate rather than limit the present invention. It will be appreciated by those skilled in the art that various modifications and different combinations of the disclosed features are possible without departing from the scope of the present invention.

Claims

1. A computer-implemented method for providing a patient with a representation of a 3D shape of a restorative dental object, the method comprising: training a machine learning system in an end-to-end manner using a plurality of pre-existing treated 3D datasets by one or more computing devices, wherein the plurality of pre-existing treated 3D datasets include a representation of a 3D scan of a patient's dentition as an input to the machine learning system and a representation of a 3D shape of a restorative dental object for the patient as an output of the machine learning system, wherein the machine learning system includes a neural network, inputting a 3D scanned representation of at least a portion of the patient's dentition into a trained machine learning system, the 3D scanned representation defining a location, orientation, and type of an implant, the trained machine learning system being installed on one or more computing devices, and identifying a representation of a 3D shape of a restorative dental object for an implant in an end-to-end manner using a trained machine learning system to generate a parameterized model of the representation of the 3D shape, wherein the parameterized model is defined by a parameter set, and wherein the range of values ​​for each parameter in the parameter set is partitioned among a set of intervals, each interval having a finite range so as to provide a desired resolution, and the trained machine learning system is adapted to estimate the correct interval to which a particular parameter belongs.

2. The computer-implemented method of claim 1 , comprising applying an end-to-end computer-based machine learning model.

3. The computer-implemented method of claim 1 , further comprising generating a representation of the 3D shape of the restorative dental object.

4. The computer-implemented method of claim 1, wherein the parameter is related to characteristics of the tooth surface anatomy, the tooth dentition, or the restoration type.

5. The computer-implemented method of claim 4, wherein the output of the trained machine learning system is a representation of a 3D shape expressed as a set of parameters defining an abutment or crown. The computer-implemented method of claim 5 , wherein the parameter set is part of parametric modeling.

7. The computer-implemented method of claim 1 , wherein the trained machine learning system is adapted to use a discriminative machine learning algorithm.

8. The computer-implemented method of claim 1, wherein the restorative dental object is an abutment or a crown for the implant.

9. The computer-implemented method of claim 1, wherein point samples suitable for input to the neural network are extracted on a 3D surface of a jaw scan geometry.

10. The computer-implemented method of claim 1 , comprising receiving, for a new clinical case of an implant-based restoration, an input of implant position and orientation from 3D scan data of a feature position object, a 3D scan of a patient's jaw, and user design preferences.

11. The computer-implemented method of claim 1 , further comprising fabricating a customized restorative element made from a representation of the 3D shape of a restorative dental object, the restorative dental object being a customized crown or abutment for an implant.

12. A computer-implemented system for providing a patient with a 3D shape representation of a restorative dental object, the system comprising: means for training a machine learning system in an end-to-end manner using a plurality of pre-existing treatment 3D datasets by one or more computing devices, wherein the plurality of pre-existing treatment 3D datasets include representations of 3D scans of a patient's dentition as input to the machine learning system and representations of 3D shapes of restorative dental objects of the patient as output of the machine learning system, wherein the machine learning system includes a neural network, means for inputting a 3D scanned representation of at least a portion of a patient's dentition, the 3D scanned representation defining a location, orientation, and type of implant, the input being to a trained machine learning system installed on one or more computing devices, and components for recognizing a representation of a 3D shape of a restorative dental object for an implant in an end-to-end manner using a trained machine learning system to generate a parameterized model of the representation of the 3D shape, wherein the parameterized model is defined by a parameter set, and wherein the range of values ​​for each parameter in the parameter set is partitioned among a set of intervals, each interval having a finite range so as to provide a desired resolution, and the trained machine learning system is adapted to estimate the correct interval to which a particular parameter belongs.

13. The computer-implemented system of claim 12, comprising means for applying an end-to-end computer-based machine learning model.

14. A computer-implemented method for training a machine learning system, the machine learning system being installed on one or more computing devices, the method comprising training the machine learning system in an end-to-end manner with a plurality of pre-existing treatment 3D datasets, the plurality of pre-existing treatment 3D datasets comprising, as input to the machine learning system, representations of 3D scans of a patient's dentition and, as output of the machine learning system, representations of 3D shapes of restorative dental objects of the patient, wherein the machine learning system is a neural network, wherein the 3D shapes are generated by a parameterized model, the parameterized model being defined by a parameter set, and wherein the range of values ​​for each parameter in the parameter set is partitioned among a set of intervals, each interval having a finite range so as to provide a desired resolution, and the trained machine learning system is adapted to estimate the correct interval to which a particular parameter falls.

15. A computer-implemented system for training a machine learning system, the machine learning system being installed on one or more computing devices, the computer-implemented system comprising components for training the machine learning system in an end-to-end manner with a plurality of pre-existing treatment 3D datasets, the 3D datasets comprising a representation of a 3D scan of a patient's dentition as input to the machine learning system and a representation of a 3D shape of a restorative dental object for the patient as output of the machine learning system, wherein the machine learning system is a neural network, wherein the 3D shape is generated by a parameterized model, the parameterized model being defined by a parameter set, and wherein the range of each parameter value in the parameter set is partitioned among a set of intervals, each interval having a finite range so as to provide a desired resolution, and the trained machine learning system is adapted to estimate the correct interval to which a particular parameter falls.

Images