Automated tooth management in dental restoration workflows

Abstract

A method and system for automating the management and restoration generation process, including the steps of forming a spline along the jaw, suggesting potential interdental gaps, weighting the potential interdental gaps to obtain one or more delimiters, automatically suggesting tooth number probabilities, and calculating a best-fit tooth number distribution to generate a patient-specific restoration.

Description

Technical Field

[0001] Cross - reference to Related Applications This patent application claims the benefit and priority of U.S. Patent Application No. 17 / 477,755, filed on September 17, 2021, and is hereby incorporated by reference in its entirety for all purposes.

[0002] The present invention generally relates to methods, systems, and computer program products for automated tooth management in a dental workflow. More specifically, the present invention relates to methods, systems, and computer program products for generating dental restorations through an automated process that includes an automated management phase.

Background Art

[0003] Currently, there are technologies that propose dental restorations for dental practitioners. For example, in a restoration workflow, an intraoral camera is used to capture a 3D image of a patient's dentition during scanning. In the design phase of the workflow, a manual management process is performed where the number of teeth for one or more restorations is input and the positions of one or more preparation sites are specified on a 3D model. In the design phase, the scan is also analyzed to generate a restoration plan.

[0004] Following the design phase, a manufacturing phase is initiated, and the generated restoration plan is generated by subtractive manufacturing or additive manufacturing. For example, a physical copy of the restoration is made using a milling unit or a grinding unit. Finally, the restoration can be sintered and polished to impart final material properties such as hardness, strength, thermal conductivity, and aesthetics to the restoration.

Summary of the Invention

[0005] Exemplary embodiments provide methods, systems, and computer program products. One relates to a method comprising the step of forming a spline along a jaw using a spline forming module. The method further includes the step of proposing potential interdental gaps by performing one or more interdental gap detection steps, which include analyzing tooth cross-sections using a cross-section module, detecting interdental papillae using an interdental papilla detection module, and classifying interdental gaps using a classification module, using an interdental gap suggestion module. The potential interdental gaps are then weighted based on one or more interdental gap detection steps to obtain one or more delimiters. Furthermore, tooth count probabilities are automatically suggested for each of one or more possible dentitions between at least a pair of fixed delimiters using an alignment module, and a best-fit tooth count distribution is determined from one or more possible dentitions in response to the step of automatically suggesting, using at least the suggested tooth count probabilities. The alignment module is a machine learning engine.

[0006] In some embodiments, patient-specific restorations are generated based on a calculated best-fit tooth distribution.

[0007] In some embodiments, all interdental gap detection steps are performed and run in parallel.

[0008] Another aspect relates to calculating the best-fit tooth distribution using a global optimization module that accelerates the calculation of the best-fit tooth distribution through dynamic programming. The global optimization module takes as input one or more factors selected from a list consisting of maxillary and mandibular match, tooth size, weights from a weighting step, and the output of a machine learning engine. The best-fit tooth distribution can further be calculated by iterating over all possible dentitions around at least a pair of fixed delimiters using an iteration module.

[0009] In some embodiments, the best-fit tooth count distribution is calculated for the entire dental cavity, including the maxilla and mandible. Multiple delimiters are obtained, and multiple possible dentitions between the at least pair of fixed delimiters are determined for both the maxilla and mandible. Tooth count probabilities may be automatically proposed for the multiple possible dentitions, from which the best-fit tooth count distribution for the entire dental cavity is calculated. The best-fit tooth count distribution includes both the best-fit maxillary tooth count distribution and the best-fit mandibular tooth count distribution.

[0010] In some embodiments, the calculation step involves calculating the best-fit tooth distribution for one jaw, and then, based on the calculated best-fit tooth distribution, inferring another best-fit tooth distribution for the opposing jaw.

[0011] In some embodiments, during the calculation step, a best-fit tooth distribution is calculated for a portion of one jaw, for example, at least 3-4 teeth. Another best-fit tooth distribution can be inferred for the opposing portion of the jaw, or for the opposing portion of the opposing jaw, based on the calculated best-fit tooth distribution. The portion of one jaw may include 8 possible teeth in the first quadrant of the jaw.

[0012] In some embodiments, the forming step is automated.

[0013] In some embodiments, one or more delimiters are potential and / or fixed delimiters based on their respective weights.

[0014] In some implementations, a preparation type probability of at least one tooth between the at least pair of fixed delimiters is proposed.

[0015] In some embodiments, at least one of the one or more delimiters indicates a space representing one or more missing teeth.

[0016] In some embodiments, the space between two delimiters that are separated by a threshold distance represents one or more possible teeth. In some embodiments, the space between two adjacent delimiters that are separated by less than a threshold distance represents an interdental gap.

[0017] In some embodiments, the automatically suggested step predicts the number of one or more possible teeth that will fit within the space.

[0018] In some embodiments, at least one of one or more delimiters indicates a boundary line between adjacent teeth.

[0019] In some embodiments, the tooth number probability is a percentage or a value between 0 and 1.

[0020] In some embodiments, the tooth number probability is a likelihood assessment or classification.

[0021] In some embodiments, the machine learning engine has models based on 3D surface-based neural networks such as convolutional neural networks (CNNs), point convolutional neural networks (PointCNNs), PointNet, and PointNet++. CNNs operate on images (or 2.5D arrays), while PointNet operates on 3D surfaces.

[0022] In another embodiment, the preprocessing step is performed by constructing an estimation model representing the current possible dentition for each of one or more possible dentitions. The constructed estimation model is provided as input to a machine learning model, and the number of teeth is obtained as output. The estimation model is represented as a 2D array (2.5 images) of height values ​​corresponding to the heights (relative to a plane parallel to the occlusal plane) of multiple surface points of the model. Alternatively, the constructed estimation model may be a 3D model. The machine learning model may be trained using a training dataset that includes an input training model and the corresponding output training number of teeth. The space within the input training model may be interpreted as corresponding to one or more missing teeth, each corresponding to one or more identical or different numbers of teeth.

[0023] In another embodiment, a computing system is disclosed. The computing system includes a processor and a memory that stores instructions that, when executed by the processor, configure the system to form splines along the jaw using a spline formation module. Potential interdental gaps are further proposed by using an interdental gap proposal module to perform one or more interdental gap detection steps, which include analyzing tooth cross-sections using a cross-section module, detecting interdental papillae using an interdental papilla detection module, and classifying interdental gaps using a classification module. The system weights the potential interdental gaps based on one or more interdental gap detection steps to obtain one or more delimiters. The system automatically proposes tooth count probabilities for each of one or more possible dentitions between at least a pair of fixed delimiters using an alignment module, and the system calculates a best-fit tooth count distribution from one or more possible dentitions in response to the automatically proposing step using at least the proposed tooth count probabilities. The alignment module is a machine learning engine. Patient-specific restorations may be generated based on the calculated best-fit tooth count distribution. The machine learning engine has a model based on a convolutional neural network (CNN) or a point convolutional neural network (PointCNN).

[0024] In yet another embodiment, a non-temporary computer-readable storage medium is disclosed. The non-temporary computer-readable storage medium includes instructions that, when executed by a computer, cause the computer to propose potential interdental spaces by performing one or more interdental space detection steps, including using a spline formation module to form splines along a jaw; using an interdental space proposal module to propose potential interdental spaces by performing one or more interdental space detection steps, including using a cross section module to analyze tooth cross sections; using an interdental papilla detection module to detect interdental papillae; and using a classification module to classify interdental spaces; weighting the potential interdental spaces based on one or more of the interdental space detection steps to obtain one or more delimiters; using an alignment module to automatically propose tooth count probabilities for each of one or more possible dentitions between at least a pair of fixed delimiters; and using at least the proposed tooth count probabilities to calculate a best-fit tooth count distribution from one or more possible dentitions. The alignment module is a machine learning engine.

[0025] These and other features and characteristics of the technology of the present invention, as well as the methods and functions of the relevant structural elements, and the economics of the assembly and manufacture of the parts, will become more apparent when considering the following description and the appended claims with reference to the accompanying drawings. All of these form part of this specification, and similar reference numbers indicate corresponding parts in various drawings. However, it should be clearly understood that the drawings are for illustrative and explanatory purposes only and are not intended as definitions of the limitations of the present invention. Where used herein and in the claims, singular forms ("a," "an," "the") include plural referents unless the context clearly indicates otherwise.

[0026] Certain novel features considered to be characteristics of the present invention are described in the appended claims. However, the present invention itself, as well as its preferred uses, further purposes and advantages, will be best understood by referring to the following detailed description of exemplary embodiments, in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0027] [Figure 1] A block diagram of a network of a data processing system in which an exemplary embodiment may be implemented. [Figure 2] A block diagram of a data processing system in which an exemplary embodiment may be implemented. [Figure 3] A block diagram of a restoration system for automatic tooth management according to an exemplary embodiment. [Figure 4] A diagram showing a method of automatic tooth management according to one or more exemplary embodiments. [Figure 5] A diagram showing a 3D model according to one or more exemplary embodiments. [Figure 6] A diagram showing a 3D model according to one or more exemplary embodiments. [Figure 7A] A diagram showing a 3D model according to one or more exemplary embodiments. [Figure 7B] A diagram showing a 2D representation of a jaw according to one or more exemplary embodiments. [Figure 8A] A diagram showing a 3D model according to one or more exemplary embodiments. [Figure 8B] A diagram showing a 2D representation of a jaw according to one or more exemplary embodiments. [Figure 8C] A diagram showing a 2D representation of a jaw according to one or more exemplary embodiments. [Figure 9A] A diagram showing a 2D representation of a jaw according to one or more exemplary embodiments. [Figure 9B] A diagram showing a 2D representation of a jaw according to one or more exemplary embodiments. [Figure 10A] A diagram showing a segmentation process according to one or more exemplary embodiments. [Figure 10B] A cross-sectional view of a jaw according to one or more exemplary embodiments. [Figure 11]This figure shows a 3D model according to one or more exemplary embodiments. [Figure 12] This figure shows a process according to one or more exemplary embodiments. [Figure 13] This figure shows an alignment module according to one or more exemplary embodiments. [Figure 14] This figure shows a 2D representation of a jaw according to one or more exemplary embodiments. [Figure 15] This figure shows a process according to one or more exemplary embodiments. [Figure 16] This figure shows a process according to one or more exemplary embodiments. [Figure 17] This figure shows a process according to one or more exemplary embodiments. [Figure 18] This figure shows a training architecture according to one or more exemplary embodiments. [Figure 19] This figure shows a process according to one or more exemplary embodiments. [Figure 20A] This figure shows a 3D model having a restoration according to one or more exemplary embodiments. [Figure 20B] This figure shows a 3D model having a restoration according to one or more exemplary embodiments. [Figure 20C] This figure shows a 3D model having a restoration according to one or more exemplary embodiments. [Figure 20D] This figure shows a 3D model having a restoration according to one or more exemplary embodiments. [Figure 21A] This figure shows a 3D model having a restoration according to one or more exemplary embodiments. [Figure 21B] This figure shows a 3D model having a restoration according to one or more exemplary embodiments. [Figure 21C] This figure shows a 3D model having a restoration according to one or more exemplary embodiments. [Figure 21D]This figure shows a 3D model having a restoration according to one or more exemplary embodiments. [Modes for carrying out the invention]

[0028] The exemplary embodiment recognizes that users of any computer-aided design (CAD) software for creating restorations unnecessarily deal with manual placement of restorations or selection of restoration types in two instances in the restoration workflow: firstly, immediately after the start of the design phase in the workflow, by defining the number and representation of teeth using a tooth drawing or tooth scheme (e.g., a two-dimensional (2D) tooth drawing); and then, after the creation of a three-dimensional (3D) model, by inputting or selecting the position of the restoration using a 3D jaw scan area (this is typically done by defining the preparation margins). The exemplary embodiment recognizes that this is not only time-consuming but also error-prone, especially for novice dental professionals. The exemplary embodiment recognizes that this is a particularly difficult problem to solve and is not known to be solved by currently available solutions. The exemplary embodiment recognizes that conventional software requires the identification of the number of prepared teeth in the 3D model, as well as the location of the prepared area. Furthermore, user modifications to these inputs must also be made. This requires considerable user interaction and therefore increases the chances of errors. It is necessary to automate these and other stages of the restoration process to improve the accuracy of the proposed restorations. There is a need for fully automated recognition and calculation that enables restoration design without user interaction after the jaw is scanned or the jaw model is imported, so that prepared areas on the tooth model can be automatically recognized and classified to assist in the generation of tooth proposals. Furthermore, there is a need for user-friendly tools that allow users to intuitively adjust cavities directly on the 3D model using simple drag-and-drop operations and provide correction tools to correct inaccurate restoration positioning.The exemplary embodiments used to illustrate the present invention generally address and resolve the aforementioned problems and other related problems by automating the tooth management stage, such as the manual management process for all teeth in the restorative workflow, and thus automating the whole or a large part of the restorative workflow, excluding, for example, the tooth scanning process.

[0029] One embodiment automatically generates a restoration without requiring prior manual input of the restoration's appearance, number of teeth, or tooth position onto the model. The embodiment displays the generated restoration on the 3D model. This embodiment provides tools for fine-tuning the cavity area, which may not be perfectly accurate. This embodiment provides tools for adjusting the restoration's position as needed.

[0030] Exemplary embodiments are described, merely as examples, with respect to specific types of data, functions, algorithms, equations, model configurations, locations of embodiments, additional data, devices, data processing systems, environments, components, and applications. Any particular embodiment of these and other similar artifacts is not intended to limit the invention. Any suitable embodiment of these and other similar artifacts may be selected within the scope of the exemplary embodiments.

[0031] Furthermore, exemplary embodiments may be implemented with respect to any type of data, data source, or access to a data source via a data network. Within the scope of the invention, any type of data storage device can provide data to embodiments of the invention locally in a data processing system or via a data network. If one embodiment is described using a mobile device, within the scope of the exemplary embodiment, any type of data storage device suitable for use with a mobile device can provide data to such embodiment locally in the mobile device or via a data network.

[0032] The exemplary embodiments are described using specific code, designs, architectures, protocols, layouts, schematics, and tools merely as examples, and are not limiting to these exemplary embodiments. Furthermore, the exemplary embodiments are described in several cases using specific software, tools, and data processing environments merely as examples to clarify the description. The exemplary embodiments may be used in conjunction with other equivalent or similar structures, systems, applications, or architectures. For example, other equivalent devices, structures, systems, applications, or architectures therefor may be used in conjunction with such embodiments of the present invention within the scope of the present invention. The exemplary embodiments may be implemented in hardware, software, or a combination thereof.

[0033] The examples provided in this disclosure are for illustrative purposes only and are not limited to these exemplary embodiments. Additional data, behaviors, actions, tasks, activities, and operations are conceived from this disclosure and are contemplated within the scope of the exemplary embodiments.

[0034] Any advantages listed herein are merely examples and are not intended to limit you to exemplary embodiments. Additional or different advantages may be realized by specific exemplary embodiments. Furthermore, specific exemplary embodiments may have some or all of the advantages listed above, or they may not.

[0035] Referring particularly to Figures 1 and 2, these drawings are illustrative diagrams of a data processing environment in which exemplary embodiments may be implemented. Figures 1 and 2 are merely examples and are not intended to claim or imply any limitations regarding environments in which different embodiments may be implemented. Many modifications can be made to the described environment based on the following description of the particular embodiment.

[0036] Figure 1 is a block diagram of a network of a data processing system in which an exemplary embodiment may be implemented. The data processing environment 100 is a network of computers in which an exemplary embodiment may be implemented. The data processing environment 100 includes a network / communication infrastructure 102. The network / communication infrastructure 102 is a medium used to provide communication links between various devices, databases, and computers connected to each other within the data processing environment 100. The network / communication infrastructure 102 may include connections such as wired, wireless, or fiber optic cables.

[0037] The terms "client" or "server" are merely examples of the roles of specific data processing systems connected to the network / communication infrastructure 102, and are not intended to exclude other configurations or roles of these data processing systems. Servers 104 and 106 are coupled to the network / communication infrastructure 102 along with the storage unit 108. Software applications may run on any computer within the data processing environment 100. Clients 110, 112, and 114 are also coupled to the network / communication infrastructure 102. Client 110 may be a dental acquisition unit with a display. Data processing systems such as Server 104 or Server 106, or clients (Clients 110, 112, and 114), may contain data and may have software applications or software tools that run on it.

[0038] As merely an example and not intended to imply any limitation to such architectures, Figure 1 shows specific components that can be used in an exemplary implementation of one embodiment. For example, the servers and clients are merely examples and not intended to imply any limitation to a client / server architecture. In another embodiment, one embodiment may be distributed across several data processing systems and data networks, as illustrated, while another embodiment may be implemented on a single data processing system within the scope of the exemplary embodiment. The data processing systems (server 104, server 106, client 110, client 112, client 114) also represent example nodes in clusters, partitions, and other configurations suitable for implementing the embodiment.

[0039] The dental scanner 122 includes one or more sensors that measure teeth by acquiring multiple images through projections that map the person's oral cavity. In one embodiment, the dental scanner 122 acquires data points at a frequency of hundreds or thousands of times per second and automatically registers the size and shape of each tooth. The dental scanner 122 continuously transmits this data to software on a connected computer, which constructs a 3D impression of the patient's oral cavity from this data.

[0040] The most widely used digital format is the STL (Standard Tessellation Language) format, but other formats are also available. This format describes a series of triangulated surfaces, where each triangle is defined by three points and one slope. STL files can describe only the surface geometry of a three-dimensional object, without representations of color, texture, or other CAD model attributes. However, other file formats (such as the Polygon File format and PLY files) have been developed to record the color, transparency, or texture of dental tissue. Regardless of the type of imaging technique used, a scanner or camera projects light, which is then recorded as individual images and compiled by software after recognition of POIs (Points of Interest). For example, two coordinates (x and y) of each point are evaluated on the image, and then a third coordinate (z) is calculated based on the distance from the scanner.

[0041] Client application 120 or any other application 116 implements one embodiment described herein. Client application 120 can use data from dental scanner 122 to generate or render a 3D model using a single frame image captured by dental scanner 122. Client application 120 can also retrieve data from storage unit 108 for rendering or characterization. Client application 120 can also run on any data processing system (server 104 or server 106, client 110, client 112, client 114), such as client application 116 in server 104, and does not need to run on the same system as client 110.

[0042] Servers 104, 106, storage unit 108, clients 110, 112, and 114 may be connected to the network / communication infrastructure 102 using wired connections, wireless communication protocols, or other suitable data connectivity. Clients 110, 112, and 114 may be, for example, personal computers or network computers.

[0043] In the illustrated embodiment, server 104 may provide data such as boot files, operating system images, and applications to clients 110, client 112, and client 114. Clients 110, client 112, and client 114 may be clients to server 104 in this embodiment. Clients 110, client 112, and client 114, or any combination thereof, may include their own data, boot files, operating system images, and applications. The data processing environment 100 may include additional servers, clients, and other devices not shown. Server 104 may include application 116 which may be configured to implement one or more of the functions described herein for displaying a live control view according to one or more embodiments.

[0044] Server 106 may include a search engine configured to retrieve or retrieve stored files, such as patient images and 3D models for dental treatment, in response to requests from an operator, as described herein with respect to various embodiments.

[0045] In the illustrated embodiment, the data processing environment 100 may be the Internet. The network / communication infrastructure 102 may represent a collection of networks and gateways that communicate with each other using the Transmission Control Protocol / Internet Protocol (TCP / IP) and other protocols. At the heart of the Internet is a backbone of data communication links between major nodes or host computers, including thousands of commercial, government, educational, and other computer systems, routing data and messages. Naturally, the data processing environment 100 may be implemented as several different types of networks, such as an intranet, a local area network (LAN), or a wide area network (WAN). Figure 1 is intended as an example and is not intended as an architectural limitation to different exemplary embodiments.

[0046] Among other uses, the data processing environment 100 can be used to implement a client / server environment in which exemplary embodiments can be implemented. The client / server environment enables the distribution of software applications and data over a network so that the application functions by utilizing interactivity between a client data processing system and a server data processing system. The data processing environment 100 can also use a service-oriented architecture that allows interoperable software components distributed over a network to be packaged together as a coherent business application. Furthermore, the data processing environment 100 can use a cloud computing model of service delivery to enable convenient on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that take the form of a cloud and can be rapidly provisioned and released with minimal management effort or interaction with service providers.

[0047] Referring to Figure 2, this figure is a block diagram of a data processing system in which an exemplary embodiment may be implemented. The data processing system 200 is an example of a computer, such as client 110, client 112, client 114, or server 104, server 106 in Figure 1, or another type of device in which computer-readable program code or instructions that implement the process for the exemplary embodiment may be located.

[0048] The data processing system 200 is described as a computer, but is not limited to this. Embodiments in the form of other devices shown in Figure 1 can be modified by adding a touch interface, for example, without departing from the overview of operation and function of the data processing system 200 described herein, and can even be made to remove certain illustrated components from the data processing system 200.

[0049] In the illustrated embodiment, the data processing system 200 employs a hub architecture including a North Bridge and Memory Controller Hub (NB / MCH) 202 and a South Bridge and Input / Output (I / O) Controller Hub (SB / ICH) 204. The processing unit 206, main memory 208, and graphics processor 210 are coupled to the North Bridge and Memory Controller Hub (NB / MCH) 202. The processing unit 206 may include one or more processors and may be implemented using one or more heterogeneous processor systems. The processing unit 206 may be a multicore processor. In certain implementations, the graphics processor 210 may be coupled to the North Bridge and Memory Controller Hub (NB / MCH) 202 via an Accelerated Graphics Port (AGP).

[0050] In the illustrated embodiment, a local area network (LAN) adapter 212 is coupled to the southbridge and input / output (I / O) controller hub (SB / ICH) 204. An audio adapter 216, a keyboard and mouse adapter 220, a modem 222, read-only memory (ROM) 224, a Universal Serial Bus (USB) and other ports 232, and PCI / PCIe devices 234 are coupled to the southbridge and input / output (I / O) controller hub (SB / ICH) 204 via bus 218. Hard disk drives (HDDs) or solid-state drives (SSDs) 226a and CD-ROMs 230 are coupled to the southbridge and input / output (I / O) controller hub (SB / ICH) 204 via bus 228. PCI / PCIe devices 234 may include, for example, Ethernet adapters, add-in cards, and PC cards for notebook computers. PCI uses a card bus controller, while PCIe does not. Read-only memory (ROM) 224 may be, for example, a flash binary input / output system (BIOS). Hard disk drives (HDDs) or solid-state drives (SSDs) 226a and CD-ROMs 230 may use, for example, integrated drive electronics (IDE), serial advanced technology attachment (SATA) interfaces, or variations such as external SATA (eSATA) and microSATA (mSATA). Super I / O (SIO) devices 236 may be coupled to the southbridge and input / output (I / O) controller hub (SB / ICH) 204 via bus 218.

[0051] Some examples of computer-usable storage devices include main memory 208, read-only memory (ROM) 224, or flash memory (not shown). Some examples of computer-usable storage devices, including hard disk drives (HDDs) or solid-state drives (SSDs) 226a, CD-ROMs 230, and other similarly usable devices, include computer-usable storage media.

[0052] The operating system runs on the processing unit 206. The operating system coordinates and controls the various components within the data processing system 200 shown in Figure 2. The operating system can be a commercially available operating system for any type of computing platform, including but not limited to server systems, personal computers, and mobile devices. Object-oriented or other types of programming systems can work with the operating system and make calls to the operating system from programs or applications running on the data processing system 200.

[0053] Instructions for an operating system, an object-oriented programming system, and applications or programs such as application 116 and client application 120 in Figure 1 may be placed on a storage device, such as in the form of code 226b on a hard disk drive (HDD) or solid-state drive (SSD) 226a, and loaded into at least one of one or more memories, such as main memory 208, for execution by the processing unit 206. The processes of an exemplary embodiment may be executed by the processing unit 206 using computer implementation instructions that may be placed in memory, for example, main memory 208, read-only memory (ROM) 224, or one or more peripheral devices.

[0054] Furthermore, code 226b may also be downloaded from a remote system 214b via network 214a, and a similar code 214c is stored in storage device 214d, and code 226b may also be downloaded to the remote system 214b via network 214a, and the downloaded code 214c is stored in storage device 214d.

[0055] The hardware in Figures 1 and 2 may vary depending on the embodiment. Other internal hardware devices or peripheral devices, such as flash memory, equivalent non-volatile memory, or optical disc drives, may be used in addition to or instead of the hardware shown in Figures 1 and 2. Furthermore, the processes of the exemplary embodiments may be applied to multiprocessor data processing systems.

[0056] In some exemplary embodiments, the data processing system 200 may generally be a personal digital assistant (PDA) comprising flash memory providing non-volatile memory for storing operating system files and / or user-generated data. The bus system may include one or more buses, such as a system bus, an I / O bus, and a PCI bus. Naturally, the bus system may be implemented using any type of communication fabric or architecture that transfers data between different components or devices attached to the communication fabric or architecture.

[0057] The communication unit may include one or more devices used to send and receive data, such as a modem or network adapter. Memory may be, for example, main memory 208, or a cache, such as the cache found in the northbridge and memory controller hub (NB / MCH) 202. The processing unit may include one or more processors or CPUs.

[0058] The embodiments shown in Figures 1 and 2 and described above are not intended to imply any architectural limitations. For example, the data processing system 200 may further take the form of a tablet computer, laptop computer, or telephone device, in addition to being in the form of a mobile or wearable device.

[0059] When a computer or data processing system is described as a virtual machine, virtual device, or virtual component, the virtual machine, virtual device, or virtual component operates like the data processing system 200, using virtual manifestations of some or all of the components represented in the data processing system 200. For example, in a virtual machine, virtual device, or virtual component, processing unit 206 is manifested as a virtual instance of all or some of the hardware processing units 206 available in the host data processing system; main memory 208 is manifested as a virtual instance of all or some of the main memory 208 that may be available in the host data processing system; and hard disk drive (HDD) or solid state drive (SSD) 226a is manifested as a virtual instance of all or some of the hard disk drive (HDD) or solid state drive (SSD) 226a that may be available in the host data processing system. In such cases, the host data processing system is represented as the data processing system 200.

[0060] Figure 3 shows a repair system 300 configured to automate the repair process according to an exemplary embodiment. In some embodiments, the repair system 300 may include one or more computing platforms 304. The computing platform 304 may be configured to communicate with one or more remote platforms 306 according to a client / server architecture, a peer-to-peer architecture, and / or other architecture. The remote platform 306 may be configured to communicate with other remote platforms via the computing platform 304 and / or according to a client / server architecture, a peer-to-peer architecture, and / or other architecture. Users can access the repair system 300 via a remote platform 104.

[0061] The computing platform 304 may consist of machine-readable instructions 308. The machine-readable instructions 308 may include one or more instruction modules. The instruction modules may include one or more of the following: a spline formation module 332, an interdental gap proposal module 310, a cross-section module 302, an interdental papilla detection module 312, a classification module 314, a weighting module 316, an alignment module 318, a preprocessing module 320, a training module 322, a best fit calculation module 324, a global optimization module 326, an iteration module 328, and / or other instruction modules. The remote platform 306 and external resources 330 may correspond to server and client applications of the data processing environment 100. In exemplary embodiments, the computing platform 304 may perform methods described herein and processes such as those shown in Figures 4, 15, 16, 19, 12, and 17. Compared to conventional workflows, the process can be performed in a way that eliminates the need to explicitly define the restoration, indicate the number of teeth, or indicate the preparation type; the restoration is directly proposed in the 3D scene on the display without user interaction; the jaw model does not need to be manually positioned; insertion axes do not need to be manually entered; preparation margins do not need to be initially entered; scan bodies (abutments) do not need to be marked; configuration axes (abutments) do not need to be entered; if necessary, cavities with restorations already in place can be intuitively adjusted; and if necessary, incorrect tooth positioning can be corrected by drag-and-drop in the 3D scene.

[0062] Next, with reference to Figure 4, the process 400 performed by the computing platform 304 in Figure 3 will be described. The process begins in step 402. In step 402, the process 400 uses the spline formation module 332 to form a spline along the jaw. In step 404, the process 400 uses the interdental gap suggestion module to suggest potential interdental gaps by performing one or more interdental gap detection steps, including (a) analyzing tooth cross-sections using the cross-section module 302, (b) detecting interdental papillae using the interdental papilla detection module 312, and (c) classifying tooth spacings using the classification module 314. In an exemplary embodiment, all three steps are performed simultaneously and in parallel. In another embodiment, any combination of steps is performed sequentially or in parallel. In step 406, the process 400 uses the weighting module 316 to weight the potential interdental gaps based on one or more interdental gap detection steps in order to obtain one or more delimiters. In exemplary embodiments, the user may scan at least three teeth, but if the scan is much shorter than required and includes only the transition between two teeth, only one delimiter may be acquired. Furthermore, it may be sufficient to scan at least three teeth on one jaw. In particular, the perforation template is in most cases composed only on the mandible or maxilla, and therefore only one side of the jaw may be scanned.

[0063] In step 408, process 400 automatically proposes tooth count probabilities for each of one or more possible tooth rows between fixed delimiters (e.g., adjacent delimiters) using the alignment module 318, as described below. In one embodiment, the alignment module 318 is a machine learning engine. In step 410, process 400 uses at least the proposed tooth count probabilities to calculate the best-fit tooth count distribution from one or more possible tooth rows using the best-fit calculation module 324.

[0064] In step 402, a spline curve 502, as shown in Figure 5, is fitted along the maxilla and mandible of a 3D model 504 obtained from a scan of the patient's dentition. The fitting results in a smooth line that can be used as a baseline for all further calculations. The fitting process may begin with a rough line fitted onto the tooth surface. The rough line is divided into two points, which are then rearranged on the jaw in a refinement process. This is done by statistical calculations that involve weighting the points on the jaw based on how important they are to the tooth (e.g., closer to the fissure / incisal edge). Generally, the higher the point, the more relevant they are. For statistical purposes, all vertices of the triangulation are used, with vertices located closer to the fissure / incisal edge having a higher weight. Alternatively, a solution is possible in which the analysis spline is fitted directly to the jaw by optimizing some of the control vertices of the spline. In this case, refinement is not necessary. The spline curve 502 can be fitted onto the upper fissures of the molars and onto the surfaces of the incisors, such as the incisal edges. Furthermore, the combined / averaged spline of the mandibular spline and maxillary spline is created as a generalized parameterized curve for the entire dentition. The spline curve 502 is used as a baseline for one or more calculations described herein.

[0065] In step 404, potential interdental gaps (spaces between teeth) are proposed using a number of interdental gap detection procedures. In an exemplary embodiment, three procedures are performed in parallel, but these procedures may be performed sequentially. This includes (a) analyzing tooth cross-sections using the cross-section module 302, (b) detecting interdental papillae using the interdental papilla detection module 312, and (c) classifying tooth spacings using the classification module 314.

[0066] The step of finding potential interdental gaps by analyzing tooth cross-sections may be performed as follows in an exemplary embodiment: The height 606 used for the cross-section is determined for all positions along the cross-section, as shown in Figure 6. Given a spline curve 502 and a transition curve 602 representing the transition between the tooth and the gingiva (ideally, the spline curve 502 and the transition curve 602 contain the same number of points), the cross-sectional curve 604 is obtained by determining the height for each pair of points on the spline curve 502 and the transition curve 602, which is a specified ratio between the pairs of points. In one example, a position on the cross-sectional curve 604 is one-third of the distance between the corresponding position on the transition curve 602 and the corresponding position on the spline curve 502. In another embodiment, the spline curve 502 is lowered to the center of the jaw or moved by a specified distance along a plane perpendicular to the occlusal plane, for example, so that the cross-sectional curve 604 appears to overlap the spline curve 502 when viewed from above / from the viewpoint of the occlusal plane 608. Naturally, the embodiments presented are not intended to be limiting, and other reference lines / curves and ratios that provide a consistent reference frame for the processes described herein may be obtained in light of the description.

[0067] The cross-sectional curve 604 remains at the same level throughout the preparation (the cross section is maintained at a fixed ratio between the spline curve 502 and the transition curve 602), and as a result, the preparation can be correctly distinguished from healthy teeth. More specifically, the cross-sectional curve 604 should function for marginal preparations, that is, analysis aided by the cross section can recognize all types of preparations, such as inlays, onlays, and partial crowns.

[0068] Next, the bilateral distance function is calculated. The calculation is shown in Figures 7A and 7B, where the cross-sectional curve 604 is sampled with a specified length resolution of, for example, about 100 samples (80 to 120 samples, etc.) per tooth 704. Vertical lines 702 perpendicular to the cross-sectional curve 604 are positioned to pass through each sample at the height of the cross-sectional curve 604. The direction of these lines is also within the occlusal plane 608. The vertical lines 702 intersect the 3D model 504, and the lingual intersection 706 and buccal / labial intersection 708 are determined each time, with potential intersections on the opposite side of the mandibular arch excluded in the exemplary embodiment. Figure 7B is a 2D (two-dimensional) diagram of the bilateral distance function calculation according to the exemplary embodiment, showing multiple vertical lines 702 passing through a single tooth 704.

[0069] By determining multiple lingual intersections 706 and buccal / labial intersections 708 for all teeth being examined and connecting them to form a contour line 710, the minimum value 804 (the minimum distance between a point on the contour line 710 on one side of the tooth and the corresponding point on the cross section curve 604) and the gap 802 (the region indicating the start or end point of the contour line 710) can be determined. The gap 802 and minimum value 804 are candidate interdental gaps, i.e., they represent potential interdental gaps / boundaries between teeth, as shown in Figures 8A, 8B, and 8C, which represent the first seven teeth of an exemplary jaw, with the remaining teeth omitted.

[0070] Figure 8A shows the buccal contour line 806 along with the corresponding gap 802 and minimum value 804. A common reference frame can be obtained so that the lingual contour line 808 on the other side of the tooth can be added by straightening the cross section curve 604 as shown in Figure 8B. By determining the gap 802 and minimum value 804 for both the buccal contour line 806 and the lingual contour line 808, the detection of potential interdental gaps is performed in a more stable manner compared to performing the process on only one side. This is based on the observation that the minimum value 804 and gap 802 from either side that are located close to each other with respect to the cross section curve 604 are statistically more likely to represent the actual interdental gap / actual boundary between teeth. Therefore, the gap 802 and minimum value 804 or their characteristics can be weighted in the weighting process by the weighting module 316, as described below.

[0071] In another interdental space detection process, interdental spaces are obtained by detecting interdental papillae. This can be achieved using an interdental papilla detection module 312. The process involves segmenting a scan of the patient's teeth into segmented gingiva 1002 and segmented teeth 1004, as shown in Figure 10A. Based on the gingival segmentation, the interdental papillae can be located. If the gingival segmentation is correct, the proximal region can also be located in the fully prepared tooth. This procedure is performed in three steps.

[0072] In the first step, the spatial curve along the boundary between the tooth and the gum is segmented. Then, two separate spatial curves along the buccal and lingual sides of the gum are determined.

[0073] In the second step, an angle function describing the path of the gingival boundary is determined along the alveolar ridge on both the buccal and lingual sides. As shown in Figure 10B, which shows a cross-section of the jaw from the crown toward the gingiva, an angle function along a reference line viewed from a position centered on the jaw 1010 is defined to achieve a combined evaluation of the occlusal margin height and the lingual-to-buccal width. The cross-section is perpendicular to the jaw midline / curve 1012 described herein. The jaw midline / curve 1012 extends in the z direction as shown in Figure 10B, and its shape corresponds to the shape of the spline curve 502 when viewed from above (towards the crown). For all sample positions zi along the jaw midline / curve 1012, the angle αi (in the cross-sectional plane xy) with respect to the lingual and buccal gingival curves is calculated and stored as the lingual and buccal angle functions. The calculation of the angles relative to the curves is made possible by a segmentation process that separates the teeth from the gingiva.

[0074] The third step is performed as follows: For two gingival boundary functions, the maximum value representing the papilla is determined. Specifically, multiple αi angles are calculated for the first tooth or preparation 1006, and another multiple αi angles are calculated for the adjacent tooth or preparation 1008. Thus, the potential interdental papilla, and consequently the potential interdental gap, between the first tooth or preparation 1006 and the adjacent tooth or preparation 1008 is determined by, or based on, the maximum αi angle or the area between adjacent maximum αi angles. Therefore, the above procedure has its maximum value at the gap where the interdental papilla and tooth are missing. Thus, the maximum value and gap are extracted and weighted by the weighting module 316 based on their shape characteristics, as described below, resulting in a fixed delimiter or a potential delimiter.

[0075] In yet another interdental gap detection process, the interdental gap may be determined based on the classification of tooth spacing / areas by the classification module 314. Given a sample along the spline curve 502, the teeth are segmented from the jaw using the Maximum Flow Minimum Cut Optimization process. In one embodiment where both the maxilla 1102 and mandible 1104 are analyzed, a modified spline curve 1106 calculated from averaging the maxillary spline curve 502 and the mandibular spline curve 502 may be used. The Maximum Flow Minimum Cut Optimization process includes a flow network in which the maximum amount of flow passing from the source or start position to the sink or end position is equal to the minimum total weight of the edges (minimum cut) that would cut the source from the sink if removed. For a tooth, Maximum Flow Minimum Cut Optimization is initiated by finding the start position on the tooth surface. The start point (source) is somewhere on the occlusal side of the tooth, and the sink is defined as all the vertices located a given distance from this start position and therefore somewhere on the gingiva or adjacent teeth. The flow edge weights are derived from the surface curvature and the strongest color transitions, resulting in a minimum cut along the edge at the gingival margin. In one embodiment, the starting position on the tooth surface can be found by projecting a line onto tooth 704 from a first position or first point on the modified spline curve 1106. This process estimates the presence of tissue / gingiva over several millimeters around the starting position of the tooth. Thus, the edges of the tooth surface are evaluated, and sharp edges / high curvature (regions with large gradient changes or large changes in the surface normal direction between adjacent points, representing the transition between the gingiva and the tooth surface) are identified. In the global optimization process, lines along the sharp edges are calculated. Lines representing the boundary between the tooth surface and the gingiva are calculated on the cervical side, and lines representing the separation between adjacent teeth on the distal and mesial sides, and regions within the lines are identified (e.g., colored) to indicate distinct teeth. This can be repeated for several starting positions based on the corresponding projections from the modified spline curve 1106. For each region, all points are projected onto the modified spline curve, but only the maximal points of the region relative to the spline are stored. These maximum values ​​represent the potential and / or fixed delimiters on a common baseline.In one embodiment, all vertices of a segmented tooth are projected onto a baseline. The maximum extension of these projections on the baseline is detected, and therefore the projections with the minimum and maximum parameter values ​​on the baseline are selected. These positions correspond to the start and end points of the tooth. Furthermore, some of the identified tooth regions terminate precisely at the gingival margin / boundary, while others do not. By discretizing the tooth region margins and performing regression analysis, latent delimiters and / or fixed delimiters can be derived.

[0076] Any combination of steps (e.g., all interdental space detection steps) may be performed in parallel, but it is assumed that any combination of steps may be performed sequentially. For example, in embodiments where real-time or high-speed automation of the management phase of the restoration workflow and subsequent generation of restoration plans are not required, any combination of steps may be performed sequentially.

[0077] Regarding the interdental gap detection step, the weighting process by the weighting module 316 weights potential interdental gaps based on one or more of the interdental gap detection steps to obtain one or more delimiters. For example, after determining gaps 802 and minimums 804, their features are characterized and weights are assigned based on one or more factors such as their geometric attributes. As shown in Figure 9A, the gaps and minimums represent fixed interdental gaps 902 or potential interdental gaps 904, where potential interdental gaps 904 have a lower confidence level (e.g., 20% to 60.99% confidence level) than fixed interdental gaps 902, which have a higher confidence level (e.g., 95% to 100% or 61% to 100% confidence level), for example, due to multiple lingual and buccal gaps or minimums coinciding. Fixed interdental gaps 902 and potential interdental gaps 904 are referred to herein as fixed delimiters and potential delimiters, and will be further detailed below.

[0078] For potential delimiter candidates from the interdental gap detection step of the cross-sectional process using the cross-sectional module 302, the gingival margin process using the interdental papilla detection module 312, and / or the interdental gap / region process using the classification module 314, weights are supported to evaluate the probability that a given delimiter candidate is in the exact location and therefore is an actual delimiter or interdental gap. These delimiter candidates are compared with each other, and an integrated delimiter is determined based on some heuristic or defined comparison algorithm configured to process the preparation. The integrated delimiter may represent, for example, a delimiter within a defined region, or a delimiter with a confidence level at least above a threshold confidence level (e.g., >60%). Thus, fixed delimiters can describe locations where various gaps or minimums coincide from lingual and / or buccal sources, while potential delimiters can describe locations with a certain uncertainty that can be stored as weights. Alternatively, fixed delimiters may have the highest weights, and potential delimiters may have relatively low weights. Naturally, these embodiments are not intended to be limiting to other embodiments, and processes and algorithms can be achieved in light of the description.

[0079] Referring to Figure 9B, one or more delimiters 906 are shown along a common baseline 918 in a 2D view of the combined delimiter arrangement 900. The one or more delimiters 906 are delimiters obtained from various interdental space detection steps and combined on the common baseline 918. The one or more delimiters may further have their respective associated weights. By using a baseline or curve in each interdental space detection step that can be easily located, such as a spline curve 502 or a curve or line having a defined spatial relationship with the spline curve 502 or other common curves, the delimiters 906 from each interdental space detection step can be aligned on the common baseline 918 in Figure 2B for further analysis. Naturally, the common baseline 918 or curve may itself be a spline curve 502 or a linear spline curve.

[0080] In step 408, tooth count probabilities are proposed for each of the one or more possible dentitions between one or more pairs of fixed delimiters using the alignment module 318 (i.e., tooth count probabilities are proposed for each possible combination of intervals between fixed delimiters). The alignment module 318 is a machine learning engine having a machine learning model trained to predict probabilities based on a large training dataset. More specifically, tooth count probabilities are automatically proposed using the alignment module 318 for each of the one or more possible dentitions between adjacent fixed delimiters. There may be several possible teeth between two fixed delimiters; for example, one or two teeth may fit in the space between two adjacent fixed delimiters. There may also be multiple adjacent pairs of fixed delimiters. Between each pair of fixed delimiters, there may also be a latent delimiter with associated uncertainty, resulting in two or more possible teeth / dentitions fitting between each pair of fixed delimiters, or only one possible tooth / dentition fitting between the pair of fixed delimiters. Each possible tooth that fits entirely or partially within the space between the two delimiters is a dentition. By looping or cycling through all possible dentitions that can fit entirely or partially within the space and repeating for all pairs of adjacent fixed delimiters, the alignment module predicts the probability that the tooth corresponds to one of the tooth counts in the cavity. In one embodiment, the alignment module 318 is a machine learning engine. By sequentially providing the alignment module with all possible dentitions within at least one pair of fixed delimiters, a set of tooth count probabilities for each possible dentition is predicted. This set represents all possible tooth types. For example, an adult cavity has 32 teeth, and the cavity has four similar quadrants, so there can be a total of eight possible tooth counts. Thus, in one embodiment, for each possible dentition, the alignment module predicts the probability of eight tooth count probabilities (P pta-t1 ,P pta-t2 ...P pta-t8 ) returns. Here, "P pta-tNThis represents the probability that the current possible dentition has N teeth. Naturally, this is not limiting, and any suitable embodiment of these and other similar artifacts may be selected within the scope of the exemplary embodiments. For example, different numbers of probabilities or different processing of input data may be obtained in light of the exemplary embodiments. The machine learning model process 1200 is described in Figure 12, and for each of one or more possible dentitions, the following steps are performed: In step 1202, a preprocessing step is performed by constructing an estimation model that represents the current possible dentition between two fixed delimiters. In step 1204, the constructed estimation model is provided as input to the machine learning model, which is a trained machine learning model. In step 1206, the number of teeth probability of the current possible dentition is obtained as output.

[0081] Referring to Figure 13, this figure shows a block diagram of an exemplary configuration for the proposed tooth count probability 1312 according to an exemplary embodiment. The alignment module 318 is a component of either the server application 116 or the client application 120 in Figure 1, depending on the particular embodiment. Briefly, a 2D array of height values ​​representing the geometric shape of the tooth row is input to the M / L model, and the M / L model determines, by ML probability, that each possible number of teeth fits this geometric shape. This classification is applied to all meaningful intervals or tooth rows between delimiters, independently of global optimization. Typically, there are many fixed delimiters, so there may not be many intervals representing potential delimiters that need to be analyzed, leaving mainly large intervals between pairs of fixed delimiters that do not have other delimiters (typically representing one possible tooth) and regions with ambiguous uncertain / potential delimiters (typically representing two or more possible teeth). Subsequently, for example, a dynamic programming approach that takes into account the probabilities calculated so far is used to analyze "multiple possible dentitions," where estimates for the minimum width, average width, and maximum width for each number of teeth are also considered.

[0082] More specifically, the alignment module 318 can sequentially obtain possible dentitions 1320 from a plurality of possible dentitions 1304 based on a combination delimiter arrangement 900. A 2D representation of the plurality of possible dentitions 1304 is shown in Figure 14, where one of the possible dentitions 1320 represents a possible arrangement of teeth around a position represented by the delimiter 906, more specifically, a possible arrangement of teeth in the space between two selected or adjacent fixed delimiters. Referring to Figure 14, Figure 14 represents the first seven possible teeth 1402 (numbered 2-8 in the ellipse of Figure 14) of an exemplary jaw, with the remaining possible teeth omitted for illustrative purposes only. Six possible dentition arrangements 1326 (numbered 1-6) are shown for illustrative purposes only. Any number of dentition arrangements 1326 required to consider all possible dentitions of the eight teeth in a quadrant, or even all 32 possible teeth in the patient's dentition around the delimiter 906, can be simultaneously analyzed by a machine learning model to automate the management phase and the restoration production process. The global optimization module can then calculate the optimal arrangement by dynamic programming to reduce the number of arrangements to consider. Returning to Figure 13, for each possible dentition 1320, a constructed estimation model 1318 (e.g., from a database of generalized models) is created using the preprocessing module 320 for use as input to the machine learning engine 1322. The constructed estimation model 1318 may represent a 2D view of the possible dentition 1320, or it may be a 2.5D estimation model constructed using it. The 2.5D model may be represented as a 2D array of height values ​​corresponding to the heights (relative to a plane parallel to the occlusal plane) of multiple surface points of the estimation model. However, this is not intended to be limiting, and other formats may be used in light of this specification, including, for example, vectors representing estimated teeth, pixels of a 2D image, a 2.5D image with depth and gray level information, and 3D points of a 3D image of an estimated / generalized tooth. In one exemplary embodiment, probabilities may be generated for each possible tooth 1402.Therefore, the exemplary embodiments are not intended to be limiting and are intended to include variations that can be obtained in light of the description herein, such as the output probability of the preparation type. Thus, for all possible dentitions of teeth between the latent delimiter and the fixed delimiter, the number of teeth or preparation type is determined with the help of a neural network. An example of a neural network for machine learning model 1306 is a CNN or PointCNN (Point Convolutional Neural Network), which is a generalization of a typical CNN (Convolutional Neural Network) for learning features from a point cloud.

[0083] In the case of a CNN, it is a feedforward artificial neural network consisting of convolutional layers in its classical form, followed by pooling layers. The CNN learns by learning the free parameters or classifiers and their weights for each layer's convolutional kernel when computing the next layer. The training of machine learning model 1306 according to an exemplary embodiment is described below.

[0084] In another embodiment, the feature extraction / selection component 1314 is configured to generate relevant features for a proposal based on data from all different available inputs (e.g., weights 1324). In this embodiment, the feature extraction / selection component 1314 receives a request that includes at least an identification of the required output type. Based on this output type, the feature extraction / selection component 1314 retrieves any combination of specific input data relevant to the required request or proposal. However, in most embodiments, feature extraction may be integrated into a deep neural network or machine learning model 1306, and feature selection may be performed outside the machine learning model, if any.

[0085] In one exemplary embodiment, the memory module 1308 stores the output of the machine learning model 1306, which is the probability or likelihood evaluation or classification of the number of teeth for each of one or more possible teeth 1402, or the probability or likelihood evaluation or classification of possible dentitions 1320.

[0086] In one exemplary embodiment, after generating an output evaluation, the alignment module 318 or the application 120 of the alignment module 318 may perform additional actions automatically or in response to a request.

[0087] Furthermore, the best fit calculation module 324 calculates the best fit tooth number distribution or arrangement according to a defined criterion (e.g., the most likely of the possible dentitions 1320) using at least the proposed tooth number probability 1312 or evaluation. To save time, the best fit tooth number distribution may be calculated in a global optimization process using a global optimization module 326 that accelerates the calculation of the best fit tooth number distribution by dynamic programming, as shown in Figure 15. This ensures that the optimal solution is reached without iterating through all possibilities. Thus, the most globally likely sequence of teeth around / between interdental spaces is calculated by dynamic programming, which includes using a recursive approach by memorizing the trends of subproblem solutions. This typically reduces the time complexity from exponential to polynomial. In this step, in addition to the availability of classification and average tooth width by a neural network, the probability distribution of occlusal opposing teeth may also be taken into consideration when determining the best fit tooth number distribution. As a result of global optimization, the jaw is segmented into 1404 segments for each number of teeth, and these segments can be used in further calculations.

[0088] Alternatively, the best-fit tooth number distribution can be calculated iteratively, though slower than global optimization, by iterating over the probabilities of all possible tooth arrangements 1320 (Figure 16) for all possible tooth arrangements 1326.

[0089] In addition to detecting possible teeth, preparations within the 3D model 504 can also be detected. Herein, a 3D surface-based neural network similar to the machine learning model 1306 automatically detects, for each tooth interval 1404, whether a tooth is present and whether this tooth is unprepared or prepared. Using a similar machine learning algorithm, the type of preparation can also be determined for each tooth interval 1404 classified as preparation.

[0090] Furthermore, the scan bodies and their implant axes can be pre-detected by separate algorithms, such as a curve detection algorithm, and assigned to a tooth spacing of 1404. Different types of scan bodies are defined by how the individual basic shapes that make up the scan body are arranged relative to each other. First, similarly curved regions are detected for the entire geometric shape of the 3D model, and then matching geometric primitives are inserted into those regions.

[0091] Furthermore, if a preparation is found very close to a toothless space (1404), the most likely combination of pontic and crown for the bridge can be automatically suggested using a well-known algorithm.

[0092] Furthermore, for each preparation found, the preparation boundary can be automatically calculated based on a well-known algorithm. After automatically determining the insertion axis of the restoration for each preparation found, a patient-specific restoration is automatically calculated based on a restoration generation algorithm. In an exemplary embodiment, the restoration generation algorithm may analyze the occlusion and anatomical structure of the individual patient and adjacent teeth based, for example, on a scanned 3D model 504, and as a result, the restoration may be designed to be patient-specific.

[0093] The machine learning model 1306 may be trained using a training dataset as described herein. In step 1702 of Figure 17, the machine learning model may be trained using a training dataset that includes the input training model and the corresponding output training tooth number probabilities.

[0094] Figure 18 shows a training architecture 1802 for training a machine learning model 1306. The machine learning model 1306 is trained using various types of training data 1804, including an estimation model constructed using a sample of possible dentitions. The samples are obtained based on actual patient data from a database. The training data 1804 may also include delimiter weights corresponding to confidence levels of tooth type or tooth spacing in the constructed estimation model. In one embodiment, when the application receives a request to provide a recommendation, it creates an array of values ​​to be input to the input neurons of the machine learning model 1306 to generate an array containing tooth count probabilities. As shown in Figure 18, in some embodiments, the program code arbitrarily extracts or selects various features / attributes 1806 from the training data 1804, which has training data entries with labels L. In the case of feature extraction, these may be incorporated into the machine learning model 1306 itself. The features are used to develop a predictive function H(x) or hypothesis that the program code utilizes as the machine learning model 1306. In identifying various features / attributes within the training data 1804, the program code may utilize a variety of techniques, including mutual information, which are examples of methods that may be used to identify features in one embodiment, but are not limited to. Other embodiments may utilize a variety of techniques for feature selection, including, but are not limited to, principal component analysis, diffusion mapping, random forests, and / or recursive feature removal (a brute-force method for selecting features). "P" is an output that may be obtained (e.g., tooth count probability or evaluation or classification), which, once received, can further start the dental restoration system 300 to perform other steps, such as finding the best-fit tooth count distribution. The program code may train the machine learning model 1306 using the machine learning algorithm 1810, including providing weights for the output so that the program code can prioritize various changes based on a prediction function that includes the machine learning model 1306. The output may be evaluated by a quality metric 1808.

[0095] By selecting a diverse set of training data 1804, the program code trains a machine learning model 1306 to identify and weight various attributes of a patient's teeth. To utilize the machine learning model 1306, the program code acquires (or derives) input data or features to generate an array of values ​​to input to the input neurons of the neural network. In response to these inputs, the output neurons of the neural network generate an array containing tooth number probabilities 1312 that should be simultaneously memorized and / or evaluated.

[0096] The constructed estimation model 1318 can be constructed for any number of teeth, but in the exemplary model, the constructed estimation model 1318 can be constructed for all teeth of the maxilla and mandible, as shown in process 1900 in Figure 19. In step 1902, process 1900 receives instructions to calculate the probability for the complete maxilla. In step 1904, process 1900 automatically proposes the number of teeth for multiple possible dentitions for the entire dental cavity. In step 1906, process 1900 calculates the best-fitting number of teeth distribution for the entire dental cavity, and the best-fitting number of teeth distribution includes both the best-fitting maxillary number of teeth distribution and the best-fitting mandibular number of teeth distribution. Process 1900 then terminates, and another process may begin creating a restoration based on the best-fitting number of teeth distribution.

[0097] The restoration system 300 may further provide tools for correcting incorrect tooth positioning and tools for intuitive fine-tuning of the dental cavity. In the rare case where the first restoration 2002 is incorrectly positioned, as shown in Figures 20A–20D, a drag-and-drop tool 2008 is used for correction. The drag-and-drop tool 2008 may be adapted to different display types, including touchscreens, and may be operated from all viewing directions. Each restoration, such as the first restoration 2002, may be moved along the jaw, as shown in Figures 20A–20B. The second restoration 2004 may be recalculated for the new position by selecting the first restoration 2002 and releasing it in the new position, as shown in Figure 20C (see Figure 20C). The recalculation may be based on one or more of the processes described herein. For example, a high degree of confidence may be given to different tooth spacings 1404, or the restoration may be directly calculated in the new position using a well-known algorithm.

[0098] By double-clicking or double-tapping the preparation location, the recalculated first restoration 2006 is added and precisely fitted into the gap, as shown in Figure 20D. Furthermore, the restoration can be removed by double-clicking or double-tapping it.

[0099] The repair system 300 may further provide intuitive tools for fine-tuning specific cavity areas, as shown in Figures 21A to 21D. In Figure 21A, the cursor 2102 is positioned on the preparation margin and a control (e.g., the left mouse button) is pressed. In Figure 21B, while holding down the mouse button, the cursor is moved along the desired new margin line 2104. As shown in Figure 21B, in the area behind 2106, the cursor 2102 jumps to the most likely alternative path of the repair margin detected by one or more of the automated algorithms described herein. In the area 2108 in front of the cursor 2102, a smooth fit of the new boundary to the previous repair margin 2112 is calculated, the length of which may depend on the distance of the current cursor position to the previous repair margin 2112. As shown in Figure 21C, in the area behind 2106, the position of the cursor 2102 can be fixed by briefly releasing the control (e.g., the left mouse button), and the fixation can be canceled by right-clicking the mouse button. The surface of the restoration 2114 is simultaneously fitted to the new restoration margin line 2110, as shown in Figure 21D. If the correction line fills the existing restoration margin, the restoration fitting may stop. The described interaction may also be performed in a similar manner by touch control, starting with a finger on the existing preparation margin followed by the new preparation margin. Releasing the finger immediately fixes the already covered portion of the new line. If it exceeds the existing preparation margin, the interaction stops. Also, to make the correction of the restoration margin easier, the restoration surface may be made transparent in certain areas when moving the cursor inside the restoration.

[0100] Accordingly, computer implementation methods, systems or devices, and computer program products are provided in exemplary embodiments for automating the tooth management stages of a restorative workflow, including the manual management process of all teeth and other related features, functions, or operations. Where an embodiment or part thereof is described in relation to a certain type of device, the computer implementation method, system or device, computer program product, or part thereof is adapted or configured for use with a preferred and equivalent embodiment of that type of device.

[0101] When an embodiment is described as being implemented in an application, the delivery of the application in a Software as a Service (SaaS) model is considered to be within the scope of an exemplary embodiment. In a SaaS model, the ability of an application to implement an embodiment is provided to the user by running the application on a cloud infrastructure. The user can access the application using various client devices via a thin client interface such as a web browser (e.g., web-based email) or other lightweight client applications. The user does not manage or control the underlying cloud infrastructure, including the network, servers, operating system, or storage of the cloud infrastructure. In some cases, the user may not even manage or control the capabilities of the SaaS application. In some other cases, a SaaS implementation of an application may allow possible exceptions to limited user-specific application configuration settings.

[0102] The present invention may be a system, method, and / or computer program product in any possible level of technical detail. The computer program product may include a computer-readable storage medium (or a plurality of computer-readable storage media) having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0103] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media may be, but are not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random-access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punched cards or raised structures in grooves on which instructions are recorded, and any suitable combination thereof. Computer-readable storage media used herein, including but not limited to computer-readable storage devices, should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0104] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0105] The computer-readable program instructions for performing the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​such as Smalltalk and C++, procedural programming languages ​​such as the C programming language, or similar programming languages). The computer-readable program instructions may run entirely on the user's computer, partially on the user's computer, run as a standalone software package, partially on the user's computer and partially on a remote computer, or run entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or may be connected to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, electronic circuits including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable instructions by personalizing the electronic circuit using state information of computer-readable program instructions in order to perform aspects of the present invention.

[0106] Aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block in a flowchart and / or block diagram, and combinations of blocks within a flowchart and / or block diagram, can be implemented by computer-readable program instructions.

[0107] These computer-readable program instructions are given to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to create a machine, thereby creating means for instructions executed via the processor of the computer or other programmable data processing device to implement functions / operations specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may further be stored in a computer-readable storage medium that can instruct computers, programmable data processing devices, and / or other devices to function in a particular way, thereby the computer-readable storage medium on which the instructions are stored constitutes a product containing instructions that implement modes of functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0108] Computer-readable program instructions can further be loaded into a computer, other programmable data processing device, or other device to generate a computer implementation process by causing the computer, other programmable device, or other device to execute a series of operational steps so that the instructions executed on the computer, other programmable device, or other device implement the functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0109] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible embodiments of the systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts and block diagrams may represent a module, segment, or a portion of instructions containing one or more executable instructions for implementing a specified logical function(s). In some alternative embodiments, the functions shown in the blocks may occur in an order other than that shown in the drawings. For example, two blocks shown consecutively may actually be executed almost simultaneously, or blocks may be executed in reverse order depending on the functions they contain. Furthermore, it should be noted that each block in the block diagrams and / or flowcharts, and any combination of blocks in the block diagrams and / or flowcharts, may be implemented by a special-purpose hardware-based system that performs a specified function or operation, or a combination of special-purpose hardware and computer instructions.

Claims

1. An information processing method performed by a computer, wherein the computer, The steps include: using a spline forming module to form a spline along the jaw, The process involves proposing potential interdental gaps by performing one or more interdental gap detection steps, including using an interdental gap suggestion module to analyze tooth cross-sections using a cross-sectional module, detecting interdental papillae using an interdental papilla detection module, and classifying tooth spacings using a classification module. To obtain one or more delimiters, a step of weighting the potential interdental gaps based on one or more of the interdental gap detection steps, The steps include: using an alignment module to automatically propose a tooth count probability for each of one or more possible tooth rows between at least one pair of fixed delimiters of the one or more delimiters; A step of calculating the best-fit tooth distribution from the one or more possible dentitions in response to the automatically suggesting step, using the proposed tooth number probabilities for each of the one or more possible dentitions. Includes, The aforementioned alignment module is an information processing method that is a machine learning engine.

2. The information processing method according to claim 1, wherein a patient-specific restoration is generated based on the calculated best-fit tooth number distribution.

3. The information processing method according to claim 1, wherein all of the interdental gap detection steps are performed and performed in parallel.

4. The information processing method according to claim 1, further comprising the step of the computer calculating the best-fit tooth count distribution using a global optimization module that accelerates the calculation of the best-fit tooth count distribution by dynamic programming.

5. The information processing method according to claim 4, wherein the global optimization module uses one or more factors selected from a list consisting of maxillary-mandibular match, tooth size, weights from a weighting step, and output of a machine learning engine as input.

6. The information processing method according to claim 1, further comprising the step of the computer calculating the best-fit tooth distribution by iterating over all possible tooth rows around the one or more delimiters using an iterative module.

7. The information processing method according to claim 1, wherein the best-fit tooth number distribution is calculated for the entire dental cavity including the maxilla and mandible, a plurality of delimiters are obtained, and a plurality of possible dentitions between the at least pair of fixed delimiters are determined for both the maxilla and the mandible.

8. The computer The steps include automatically proposing the number of teeth for each of the multiple possible dentitions for the entire dental cavity, A step of calculating the best-fit tooth number distribution for the entire tooth cavity, wherein the best-fit tooth number distribution includes both the best-fit maxillary tooth number distribution and the best-fit mandibular tooth number distribution. The information processing method according to claim 7, further comprising:

9. The information processing method according to claim 1, wherein in the calculation step, the best-fit tooth distribution is calculated for one jaw, and based on the calculated best-fit tooth distribution, another best-fit tooth distribution is estimated for the opposing jaw.

10. The information processing method according to claim 1, wherein in the calculation step, the best-fit tooth distribution is calculated for a portion of one jaw.

11. The information processing method according to claim 10, wherein another best-fit tooth number distribution is estimated for opposing jaws or for opposing jaws based on the calculated best-fit tooth number distribution.

12. The information processing method according to claim 10, wherein a portion of one jaw includes eight possible teeth in the first quadrant of the jaw.

13. The information processing method according to claim 1, wherein the aforementioned forming step is automated.

14. The information processing method according to claim 1, wherein the one or more delimiters are determined to be potential delimiters and / or fixed delimiters based on their respective weights.

15. The information processing method according to claim 1, further comprising the step of the computer automatically proposing the preparation type probability of at least one tooth between the at least pair of fixed delimiters.

16. The information processing method according to claim 1, wherein at least one of the one or more delimiters indicates a space representing one or more missing teeth.

17. The information processing method according to claim 1, wherein the space between two delimiters that are separated by a threshold distance represents one or more possible teeth.

18. The information processing method according to claim 17, wherein the automatically suggesting step predicts the number of one or more possible teeth that fit within the space.

19. The information processing method according to claim 1, wherein at least one of the one or more delimiters indicates a boundary between adjacent teeth.

20. The information processing method according to claim 1, wherein the tooth number probability is a percentage or a value between 0 and 1.

21. The information processing method according to claim 1, wherein the tooth number probability is a likelihood evaluation or classification.

22. The information processing method according to claim 1, wherein the machine learning engine has a model based on a 3D surface-based neural network.

23. For each of the one or more possible dentitions, The computer performs a preprocessing step by constructing an estimation model that represents the currently possible dentition, The steps include: providing the constructed estimation model as input to a machine learning model; The computer obtains the tooth number probability as output. The information processing method according to claim 1, further comprising:

24. The information processing method according to claim 23, wherein the estimation model is represented as a 2D array of height values ​​corresponding to the heights of multiple surface points of the model.

25. The information processing method according to claim 23, wherein the constructed estimation model is a 3D model.

26. The information processing method according to claim 1, further comprising the step of the computer training a machine learning model using a training dataset which includes an input training model and a corresponding output training tooth number probability.

27. The information processing method according to claim 26, wherein the spaces within the input training model are interpreted as corresponding to one or more missing teeth, each corresponding to one or more identical or different numbers of teeth.

28. The information processing method according to claim 2, wherein, in response to the inaccurate generation of the patient-specific restoration, another patient-specific restoration is regenerated based on the selection of a new tooth position for another patient-specific restoration.

29. The information processing method according to claim 2, wherein the inaccurate margin of the patient-specific restoration is corrected by adjusting the inaccurate margin to a new position based on possible margin courses.

30. Processor and A memory for storing instructions, wherein, when the instructions are executed by the processor, the system... Using the spline forming module, splines are formed along the jaw. By using the interdental gap suggestion module to perform one or more interdental gap detection steps, including the steps of analyzing tooth cross-sections using the cross-section module, detecting interdental papillae using the interdental papilla detection module, and classifying tooth spacings using the classification module, potential interdental gaps are suggested. To obtain one or more delimiters, the potential interdental gaps are weighted based on one or more of the interdental gap detection steps. Using the alignment module, the number of teeth probability is automatically proposed for each of the one or more possible tooth rows between at least one pair of fixed delimiters of the one or more delimiters. Using the proposed tooth count probabilities for at least one or more possible tooth rows, a memory and, in response to the automatically suggesting step, calculate the best-fit tooth count distribution from the one or more possible tooth rows. A computing system comprising, The alignment module is a computing system that is a machine learning engine.

31. The computing system according to claim 30, wherein patient-specific restorations are generated based on the calculated best-fit tooth distribution.

32. The computing system according to claim 30, wherein the machine learning engine has a model based on a 3D surface-based neural network.

33. A non-temporary computer-readable storage medium for storing instructions, wherein, when the instructions are executed by the computer, the computer... Using the spline forming module, splines are formed along the jaw. By using the interdental gap suggestion module to perform one or more interdental gap detection steps, including the steps of analyzing tooth cross-sections using the cross-section module, detecting interdental papillae using the interdental papilla detection module, and classifying tooth spacings using the classification module, potential interdental gaps are suggested. To obtain one or more delimiters, the potential interdental gaps are weighted based on one or more of the interdental gap detection steps. Using the alignment module, the number of teeth probability is automatically proposed for each of the one or more possible tooth rows between at least one pair of fixed delimiters of the one or more delimiters. Using the proposed tooth count probabilities for at least one or more possible tooth rows, in response to the automatically proposing step, the best-fit tooth count distribution is calculated from the one or more possible tooth rows. The alignment module is a non-temporary, computer-readable storage medium that is a machine learning engine.

Images