Visual presentation of gingival lines generated based on 3D tooth models

Abstract

Systems and methods of simulating dental treatment are disclosed. A method may include capturing a first 2D image of a patient's face including patient's teeth; constructing a parameter 3D model of the teeth and gingiva of the patient based on the first 2D image; displaying a simulation result of the dental treatment of the patient's teeth by rendering a parameter 3D model of the patient's teeth at one or more positions and / or orientations corresponding to the treatment target of the dental treatment plan; and rendering a second 2D image of the patient's face having gums and teeth according to the simulation of the dental treatment plan. As described herein, a dental treatment plan may include an orthodontic element and / or a repair element. The simulation results may correspond to estimated results and / or expected results of the dental treatment plan.

Description

[0001] This application is a divisional application of the patent application with application date of May 12, 2020, application number 2020800362364, and invention name “Visual presentation of gum line generated based on 3D tooth model”.

[0002] Cross-references

[0003] This application claims the benefit of U.S. Provisional Application No. 62 / 847,780, filed May 14, 2019, which is incorporated herein by reference. Technical Field

[0004] The technical field relates to digital dentistry, and more particularly to providing simulated results of dental (eg, orthodontic, restorative, etc.) treatments by evaluating a two-dimensional (2D) depiction of a patient's untreated teeth with reference to parameters associated with a model dental arch. Background Art

[0005] Orthodontic treatment generally involves addressing tooth and / or jaw misalignment and may include the diagnosis, prevention, and / or correction of malocclusions. A person seeking orthodontic treatment may seek a treatment plan from an orthodontist (a professional who has received special training after graduating from dental school, for example). Many orthodontic treatment plans include treatment with braces, brackets, wires, and / or polymer appliances. The orthodontic professional who designs and / or implements the orthodontic appliances may adjust the appliances at various times for the person seeking orthodontic treatment.

[0006] Many people are referred for orthodontic treatment by their dentist, other treating professionals, or others. For example, many adolescents or people with severe malocclusion may have been referred for orthodontic treatment by their dentist or parents. However, many others may not know whether they should receive orthodontic treatment. For example, many people with mild malocclusion may not know whether orthodontic treatment is appropriate or advisable for them.

[0007] Additionally, many people may imagine how their smile would look without tooth and / or jaw misalignment, for example, after undergoing estimated dental treatment(s) and / or anticipated dental treatment(s), after inserting implants or other devices having configurations appropriate for their face, age, heritage, and / or lifestyle, etc. While it may be desirable to enable people to imagine what their smile and / or appearance would look like after completing available treatment options, the computational burden and / or computational expense of existing tools makes this difficult to do. It is also difficult for people to imagine how dental treatment would have a meaningful impact on a patient's life with existing tools. Summary of the Invention

[0008] The present disclosure generally relates to systems, methods, and / or computer-readable media related to simulating dental treatment of a patient's teeth, and more particularly to providing a photorealistic rendering of a two-dimensional (2D) image of a patient, the two-dimensional image representing one or more simulated (e.g., estimated and / or expected) results of a dental treatment plan. Embodiments herein produce nearly accurate and realistic renderings of simulated results of dental treatment and / or animations of three-dimensional (3D) models that may not have been previously possible to generate, or could only be generated in a minimal manner using manual photo editing tools. As described herein, the described embodiments use automated agents and / or rules to provide accurate and realistic renderings of simulated results of dental (e.g., orthodontic, restorative, etc.) treatment and / or animations of 3D models that were previously impossible to achieve. Embodiments herein enable people who are considering orthodontic treatment and / or are currently undergoing orthodontic treatment to visualize automatically generated simulations of simulated orthodontic treatment results on a computer, and can inform a person's choice of whether to seek orthodontic treatment during a general course of orthodontic treatment and / or a specific course of orthodontic treatment. As described herein, the present disclosure also relates to systems and methods for accurately and realistically simulating a 3D model of teeth in a final orthodontic position in a 2D image of a person.

[0009] A computer-implemented method for simulating one or more simulated results of a dental treatment is disclosed. In some embodiments, the computer-implemented method for simulating orthodontic treatment may include capturing a first 2D image. In some embodiments, the first 2D image may include a representation of a patient's face and the patient's teeth. The method may include identifying one or more shapes associated with at least one of the patient's teeth. The method may also include constructing a parametric 3D model of the patient's teeth based on the first 2D image using one or more case-specific parameters of the one or more shapes associated with at least one of the patient's teeth. The method may also include: simulating the results of a dental treatment plan for the patient's teeth to produce a simulated result of the dental treatment plan; and modifying the parametric 3D model to provide a modified 3D model representing the simulated result of the dental treatment plan. The method may also include rendering a second 2D image representing the patient's face using the modified 3D model, wherein the second 2D image represents the patient's teeth according to the simulated result of the dental treatment plan.

[0010] In some embodiments, constructing the parametric 3D model includes: finding edges of teeth and lips in the first 2D image; aligning the parametric dental model with the edges of teeth and lips in the first 2D image to determine case-specific parameters; and storing case-specific parameters of the parametric dental model that align the parametric dental model with the edges of teeth, gums, and lips in the first 2D image.

[0011] In some embodiments, rendering the second 2D image includes: accessing a parametric 3D model of the patient's teeth; projecting one or more tooth locations from the first 2D image onto the parametric 3D model; and mapping color data from the 2D image to corresponding locations on the parametric 3D model to generate a texture of the parametric 3D model; and using the texture as part of the second 2D image of the patient's face.

[0012] In some embodiments, the predetermined position is based on an average position of multiple teeth of previous patients after dental treatment.

[0013] In some embodiments, the predetermined position is based on an average position of multiple teeth of previous patients prior to dental treatment.

[0014] In some embodiments, a computer-implemented method may include finding edges of teeth and lips in a first 2D image; and aligning the parametric 3D tooth model with the edges of the teeth, gums, and lips in the first 2D image.

[0015] In some embodiments, the first 2D image may include a side image representing a profile of the patient's face.

[0016] In some embodiments, the simulated results of the dental treatment plan may include estimated results of the dental treatment plan.

[0017] In some embodiments, the simulated results of the dental treatment plan may include the expected results of the dental treatment plan.

[0018] In some embodiments, the dental treatment plan may include an orthodontic treatment plan, a restorative treatment plan, or some combination thereof.

[0019] In some embodiments, capturing the first 2D image may include instructing a mobile phone or a camera to image the patient's face, or collecting the first 2D image from a storage device or a network system.

[0020] In some embodiments, constructing a parametric 3D model of the patient's teeth based on a 2D image using one or more case-specific parameters of one or more shapes associated with at least one tooth of the patient may include: roughly aligning the teeth represented in the 3D parametric model with the patient's teeth represented in the 2D image; and performing an expectation step to first determine a probability that a projection of a silhouette of the 3D parametric model matches one or more edges of the 2D image.

[0021] In some embodiments, constructing a parametric 3D model of a patient's teeth based on a 2D image using one or more case-specific parameters of one or more shapes associated with at least one tooth of the patient may include: performing a maximization step using a small angle approximation to linearize a rigid transformation of the teeth in the 3D model; and performing an expectation step to secondly determine a probability that a projection of a silhouette of the 3D parametric model matches an edge of the 2D image.

[0022] In some embodiments, the computer-implemented method may include iterating through the expectation step and the maximization step for a first round of multiple times using a first subset of parameters; and after iterating through the expectation step and the maximization step for the first round of multiple times using the first subset of parameters of the 3D parametric model, iterating through the expectation step and the maximization step for a second round of multiple times using the first subset and the second subset of parameters.

[0023] In some embodiments, a computer-implemented method may include capturing a first 2D image of a patient's face (including the patient's teeth). The method may include constructing a parametric 3D model of the patient's teeth based on the 2D image, the parametric 3D model including case-specific parameters of a shape of at least one of the patient's teeth. The method may also include simulating an estimated (e.g., estimated final) orthodontic position and / or an expected (e.g., expected final) orthodontic position of the patient's teeth by collecting information about one or more model dental arches representing a smile without tooth misalignment and / or jaw misalignment, and by rendering a 3D model of the patient's teeth in predetermined positions (e.g., positions corresponding to the positions of the teeth in the model dental arches), and rendering a second 2D image of the patient's face with the teeth in the estimated orthodontic position.

[0024] In some embodiments, constructing the parametric 3D model includes: finding edges of teeth and lips in the first 2D image; aligning the parametric dental model with the edges of teeth and lips in the first 2D image to determine case-specific parameters; and storing case-specific parameters of the parametric dental model that align the parametric dental model with the edges of teeth, gums, and lips in the first 2D image.

[0025] In some embodiments, rendering a second 2D image includes: rendering a parametric model of the patient based on the positions of the teeth in the first 2D image; projecting the 2D image onto the rendered parametric model of the patient based on the positions of the teeth in the first 2D image; and mapping color data from the 2D image to corresponding positions on the 3D model to generate a texture of the 3D model; and using the generated texture to render a second 2D image of the patient's face with the teeth in estimated orthodontic positions.

[0026] In some embodiments, rendering the second 2D image further includes applying a simulated treatment to the second 2D image or viewing customization options for the second 2D image.

[0027] In some embodiments, simulating a treatment or viewing customized options may include one or more of changing the margin of the gums, replacing teeth, adjusting jaw position, or adjusting color data.

[0028] In some embodiments, the predetermined position is based on a combination of the (eg, average) positions of multiple teeth of previous patients after orthodontic treatment and / or in the absence of tooth or jaw misalignment.

[0029] In some embodiments, the predetermined position is based on a combination of the (eg, average) positions of multiple teeth of previous patients prior to orthodontic treatment.

[0030] In some embodiments, the method may include: finding edges of teeth and lips in the first 2D image; and aligning the parametric tooth model with the edges of the teeth, gums, and lips in the first 2D image.

[0031] In some embodiments, the first 2D image comprises a side image.

[0032] A computer-implemented method for constructing a 3D model of teeth from a 2D image is disclosed. The method may include: capturing a 2D image of a patient's face (including the patient's teeth); determining the margins of the teeth and gums within the first 2D image; fitting the teeth in a 3D parametric model of the teeth to the margins of the teeth and gums within the first 2D image, the 3D parametric model including case-specific parameters of the shape of the patient's teeth; and determining values ​​for the case-specific parameters of the 3D parametric model based on the fitting.

[0033] In some embodiments, fitting teeth in a 3D parametric model of teeth to the edges of teeth and gums within a first 2D image includes: roughly aligning the teeth in the 3D parametric model with the teeth in the 2D image; and performing desired steps to determine a probability that a projection of a silhouette of the 3D parametric model matches the edges of the 2D image.

[0034] In some embodiments, fitting the teeth in the 3D parametric model of the teeth to the edges of the teeth and gums within the first 2D image also includes: performing a maximization step using a small angle approximation to linearize the rigid transformation of the teeth in the model; and performing an expectation step to again determine the probability that the projection of the silhouette of the 3D parametric model matches the edges of the 2D image.

[0035] In some embodiments, the computer-implemented method further includes: performing a first round of multiple iterations on the expectation step and the maximization step using a first subset of parameters; and after performing a first round of multiple iterations on the expectation step and the maximization step using a first subset of parameters of the 3D parametric model, performing a second round of multiple iterations on the expectation step and the maximization step using a first subset and a second subset of parameters.

[0036] In some embodiments, the first multiple times are the same as the second multiple times.

[0037] In some embodiments, the first subset of case-specific parameters of the 3D parametric model is one or more of a scale factor, tooth position, and tooth orientation.

[0038] In some embodiments, the second subset of parameters of the 3D parametric model is tooth shape and one or more of tooth position and tooth orientation.

[0039] A computer-implemented method for providing simulated results of orthodontic treatment is disclosed. The method may include: constructing a 3D parametric model of a dental arch, the 3D parametric model including general parameters of tooth shape, tooth position, and tooth orientation; capturing a 2D image of a patient; constructing a case-specific 3D parametric model of the patient's teeth from the 2D image; determining case-specific parameters of the constructed parametric model; rendering the 3D parametric model of the patient's teeth in an estimated and / or intended final position (e.g., without tooth and / or jaw misalignment); and inserting the rendered 3D model into the 2D image of the patient.

[0040] In some embodiments, constructing a 3D parametric model of the patient's teeth from a 2D image includes: finding edges of the teeth, gums, and lips in the first 2D image; and aligning the 3D parametric model with the edges of the teeth, gums, and lips in the first 2D image.

[0041] In some embodiments, the method further includes applying a texture to the rendered 3D parametric model of the patient's teeth in an estimated and / or intended final position (e.g., without tooth misalignment and / or jaw misalignment), wherein the texture is derived from a 2D image of the patient.

[0042] In some embodiments, texture is derived from a 2D image of a patient by projecting the 2D image onto a rendered parametric model of the patient based on the positions of the teeth in the first 2D image; and mapping color data from the 2D image to corresponding positions on the 3D model to derive texture of the 3D model.

[0043] In some embodiments, rendering of a 3D parametric model of a patient's teeth in an estimated and / or expected final position includes: generating an average shape of the teeth based on the 3D parametric model; adjusting the shape of the teeth in the 3D parametric model based on case-specific tooth shape parameters; positioning the teeth with an average tooth position and orientation based on average position and orientation parameters so that the teeth have a case-specific shape and an average position and orientation; and scaling the dental arch based on case-specific dental arch scaling parameters.

[0044] A non-transitory computer-readable medium includes instructions that, when executed by a processor, cause the processor to perform any of the methods described herein.

[0045] A system is disclosed. The system may include: a photo parameterization engine configured to generate a 3D parametric dental arch model from a 2D image of a patient's face and teeth, the parametric 3D model including case-specific parameters of a shape of at least one tooth of the patient; and a parametric treatment prediction engine configured to identify an estimated outcome and / or expected outcome of orthodontic treatment for the patient based on the 3D parametric dental arch model and historical and / or ideal dental arch models of a plurality of patients.

[0046] In some embodiments, the system includes a treatment projection rendering engine configured to render the 3D parametric dental arch model.

[0047] In some embodiments, the photo parameterization engine, the parametric treatment prediction engine, and the treatment projection rendering engine are together configured to perform the methods described herein.

[0048] Incorporation by Reference

[0049] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The novel features of the present invention are particularly set forth in the appended claims. A better understanding of the features and advantages of the present invention may be obtained by reference to the following detailed description which illustrates illustrative embodiments in which the principles of the invention are utilized, and the accompanying drawings in which:

[0051] Figure 1 A method of providing an estimated outcome of orthodontic treatment according to one or more embodiments herein is shown;

[0052] Figure 2 shows a parametric tooth model according to one or more embodiments herein;

[0053] Figure 3A shows an example of how well a parametric tooth model matches an original 3D model according to one or more embodiments herein;

[0054] Figure 3B A method for determining universal parameters from historical cases and / or ideal cases according to one or more embodiments herein is shown;

[0055] Figure 4 illustrates alignment of past cases used in determining parameters of a parametric model according to one or more embodiments herein;

[0056] Figure 5 Depicted is a method of generating a parametric model of a patient's teeth and converting the parametric model into a 3D model of a dental arch according to one or more embodiments herein;

[0057] Figure 6 Describes a method for constructing a 3D model from a 2D image according to one or more embodiments herein;

[0058] Figure 7A Described is a method of constructing a patient-specific parametric model of a patient's teeth according to one or more embodiments herein;

[0059] Figure 7B depicts a tooth model having a gum margin and a lip margin according to one or more embodiments herein;

[0060] Figure 8 Depicted is a method of rendering a patient's teeth in an initial position using a parametric model of the patient's dental arch according to one or more embodiments herein;

[0061] Figure 9 Depicted are methods of constructing a 3D model and applying a texture to the 3D model according to one or more embodiments herein;

[0062] Figure 10A Depicted is a method of simulating an estimated outcome of orthodontic treatment of a patient's teeth according to one or more embodiments herein;

[0063] Figure 10B Depicted is a method of simulating orthodontic treatment of a patient based on matching tooth shape parameters according to one or more embodiments herein;

[0064] Figure 11 An example of a method of rendering teeth according to an estimated result of a dental treatment plan according to one or more embodiments herein is shown;

[0065] Figure 12 Depicted is a system for simulating estimated outcomes of orthodontic treatment according to one or more embodiments herein;

[0066] Figure 13 depicts an example of one or more elements of an estimated orthodontic treatment simulation system according to one or more embodiments herein;

[0067] Figure 14 A tooth repositioning appliance according to one or more embodiments herein is shown;

[0068] Figure 15 A tooth repositioning system according to one or more embodiments herein is shown;

[0069] Figure 16 A method of performing orthodontic treatment using multiple appliances according to one or more embodiments herein is shown;

[0070] Figure 17 A method of designing an orthodontic appliance according to one or more embodiments herein is shown;

[0071] Figure 18 A method of planning orthodontic treatment according to one or more embodiments herein is shown;

[0072] Figure 19 is a simplified block diagram of a system for designing orthodontic appliances and planning orthodontic treatment according to one or more embodiments herein.

[0073] Figure 20 A method of simulating a gum line according to one or more embodiments herein is shown;

[0074] Figure 21A shows a gum line input determined from a 2D image of a patient's teeth according to one or more embodiments herein;

[0075] Figure 21B Depicts a process for modeling a gum line based on input determined from a 2D image of a patient's teeth in accordance with one or more embodiments herein;

[0076] Figure 22 illustrates gum line parameters according to one or more embodiments herein;

[0077] Figure 23 A method of leveling a gum line according to one or more embodiments herein is shown;

[0078] Figure 24 Described is a process for rendering a realistic composite image of a patient's face and a model of the patient's teeth, according to one or more embodiments herein. DETAILED DESCRIPTION

[0079] The embodiments discussed herein provide tools such as automated agents to visualize the effects of correction of tooth / jaw misalignment, malocclusion, etc., without the computational burden and / or expense of scanning a patient's dentition or dental impressions, and also to calculate the final position of a treatment plan for the patient's dentition. As discussed in detail herein, these techniques may involve obtaining a two-dimensional (2D) representation (e.g., an image) of the patient's dentition, obtaining one or more parameters representing attributes of the patient's dentition in the 2D representation, and using the one or more parameters to compare the attributes of the patient's dentition with attributes of a model dental arch (e.g., attributes of a historical case and / or attributes representing an ideal dental arch morphology). The techniques herein may provide a basis for simulating the simulated results of a dental treatment plan.

[0080] As used herein, "simulated results of a dental treatment plan" may include, for example, estimated results and / or expected results of a dental treatment plan after performing one or more dental procedures (such as an orthodontic procedure, a restorative procedure, etc.). As used herein, "estimated results of a dental treatment plan" may include an estimate of the state of a patient's dentition after the dental procedure. In certain circumstances, the estimated results of a dental treatment plan as used herein may differ from the "actual results of a dental treatment plan," which may represent the state of a patient's dentition after the dental treatment plan is implemented. In various circumstances, the estimated results and actual results of a dental treatment plan as used herein may differ from the "expected results of a dental treatment plan," which may represent the expected state of a patient's dentition after the dental treatment plan is implemented. It should also be noted that the "estimated results of an orthodontic treatment plan" may include an estimate of the state of a patient's dentition after correction of any tooth / jaw misalignments, malocclusions, etc. suffered by the patient. In some embodiments, the estimated result of the orthodontic treatment plan includes an estimated state of the patient's dentition if the patient's dentition has been modified to have a model and / or ideal dental arch form reflected by one or more databases of historical cases and / or ideal dental arch forms. The "actual result of the orthodontic treatment plan" may represent the state of the patient's dentition after the orthodontic treatment plan is implemented; the "expected result of the orthodontic treatment plan" may represent the expected state of the patient's dentition after the orthodontic treatment plan is implemented.

[0081] The features and advantages of the present disclosure may be better understood by referring to the following detailed description and accompanying drawings that set forth illustrative embodiments, in which the principles of embodiments of the present disclosure are utilized.

[0082] Figure 1 An example of a method 100 for providing an estimated result and / or expected result of a dental treatment plan according to one or more embodiments of the present invention is shown. The method 100 can be performed by any system disclosed herein. It should be noted that various examples may include, for example, Figure 1More or fewer blocks than those shown in FIG.

[0083] In block 110, one or more two-dimensional (2D or 2-D) images of the patient are captured. In some embodiments, the 2D image(s) depict the patient's mouth and include one or more images of the face, head, neck, shoulders, torso, or the entire patient. The 2D image(s) of the patient may include images of the patient with the patient's mouth in one or more positions; for example, the patient's mouth may be in a smiling position (e.g., a social smile position), a resting position with muscles relaxed and lips slightly open, or a retracted anterior open bite or anterior closed bite position.

[0084] In some embodiments, an image of the patient is acquired using an image capture device. As used herein, an "image capture device" (i.e., an "image capture system") may include any system capable of capturing images. Examples of image capture devices include cameras, smartphones, digital imaging devices, components of a computer system configured to capture images, and the like. Images may be captured using a lens with a predetermined focal length at a distance from the patient. Images may also be captured remotely and then received for processing. In some embodiments, images of the patient are acquired from a computer storage device, a network location, a social media account, and the like. The images may be a series of images or videos captured from one or more perspectives. For example, the images may include one or more of a frontal face image and a side image, the side image including one or more three-quarter side images and a full side image.

[0085] At block 120, a three-dimensional (3D or 3-D) model of the patient's teeth is generated based on the 2D image of the patient. Figure 6 As discussed in more detail elsewhere herein, generating the 3D model can include identifying the patient's teeth. Generating the 3D model can also include identifying the patient's gums, lips, and / or mouth opening, forming a parametric model of each identified tooth, and combining the parametric tooth models into one or more (e.g., two, three, etc.) parametric dental arch models. As further discussed herein, the parametric models of the patient's teeth and dental arch can be based on or referenced to an average parametric tooth and dental arch model.

[0086] As used herein, a "parametric model of a patient's teeth" (e.g., a "parametric model of a patient's dentition") may include a model of a patient's dentition characterized by a probability distribution having a finite number of parameters (e.g., a statistical model). A parametric model of a patient's dentition may include a parametric model of the patient's teeth and / or dental arch. A parametric model may include models representing objects of various dimensions and may include parametric 2D models, parametric 3D models, and the like. Modeling a patient's teeth using a parametric model of the patient's dentition may reduce memory and computational requirements when manipulating, comparing, or otherwise using digital models (as described herein) and simplify comparisons between different models. In some embodiments, a parametric model of a tooth may be represented as:

[0087]

[0088] in is the average tooth shape, which is a universal parameter. As described herein, each tooth (e.g., the upper right canine, which is tooth number 6 in the universal tooth numbering system) has its own average tooth shape, which is calculated from thousands of teeth with the same tooth number. The symbol τ can represent the index of each tooth, which can be based on the universal tooth numbering system or another numbering system, can be the principal component of the shape of each tooth, as well as a universal parameter, and The coefficients of the principal components of the tooth shapes are case-specific parameters. Thus, equation (1) can be used to represent a parametric model for each patient-specific tooth (e.g., left lower incisor, right upper canine, etc.) based on the specific shape of each tooth relative to the average tooth shape of that tooth.

[0089] The parametric model of the patient's dentition can be expressed as:

[0090]

[0091] in is described above with reference to equation (1), is the average tooth position, which is a universal parameter; T τ is the deviation of the patient's tooth position from the corresponding mean tooth position and is a case-specific parameter; and Φ is the arch scaling factor that scales the unitless parameter value to the real-world value and is also a case-specific parameter. T is the global perspective of the dental arch from a certain viewpoint and is a case-specific parameter that, in some embodiments, is only used when matching with a 2D image, as dental arch scans typically do not have perspective, while 2D images (e.g., camera images) do.

[0092] To generate a parametric 3D model of a patient's teeth from a 3D tooth model (derived from an image of the patient's teeth or from another method known in the art), the teeth may be modeled based on displacement of the scanned tooth surface from a fixed shape (e.g., a fixed sphere). To illustrate this, refer to Figure 2 , which shows a parametric tooth model according to one or more embodiments herein. Figure 2 In the example of , a sphere 210 is shown with a plurality of vertices in fixed or known positions and orientations. A tooth 220 may be placed or otherwise modeled in the center of the sphere 210. In some embodiments, the volumetric center of the tooth 220, a scanned portion of the tooth 220, or the crown of the tooth 220 may be aligned with the center of the sphere 210. Each vertex 230a, 230b of the sphere 210 may then be mapped to a position on the surface of the tooth model. In some embodiments, the mapping may be represented by an n*3 matrix, where n represents the number of points on the sphere (e.g., 2500), and then for each point, the x, y, and z positions are recorded. In some embodiments, the matrix stores the difference between the positions on the average tooth and the corresponding positions on the actual model. In this way, each tooth is represented by the same 2500 points, and it is easy to compare the differences between teeth. This difference can be represented by a PCA component as Each specific case ultimately has a unique Collection, because is common to all cases. To illustrate this, refer to Figure 3A , which shows an example of how well the parametric model 320 of a tooth matches the original 3D model 310. The parameters of a particular tooth may be stored in a data repository (e.g., a database) and may be retrieved by express.

[0093] A parametric model of a patient's dental arch may involve parameterizing the position and orientation of each tooth in the dental arch and the scale of the dental arch. The case-specific parameters of a particular tooth may be stored in a matrix or other data repository as shown in the following equation (3):

[0094]

[0095] where ξ ij is the rotational component that defines the orientation of the tooth relative to the dental arch and can be the three rotation angles α of the tooth τ , β τ , γ τ function of Δ τ,x , Δ τ,y and Δ τ,zis the translation of the center point of the tooth relative to the origin of the dental arch. Rotation can be based on the orientation of the orthogonal axes of each tooth (e.g., the long axis, buccal-lingual axis, and mesio-distal axis) relative to a fixed reference or relative to an average rotation. In some embodiments, one or more of these components can be expressed as a deviation or change from the average dental arch discussed herein.

[0096] Similarly, a scaling factor Φ is applied to the teeth and their arch positions to scale the arch from a generic or unitless representation to a real-world scale representing the actual size and position of the teeth in the arch. In some embodiments, the scaling can be between projected 3D units (e.g., millimeters) and image size (e.g., pixels).

[0097] As discussed above and elsewhere herein, a parametric model of a dental arch including teeth can be represented based on an average dental arch model including teeth. The average dental arch model can be determined based on an average of a large dataset of patient dental arch scans. In some embodiments, the average dental arch model can be based on a large dataset of parameterized dental arches.

[0098] For example, an average dental arch may be constructed from a set of previously scanned and / or segmented dental arches. Figure 3B An example of a method 350 of determining an average dental arch model is depicted and discussed in further detail herein.

[0099] In some embodiments, the parametric model of the dental arch may be converted into a 3D model of the dental arch. Figure 5 A method 500 of converting a parametric model of one or more teeth into a 3D model of a dental arch according to some embodiments is depicted and discussed in further detail herein.

[0100] Return to Figure 1 At block 130, an estimated result and / or expected result of the dental treatment plan is identified on the 3D model to obtain an estimated result and / or expected result of the orthodontic treatment plan. Figures 9-11As discussed, the estimated and / or expected results of a dental treatment plan can be based on a parametric model of the patient's teeth at positions indicated according to a predetermined dental arch model. In some embodiments, the predetermined dental arch model can be based on an average dental arch model, which can be based on a historical average of dental arch scans collected from the patient or other dental arch models from the patient. In some embodiments, for example, the predetermined dental arch model can be based on an idealized model, such as a clinically ideal dental arch or dental arch model, wherein the tooth position and / or orientation is predetermined based on, for example, one or more of the aesthetic and clinical characteristics of the teeth (e.g., correct occlusion). In some embodiments, the estimated and / or expected results of a dental treatment plan can correspond to the expected final position (or an estimate thereof) of the patient's teeth after implementation of the treatment plan. In some embodiments, the estimated and / or expected results of a dental treatment can correspond to estimates of the final position and intermediate positions during the treatment planning process. As described herein, a dental treatment plan can include an orthodontic treatment plan, a restorative treatment plan, some combination thereof, and the like.

[0101] At block 140, a second 2D image(s) is generated that shows the estimated and / or expected results of the orthodontic treatment. As discussed herein, the second 2D image(s) may be a 2D facial image of the patient with the teeth aligned according to the estimated and / or expected results of the dental treatment plan. In some embodiments, for example, as described below with reference to Figure 9 As discussed elsewhere herein, the image can include an estimated texture and / or projected texture of the patient's teeth. As used herein, a "projected texture" or "estimated texture" of a patient's teeth can include a projection / estimate of the tooth texture and can include the feel, appearance, consistency, and / or other properties of the tooth surface. At block 150, the second 2D image(s) generated at block 140 are provided to a user. For example, the image can be rendered for viewing by a patient or dental professional. In some embodiments, the second 2D image can be loaded into or retrieved from memory.

[0102] Steering Figure 3B , Figure 3BA method for determining universal parameters from historical cases and / or ideal cases according to one or more embodiments of the present invention is shown. At box 360, historical cases and / or ideal cases are obtained. Historical cases and / or ideal cases can be retrieved from a data warehouse. Historical cases and / or ideal cases can include cases representing previously scanned and segmented dental arch models. In some embodiments, historical cases and / or ideal cases can represent dental arch models of treated patients (e.g., patients who have been treated in the past) and / or ideal dental arch models representing expected results of various forms of orthodontic treatment. In various embodiments, historical cases and / or ideal cases can represent dental arch models having ideal dental arch morphology. In some embodiments, historical cases and / or ideal cases can include multiple areas where the model dental arch has teeth that correspond to the positions of implants to be implanted in the patient's dental arch.

[0103] At block 370, the historical case and / or ideal case are aligned. In some embodiments, each dental arch of the historical case and / or ideal case is aligned at multiple locations. As an example, each dental arch of the historical case and / or ideal case can be aligned at the following three locations: between the central incisors, and at each distal end of the left and right sides of each dental arch. For example, Figure 4 A set of dental arches 400 are shown aligned at a position between the central incisors 410, at a left distal end 430 of the dental arch, and at a right distal end 420 of the dental arch. It should be noted that historical cases and / or ideal cases may be aligned at a variety of positions and a different number of positions without departing from the scope and spirit of the inventive concepts described herein.

[0104] return Figure 3B , determining the average dental arch model and determining the dental arch model distribution may include performing the sub-operations at block 370. For example, averaging each dental arch to determine Then determine the local deformation of each tooth and compare it with Compare to determine T τ , then after aligning each tooth, β τ .

[0105] At block 380, an estimate of the distribution of the case-specific parameters is determined. For example, the relative shape, position, and rotation of each tooth is determined to construct a distribution of each case-specific parameter. For example, the coefficients of the principal components of the surface model of each corresponding tooth in all retrieved models are determined. distribution.

[0106] The positions and orientations of each tooth can be averaged to determine the average position of each tooth, and the orientations of each tooth can be averaged to determine the average orientation of each tooth. The average positions and average orientations are used to determine

[0107] At block 390 , the mean of the estimated distribution of the case-specific parameters is used as the universal parameter for mean tooth position and mean tooth shape.

[0108] Figure 5 Depicted is a method 500 of generating a parametric model of one or more teeth of a patient and converting the parametric model of the one or more teeth into a 3D model of a dental arch, according to some embodiments. The method 500 can be used to generate a 3D model of a patient's dental arch based on the parametric model of the patient's teeth.

[0109] At block 510, the average shape of each tooth is determined. As an example, the average shape of each tooth is determined In some embodiments, as discussed herein, the average shape can be based on the average shape of a set of dental arches from historical cases and / or ideal cases. In some embodiments, for example, the average shape can be rendered on a screen for viewing by a patient or dental professional. For example, average tooth shape 512 is rendered for viewing. In some embodiments, the average shape can be loaded into memory or retrieved from memory. The average shape can also be initialized as a set of matrices, one for each tooth.

[0110] At block 520, a principal component analysis shape adjustment is performed on the mean shape of the teeth. As discussed herein, this adjustment adjusts the shape of the teeth based on the patient's specific teeth, e.g., based on a scan, 2D image, or other imaging technique of the patient's teeth. As an example, the mean shape of the teeth is adjusted. Perform principal component analysis shape adjustment. For each tooth in the model, the case-specific coefficients of the principal components Apply to principal components After the shape adjustment is completed, in some embodiments, the adjusted shape can be rendered on a screen for viewing by a patient or dental professional, for example. For example, the adjusted tooth shape 522 is rendered for viewing. In some embodiments, the adjusted shape can be stored in a memory. Alternatively, the adjusted shape can be stored as a set of matrices, with each matrix corresponding to a tooth.

[0111] At block 530, an average tooth pose is determined. In some embodiments, as discussed herein, the average tooth pose can be based on the average tooth pose of a set of dental arches from historical cases and / or ideal cases. In some embodiments, at block 530, each adjusted tooth 522 is placed in its corresponding average position and orientation determined by the average dental arch. In some embodiments, for example, the average tooth pose can be rendered on a screen for viewing by a patient or dental professional. For example, the average tooth pose 532 is rendered for viewing. In some embodiments, the average tooth pose can be loaded into or retrieved from a memory. The average tooth pose can also be initialized as a set of matrices, each matrix corresponding to a tooth in a tooth pose. In some embodiments, before adjusting the shape of the tooth at block 520, the average tooth shape from block 510 can be placed at its corresponding average tooth pose at block 530. In other words, the order of blocks 520 and 530 can be swapped.

[0112] At block 540, a tooth pose adjustment is performed on the average tooth pose. As discussed herein, this adjustment is based on the patient's specific teeth, for example, based on a scan, 2D image, or other imaging technology of the patient's teeth to adjust the shape of the teeth. In some embodiments, as discussed above, the pose adjustment T τ Based on the specific tooth pose of the patient's dental arch. In some embodiments, at block 540, as discussed herein, the position and orientation of each tooth 522 is adjusted so that the tooth is placed in a position and orientation determined by the position and orientation of the teeth in the imaged patient's dental arch or otherwise determined. In some embodiments, for example, the adjusted tooth pose can be rendered on a screen for viewing by the patient or dental professional. For example, the adjusted tooth pose 542 is rendered for viewing. In some embodiments, the adjusted tooth pose can be stored in a memory. The adjusted tooth pose can also be stored as a set of matrices and / or data structures, for example, each matrix and / or data structure corresponding to a tooth in a particular tooth pose. In some embodiments, before adjusting the tooth shape at block 520, the average tooth shape from block 510 can be placed in its corresponding adjusted tooth pose at block 540. In other words, the order of block 520 and blocks 530 and 540 can be swapped so that block 520 occurs after blocks 530 and 540.

[0113] At block 550, the dental arch is scaled so that it is generated based on the patient's teeth and arch dimensions. In some embodiments, as discussed above, the arch scaling factor Φ is based on the patient's specific teeth and arch. In various embodiments, for example, when scaling the 3D model for integration into a 2D image, the arch scaling factor can also be based on one or more of the image dimensions of the patient's 2D image. In some embodiments, at block 550, as discussed herein, the dimensions of each tooth 522 and the dental arch are adjusted so that the scaled dental arch matches the dimensions of the patient's dental arch, such as determined by the dimensions of the teeth and arch in the imaged patient arch or otherwise. In some embodiments, for example, the scaled dental arch can be rendered on a screen for viewing by the patient or dental professional. For example, the scaled dental arch 552 is rendered for viewing. In some embodiments, the scaled dental arch can be stored in a memory. Alternatively, the scaled dental arch can be stored as a set of matrices and / or data structures, each matrix and / or data structure corresponding to a tooth in a dental pose. In some embodiments, the average tooth shapes from block 510 may be placed into their corresponding scaled positions and sizes at block 550 before adjusting the shapes of the teeth at block 520. In other words, the order of block 520 and blocks 530, 540, and 550 may be swapped so that block 520 occurs after blocks 530, 540, and 550.

[0114] You can follow except Figure 5 The blocks of method 500 may be performed in an order other than the order shown. For example, blocks 510 and 530 may be performed before blocks 520, 540, and 550. In some embodiments, blocks 510, 520, 530, and 550 may be performed before block 540. These and other modifications may be made to the order of the blocks in method 500 without departing from the spirit of the present disclosure.

[0115] Notice Figure 6 , Figure 6 A method 600 for constructing a 3D model from a 2D image is shown, according to one or more embodiments disclosed herein.

[0116] At block 610, a 2D image of the patient is captured. In some embodiments, the 2D image includes the patient's mouth and one or more images of the patient's face, head, neck, shoulders, torso, or the entire patient. The 2D image of the patient may include an image of the patient with the patient's mouth in one or more positions. For example, the patient's mouth may be in a smiling position (e.g., a social smile position), a resting position with muscles relaxed and lips slightly open, or a retracted anterior open bite or anterior closed bite position.

[0117] In some embodiments, images of the patient are captured using an image capture system. Images can be captured using a lens with a predetermined focal length at a distance from the patient. Images can also be captured remotely and then received for processing. Images can be collected from a storage system, a network location, a social media site, or the like. The images can be a series of images from a video captured from one or more perspectives. For example, the images can include one or more frontal facial images and profile images, including one or more three-quarter profile images and full profile images.

[0118] At block 620, edges of features of the patient's mouth are determined. For example, edges of one or more of the patient's teeth, lips, and gums can be determined. A preliminary determination of the patient's lip (e.g., defining the inner edges of the lips of an open mouth), teeth, and gum contours can be identified by a machine learning algorithm, such as a convolutional neural network. The machine learning algorithm can be trained based on pre-identified landmarks of the patient's lips, teeth, and gums visible within the 2D image. The initial contour can be a weighted contour such that the machine learning algorithm is confident that a given location in the image (e.g., at each pixel) is an edge or contour of the patient's lips, teeth, or gums.

[0119] Initial contours can be extracted from the image. The initial contours can have a brightness or other scale applied to them. For example, in a grayscale image of contours, each pixel can be assigned a value between 0 and 255, which can indicate the confidence that the pixel is a contour, or can indicate the magnitude of the contour at that location.

[0120] The pixels representing the contour can then be binarized to change the pixels from a scale of, for example, 0 to 255 to a binary scale of, for example, 0 or 1, thereby creating a binarized tooth contour. During the binarization process, the value of each pixel is compared to a threshold. If the value of the pixel is greater than the threshold, it can be assigned a new first value, for example, 1; and if the pixel is less than the threshold, it can be assigned a new second value, for example, 0.

[0121] The binary tooth contour can be thinned to reduce the thickness of the contour to, for example, a single pixel width, thereby forming a thinned contour. The width of the thinned contour can be measured as the shortest distance from a contour pixel adjacent to a non-contour pixel on a first side of the contour to a contour pixel adjacent to a non-contour pixel on a second side of the contour. A single pixel representing the thinned contour at a particular location can be located at the midpoint of the width between the pixel on the first side and the pixel on the second side. After thinning the binary tooth contour, the thinned contour can be a single-width contour at a location corresponding to the midpoint of the binary contour.

[0122] At block 630 , the parameterized 3D tooth and arch model is matched to each of the patient's teeth depicted in the 2D image of the patient. The matching may be based on the edges determined at block 630 , also referred to as contours. Figure 7A An example of the process of matching tooth and arch models to edges is shown. Figure 6 After the teeth and dental arches have been identified and modeled, or as part of such a process, missing or broken teeth can be inserted into the parametric 3D tooth and dental arch model. For example, when a tooth is missing or severely damaged, the parametric model of the tooth can be replaced with an average tooth model to simulate a denture, such as a veneer, crown, or implant. In some embodiments, the missing tooth can be left as a space in the dental arch. In some embodiments, a broken tooth can be left as is without replacing it with an average-shaped tooth.

[0123] During the matching process, case-specific parameters of the parametric arch model are changed and iterated until a match exists between the parametric arch model and the teeth depicted in the 2D image. This match can be determined based on matching the projection of the edges of the parametric model silhouette with the edges of the lips, teeth, and gums identified in the 2D image.

[0124] At block 640, a parametric model of the patient's teeth is rendered. Figure 8 Examples of processes for rendering a parametric model of teeth are described elsewhere herein. During the rendering process, a 3D model of the patient's teeth is formed based on data describing the patient's teeth and dental arch. For example, the 3D model of the patient's teeth is formed based on the parametric 3D model formed at block 630 or described elsewhere herein. In some embodiments, the 3D model is inserted directly into a 2D image of the patient. In these embodiments, the actions in block 640 may be omitted or combined with the actions in block 650, such that, for example, the 3D tooth model is rendered in 2D for insertion into the 2D image.

[0125] Optionally, at box 640, simulated treatments or viewing customization options, such as gum line adjustments, jaw position adjustments, missing tooth insertion, or broken tooth restoration, can be applied to the 3D model before rendering into 2D form. In some embodiments, the edges of the parametric model (e.g., lips, gum line, missing teeth, or broken teeth) can be changed to display simulated results of cosmetic or other treatments and procedures, or to adjust the 2D patient image for customized viewing. For example, a user such as a dental professional or patient can adjust the gum line to simulate gum treatment. As another example, a user can choose to show, hide, or restore missing or broken teeth to simulate a tooth restoration or replacement procedure. In another example, a user can choose to adjust the jaw position in a simulated image before or after treatment to simulate the appearance of an open jaw, closed jaw, or partially open jaw. The jaw position parameters can be defined by the distance between the tooth surfaces in the upper jaw and the tooth surfaces in the lower jaw, for example, the distance between the incisor surface of the upper central incisor relative to the incisor surface of the lower central incisor. The jaw position parameters can be defined and changed by the user. For example, the distance between the incisor surface of the upper central incisor and the incisor surface of the lower central incisor can vary between -5 mm (which means that the lower incisor overlaps the upper incisor by 5 mm) and 10 mm (which means that the gap between the upper incisor and the lower incisor is 10 mm). The gum line can be adjusted by replacing the patient's gum line mask shape with an average gum line mask shape from a historical shape data warehouse. The display of the gum line adjustment can be optionally selected or adjusted by the user. Missing or broken teeth can be replaced with an average tooth shape from a historical shape data warehouse. The display of missing tooth replacements or broken tooth restorations can be optionally selected by the user. For example, a simulated treatment or viewing customization options can be rendered on the screen for viewing by the user, patient, or dental professional.

[0126] At block 650, the 3D dental model is inserted into the 2D image of the patient. The inner lip edge determined at block 620 can be used to define the outline of the patient's mouth in the 2D image. At block 650, the area of ​​the 2D image defined by the open mouth can be removed, and the 3D model can be placed behind the 2D image and within the open mouth. In some embodiments, inserting the 3D dental model into the 2D image also includes rendering an image of the parameterized gum line in the 2D image. For example, the 3D dental model can be inserted into the 2D image as described above. Figure 20 The method 2001 is shown and described for determining a parameterized gum line.

[0127] At block 660, the texture is applied to the teeth. Figure 9An example of applying texture is discussed in more detail. In some embodiments, when applying texture to teeth, a 2D image of the patient is projected onto a 3D model of the teeth, such as, for example, a parametric 3D model of the patient's teeth determined at block 630. When projecting the 2D image into the 3D model, a pixel at each location in the 2D image is assigned to a location on the 3D model. In some embodiments, the pixel values ​​or texture information from the 2D image are processed before being applied to the 3D model. Other techniques such as image inpainting, blurring, image processing filters, pix2pix transformation techniques, etc. may also be used to generate a texture applied to the surface of the 3D model. The projected pixels at each location on the surface of the 3D model form a texture for the model. For example, pixels projected onto a particular tooth of the patient form a texture for the tooth model. This texture can be applied to the 3D tooth model.

[0128] Figure 7A A method 700 of constructing a patient-specific parametric model of a patient's teeth is shown according to some embodiments.

[0129] At block 710, a rough alignment of each corresponding average parameter tooth is aligned with the corresponding center of the tooth in the 2D image. The center of the parameter tooth can be determined based on the center of the projection of the parameter tooth silhouette. The center of the tooth identified in the patient's 2D image can be determined based on the center of the area defined by the tooth edge and the corresponding lip edge and / or gum edge. Prior to the expectation and maximization steps of blocks 720 and 730, respectively, the center of the 2D image tooth and the center of the parameter tooth can be aligned.

[0130] At block 710, method 700 can dynamically generate a parametric tooth model that matches the lip and gum margins to the teeth in the 2D image. Additionally or alternatively, the lip and gum margins can be applied and / or dynamically adjusted at any of blocks 720, 730, and 740. A 3D tooth model can be calculated on the fly to adjust the lip and gum line placement. Using these models provides a parametric model that is significantly more accurate than using the gum line alone.

[0131] When the lip and gum information is applied at block 710, in some embodiments, only the portion of the parametric model beyond the lip edge and / or gum edge is used to determine the regional center of the teeth. Figure 7B , which depicts an example of a tooth model with a gum margin 760 and a tooth model with a lip margin 770. At any of blocks 710, 720, 730, and 740, the position of the gum margin and / or the lip margin may be modified to adjust the fit of the tooth silhouette to the visible portion of the corresponding tooth in the 2D image.

[0132] Adding the lip margin or gum margin of the teeth can significantly improve efficiency and the operation process. As a result, the process matches the tooth model to the 2D image much more realistically than a process that does not apply the lip margin and gum margin to the tooth model, and the process uses a wider variety of photos.

[0133] Return to Figure 7A , at block 720, the expectation step is performed. In some embodiments, an expectation management (EM) engine and / or an engine configured to create a 3D model performs block 720. At the expectation step, a silhouette of the tooth is projected onto a 2D image, and the edge of the silhouette is evaluated against the edge of the tooth in the 2D image determined based on the lip edge, the gum edge, and the tooth edge. The evaluation can be to determine the normal of the position at the edge of the silhouette and the nearest position in the 2D image with a similar normal. The probability that the two edges are the same is then determined. This process can be repeated for each position at the edge of the silhouette.

[0134] At block 730, the EM engine's maximization step is executed. In the maximization step, a small-angle approximation is used to provide a maximized analytical solution. Compared to other methods, such as the Gauss-Newton iteration method, the small-angle approximation and the analytical solution provide an improved solution. The small-angle approximation significantly reduces computation time and yields an exact solution more quickly.

[0135] Blocks 720 and 730 may be iteratively performed for a single parameter or a subset of parameters, performing the desired step, then performing the maximize step, then returning to the desired step, and so on, until a threshold is reached that converges the single parameter or the subset of parameters. The process may then proceed to block 740.

[0136] At block 740, the optimized parameters or subset of parameters are added to the parametric model. For example, the optimization of the parametric model may start with Φ and T, and then after looping through EM blocks 720 and 730, additional parameters may be added. For example, T may be added. τ , and then optimized by EM blocks 720 and 730 for additional iterations. The number of iterations before adding additional parameters can be changed. In some embodiments, EM blocks 720 and 730 can be looped through 3 times, 5 times, 7 times, 10 times, 15 times, 20 times, or any number of times (e.g., any integer). Finally, The updated parametric model is added to the parameter model, which is processed through EM blocks 720 and 730 until convergence is achieved. During this process, outliers can be identified and filtered out. In some embodiments, after block 740, the process 700 can loop back to block 710, where a coarse alignment process is performed based on the updated parametric model, rather than looping back to block 720 and proceeding directly to the desired step.

[0137] At block 750, the parameters of the parametric model are exported to another engine, or even to a data warehouse, for later retrieval. Figure 8 As described elsewhere herein, the rendering engine can retrieve parameters of the parametric model for use in rendering.

[0138] Figure 8 Depicted is a method 800 of rendering a patient's teeth in an initial position using a parametric model of the patient's dental arch, according to one or more embodiments herein.

[0139] At block 810, the average shape of each tooth is determined. In some embodiments, as discussed herein, the average shape can be based on a set of average shapes of dental arches, for example, obtained from historical cases and / or cases representing ideal dental arch morphology. In some embodiments, the average shape can be rendered on a screen for viewing by a patient or dental professional, for example. In some embodiments, the average shape can be loaded into or retrieved from memory. The average shape can also be initialized as a set of matrices, one for each tooth.

[0140] At block 820, the average shape of the teeth Perform principal component analysis shape adjustment. For each tooth in the model, the case-specific coefficients of the principal components Apply to principal components After the shape adjustment is completed, in some embodiments, the adjusted shape can be rendered on a screen for viewing by the patient or dental professional, for example. For example, the adjusted tooth shape can be rendered for viewing. In some embodiments, the adjusted shape can be stored in a memory. Alternatively, the adjusted shape can be stored as a set of matrices, with each matrix corresponding to a tooth.

[0141] At block 830, an average tooth pose is determined. In some embodiments, the average tooth pose can be based on an average tooth pose of a set of scanned dental arches, as discussed herein. In some embodiments, at block 830, each adjusted tooth is placed at its corresponding average position and orientation determined by the average dental arch. In some embodiments, the average tooth pose can be rendered on a screen, for example, for viewing by a patient or dental professional. In some embodiments, the average tooth pose can be loaded into or retrieved from a memory. The average tooth pose can also be initialized as a set of matrices and / or other data structures, for example, each matrix and / or other data structure corresponding to a tooth in a dental pose. In some embodiments, the average tooth shapes from block 810 can be placed at their corresponding average tooth pose at block 830 before the shapes of the teeth are adjusted at block 820. In other words, the order of blocks 820 and 830 can be swapped.

[0142] At block 840, a tooth posture adjustment is performed based on the average tooth posture. In some embodiments, as discussed above, the posture adjustment T τ Based on the specific tooth posture of the patient's dental arch. In some embodiments, as discussed herein, at box 840, the position and orientation of each tooth is adjusted so that it is placed in a position and orientation determined by the position and orientation of the teeth in the patient's dental arch or determined in other ways. In some embodiments, for example, the adjusted tooth posture can be rendered on a screen for viewing by the patient or dental professional. In some embodiments, the adjusted tooth posture can be stored in a memory. The adjusted tooth posture can also be stored as a set of matrices and / or other data structures, for example, each matrix and / or data structure corresponding to a tooth in a tooth posture. In some embodiments, before adjusting the shape of the teeth at box 820, the average tooth shape from box 810 can be placed at box 840 with its corresponding adjusted tooth posture. In other words, the order of box 820 and boxes 830 and 840 can be swapped so that box 820 occurs after boxes 830 and 840.

[0143] At block 850, the dental arch is scaled so that the generated dental arch has tooth dimensions based on the patient's teeth and arch dimensions. In some embodiments, as discussed above, the dental arch scaling factor Φ is based on specific teeth and arches specific to the patient. In some embodiments, at block 850, the size of the dental arch is adjusted, as discussed herein, so that the scaled dental arch matches the dimensions of the patient's dental arch, the dimensions of the patient's dental arch being determined, for example, by the dimensions of the teeth and arches in the patient's dental arch or otherwise. In some embodiments, for example, the scaled dental arch can be rendered on a screen for viewing by the patient or a dental professional. In some embodiments, the scaled dental arch can be stored in a memory. The scaled dental arch can also be stored as a set of matrices and / or data structures, for example, each matrix and / or data structure corresponding to a tooth in a dental pose. In some embodiments, before adjusting the shape of the teeth at block 820, the average tooth shapes from block 810 can be placed at their corresponding scaled positions and sizes at block 850. In other words, the order of block 820 and blocks 830 , 840 , and 850 may be swapped such that block 820 occurs after blocks 830 , 840 , and 850 .

[0144] You can follow Figure 8 The blocks of method 800 may be performed in an order other than the order shown. For example, blocks 810 and 830 may be performed before blocks 820, 840, and 850. In some embodiments, blocks 810, 820, 830, and 850 may be performed before block 840. These and other modifications may be made to the order of the blocks in method 800 without departing from the spirit of the present disclosure.

[0145] Figure 9 A method 900 for constructing a 3D model and applying texture to the 3D model according to one or more embodiments herein is depicted. Texture can help provide details, such as color information, to the 3D model of a patient's teeth. As described herein, process 900 uses an image of a patient's teeth to provide realistic textures to the patient's teeth.

[0146] Thus, as described elsewhere herein, e.g., in conjunction with Figure 5 In related discussion, a 3D model of the patient's teeth and a 2D image of the patient are acquired at block 910. The 2D image and the 3D model should depict the teeth in the same position, for example, the 3D model can be a parametric model derived from the 2D image of the patient.

[0147] At block 920, a 2D image of the patient's teeth is projected onto the 3D model and the image is aligned with the model. This alignment can be performed by matching the contours in the 3D model with the contours in the 2D image. As described elsewhere herein, the contours can include the determined lip margins, gum margins, and tooth margins.

[0148] At block 930, color information from the 2D image is mapped as a texture to the 3D model. The color information may include lighting conditions such as specular highlights, accurate tooth coloring and texture, and other tooth features such as veneers, gold, etc. Once the texture is mapped to the 3D model, the teeth in the 3D model may be repositioned, for example, to depict an estimated and / or expected outcome of a dental treatment plan (e.g., an orthodontic treatment plan, a restorative treatment plan, some combination thereof, etc.). As described in reference Figure 10A 、 Figure 10B And / or as described elsewhere herein, such a final position includes both accurate color and tooth feature information and accurate 3D positioning of the patient's teeth, and can, for example, be used to simulate an estimated and / or expected outcome of a dental treatment plan (e.g., a final orthodontic position) for the patient's teeth.

[0149] In some embodiments, the texture model is adjusted to simulate a clinical treatment or aesthetic treatment. For example, the color can be adjusted to simulate a teeth whitening procedure. Simulating these treatments can include generating a mask from a 2D projection of a 3D tooth model, wherein the model is in an arrangement that is inserted into the 2D image, such as in an initial position or a final position. The mask can be applied to the 2D image of the patient's teeth in the initial position or the final position. Color adjustments or whitening can also be applied to the masked area. The color adjustment and whitening parameters can be optionally selected and adjusted by the user. For example, the color adjustment and whitening can be rendered on a screen for viewing by the user, patient, or dental professional.

[0150] Figure 10A A method 1000 is shown for simulating estimated and / or expected results of a dental treatment plan for a patient's teeth, according to one or more embodiments herein.

[0151] At block 1010, a model of an average dental arch is constructed. In some embodiments, the model is a parametric 3D model of an average dental arch based on a set of scanned historical dental arches and / or dental arches that represent an ideal dental arch morphology. In some embodiments, the scans are from the initial position of the patient's teeth. In some embodiments, the scans are from the patient after the patient has completed orthodontic treatment. In other embodiments, the scans are taken without regard to whether the patient has already received orthodontic treatment. In some embodiments, the historical cases and / or ideal cases can represent dental arch models of treated patients (e.g., patients who have been treated in the past) and / or ideal dental arch models that represent expected results of various forms of orthodontic treatment. Block 1010 can include a reference Figure 3B Described process 350. At block 1010, cases are acquired. These cases may be retrieved from a data repository and may include previously scanned and segmented dental arch models.

[0152] The arches can also be aligned. Each arch in the data warehouse can be aligned in multiple locations. For example, each arch can be aligned in three locations: between the central incisors, and at each distal end of the left and right sides of each arch. For example, Figure 4 A set of dental arches 400 are shown aligned at a location between the central incisors 410, at a left distal end 430 of the dental arch, and at a right distal end 420 of the dental arch.

[0153] In some embodiments, determining the average dental arch model and determining the dental arch model distribution include performing sub-steps. For example, each historical dental arch may be rescaled to determine Φ, and then averaged across each dental arch to determine Then determine the local deformation of each tooth and Compare to determine T τ , then after aligning each tooth, determine β τ .

[0154] The distribution estimates of the case-specific parameters are also determined. For example, the relative shape, position, and rotation of each tooth are determined to construct the distribution of each case-specific parameter. For example, for the surface model of each corresponding tooth in all retrieved models, the coefficients of the principal components are determined. distribution.

[0155] The positions and orientations of each tooth can be averaged to determine the average position of each tooth, and the orientations of each tooth can be averaged to determine the average orientation of each tooth. The average positions and average orientations are used to determine

[0156] Finally, the mean of the distribution of estimates for the case-specific parameters can be used as the universal parameters for mean tooth position and mean tooth shape.

[0157] At block 1020, a 2D image of the patient is captured. In some embodiments, the 2D image includes the patient's mouth and one or more images of the patient's face, head, neck, shoulders, torso, or the entire patient. The 2D image of the patient may include an image of the patient with the patient's mouth in one or more positions. For example, the patient's mouth may be in a smiling position (e.g., a social smile position), a resting position with muscles relaxed and lips slightly open, or a retracted anterior open bite or anterior closed bite position.

[0158] In some embodiments, an image of the patient is acquired, for example, by capturing the image of the patient using an image capture system. The image can be captured using a lens having a predetermined focal length at a distance from the patient. In some embodiments, the image of the patient is acquired from a computer storage device, a network location, a social media account, or the like. The image can also be captured remotely and then received for processing. The image can be a series of images from a video captured from one or more perspectives. For example, the images can include one or more of a frontal facial image and a profile image, the profile image including one or more three-quarter profile images and a full profile image.

[0159] At block 1030, a 3D model of the patient's teeth is constructed from the 2D image of the patient's teeth. Figure 6 A parametric model is constructed using the process described herein, wherein edges of features of the patient's oral cavity are determined. For example, edges of one or more of the patient's teeth, lips, and gums can be determined. The initial determination of the patient's lip (e.g., the inner edges of the lips defining an open mouth), teeth, and gum contours can be identified using a machine learning algorithm such as a convolutional neural network.

[0160] Then, an initial contour can be extracted from the image. The pixels representing the contour can then be binarized. The binarized tooth contour is thinned to reduce the thickness of the contour to, for example, a single pixel width to form a thinned contour.

[0161] Using the thinned or otherwise processed contours, the parametric 3D tooth and arch model is matched to each of the patient's teeth depicted in the 2D image of the patient. This matching is based on the contours determined above. After the teeth and arches are identified and modeled, or as part of this process, missing or broken teeth can be inserted into the parametric 3D tooth and arch model.

[0162] At block 1040, case-specific parameters are estimated. In some embodiments, case-specific parameters may be determined at block 1030 based on, for example, Figure 6During the matching process or at block 1040, case-specific parameters of the parametric dental arch model are varied and cycled through until a match is found between the parametric dental arch model and the teeth depicted in the 2D image. As described herein, such a match can be determined based on matching the projection of the edges of the silhouette of the parametric model with the edges of the lips, teeth, and gums identified in the 2D image.

[0163] At block 1050 , the patient's teeth are rendered according to simulation results of a dental treatment plan using the patient's case-specific parameters for tooth shape or dental arch scaling and using average values ​​for tooth positions. Figure 11 An example of a method of rendering teeth according to a dental treatment plan and / or an expected outcome of the dental treatment plan using appropriate case-specific and average parameter values ​​is shown.

[0164] Optionally, at block 1050, other changes such as gum line adjustments, jaw position adjustments, missing tooth replacements, or broken tooth restorations may be applied to the 3D model before rendering into 2D form. In some embodiments, the edge features and other features of the model, such as lips, gum line, and missing or broken teeth, may be altered to display a simulated result of a treatment procedure, or to adjust the 2D image of the patient for customized viewing. For example, a user may adjust the gum line to simulate a gum treatment. In another example, a user may choose to show, hide, or restore a missing or broken tooth to simulate a tooth restoration or replacement procedure. In this case, the missing or broken tooth may be replaced with an ideal tooth based on the ideal parameters discussed above. The missing or broken tooth may be replaced based on a tooth from a historical data repository, or with the patient's own tooth, for example, a corresponding tooth from the opposite side of the patient's dental arch may be used. For example, if the upper left canine is missing or broken, a mirrored model of the patient's upper right canine may be used in its place. In another example, the user may choose to adjust the jaw position in a pre-treatment or post-treatment simulated image to simulate an open, closed, or partially open jaw appearance. The jaw position parameters may be defined by the distance between the tooth surface in the upper jaw and the tooth surface in the lower jaw, for example, the distance between the incisor surface of the upper central incisor relative to the incisor surface of the lower central incisor. The jaw position parameters may be defined and changed by the user. The gum line may be adjusted by replacing the patient's gum line mask shape with an average gum line mask shape from a historical average shape database. The display of the gum line adjustment may be optionally selected or adjusted by the user. Missing or broken teeth may be replaced with an average tooth shape from a historical average shape database. The display of missing tooth replacements or broken tooth restorations may be optionally selected by the user. For example, a simulated treatment or viewing customization options may be rendered on a screen for viewing by the user, patient, or dental professional.

[0165] At block 1060, the 3D model is inserted into the 2D image of the patient. For example, as part of the model construction process, a 3D rendering of the teeth may be placed in the patient's mouth as defined by the lip contour determined at block 1030. In some embodiments, inserting the 3D tooth model into the 2D image further includes rendering an image of the parameterized gum line in the 2D image. For example, the 3D rendering may be as described with respect to Figure 20The parameterized gum line is determined as shown and described in method 2001. In some embodiments, in block 1060, the texture model is adjusted to simulate a clinical aesthetic treatment. For example, color can be adjusted to simulate a teeth whitening procedure. A mask can be generated from a 2D projection of the 3D tooth model. The mask can be applied to the 2D image of the patient's teeth in an initial position or a final position. Color adjustment or whitening can also be applied to the masked area. Color adjustment and whitening parameters can be optionally selected and adjusted by the user.

[0166] The user can select, deselect or adjust the simulated treatment performed at box 1050 or box 1060 or view custom options on the projected 2D image. Parameters can be adjusted to simulate treatments such as whitening, gum line treatment, tooth replacement or tooth restoration, or to view custom options, such as displaying an open jaw, closed jaw or partially open jaw. In one example, the user can optionally adjust the color parameters to simulate tooth whitening. The user can adjust the color parameters to increase or decrease the degree of tooth whitening. In another example, the user can optionally adjust the gum line parameters to simulate gum line treatment or change for oral hygiene. The user can adjust the gum line parameters to raise the gum line (reduce the amount of exposure of the tooth surface), thereby simulating gum restoration due to, for example, improved oral hygiene or gum line treatment, or to lower the gum line (increase the amount of exposure of the tooth surface), thereby simulating gum recession due to, for example, poor oral hygiene. In another example, the user can optionally select tooth replacement parameters. The user can select tooth replacement parameters to replace one or more missing, broken, or damaged teeth with ideal teeth (e.g., teeth from a historical data warehouse or the patient's own teeth). In another example, the user can optionally adjust jaw line parameters for a customized view of the jaw position. The user can increase the distance between the teeth in the upper jaw relative to the teeth in the lower jaw to simulate an open bite, or decrease the distance between the teeth in the upper jaw relative to the teeth in the lower jaw to simulate a closed bite. In this case, the user can simulate the appearance of orthodontic treatment, aesthetic treatment, or clinical treatment in various jaw positions.

[0167] Figure 10B Depicted is a process 1005 for simulating a final treatment position of a patient's teeth using matching dental arches identified from a parametric search of a treatment plan data repository.

[0168] As in Figure 10A As described at block 1020 of FIG. , at block 1015, a 2D image of the patient is captured, and as in FIG. Figure 10A At block 1030 of FIGURE 1, a 3D model of the patient's teeth is constructed from the 2D image of the patient's teeth at block 1025. Figure 10A At block 1040 , at block 1035 , case-specific parameters are estimated.

[0169] At block 1045, the tooth shape parameters of the patient's teeth from the 3D model are used to perform a parametric search of the treatment planning data repository. When matching the shapes of the patient's teeth to the shapes of historically treated teeth in the data repository, the shapes of the teeth in the patient's parametric 3D model are compared to the parametric shapes of the teeth in each historical record in the treatment data repository. When a match between the tooth shapes is found, the final tooth position and orientation or the final dental arch model of the matching record can be used to simulate the treatment plan. A parametric template dental arch can be identified based on the comparison of the tooth shape parameters, thereby identifying a matching template dental arch. In some embodiments, for example, the match can be based on the closest matching template dental arch. In some embodiments, the template shapes can be loaded into memory or retrieved from memory. The template shapes can also be initialized as a set of matrices, each corresponding to a tooth. Once a match is found in a record within the treatment data repository, the final dental arch model is retrieved from the matching record and can be used as the basis for the patient's target final tooth position.

[0170] In some embodiments, the tooth positions and orientations in the matching record are used as the basis for the tooth positions and orientations of the patient's target final tooth positions. When using the tooth positions and orientations, the patient's parameterized dental arch model can be modified with the tooth positions and orientations in the matching record, or the matching record can be updated with the shapes of the patient's teeth. In some embodiments, a model of each of the patient's teeth with unchanged tooth shape is placed in the final tooth pose determined from the matching record. In some embodiments, a model of each of the patient's teeth with an adjusted shape (e.g., a tooth shape from the matching record) is placed in the final tooth pose determined from the matching record. Optionally, a tooth pose adjustment can be performed to adjust the position of the teeth in the final tooth pose.

[0171] In some embodiments, the final dental arch model from the matching record is used as the final position for the simulation of the patient's treatment in the 2D rendering. In some embodiments, the teeth of the final dental arch model from the matching record are positioned based on the positions of the teeth in the patient's 2D photograph. Alternatively, the teeth of the final dental arch model are positioned based on the positions of the teeth in the patient's parametric 3D model. Optionally, the final dental arch model, with its position adjusted based on the patient's 2D photograph or 3D model, can be used as the simulation position in the 2D rendering.

[0172] The shape of the template teeth can be adjusted based on the patient's parameterized 3D model to more closely match the patient's tooth shape. Optionally, principal component analysis shape adjustment can be performed on the template tooth shape. After the shape adjustment is completed, in some embodiments, for example, the adjusted shape can be rendered on a screen for viewing by a user, patient, or dental professional. In some embodiments, the adjusted shape can be stored in a memory. Alternatively, the adjusted shape can be stored as a set of matrices, each matrix corresponding to a tooth. In some embodiments, the patient's teeth are used to replace the teeth in the matching template from the treatment data repository.

[0173] The template dental arch can be scaled so that the generated dental arch is based on the patient's teeth and dental arch dimensions. In some embodiments, as discussed above, the template dental arch scaling factor Φ is based on specific teeth and dental arches specific to the patient. In some embodiments, for example, when the 3D model is scaled for integration into a 2D image, the template dental arch scaling factor Φ can also be based on one or more of the image dimensions of the patient's 2D image. In some embodiments, as discussed herein, the dimensions of each template tooth and template dental arch are adjusted so that the scaled template dental arch matches the dimensions of the patient's dental arch, the dimensions of the patient's dental arch being determined, for example, by the dimensions of the teeth and dental arch in the scanned patient's dental arch or otherwise. In some embodiments, for example, the scaled template dental arch can be rendered on a screen for viewing by a user, patient, or dental professional. In some embodiments, the scaled template dental arch can be stored in a memory. The scaled template dental arch can also be stored as a set of matrices, each matrix corresponding to a tooth in a dental pose. In some embodiments, the ideal template tooth shape can be placed in its corresponding scaled position and size before the shape of the tooth is adjusted.

[0174] At block 1045, simulation treatment or viewing customization options such as gum line adjustment, jaw position, missing tooth implantation or fractured tooth restoration may be applied to the 3D model before rendering into 2D form. As described in block 640, simulation treatment or viewing customization option changes may be applied and adjusted.

[0175] At block 1055, the 3D model is inserted into the 2D image of the patient. For example, a 3D rendering of the teeth may be placed in the patient's mouth defined by the lip contours determined as part of the model building process at block 1025. The rendering may be performed at block 1045 with or without enabling a simulated treatment or viewing customization options. In some embodiments, inserting the 3D tooth model into the 2D image also includes rendering an image of the parameterized gum line in the 2D image. For example, the 3D rendering may be performed as described with respect to Figure 20The parameterized gum line is determined as shown and described in method 2001. In some embodiments, at block 1055, the texture model is adjusted to simulate a clinical treatment. For example, color can be adjusted to simulate a teeth whitening procedure. A mask can be generated from a 2D projection of the 3D tooth model. The mask can be applied to the 2D image of the patient's teeth in an initial position or a final position. Color adjustment or whitening can also be applied to the masked area. Color adjustment and whitening parameters can be optionally selected and adjusted by the user.

[0176] The simulated treatments performed at block 1045 or block 1055 and the viewing customization options may be selected, deselected, or adjusted by the user on the projected 2D image. Parameters may be adjusted to simulate treatments such as whitening, gum line treatment, tooth replacement, or tooth restoration, or for viewing customization options such as displaying an open, closed, or partially open bite.

[0177] Figure 11 An example of a method 1100 for rendering teeth based on estimated and / or expected results of a dental treatment plan based on a parametric model of the patient's dental arch using patient-derived values ​​for case-specific parameters of scale and tooth shape, and using average values ​​for tooth positions (e.g., average values ​​derived from historical and / or ideal arches).

[0178] At block 1110, the average shape of each tooth is determined In some embodiments, as discussed herein, the average shape can be based on the average shape of a set of scanned dental arches. In some embodiments, the average shape can be rendered on a screen, for example, for viewing by a patient or dental professional. In some embodiments, the average shape can be loaded into or retrieved from memory. The average shape can also be initialized as a set of matrices, one for each tooth.

[0179] At block 1120, the average shape of the teeth Perform principal component analysis shape adjustment. For each tooth in the model, the case-specific coefficients of the principal components Apply to principal components After the shape adjustment is completed, in some embodiments, the adjusted shape can be rendered on a screen for viewing by a patient or dental professional, for example. In some embodiments, the adjusted shape can be stored in a memory. Alternatively, the adjusted shape can be stored as a set of matrices and / or data structures, for example, each matrix and / or data structure corresponding to a tooth.

[0180] At box 1130, an average tooth pose is determined. In some embodiments, as discussed herein, the average tooth pose can be based on the average tooth pose of a set of scanned dental arches. In some embodiments, at box 1130, each adjusted tooth is placed in its corresponding average position and orientation determined by the average dental arch. In some embodiments, for example, the average tooth pose can be rendered on a screen for viewing by a patient or dental professional. In some embodiments, the average tooth pose can be loaded into or retrieved from a memory. The average tooth pose can also be initialized as a set of matrices, each matrix corresponding to a tooth in a dental pose. In some embodiments, before the shape of the tooth is adjusted at box 1120, the average tooth shape from box 1110 can be placed in its corresponding average tooth pose at box 1130. In other words, the order of boxes 1120 and box 1130 can be swapped.

[0181] At box 1140, the dental arch is scaled so that the generated dental arch has tooth dimensions based on the patient's teeth and dental arch dimensions. In some embodiments, as discussed above, the dental arch scaling factor Φ is based on the specific teeth and dental arch of a specific patient. In some embodiments, as discussed herein, at box 1140, the dental arch is adjusted so that the scaled dental arch matches the dimensions of the patient's dental arch, the dimensions of the patient's dental arch being determined, for example, by the dimensions of the teeth and dental arch in the scanned patient's dental arch or otherwise. In some embodiments, for example, the scaled dental arch can be rendered on a screen for viewing by a patient or dental professional. In some embodiments, the scaled dental arch can be stored in a memory. The scaled dental arch can also be stored as a set of matrices, each matrix corresponding to a tooth in a dental pose. In some embodiments, the average tooth shape from box 1110 can be placed in its corresponding scaled position and size at box 1140 before the shape of the tooth is adjusted at box 1120. In other words, the order of block 1120 and blocks 1130 and 1140 may be swapped such that block 1120 occurs after blocks 1130 and 1140 .

[0182] Simulation treatment or viewing customization options, such as gum line adjustment, jaw position, missing tooth insertion, or fractured tooth restoration, may be applied to the 3D model prior to rendering at block 1150. Simulation treatment or viewing customization options may be applied and adjusted as described in block 640. Simulation treatment or viewing customization options may be applied before or after arch scaling (e.g., before or after block 1140).

[0183] At block 1150, the 3D model of the patient's teeth determined at block 1140 is rendered for viewing, or the final 3D model is stored for later on-screen rendering. The 3D model may be rendered for viewing, or stored for later on-screen rendering, with or without performing a simulated treatment or viewing custom options.

[0184] At block 1160, the texture is applied to the 3D model of the patient's teeth at the estimated final position and / or the expected final position. Figure 9 The texture determined based on the 2D image of the patient's teeth in process 900 can be applied to the 3D model. In some embodiments, the texture application process of block 1160 can occur during or as part of the rendering process of block 1150. The texture can be applied to a mask area, where the mask area can define an area corresponding to an oral feature (e.g., lips, gums, or teeth). For example, the texture can be applied to a 3D model by, for example, Figure 20 The mask area or portion thereof is determined using the simulated gingiva shown and described in method 2001 of FIG. During texture application, either at block 1160 or during the rendering process at block 1150, the texture model can be adjusted to simulate treatment or for customizable viewing. Simulated treatment or viewing customization options, including color adjustments to simulate teeth whitening, for example, can be applied to the 3D model or can occur as part of the rendering process. Alternatively, color adjustments can be made as described at block 1060. Simulated treatment and viewing customization options, color adjustments, can be adjusted by the user.

[0185] Alternatively, step 1110 may be replaced with a parametric search of the treatment planning data repository to identify a matching template model based on the patient's tooth shape. As described for using template dental arch and tooth shapes instead of average dental arch or tooth shapes, steps 1120 through 1160 may then be performed to determine an ideal tooth pose, rather than an average tooth pose.

[0186] Notice Figure 20 , which depicts a method 2001 for simulating a gum line using a combination of patient-specific input parameters and simulation input parameters. At block 2011, one or more 2D images of a patient are captured or retrieved. Figure 1 Capturing 2D images as described at block 110 of the present invention. Capturing one or more 2D images of the patient. The 2D images may depict, for example, the patient's mouth in a smiling position. The 2D images may also be used as Figure 6 Block 610, Figure 10A Box 1020 or Figure 10BFor example, the 2D image captured at block 1020 may be retrieved at block 2011 for use in generating the gingiva for use in the 2D image generated at block 1050 or block 1060.

[0187] At block 2021, the edges of the patient's oral features (e.g., teeth, gums, and lips) are determined from a 2D image of the patient's mouth. Figure 6 The edges of oral features (e.g., teeth, lips, and gums) are identified as described at block 620 of FIG. Facial landmarks from the 2D image of the patient can be used to identify a mouth opening within which oral features (including teeth and gums) are located.

[0188] An initial determination of the edges of oral features may be identified by a machine learning algorithm. An initial tooth outline is extracted from the image and a brightness or other scale may be applied to it. The initial outline may then be subjected to binarization to change the pixels of the image to a binary scale. The binarized tooth outline is thinned to, for example, a single pixel width, thereby forming a thinned tooth outline. An initial determination of the patient's lips may be based on facial feature points, such as lip feature points determined according to a machine learning algorithm (e.g., a convolutional neural network). As with the tooth outline, a brightness or other scale is also applied to the lip outline. As with the initial tooth outline, the lip outline is binarized, thinned, and contoured. The binarized and thinned tooth and lip outlines may define the edges of oral features. For example, the image may be imaged as described above with respect to Figure 6 Determine the edges as described.

[0189] At block 2031, patient-specific input parameters for the gums are determined. The input parameters may include Figure 21A In some embodiments, the patient-specific input parameters may be determined by the shape and scale of one or more oral features of the captured 2D image of the patient. The patient-specific input parameters may be determined based on the edges of the oral features, for example, Figure 21B 2100. The edges of the oral features may include, for example, the lip edge 2110, the tooth edge 2120, and the gum edge 2130 determined at block 2021. The patient-specific input parameters may include the patient's tooth height, which may be the distance between the incisaledge 2120 of the tooth and the edge 2130 of the gum or gum line, and may be determined for each tooth of the patient's dental arch. For example, Figure 21A As shown in , the patient's tooth height can be determined by h gIn some embodiments, the patient's tooth height can be the maximum distance between the incisal edge 2120 of the tooth and the edge 2130 of the gum. In some embodiments, the maximum distance can be measured as the distance between the incisal edge 2120 of the tooth and the edge 2130 of the gum at the highest point of the gum.

[0190] In some embodiments, the patient-specific input parameters may include the gum tip distance, which may be the distance between the edge of the lip 2110 and the gum line 2140. This distance may be determined at a location between each pair of adjacent teeth in the patient's dental arch. Figure 21A As shown, the gingival tip distance can be expressed by h b In some embodiments, the patient's tooth height can be the maximum distance between the edge of the lips 2110 and the edge of the gums 2140. The maximum value can be measured as the distance between the lips and the edge of the gums 2140 at the lowest point of the gums. The patient-specific input parameters can include the gum height, which can be the distance between the edge of the lips 2110 and the gum line 2130 determined for each tooth of the patient's dental arch, for example, Figure 21A As shown, the gingival height can be expressed by h t In some embodiments, the gum height can be the minimum distance between the edge of the lip 2110 and the highest point of the gum 2130. The patient-specific input parameters can include a tooth width determined for each tooth of the patient's dental arch. For example, Figure 21A As shown, the tooth width can be represented by w. In some embodiments, the tooth width can be the width of the tooth at the incisal edge 2120. In some embodiments, the tooth width can be the width of the tooth at the maximum tooth width.

[0191] At block 2041, simulation parameters of the gingiva are determined. The simulation parameters may be derived based on the patient-specific input parameters and may be used in generating the simulated gingiva. In some embodiments, the parameters of the gingival tip represented by h, the visible tooth height represented by g, and the tooth curvature represented by R may be a convolution of the patient-specific input parameters, such as Figure 22 For example, they can be a convolution of patient-specific parameters and simulation input parameters or ideal aesthetic parameters. "Ideal teeth" can include a representation of teeth or parameters taken from a model dental arch (e.g., a model dental arch of a historical case and / or a model dental arch representing an ideal dental arch form). Ideal teeth can be characterized by various parameters, including but not limited to: gum tip parameters, ideal tooth height, tooth thickness, distance from gum line to lip, or visible tooth height. Patient-specific parameters can be, for example, parameters of the following as discussed above: g The height of the patient's teeth is represented by h b The distance to the gingival tip is represented by ht The gingival height represented by H and the tooth width represented by w. The simulation input parameters or ideal aesthetic parameters can be, for example, the following parameters: g The ideal tooth height represented by , the customizable tooth height scaling parameters represented by c1 and c2, the customizable gum tip scaling parameter represented by s, and the tooth shape scaling parameter represented by m. The visible tooth height represented by g can be the visible tooth height of the patient's teeth determined based on the 2D image (represented by h g ) and ideal tooth height (denoted by H g The visible tooth height represented by g can be determined based on the weighted average, for example, g=c1H g +c2h g , where c1 and c2 are customizable tooth height scaling parameters, and where c1 + c2 = 1. For example, c1 can be 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, or 0.9. For example, c1 can be 0.1 to 0.5, 0.2 to 0.6, 0.3 to 0.7, 0.4 to 0.8, or 0.5 to 0.9. In some embodiments, c1 is adjusted by a user, such as a dental professional or patient.

[0192] In some embodiments, the visible tooth height represented by g represents the visible tooth height of the patient's teeth determined based on the 2D image (represented by h g The gingival tip parameter h of each tooth can be determined based on patient-specific input parameters, for example, where h = h b –h t The gum tip parameter describes the shape of the gums around the teeth. A gum tip parameter of 0 describes a gum line that is a straight line across the teeth, and a positive gum tip parameter describes a gum line that is a curved shape, where the distance between the lips and the gum line is greater between the teeth than at the midline of the teeth. In some embodiments, the gum tip parameter h may be based on the value given by h g The patient-specific tooth height, represented by H g The difference between the ideal tooth height represented by and the customizable gingival tip scaling parameter represented by s is determined, for example, where h = s(H g –h g )(h b –h t ), where s is determined by H g and h g For example, when the difference is large, it is likely to be a rectangular tooth, and s takes a larger value; when the difference is small, s takes a relatively small value. In some embodiments, it can be based on the ideal tooth length H g The length of the patient's tooth h g. For example, s can be 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, or 2.0. For example, s can vary from 0.8 to 1.0, from 0.9 to 1.1, from 1.0 to 1.2, from 1.1 to 1.3, from 1.2 to 1.4, from 1.3 to 1.5, from 1.4 to 1.6, from 1.5 to 1.7, from 1.6 to 1.8, from 1.7 to 1.9, from 1.8 to 2.0, or from 0.8 to 2.0. In some embodiments, s is adjusted by a user, such as a dental professional or patient.

[0193] For each tooth, the maximum threshold of the gingival tip parameter represented by h can be set to g The part of the patient's tooth height that is represented by the gingival tip parameter h can be set to no more than 0.2h. g , 0.3h g , 0.4h g , 0.5h g , 0.6h g , 0.7h g , 0.75h g , 0.8h g , 0.85h g or 0.9h g The maximum threshold for the gingival tip parameter h can be set as a fraction of the visible tooth height represented by g. For example, h may not exceed 0.2g, 0.3g, 0.4g, 0.5g, 0.6g, 0.7g, 0.75g, 0.8g, 0.85g, or 0.9g.

[0194] The tooth curvature parameter represented by R can be expressed by h t The tooth curvature parameter R may be a function of the patient-specific gum width parameter denoted by , the tooth thickness denoted by T, and the tooth shape scaling parameter denoted by m. In some embodiments, the tooth curvature parameter R may be given by R=mh t In some embodiments, the tooth thickness represented by T can be determined based on a parameterized 3D model of the patient's teeth, for example, where T is the distance between the lingual and buccal surfaces of the tooth at the midpoint of the gum line. g m can be adjusted based on the length of the patient's teeth as indicated by the tooth length. For example, m can be 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, or 1.0. For example, m can vary from 0.1 to 0.3, from 0.2 to 0.4, from 0.3 to 0.5, from 0.4 to 0.6, from 0.5 to 0.7, from 0.6 to 0.8, from 0.7 to 0.9, or from 0.8 to 1.0. In some embodiments, m is adjusted by a user, such as a dental professional or patient.

[0195] In some embodiments, input parameters (e.g., parameters represented by h, g, and R) are used to determine a parameterized gum line. Patient-specific input parameters, convolution of patient-specific input parameters, and simulated input parameters, or any combination thereof, can be used to determine the gum line. In some embodiments, determining the parameterized gum line includes approximating a parametric 3D model of a tooth from a plurality of teeth of the patient as a cylinder. The cylinder can be cross-sectioned by an inclined plane, wherein the orientation and position of the inclined plane are determined based on the input parameters such that an intersection line between the inclined plane and the cylinder passes through coordinates defined by h, g, and R. The intersection line between the inclined plane and the cylinder can be used to define an edge of the gum. In some embodiments, determining the parameterized gum line includes cross-sectioning the 3D model of the patient's teeth by an inclined plane, wherein the orientation and position of the inclined plane are determined based on the input parameters such that an intersection line between the inclined plane and the cylinder passes through coordinates defined by h, g, and R. The intersection line between the inclined plane and the 3D model of the patient's teeth can be used to define an edge of the gum. Points on the gingival side of the intersection line between the inclined plane and the cylinder or the 3D model of the patient's teeth are rendered as gingiva, while points on the incisal side of the intersection line between the inclined plane and the cylinder are rendered as teeth. In some embodiments, the points are a 3D point cloud of points. In some embodiments, the points are pixels of a 2D image.

[0196] At block 2051, the gum line is optionally leveled. The gum line that intersects the corresponding coordinates of the gum line of each tooth can be adjusted by fitting to a quadratic polynomial, such as Figure 23 The gum line can be adjusted so that the corresponding coordinates of the gum line are located along the line defined by the quadratic polynomial fit. In some embodiments, the corresponding coordinates of each tooth are the endpoints of the gum curve or Figure 21B The highest point of the gum 2130 in the tooth (i.e., the point on the gum line of the tooth where the distance between the gum line and the lips is shortest). In some embodiments, the corresponding coordinates of each tooth are the gum curve tip point or Figure 21B The lowest point of the gums 2140 (ie, the point on the gum line between two teeth where the distance between the gum line and the lips is the longest).

[0197] As described in method 2001, based on patient-specific input parameters and simulated input parameters and optionally as Figure 23 The leveled gum line described in can be used to render the gums in a final 2D image of the patient's teeth in an initial or final position, followed by a simulated treatment, such as Figure 1 Box 140, Figure 6 Box 650, Figure 10A Box 1060 or Figure 10B as described in block 1055 of .

[0198] like Figure 20 The parametric gum line simulation using a combination of patient-specific parameters, simulation parameters and ideal aesthetic parameters is shown and optionally Figure 23 The result of gum line leveling is that the gum line in the rendered 2D image more closely matches the initially captured 2D image, resulting in a more aesthetically pleasing corrected simulated 2D image, compared to a parametric gum line defined entirely by patient-specific input parameters. This improvement may be more pronounced in cases where the lip partially covers one or more teeth, one or more teeth are chipped or damaged, one or more teeth are crooked, or the patient's teeth have a significantly different aspect ratio. Furthermore, parametric gum line simulation is less computationally intensive than generating a 3D gum model using a 3D mesh template.

[0199] At block 2061, the simulated position of the parameterized gum line and 3D tooth model is rendered and inserted into the 2D image of the patient in either the initial position or the final matching position. Figure 10A The rendering of the 2D image is described in more detail at block 1060 of FIG. The 3D rendering of the teeth can be placed in the patient's mouth as defined by the lip contour. Figure 20 An image of the parameterized gum line identified in method 2001.

[0200] In some embodiments, a neural network such as a generative adversarial network or a conditional generative adversarial network can be used to integrate a 3D model of the teeth in their final position with an image of the patient's face and match the color, tone, shading, and other aspects of the 3D model to the facial photograph. The neural network is trained using the facial images. In some embodiments, the facial images can include images of a person's face with a social smile. In some embodiments, the facial images can include facial images of the patient's teeth before orthodontic treatment. During training, the patient's teeth and their outlines can be identified. For example, each tooth can be identified by type (e.g., upper left central incisor, lower right canine). Other aspects and features of the image can also be identified during training, such as the position and color of the gums, the color of the teeth, the relative brightness of the surfaces inside the mouth, etc.

[0201] refer to Figure 24After training, the neural network receives input at block 4804 for generating a realistic rendering of the patient's teeth in a clinical final position. In some embodiments, the input may include one or more of a rendered image of a 3D model of the patient's teeth in a clinical final position, such as determined based on an orthodontic treatment plan, a parameterized gum line, a blurred initial image 4810 of the patient's teeth, and a color-coded image 4812 of the 3D model of the patient's teeth in the clinical final position, or a 3D rendered model 4808 of the patient's teeth in the clinical final position.

[0202] A rendered image of a 3D model of the patient's teeth in a clinical final position or a rendered 3D model 4808 of the patient's teeth in a clinical final position can be determined based on a clinical orthodontic treatment plan for moving the patient's teeth from an initial position toward a final position, as described above. The image or rendering 4808 can be generated based on the imaging perspective. For example, one or more of the imaging distance, the focal length of the imaging system, and the size of the patient's teeth in the initial facial image can be used to generate the image or rendering.

[0203] The blurred image 4810 of the patient's teeth can be generated using one or more blur algorithms (e.g., a Gaussian blur algorithm). In some embodiments, the Gaussian blur can have a large radius, for example, a radius of at least 5, 10, 20, 40, or 50 pixels. In some embodiments, the blur is large enough that tooth structure cannot be discerned in the blurred image.

[0204] The encoded model of the patient's teeth 4812 can be a red-green-blue (RGB) color-coded image of the patient's teeth model, where each color channel corresponds to a different quality or feature of the model. For example, the green channel, which can be an 8-bit color channel, indicates the brightness of the blurred image 4810 on a scale of 0 to 255, for example, as overlaid on the 3D model.

[0205] The red channel can be used to distinguish each tooth and gum from each other. In such an embodiment, the gums 4813 can have a red channel value of 1, the upper left central incisor 4814 can have a red value of 2, the lower right canine can have a red channel value of 3, portions of the model that are not teeth or gums have a red channel value of 0, and so on, so that the red channel value of each pixel identifies the dental anatomy associated with the pixel.

[0206] The blue channel can be used to identify the angle of the teeth and / or gums relative to the plane of the face. For example, the angle perpendicular to the surface of the tooth structure is determined at each pixel location, and a value between 0 and 255 (for an 8-bit color channel) is assigned to that pixel. This information allows the neural network to model the reflectivity of light from the tooth surface, for example.

[0207] The neural network then uses the input and its training to render a realistic image of the patient's teeth in their final position 4806. This realistic image is then integrated into the open mouth in the facial image and an alpha channel blur is applied.

[0208] Figure 12 Depicted is a system 1200 for simulating estimated and / or expected results of orthodontic treatment, according to some embodiments. Figure 12 In the example of FIG. 1 , the system 1200 includes a computer readable medium 1210, a dental scanning system 1220, a dental treatment planning system 1230, a dental treatment simulation system 1240, and an image capture system 1250. One or more elements of the system 1200 may include, for example, a reference Figure 20 One or more elements of system 1200 may also include one or more computer-readable media that include instructions that, when executed by a processor (e.g., a processor of any of systems 1220, 1230, 1240, and 1250), cause the corresponding system or systems to perform the processes described herein.

[0209] The dental scanning system 1220 may include a computer system configured to capture one or more scans of a patient's dentition. The dental scanning system 1220 may include a scanning engine for capturing 2D images of the patient. These images may include images of the patient's teeth, face, and jaws. These images may also include x-rays or other sub-surface images of the patient. The scanning engine may also capture 3D data representing the patient's teeth, face, gums, or other aspects of the patient.

[0210] The dental scanning system 1220 may also include a 2D imaging system, such as a still or video camera, an x-ray machine, or other 2D imager. In some embodiments, the dental scanning system 1220 also includes a 3D imager, such as an intraoral scanner or an impression scanner. Figures 1-11 As described herein, the dental scanning system 1220 and associated engines and imagers may be used to capture historical scan data for use in determining historical average parameters of a 3D parametric dental model. Figures 1-11 As described, the dental scanning system 1220 and associated engines and imagers may be used to capture 2D images of a patient's face and dentition for use in constructing a 3D parametric model of the patient's teeth.

[0211] The dental treatment simulation system 1240 may include a computer system configured to simulate one or more estimated outcomes and / or expected outcomes of a dental treatment plan. In some embodiments, the dental treatment simulation system 1240 obtains a photograph and / or other 2D image of a customer / patient. The dental treatment simulation system 1240 may also be configured to determine teeth, lips, gums, and / or other edges associated with the teeth in the 2D image. As described herein, the dental treatment simulation system 1240 may be configured to match teeth and / or dental arch parameters to the teeth, lips, gums, and / or other edges. The dental treatment simulation system 1240 may also render a 3D dental model of the patient's teeth. The dental treatment simulation system 1240 may collect information related to historical dental arches and / or ideal dental arches that represent estimated treatment outcomes. In various embodiments, the dental treatment simulation system 1240 may insert the 3D dental model into a 2D image of the patient, align it with the 2D image of the patient, etc., to render a 2D simulation of the estimated outcome of the orthodontic treatment. The dental treatment simulation system 1240 may include a photo parameterization engine, which may also include an edge analysis engine, an EM analysis engine, a tooth rough alignment engine, and a 3D parameterization conversion engine. The dental treatment simulation system 1240 may also include a parameter treatment prediction engine, which may also include a treatment parameterization engine, a scanned tooth normalization engine, and a treatment plan remodeling engine. The dental treatment simulation system 1240 and its related engines may perform the above-referenced Figure 5-Figure 9 Describe the process.

[0212] The dental treatment planning system 1230 may include a computer system configured to implement a treatment plan. The dental treatment planning system 1230 may include a rendering engine and interface for visualizing or otherwise displaying simulation results of a dental treatment plan. For example, the rendering engine may render a visualization of a 3D model as described herein, e.g., Figure 1 At block 140, Figure 6 At blocks 640, 650, and 660, reference is made to Figure 8 The process 800 described, Figure 9 At blocks 910, 920, and 930 of FIG. 10, block 1150, Figure 11 Block 1150 and Figure 20 Block 2061. The rendering engine can render a realistic rendering of the patient's teeth in their final clinical position, as shown in FIG. Figure 24As described. The dental treatment planning system 1230 can also determine an orthodontic treatment plan for moving the patient's teeth from an initial position to a final position, for example, based in part on a 2D image of the patient's teeth. The dental treatment planning system 1230 can be operable to provide image viewing and manipulation so that the rendered image can be scrollable, pivotable, scalable, and interactive. The dental treatment planning system 1230 can include graphics rendering hardware, one or more displays, and one or more input devices. Some or all of the dental treatment planning system 1230 can be implemented on a personal computing device such as a desktop computing device or a handheld device such as a mobile phone. In some embodiments, at least a portion of the dental treatment planning system 1230 can be implemented on a scanning system such as a dental scanning system 1220. The image capture system 1250 can include a device configured to acquire images (including images of the patient). The image capture system can include any type of mobile device (iOS device, iPhone, iPad, iPod, etc., Android device, portable device, tablet computer), PC, camera (DSLR camera, film camera, video camera, still camera, etc.). In some implementations, image capture system 1250 includes a set of stored images, such as images stored on a storage device, a network location, a social media site, or the like.

[0213] Figure 13 An example of one or more elements of a dental treatment simulation system 1240 is shown in accordance with some embodiments. Figure 13 In the example of , the dental treatment simulation system 1240 includes a photo collection engine 1310 , a photo data repository 1360 , a photo parameterization engine 1340 , a case management engine 1350 , a reference case data repository 1370 , and a treatment rendering engine 1330 .

[0214] The photo collection engine 1310 may implement one or more automated agents configured to retrieve photos of the selected patient from the photo data repository 1360, the image capture system 1250, and / or scans from the dental scanning system 1220. The photo collection engine may then provide the one or more photos to the photo parameterization engine 1340. The photo collection engine may then provide the one or more photos to the photo parameterization engine 1340 and / or other modules of the system.

[0215] The photo data warehouse 1360 may include a data warehouse configured to store photos of patients (e.g., photos of their faces). In some embodiments, these photos are 2D images that include the patient's mouth and one or more images of the face, head, neck, shoulders, torso, or the patient as a whole. The 2D images of the patient may include images of the patient with the patient's mouth in one or more positions. For example, the patient's mouth may be in a smiling position (such as a social smile position), a resting position with muscles relaxed and lips slightly parted, or a retracted anterior open bite or anterior closed bite position. In some embodiments, images of the patient are captured using an image capture system. The images may be captured using a lens with a predetermined focal length at a distance from the patient. The images may also be captured remotely and then received for processing. The images may be a series of images from a video captured from one or more perspectives. For example, the images may include one or more of a frontal facial image and a profile image, the profile image including one or more three-quarter profile images and a full profile image.

[0216] The photo parameterization engine 1340 may implement one or more automated agents configured to construct a parametric 3D model based on the patient's 2D images from the photo data repository 1360 and the parametric model and average data from the parametric treatment prediction engine. The edge analysis engine 1342 may implement one or more automated agents configured to determine the edges of the teeth, lips, and gums within the patient's photo, for example, as referenced in FIG. Figure 6 In particular, an initial step of identifying the patient's lips (e.g., the inner edges of the lips defining an open mouth), teeth, and gum contours can be determined by a machine learning algorithm such as a convolutional neural network.

[0217] Then, an initial contour can be extracted from the image. The pixels representing the contour can then be binarized. The binarized tooth contour is thinned to reduce the thickness of the contour to, for example, a single pixel width, thereby forming a thinned contour.

[0218] Using the thinned or otherwise derived contours, the parameterized 3D tooth and arch model is matched to each of the patient's teeth depicted in the 2D image of the patient. This matching is based on the contours determined above.

[0219] The rough alignment engine 1344 may implement one or more automated agents configured to receive a 2D image and / or associated edges and perform a rough alignment between the identified edges and an average tooth model, e.g., a reference image. Figure 7AAs described, the respective centers of the teeth in the 2D image and the parametric tooth silhouette may be aligned before sending the 2D image and / or associated edges along with the silhouette and its associated tooth parameters to the EM engine 1346 .

[0220] The expectation maximization (EM) engine 1346 may implement one or more automated agents configured to perform an expectation maximization analysis between edges of the 2D image and parameters of the 3D parametric model, e.g., referring to Figure 7A , and in particular as described in blocks 720 , 730 , and 740 .

[0221] Once the EM analysis engine 1346 completes matching the parametric 3D model with the patient's 2D image, the parametric results are sent to the 3D parameter conversion engine 1348, which can convert the principal component analysis output by the EM analysis engine into case-specific parameters that define a 3D model of the patient's teeth based on the parametric model for use by the parametric treatment prediction engine 1350, as described above.

[0222] The gingival parameterization engine 1345 may implement one or more automated agents configured to perform Figure 20 as described, and specifically as regards Figure 20 The gum line model is constructed using the patient-specific input parameters and, optionally, the simulation input parameters as described in blocks 2031 and 2041 of the present invention. The gum line model is generated using inputs from an initial 2D image of the patient's teeth, a 3D parametric model of the patient's teeth, and optional ideal tooth shape parameters. The gum line model can then be rendered into a 2D image using the treatment rendering engine as described in block 1330.

[0223] The gum line leveling engine 1347 may implement one or more automated agents configured to level the gum line based on the gum line parameters generated at block 1345, the gum parameterization engine. Optionally, the gum line leveling may be as described with respect to Figure 20 2051. The leveled gum line may then be rendered into the 2D image using a treatment rendering engine as described at block 1330.

[0224] The case management engine 1350 may include a case parameterization engine 1352, a scanned teeth normalization engine 1354, and a treatment plan simulation engine 1356. The case management engine 1350 may implement one or more automated agents configured to define a parametric model for representing the patient's teeth and dental arch, determine average data parametrically representing the average position of the patient's teeth and dental arch based on a treatment plan retrieved from a treatment plan data repository 1370, and simulate treatment of a specific patient's teeth based on the parametric model, the average tooth position, and the patient's specific teeth.

[0225] The case parameterization engine 1352 may implement one or more automated agents configured to define a parametric 3D model for use in modeling the teeth. For example, the definition of the parametric 3D model may be as described above with reference to equation (2). The treatment parameterization engine 1352 may define not only equations but also a schema for the parameters. For example, a T representing the position of the tooth may be τ is defined as a 4*4 matrix including tooth position and 3-axis rotation. Similarly, the tooth shape can be represented by is defined as a 2500*3 matrix where each vertex of the sphere is mapped to a position on the surface of the tooth model, where each vertex is a specific x, y, and z position. For lower or higher definition models, fewer or more vertices may be used. Further discussion of the parameters defined by the treatment parameterization engine 1352 is described elsewhere herein (e.g., with reference to Figure 2-Figure 5 ).

[0226] The scanned teeth normalization engine 1354 may implement one or more automated agents configured to parameterize a set of scanned teeth for treatment plans collected from the treatment plan data repository 1370 and then determine an average set of general common parameters for the parametric model. Figure 3B The process 350 described above. For example, the scanned teeth normalization engine 1354 can align the dental arches within the historical cases retrieved from the data repository 1370. Each dental arch in the data repository is aligned at three locations: between the central incisors and at each distal end on the left and right sides of each dental arch. The position and orientation of each tooth is then determined based on the position and orientation of each tooth relative to the alignment points.

[0227] The scanned teeth normalization engine 1354 may implement one or more automated agents configured to determine the distribution of parameters across cases. For example, each historical arch may be rescaled to determine Φ, and then averaged across arches to determine Then determine the local deformation of each tooth and Compare to determine T τ , then after aligning each tooth, determine β τ .

[0228] From this data, the relative shape, position and rotation of each tooth are determined in order to construct the distribution of each case-specific parameter. For example, the coefficients of the principal components of the surface model of each corresponding tooth in all retrieved models are determined. distribution.

[0229] The positions and orientations of each tooth can be averaged to determine the average position of each tooth, and the orientations of each tooth can be averaged to determine the average orientation of each tooth. The average positions and average orientations are used to determine

[0230] In some embodiments, a template data repository of parameterized template models can be generated using a set of scanned teeth for treatment plans collected from a treatment plan data repository. The scanned teeth normalization engine 1354 can align the dental arches in historical cases retrieved from the reference case data repository 1370. Each dental arch in the reference case data repository is aligned at three locations: between the central incisors and at each distal end on the left and right sides of each dental arch. The position and orientation of each tooth is then determined based on the position and orientation of each tooth relative to the alignment points. The parameters of the parameterized template models in the data repository are searchable. The data repository can be used to match a patient 3D model (e.g., generated from a 2D model or from a 3D scan) with the closest template in the data repository based on tooth shape and dental arch shape to identify a matching template model. The closest matching template model can be used to determine the final tooth position and orientation of the patient's teeth.

[0231] The treatment plan simulation engine 1356 may implement one or more automated agents configured to simulate estimated and / or expected outcomes of a dental treatment plan for a specific patient's teeth using the parametric model defined by the treatment parameterization engine 1352, the generic parameters from the scanned teeth normalization engine, and the case-specific parameters from the photo parameterization engine 1340, e.g., as described with reference to FIG. Figure 11 Descriptive.

[0232] The treatment planning simulation engine 1356 retrieves or otherwise determines the average shape In some embodiments, the mean shape can be retrieved from the scanned tooth normalization engine 1354. In some embodiments, the mean shape can be loaded into memory or retrieved from memory. The mean shape can also be initialized as a set of matrices, each matrix corresponding to a tooth.

[0233] Treatment planning simulation engine 1356 average shape of teeth Perform principal component analysis shape adjustment. For each tooth in the model, the case-specific coefficients determined from the photo parameterization engine 1340 for the principal components Apply to principal components After the shape adjustment is completed, in some embodiments, the adjusted shape can be rendered on a screen for viewing by a patient or dental professional, for example. In some embodiments, the adjusted shape can be stored in a memory. Alternatively, the adjusted shape can be stored as a set of matrices, each matrix corresponding to a tooth.

[0234] The treatment plan simulation engine 1356 determines an average tooth pose. In some embodiments, as discussed above, the average tooth pose can be based on an average tooth pose of a set of scanned dental arches. In some embodiments, each adjusted tooth is placed at its corresponding average position and orientation determined by the average dental arch. In some embodiments, the average tooth pose can be rendered on a screen, for example, for viewing by a patient or dental professional. In some embodiments, the average tooth pose can be loaded into memory or retrieved from memory. The average tooth pose can also be initialized as a set of matrices, each matrix corresponding to a tooth in a dental pose.

[0235] In some embodiments, the treatment plan remodeling engine 1356 uses the tooth position and orientation coordinates from the ideal parameterized data repository match and applies the shape and texture of the patient's 3D model generated from the 2D image to the tooth positions and orientations to generate the dental arch.

[0236] The treatment plan simulation engine 1356 scales the dental arch so that the generated dental arch is based on the patient's teeth and dental arch dimensions. In some embodiments, as discussed above, the dental arch scaling factor Φ is based on the specific teeth and dental arch of a specific patient. In some embodiments, as discussed herein, the dental arch is adjusted so that the scaled dental arch matches the dimensions of the patient's dental arch, the dimensions of the patient's dental arch being determined, for example, by the dimensions of the teeth and dental arch in the scanned patient's dental arch or otherwise. In some embodiments, for example, the scaled dental arch can be rendered on a screen for viewing by a patient or dental professional. In some embodiments, the scaled dental arch can be stored in a memory. The scaled dental arch can also be stored as a set of matrices, each matrix corresponding to a tooth in a dental pose. The scaled dental arch can be sent to the treatment rendering engine 1330.

[0237] The treatment rendering engine 1330 renders the patient's teeth and 2D images in the final position as determined, for example, by the parametric treatment prediction engine 1350, and in particular by the treatment plan simulation engine 1356 located within the parametric treatment prediction engine 1350. The treatment rendering engine 1330 may perform a reference Figure 9 , Figure 10 and Figure 11 In particular, see Figure 9 Figure 10 Figure 11 and the rendering process described elsewhere in this article.

[0238] Figure 14An exemplary tooth repositioning device or appliance 1500 that can be worn by a patient to achieve gradual repositioning of individual teeth 1502 in the jaw is shown. The device may include a housing (e.g., a continuous polymeric housing or a segmented housing) having tooth-receiving cavities that accommodate and elastically reposition the teeth. The device or one or more portions thereof may be indirectly manufactured using a physical model of the teeth. For example, the device (e.g., a polymeric device) may be formed using a physical model of the teeth and an appropriate number of layers of polymeric material sheets. The physical model of the teeth (e.g., a physical mold) may be formed by various techniques including 3D printing. The device may be formed by heat-forming the device on the physical model. In some embodiments, the physical device is directly manufactured from a digital model of the device, for example using additive manufacturing techniques. In some embodiments, the physical device may be manufactured by various direct forming techniques such as 3D printing. The device may be mounted on all teeth present in the upper or lower jaw, or on less than all teeth. The appliance can be specifically designed to accommodate the patient's teeth (e.g., the topography of the tooth-receiving cavity matches the topography of the patient's teeth) and can be manufactured based on positive or negative models of the patient's teeth generated by impressions, scans, etc. Alternatively, the appliance can be a universal appliance that is configured to accommodate teeth but is not necessarily shaped to match the topology of the patient's teeth. In some cases, only some of the teeth accommodated by the appliance are repositioned by the appliance, while other teeth can provide a base or anchoring area that holds the appliance in place when the appliance applies force to one or more teeth that are targeted for repositioning. In some cases, some teeth, most teeth, or even all teeth are repositioned at some point during treatment. The teeth that are moved can also serve as a base or anchoring portion to hold the appliance in place when the appliance is worn by the patient. In some embodiments, a wire or other device to secure the appliance in place on the teeth is no longer provided. However, in some cases, it may be desirable or necessary to provide a separate attachment or other anchoring element 1504 on the tooth 1502 using a corresponding receptacle or aperture 1506 in the device 1500 so that the device can apply a selected force on the tooth. Alignment systems, including those described in numerous patents and patent applications assigned to Align Technologies, Inc. (including, for example, U.S. Patent Nos. 6,450,807 and 5,975,893) and on the company's website accessible on the World Wide Web (see, for example, the url "invisalign.com"). Examples of tooth-mounted attachments suitable for use with orthodontic appliances are also described in patents and patent applications assigned to Align Technologies, Inc., including, for example, US Pat. No. 6,309,215 and US Pat. No. 6,830,450.

[0239] Alternatively, in situations involving more complex movements or treatment plans, the use of auxiliary components (e.g., features, accessories, structures, devices, components, etc.) in conjunction with the orthodontic appliance may be advantageous. Examples of such accessories include, but are not limited to, elastic bands, wires, springs, rods, arch expanders, palatal expanders, bi-occlusal splints, bite blocks, bite ramps, mandibular advancement splints, bite plates, bridges, hooks, brackets, headgear tubes, springs, bumper tubes, palatal bars, frames, nail tube devices, buccal shields, buccinator bows, wire shields, lingual flanges and pads, lip pads or lip stops, protrusions, divots, etc. In some embodiments, the appliances, systems, and methods described herein include improved orthodontic appliances having integrally formed features that are shaped to couple with or replace such auxiliary components.

[0240] Figure 15A tooth repositioning system 1510 is shown that includes a plurality of appliances 1512, 1514, 1516. Any appliance described herein may be designed and / or provided as part of a set of a plurality of appliances for use in a tooth repositioning system. Each appliance may be configured so that a tooth-receiving cavity has a geometry corresponding to an intermediate tooth arrangement or a final tooth arrangement intended for the appliance. By placing a series of stepwise position adjustment appliances on the patient's teeth, the patient's teeth may be gradually repositioned from an initial tooth arrangement toward a target tooth arrangement. For example, the tooth repositioning system 1510 may include a first appliance 1512 corresponding to an initial tooth arrangement, one or more intermediate appliances 1514 corresponding to one or more intermediate arrangements, and a final appliance 1516 corresponding to a target arrangement. The target tooth arrangement may be a planned final tooth arrangement selected for the patient's teeth at the conclusion of all planned orthodontic treatment. Alternatively, the target arrangement can be one of several intermediate arrangements of the patient's teeth during the course of orthodontic treatment, which can include a variety of different treatment plans, including but not limited to the following examples: surgery is recommended, interproximal surface removal (IPR) is appropriate, a progress check is scheduled, anchor placement is optimal, palatal expansion is required, dental restorations (e.g., inlays, onlays, crowns, bridges, implants, veneers, etc.) are involved, etc. For example, it will be understood that the target tooth arrangement can be any planned resulting arrangement for the patient's teeth after one or more step-by-step repositioning stages. Likewise, the initial tooth arrangement can be any initial arrangement of the patient's teeth followed by one or more step-by-step repositioning stages.

[0241] Figure 16A method 1550 for performing orthodontic treatment using multiple appliances is shown, according to many embodiments. Method 1550 can be practiced using any appliance or appliance set described herein. In block 1560, a first orthodontic appliance is applied to the patient's teeth to reposition the teeth from a first tooth arrangement to a second tooth arrangement. In block 1570, a second orthodontic appliance is applied to the patient's teeth to reposition the teeth from the second tooth arrangement to a third tooth arrangement. Method 1550 can be repeated as needed using any suitable number of sequential appliances and any suitable combination of sequential appliances to progressively reposition the patient's teeth from an initial arrangement to a target arrangement. Appliances can be generated all at the same stage, or in groups or batches (e.g., at the beginning of treatment, in the middle of treatment, etc.), or one appliance can be manufactured at a time, and the patient can wear each appliance until the pressure of each appliance on the teeth is no longer felt, or until the maximum amount of tooth movement exhibited for that given stage has been achieved. The multiple appliances (e.g., a set) can be designed and even manufactured before the patient wears any appliance from the multiple different appliances (e.g., a set). After wearing the appliance for an appropriate period of time, the patient can replace the current appliance with the next appliance in the series until there are no more appliances left. These appliances are generally not attached to the teeth, and the patient can install and replace the appliance at any time during the procedure (e.g., patient-removable appliances). The final appliance or some appliances in the series may have one or more geometries selected to overcorrect the tooth arrangement. For example, one or more appliances may have a geometry that will cause the movement of each tooth to exceed the tooth arrangement selected as the "final" (if fully achieved). This overcorrection may be desirable to offset potential relapse after the repositioning method is completed (e.g., allowing each tooth to move back toward their pre-correction position). Overcorrection is also beneficial in speeding up the correction process (e.g., an appliance whose geometry is positioned beyond the desired intermediate or final position can shift each tooth toward that position at a faster rate). In this case, the use of the appliance can be stopped before the teeth reach the position defined by the appliance. In addition, overcorrection can be applied intentionally to compensate for any inaccuracies or limitations of the appliance.

[0242] The various embodiments of the orthodontic appliances presented herein can be manufactured in a variety of ways. In some embodiments, the orthodontic appliances (or portions thereof) described herein can be produced using direct manufacturing techniques such as additive manufacturing (also referred to herein as "3D printing") or subtractive manufacturing (e.g., milling). In some embodiments, direct manufacturing involves forming an object (e.g., an orthodontic appliance or portion thereof) without using a physical template (e.g., a mold, a mask, etc.) to define the geometry of the object.

[0243] In some embodiments, the orthodontic devices herein can be manufactured using a combination of direct and indirect manufacturing techniques, such that different parts of the device can be manufactured using different manufacturing techniques and assembled to form the final device. For example, the device housing can be formed by indirect manufacturing (e.g., thermoforming), and one or more structures or components described herein (e.g., auxiliary components, power arms, etc.) can be added to the housing by direct manufacturing (e.g., printed onto the housing).

[0244] The configuration of the orthodontic appliances described herein can be determined based on the patient's treatment plan, for example, a treatment plan involving the sequential implementation of multiple appliances for progressively repositioning teeth. Computer-based treatment planning and / or appliance manufacturing methods can be used to facilitate the design and manufacture of appliances. For example, one or more appliance components described herein can be digitally designed and manufactured using computer-controlled manufacturing equipment (e.g., computer numerical control (CNC) milling, computer-controlled additive manufacturing, such as 3D printing, etc.). The computer-based methods proposed herein can improve the accuracy, flexibility, and convenience of appliance manufacturing.

[0245] In some embodiments, computer-based 3D planning / design tools (e.g., Treat TM Software) can be used to design and manufacture the orthodontic appliances described herein.

[0246] Figure 17 A method 1800 for designing an orthodontic appliance to be manufactured according to an embodiment is shown. The method 1800 can be applied to any embodiment of the orthodontic appliance described herein. Some or all of the operations of the method 200 can be performed by any suitable data processing system or device (e.g., one or more processors configured with suitable instructions).

[0247] In block 1810, a movement path is determined for moving one or more teeth from an initial arrangement to a target arrangement. The initial arrangement can be determined from a mold or scan of the patient's teeth or oral tissue, for example, using wax articulation, direct contact scanning, x-ray imaging, tomography, ultrasound imaging, and other techniques for obtaining information about the position and structure of teeth, jaws, gums, and other orthodontically relevant tissues. A digital data set can be obtained from the acquired data, which represents the initial (e.g., before treatment) arrangement of the patient's teeth and other tissues. Optionally, the initial digital data set is processed to segment tissue constituents from each other. For example, a data structure digitally representing each crown can be generated. Advantageously, a digital model of the entire tooth can be generated, including measured or inferred hidden surface and root structures, as well as surrounding bone and soft tissue.

[0248] The target arrangement of teeth (e.g., the desired and anticipated end result of orthodontic treatment) can be received from the clinician in the form of a prescription, can be calculated based on basic orthodontic principles, and / or can be computationally extrapolated from the clinical prescription. With the specification of the desired final positions of the teeth and the digital representation of the teeth themselves, the final position and surface geometry of each tooth can be specified to form a complete model of the dental arrangement at the desired end of treatment.

[0249] Each tooth has both an initial position and a target position, and a movement path can be defined for the movement of each tooth. In some embodiments, these movement paths are configured to move the teeth in the fastest manner with a minimum amount of round-tripping to move the teeth from their initial positions to their desired target positions. Optionally, the tooth paths can be segmented, and the segments can be calculated so that the movement of each tooth within a segment remains within threshold limits for linear and rotational translation. In this way, the endpoints of each path segment can constitute a clinically feasible repositioning, and the collection of segment endpoints can constitute a clinically feasible sequence of tooth positions such that movement from one point in the sequence to the next does not cause collision of the teeth.

[0250] In box 1820, a force system is determined for moving one or more teeth along a movement path. The force system may include one or more forces and / or one or more torques. Different force systems will cause different types of tooth movement, such as tilting, translation, rotation, extrusion, intrusion, root movement, etc. Biomechanical principles, modeling techniques, force calculation / measurement techniques, etc. (including knowledge and schemes commonly used in orthodontics) can be used to determine the appropriate force system to be applied to the teeth to achieve tooth movement. When determining the force system to be applied, resources that may be considered include literature, force systems determined by experiments or virtual modeling, computer-based modeling, clinical experience, minimizing unnecessary forces, etc.

[0251] The force system can be determined in various ways. For example, in some embodiments, the force system is determined on a patient-by-patient basis (e.g., using patient-specific data). Alternatively, or in combination, the force system can be determined based on a generalized model of tooth movement (e.g., based on experiments, modeling, clinical data, etc.), thereby not necessarily using patient-specific data. In some embodiments, the determination of the force system includes calculating specific force values ​​to be applied to one or more teeth to produce a specific movement. Alternatively, the determination of the force system can be performed at a high level without calculating specific force values ​​for the teeth. For example, box 1820 can include determining a specific type of force to be applied (e.g., an extrusion force, an intrusion force, a translational force, a rotational force, a tilting force, a torque force, etc.) without calculating the specific magnitude and / or direction of the force.

[0252] In block 1830, an appliance geometry and / or material composition of an orthodontic appliance configured to generate a force system is determined. The appliance can be any embodiment of an appliance discussed herein, such as an appliance having variable local properties, integrally formed components, and / or a power arm.

[0253] For example, in some embodiments, the device comprises a heterogeneous thickness, a heterogeneous hardness, or a heterogeneous material composition. In some embodiments, the device comprises two or more of a heterogeneous thickness, a heterogeneous hardness, or a heterogeneous material composition. In some embodiments, the device comprises a heterogeneous thickness, a heterogeneous hardness, and a heterogeneous material composition. The heterogeneous thickness, hardness, and / or material composition can be configured to generate a force system for moving teeth, such as by preferentially applying force at certain locations on the teeth. For example, an device with a heterogeneous thickness can include a thicker portion that exerts a greater force on the teeth than a thinner portion. As another example, an device with a heterogeneous hardness can include a harder portion that exerts a greater force on the teeth than a more elastic portion. As described herein, changes in hardness can be achieved by changing the device thickness, material composition, and / or degree of photopolymerization.

[0254] In some embodiments, determining the device geometry and / or material composition includes determining the geometry and / or material composition of one or more integrally formed components to be manufactured directly with the device housing. The integrally formed components can be any of the embodiments described herein. The geometry and / or material composition of the integrally formed component(s) can be selected to facilitate application of the force system to the patient's teeth. The material composition of the integrally formed components can be the same as or different from the material composition of the housing.

[0255] In some embodiments, determining the instrument geometry includes determining a geometry of a variable gable bend.

[0256] Box 1830 may include analyzing the desired force system to determine the instrument geometry and material composition that will produce the force system. In some embodiments, the analysis includes determining the instrument characteristics (e.g., hardness) at one or more locations that will produce the desired force at the one or more locations. The analysis then includes determining the instrument geometry and material composition at the one or more locations to achieve the specified characteristics. The determination of the instrument geometry and material composition can be performed using a treatment or force simulation environment. The simulation environment can include, for example, a computer modeling system, a biomechanical system or device, etc. Optionally, a digital model of the instrument and / or tooth, such as a finite element model, can be made. The finite element model can be created using computer program application software available from various suppliers. To create the solid geometry model, a computer-aided engineering (CAE) or computer-aided design (CAD) program can be used, such as the Autodesk® software available from Autodesk, Inc. of San Rafael, Canada. Software Products. To create finite element models and analyze them, program products from several vendors may be used, including the finite element analysis package from ANSYS, Inc., Canonsburg, Pennsylvania, and the SIMULIA (Abaqus) software product from Dassault Systèmes, Inc., Waltham, Massachusetts.

[0257] Optionally, one or more device geometries and material compositions can be selected for testing or force modeling. As described above, the desired tooth movement and the force system required or desired to cause the desired tooth movement can be identified. Using a simulation environment, the geometry and composition of the candidate device can be analyzed or modeled to determine the actual force system generated by using the candidate device. One or more modifications can optionally be made to the candidate device, and the force modeling (as described) can be further analyzed, for example, to iteratively determine the device design that produces the desired force system.

[0258] Optionally, box 1830 may also include determining the geometry of one or more auxiliary components to be used in conjunction with the orthodontic appliance to apply a force system on one or more teeth. These auxiliary devices may include one or more of the following: tooth-mounted accessories, elastics, wires, springs, bite blocks, arch expanders, wire-bracket appliances, shell appliances, head caps, or any other orthodontic devices or systems that can be used in conjunction with the orthodontic appliances herein. The use of these auxiliary components may be advantageous in situations where it is difficult for the appliance to generate a force system alone. In addition, auxiliary components may be added to the orthodontic appliance to provide other desired functions in addition to generating a force system, such as a mandibular advancement splint for treating sleep apnea, a bridge for improving aesthetic appearance, and the like. In some embodiments, the auxiliary components are manufactured and provided separately from the orthodontic appliance. In addition, the geometry of the orthodontic appliance may be modified to include one or more auxiliary components that are integrally formed components.

[0259] In box 1840, instructions for manufacturing an orthodontic appliance having an appliance geometry and material composition are generated. These instructions can be configured to control a manufacturing system or device to produce an orthodontic appliance having a specified appliance geometry and material composition. In some embodiments, these instructions can be configured to manufacture an orthodontic appliance using direct manufacturing (e.g., stereolithography, selective laser sintering, fused deposition modeling, 3D printing, continuous direct manufacturing, multi-material direct manufacturing, etc.). Alternatively, as discussed above and herein, these instructions can be configured to cause a manufacturing machine to directly manufacture an orthodontic appliance having a tooth-receiving cavity having a variable roof-shaped curvature. In an alternative embodiment, the instructions can be configured to manufacture the appliance indirectly (e.g., by thermoforming).

[0260] Although the above blocks illustrate a method 1800 for designing an orthodontic appliance according to some embodiments, those of ordinary skill in the art will recognize some variations based on the teachings described herein. Some blocks may include sub-blocks. Some blocks may be repeated as frequently as desired. One or more blocks of method 200 may be performed using any suitable manufacturing system or equipment, such as the embodiments described herein. Some blocks are optional, and the order of the blocks may be changed as desired. For example, in some embodiments, block 1820 is optional, such that block 1830 includes determining the appliance geometry and / or material composition based directly on the tooth movement path rather than based on a force system.

[0261] Figure 18 A method 1900 for digitally planning orthodontic treatment and / or appliance design or manufacture, according to an embodiment, is shown. The method 1900 can be applied to any treatment procedure described herein and can be executed by any suitable data processing system.

[0262] In block 1910, a digital representation of a patient's teeth is received. The digital representation may include surface topography data of the patient's oral cavity (including teeth, gum tissue, etc.). The surface topography data may be generated by directly scanning the oral cavity, a physical model (positive or negative) of the oral cavity, or an impression of the oral cavity using a suitable scanning device (e.g., a handheld scanner, a desktop scanner, etc.).

[0263] In block 1920, one or more treatment phases are generated based on the digital representation of the teeth. These treatment phases can be step-by-step repositioning phases of an orthodontic treatment procedure designed to move one or more of the patient's teeth from an initial tooth arrangement to a target tooth arrangement. For example, these treatment phases can be generated by determining an initial tooth arrangement indicated by the digital representation, determining a target tooth arrangement, and determining a movement path for one or more teeth in the initial arrangement to achieve the target tooth arrangement. The movement path can be optimized based on minimizing the total movement distance, preventing collisions between teeth, avoiding difficult tooth movements, or any other suitable criteria.

[0264] In block 1930, at least one orthodontic appliance is manufactured based on the generated treatment phase. For example, a set of appliances can be manufactured, each appliance being shaped according to the tooth arrangement specified for a treatment phase, such that the appliances can be worn sequentially by the patient to gradually reposition the teeth from an initial arrangement to a target arrangement. The set of appliances can include one or more of the orthodontic appliances described herein. The manufacture of the appliance can include creating a digital model of the appliance to be used as input to a computer-controlled manufacturing system. As desired, the appliance can be formed using direct manufacturing methods, indirect manufacturing methods, or a combination of both.

[0265] In some cases, staging the various arrangements or treatment phases may not be necessary for device design and / or manufacture. Figure 18 As shown by the dashed lines in , the design and / or manufacture of orthodontic appliances (and possibly particular orthodontic treatments) can include using a representation of a patient's teeth (e.g., receiving a digital representation of the patient's teeth 1910), followed by designing and / or manufacturing the orthodontic appliance based on the representation of the patient's teeth in the arrangement represented by the received representation.

[0266] Optionally, some or all of the blocks of method 1900 are performed locally at the site where the patient is treated and during a single visit by the patient, referred to herein as "chairside manufacturing." For example, chairside manufacturing can include scanning the patient's teeth, automatically generating a treatment plan with various treatment stages, and immediately manufacturing one or more orthodontic appliances to treat the patient using a chairside direct manufacturing machine, all during a single appointment in the treating professional's office. In an embodiment where a series of appliances are used to treat a patient, the first appliance can be manufactured at the chairside for immediate delivery to the patient, while the remaining appliances are manufactured separately (e.g., off-site in a laboratory or central manufacturing facility) and delivered at a later time (e.g., in a subsequent appointment, mailed to the patient). Alternatively, the methods herein can be adapted to produce and immediately deliver an entire series of appliances on-site during a single visit. Thus, chairside manufacturing can improve the convenience and speed of the treatment procedure by immediately beginning treatment for the patient in the physician's office, rather than having to wait for the appliances to be manufactured and delivered at a later date. Furthermore, chairside manufacturing can provide improved flexibility and efficiency in orthodontic treatment. For example, in some embodiments, the patient is rescanned at each consultation to determine the actual position of the teeth and the treatment plan is updated accordingly. New appliances can then be fabricated and delivered immediately, chairside, to accommodate any changes or deviations in the treatment plan.

[0267] Figure 19 2 is a simplified block diagram of a data processing system 2000 that can be used to perform the methods and processes described herein. The data processing system 2000 generally includes at least one processor 2002 that communicates with one or more peripheral devices via a bus subsystem 2004. These peripheral devices generally include a storage subsystem 2006 (a memory subsystem 2008 and a file storage subsystem 2014), a set of user interface input and output devices 2018, and an interface to an external network 2016. This interface is schematically shown as a "network interface" block 2016 and is coupled to corresponding interface devices in other data processing systems via a communication network interface 2024. The data processing system 2000 may include, for example, one or more computers, such as personal computers, workstations, mainframes, laptop computers, and the like.

[0268] The user interface input device 2018 is not limited to any particular device and may generally include, for example, a keyboard, a pointing device, a mouse, a scanner, an interactive display, a touchpad, a joystick, etc. Similarly, a variety of user interface output devices may be used in the system of the present invention and may include, for example, one or more of a printer, a display (e.g., visual, non-visual) system / subsystem, a controller, a projection device, an audio output, etc.

[0269] The storage subsystem 2006 holds basic programming requirements and includes computer-readable media containing instructions (e.g., operating instructions, etc.) and data structures. The program modules discussed herein are typically stored in the storage subsystem 2006. The storage subsystem 2006 typically includes a memory subsystem 2008 and a file storage subsystem 2014. The memory subsystem 2008 typically includes multiple memories (e.g., RAM 2010, ROM 2012, etc.), including computer-readable memory for storing fixed instructions, instructions and data during program execution, basic input / output systems, and the like. The file storage subsystem 2014 permanently (non-volatile) stores program and data files and may include one or more removable or fixed drives or media, such as hard disks, floppy disks, CD-ROMs, DVDs, optical drives, and the like. One or more of the storage systems, drives, and the like may be located remotely, such as via a server on a network or coupled via the Internet / World Wide Web. In this context, the term "bus subsystem" is used generally to include any mechanism for enabling various components and subsystems to communicate with each other as intended, and may include various suitable components / systems that would be considered or deemed suitable for use therein. It should be appreciated that the various components of the system may, but need not, be in the same physical location, but may be connected via various local or wide area network media, transmission systems, etc.

[0270] Scanner 2020 comprises any device for acquiring a digital representation (e.g., image, surface topography data, etc.) of a patient's teeth (e.g., by scanning a physical model of the teeth, such as a cast 2027, by scanning an impression taken from the teeth, or by directly scanning the oral cavity), which may be obtained from the patient or a treating professional (e.g., an orthodontist), and also comprises a device for providing the digital representation to data processing system 2000 for further processing. Scanner 2020 may be located at a remote location relative to the other components of the system and may transmit image data and / or information to data processing system 2000, for example, via network interface 2024. Fabricator 2022 fabricates appliance 2023 based on a treatment plan that includes data set information received from data processing system 2000. For example, fabricator 2022 may be located at a remote location and receive data set information from data processing system 2000 via network interface 2024. Camera 2025 may comprise any image capture device configured to capture still images or movies. The camera 2025 can capture various perspectives of the patient's dentition. In some embodiments, the camera 2025 can capture images at different distances from the patient and at different focal lengths.

[0271] The data processing aspects of the methods described herein can be implemented in digital electronic circuits, or in computer hardware, firmware, software, or a suitable combination thereof. The data processing apparatus can be implemented in a computer program product that is tangibly embodied in a machine-readable storage device for execution by a programmable processor. The data processing blocks can be executed by a programmable processor that executes program instructions, thereby performing functions by operating on input data and generating outputs. The data processing aspects can be implemented on one or more computer programs that can be executed on a programmable system that includes one or more programmable processors operably coupled to a data storage system. Typically, the processor will receive instructions and data from a read-only memory and / or a random access memory. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, for example: semiconductor memory devices (such as EPROM, EEPROM, and flash memory devices); magnetic disks (such as internal hard disks and removable disks); magneto-optical disks; and CD-ROM disks.

[0272] Although the detailed description contains many specific details, these specific details should not be interpreted as limiting the scope of the present disclosure, but are merely interpreted as illustrating different examples and aspects of the present disclosure. It should be understood that the scope of the present disclosure includes other embodiments not discussed in detail above. Various other modifications, variations and variations that are obvious to those skilled in the art can be made in the arrangement, operation and details of the methods, systems and devices of the present disclosure provided herein without departing from the spirit and scope of the present invention as described herein.

[0273] As used herein, the terms "dental appliance" and "tooth receiving appliance" are considered synonymous. As used herein, "dental positioning appliance" or "orthodontic appliance" may be considered synonymous and may include any dental appliance configured to change the position of a patient's teeth according to a plan, such as an orthodontic treatment plan. As used herein, "patient" may include any person, including a person seeking dental / orthodontic treatment, a person currently receiving dental / orthodontic treatment, and a person who has previously received dental / orthodontic treatment. "Patient" may include a customer or potential customer for orthodontic treatment, such as a person using the visualization tools herein to inform their decision to undergo orthodontic treatment at all or to select a specific orthodontic treatment plan. As used herein, "dental positioning appliance" or "orthodontic appliance" may include a set of dental appliances configured to gradually change the position of a patient's teeth over time. As described herein, dental positioning appliances and / or orthodontic appliances may include polymer appliances configured to move a patient's teeth according to an orthodontic treatment plan.

[0274] As used herein, the term "and / or" can be used as a function word to indicate that two words or phrases are taken together or separately. For example, the phrase "A and / or B" includes A alone, B alone, and A and B together. Depending on the context, the term "or" does not exclude one of the multiple words / phrases. For example, the phrase "A or B" does not exclude A and B together.

[0275] As used herein, the terms "torque" and "moment" are considered synonymous.

[0276] As used herein, a "torque" may include a force acting on an object, such as a tooth, at a distance from a center of resistance. For example, a torque may be calculated as a vector cross product of a vector force applied to a location corresponding to a displacement vector from the center of resistance. A torque may include a vector pointing in one direction. A torque that is opposite to another torque may include one of a torque vector oriented toward a first side of an object (such as a tooth) and another torque vector oriented toward an opposite side of the object (such as a tooth). Any discussion herein about applying a force on a patient's teeth applies equally to applying a torque on the teeth, and vice versa.

[0277] As used herein, "a plurality of teeth" may include two or more teeth. A plurality of teeth may include adjacent teeth, but need not include adjacent teeth. In some embodiments, the one or more posterior teeth include one or more of a molar, a premolar, or a canine, and the one or more anterior teeth include one or more of a central incisor, a lateral incisor, a molar, a first bicuspid, or a second bicuspid.

[0278] The embodiments disclosed herein may be well suited for moving one or more teeth of a first set of one or more teeth or moving one or more teeth of a second set of one or more teeth, and combinations thereof.

[0279] The embodiments disclosed herein may be well suited for integration with one or more commercially available tooth movement components, such as attachments and polymer shell appliances. In some embodiments, the appliance and one or more attachments are configured to move one or more teeth along a tooth movement vector comprising six degrees of freedom, wherein three degrees of freedom are rotational and three degrees of freedom are translational.

[0280] Repositioning of the teeth can be accomplished using a series of removable elastic positioning devices, such as those available from Align Technologies, Inc., the assignee of the present disclosure. Systems. These appliances may have a thin shell made of a resilient material that generally conforms to the patient's teeth but is slightly misaligned from the initial or immediately previous tooth configuration. Placing the appliance on the teeth allows controlled forces to be applied at specific locations to gradually move the teeth into a new configuration. This process is repeated with successive appliances containing new configurations, ultimately moving the teeth through a series of intermediate configurations or alignment patterns to the final desired configuration. Repositioning of teeth may be accomplished with a series of other removable orthodontic appliances and / or dental appliances, including polymer-shell appliances.

[0281] The term "computer system" as used herein is intended to be broadly interpreted. Generally speaking, a computer system includes a processor, memory, non-volatile memory, and an interface. A typical computer system typically includes at least a processor, memory, and a device (e.g., a bus) coupling the memory to the processor. For example, the processor can be a general-purpose central processing unit (CPU) such as a microprocessor or a special-purpose processor such as a microcontroller.

[0282] By way of example and not limitation, memory can include random access memory (RAM), such as dynamic RAM (DRAM) and static RAM (SRAM). Memory can be local, remote, or distributed. The bus can also couple the processor to non-volatile memory. Non-volatile memory is typically a magnetic floppy or hard disk, a magneto-optical disk, an optical disk, a read-only memory (ROM) such as a CD-ROM, EPROM, or EEPROM, a magnetic or optical card, or other forms of memory for large amounts of data. During the execution of software on a computer system, some of this data is typically written to memory via a direct memory access process. Non-volatile memory can be local, remote, or distributed. Non-volatile memory is optional because a system can be created using all applicable data available in memory.

[0283] Software is typically stored in non-volatile memory. In fact, for large programs, it may not even be possible to store the entire program in memory. Nevertheless, it should be understood that in order for the software to run, if necessary, the software is moved to a computer-readable location suitable for processing, and for the purposes of illustration, this location is referred to as memory herein. Even when the software is moved to memory for execution, the processor typically utilizes hardware registers to store values ​​associated with the software, as well as local caches that ideally speed up execution. As used herein, when a software program is referred to as being "implemented in a computer-readable storage medium," it is assumed that the software program is stored in a known or convenient location (from non-volatile memory to hardware registers) that is applicable. A processor is considered to be "configured to execute the program" when at least one value associated with the program is stored in a register readable by the processor.

[0284] In one example of operation, a computer system may be controlled by operating system software, which is a software program that includes a file management system such as a disk operating system. An example of operating system software with associated file management system software is Microsoft Corporation of Redmond, Washington. Another example of an operating system family with associated file management systems is the Linux operating system and its associated file management system. The file management system is typically stored in non-volatile memory and causes the processor to perform various actions required by the operating system to input and output data and store data in memory, including storing files on the non-volatile memory.

[0285] The bus can also couple the processor to an interface. The interface can include one or more input and / or output (I / O) devices. Depending on the considerations of a particular embodiment or other considerations, by way of example and not limitation, the I / O devices can include a keyboard, a mouse or other pointing device, a disk drive, a printer, a scanner, and other I / O devices including a display device. By way of example and not limitation, the display device can include a cathode ray tube (CRT), a liquid crystal display (LCD), or some other applicable known or convenient display device. The interface can include one or more of a modem or a network interface. It is understood that a modem or a network interface can be considered as part of a computer system. The interface can include an analog modem, an ISDN modem, a cable modem, a token ring interface, a satellite transmission interface (e.g., "direct PC"), or other interfaces for coupling a computer system to other computer systems. The interface enables a computer system and other devices to be coupled together in a network.

[0286] The computer system may be compatible with or implemented as part of or by a cloud-based computing system. As used herein, a cloud-based computing system is a system that provides virtualized computing resources, software, and / or information to end-user devices. Computing resources, software, and / or information may be virtualized by maintaining centralized services and resources that edge devices may access through a communication interface (e.g., a network). "Cloud" may be a marketing term and, for the purposes of this document, may include any network described herein. A cloud-based computing system may involve a subscription service or use a utility pricing model. A user may access the protocol of a cloud-based computing system through a web browser or other container application located on their end-user device.

[0287] The computer system can be implemented as an engine, a part of an engine, or by multiple engines. As used herein, an engine includes one or more processors or a part of one or more processors. A part of one or more processors may include certain parts of hardware, rather than all hardware, that includes any given one or more processors, such as a subset of registers, a portion of a processor dedicated to one or more threads of a multi-threaded processor, a time segment in which all or part of a processor is dedicated to executing part of the functions of an engine, etc. Therefore, the first engine and the second engine may have one or more dedicated processors, or the first engine and the second engine may share one or more processors with each other or with other engines. Depending on the considerations of a particular embodiment or other considerations, the engine may be centralized, or its functions may be distributed. The engine may include hardware, firmware, or software specifically implemented in a computer-readable medium for execution by a processor. For example, as described with reference to the figures herein, the processor converts data into new data using implemented data structures and methods.

[0288] The engine described herein or the engine that can implement the systems and devices described herein can be a cloud-based engine. As used herein, a cloud-based engine is an engine that can run applications and / or functions using a cloud-based computing system. All or part of the applications and / or functions can be distributed across multiple computing devices and need not be limited to only one computing device. In some embodiments, the cloud-based engine can execute functions and / or modules that an end user accesses through a web browser or container application without having to install the functions and / or modules locally on the end user's computing device.

[0289] As used herein, a data warehouse is intended to include a repository having any applicable data organization, including tables, comma-separated value (CSV) files, traditional databases (e.g., SQL), or other applicable known or convenient organizational formats. For example, a data warehouse can be implemented as software that is embodied in a physical computer-readable medium located on a dedicated machine, in firmware, in hardware, in a combination thereof, or in an applicable known or convenient device or system. Components associated with a data warehouse (e.g., a database interface) can be considered to be "part of" the data warehouse, part of some other system component, or a combination thereof, although the physical location and other characteristics of the components associated with the data warehouse are not critical to understanding the technology described herein.

[0290] A data warehouse may include data structures. As used herein, a data structure is associated with a specific way of storing and organizing data in a computer so that it can be effectively used in a given context. Data structures are generally based on the ability of a computer to access and store data at any location in its memory, where the location is specified by an address, which is a bit string that can itself be stored in memory and manipulated by a program. Thus, some data structures are based on using arithmetic operations to calculate the addresses of data items; while other data structures are based on storing the addresses of data items in the structure itself. Many data structures use both principles, sometimes combined in non-trivial ways. The implementation of a data structure typically involves writing a set of programs that create and manipulate instances of the structure. The data warehouse described herein may be a cloud-based data warehouse. A cloud-based data warehouse refers to a data warehouse that is compatible with cloud-based computing systems and engines.

[0291] Although reference is made to instruments comprising polymeric shell instruments, the embodiments disclosed herein are well suited for use with many instruments that house teeth (e.g., instruments that do not have one or more of a polymer or a shell). For example, the instruments can be made from one or more of many materials, such as metal, glass, reinforced fibers, carbon fibers, composite materials, reinforced composite materials, aluminum, biomaterials, and combinations thereof. For example, the instruments can be formed in a variety of ways, such as by thermoforming as described herein or directly manufactured. Alternatively or in combination, the instruments can be manufactured by machining, such as by computer numerical control machining of an instrument from a block of material. In addition, although orthodontic instruments are mentioned herein, at least some of the techniques described herein may be applicable to restorative instruments and / or other dental instruments, including but not limited to crowns, ornaments, tooth whitening instruments, tooth protection instruments, and the like.

[0292] Although preferred embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that these embodiments are provided by way of example only. Without departing from the present invention, those skilled in the art may now make many modifications, variations, and substitutions. It should be understood that various alternatives to the embodiments of the present invention described herein may be adopted in the process of practicing the present invention. The following claims are intended to define the scope of the present invention, and methods and structures within the scope of these claims and their equivalents will thus be encompassed.

Claims

1. A computer-implemented method for simulating a patient's gums, the computer-implemented method comprising: capturing a first 2D image of the patient's face including the patient's teeth, lips, and gums; constructing a parametric model of the patient's gums based on the first 2D image, the parametric model comprising patient-specific input parameters for the patient's gums; as well as A second 2D image of the patient's face having teeth and gums is rendered based on the parametric model of the patient's gums.

2. A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform a method comprising: capturing a first 2D image of the patient's face including the patient's teeth and gums; constructing a parametric model of the patient's gums based on the first 2D image, wherein the parametric model includes patient-specific input parameters of the patient's gums; aligning coordinates of patient-specific parameters of the patient's gums; and A second 2D image of the patient's face is rendered based on the aligned coordinates of the patient's gums.

3. A computer-implemented method for simulating orthodontic treatment, the computer-implemented method comprising: rendering a first color 2D image comprising a color representation of the patient's face, teeth, gums, lips, or any combination thereof; identifying one or more edges of the patient's face, teeth, gums, lips, or any combination thereof; generating a parametric 3D model of the patient's face, teeth, gums, lips, or any combination thereof from the identified one or more edges; simulating the position of the patient's teeth by rendering the parametric 3D model of the patient's teeth in predetermined positions of the treatment plan; mapping color information from the first color 2D image onto the parametric 3D model; as well as A second color 2D image is generated based on the predetermined positions of the treatment plan of the parametric 3D model and the mapped color information of the parametric 3D model, the second color 2D image representing the patient's face, teeth, gums, lips, or any combination thereof.

4. A computer-implemented method of rendering a patient's gums, the computer-implemented method comprising: capturing a first 2D image of the patient's face including the patient's teeth and gums; A parametric model of the patient's gingiva comprising patient-specific input parameters for the patient's gingiva is constructed based on the first 2D image by: Finding the edges of the teeth, gums, and lips in the first 2D image, and determining the patient-specific input parameters from the edges of the teeth, gums, and lips; aligning coordinates of patient-specific input parameters from a parametric model of the patient's gingiva; as well as A second 2D image of the patient's face with teeth and gums is rendered based on the aligned coordinates of the patient's gums.

5. A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform the method according to claim 4.

Images