Automatic alignment and orientation of digital 3D dental arch pairs

Abstract

A method for aligning a digital 3D model of a pair of dental arches, including a mandible and a maxilla, to produce an occlusion. The method estimates representative planes for the mandible and maxilla of the 3D model and transforms the model so that the representative planes are aligned with their respective coordinate systems and represented in a horizontal view. The method also transforms the model so that the mandible and maxilla are aligned with the same coordinate system and represented in a front view. The digital 3D model is then occluded so that the mandible is in occlusion with the maxilla or within a near-occlusal distance. The digital model of the occluded pair of dental arches can also be translated or rotated so that it is displayed centered in the front view.

Description

[Background technology]

[0001] The use of intraoral three-dimensional (3D) scanners is becoming increasingly popular. These scanners generate digital 3D models that represent the 3D structure of a patient's dentition, including both hard and associated soft tissues. These 3D scans are useful in many applications, most commonly used in digital dentistry and orthodontic workflows, such as the fabrication of crowns, implants, and appliances. Using these intraoral scanners, 3D scans of both the upper and lower arches can be obtained. Generally, the digital 3D representations of the arches may not be oriented relative to a desired viewpoint or relative to each other. Therefore, a need exists to automatically detect and adjust these orientations of digital 3D dental arch pairs. Summary of the Invention

[0002] A method of aligning a pair of dental arches consistent with the present invention includes receiving a first digital 3D model of at least a portion of a person's lower jaw and a second digital 3D model of at least a portion of the person's upper jaw. First and second representative surfaces are estimated for the lower jaw and the upper jaw. The first and second digital 3D models are transformed so that the first and second representative surfaces are aligned with respective coordinate systems. The first and second digital 3D models are also transformed so that the lower jaw and the upper jaw are aligned with the same coordinate system. After transformation, the aligned digital 3D models resulting in an alignment of the first and second digital 3D models may also be translated or rotated so that the lower jaw and the upper jaw are represented in a frontal view, such that the lower jaw is aligned with the upper jaw. [Brief explanation of the drawings]

[0003] The accompanying drawings, which are incorporated in and constitute a part of this specification, and together with the description, serve to explain the advantages and principles of the present invention.

[0004] [Figure 1]FIG. 1 is a diagram of a system for receiving and processing a digital 3D model based on an intraoral 3D scan or a 3D scan of an impression. [Figure 2] 1 is a flowchart of a method for automatic alignment and orientation of a pair of dental arches. [Figure 3] This is an example of parameterization of a bow curve. [Figure 4] 10 is an example of a full arch scan and a quadrant scan. [Figure 5] An example of a non-engaged arch pair with estimated representative surfaces. [Figure 6] This is an example of an interlocked pair of bows. [Figure 7] This is an example of a pair of arches before they are brought into engagement. [Figure 8] 8 illustrates the pair of arches of FIG. 7 in schematic engagement. DETAILED DESCRIPTION OF THE INVENTION

[0005] Described herein is a technique for automatically aligning and orienting a pair of three-dimensional dental arches, such as mandibular and maxillary scans (a person's upper and lower arches). This technique can result in the arch pair being oriented to face in a standard direction and in close proximity to each other, with the bite shown in a horizontal front view. This technique can also be used to align and orient the arch pair in other directions and other perspectives.

[0006] FIG. 1 is a diagram of a system 10 for receiving and processing a digital 3D model based on an intraoral 3D scan. The system 10 includes a processor 20 that receives a digital 3D model of a tooth (12) from an intraoral 3D scan or a dental impression scan. The system 10 can also include an electronic display device 16, such as a liquid crystal display (LCD) device, and an input device 18 that receives user commands or other information. Systems for generating digital 3D images or models based on a set of images from multiple perspectives are disclosed in U.S. Patent Nos. 7,956,862 and 7,605,817, both of which are incorporated by reference herein as if fully set forth. These systems can use an intraoral scanner to obtain digital images from multiple views of a tooth or other intraoral structure, which are then processed to generate a digital 3D model representing the scanned tooth. The system 10 can be implemented, for example, using a desktop, notebook, or tablet computer. The system 10 can receive the 3D scan locally or remotely over a network.

[0007] The 3D scans discussed in this paper are represented as triangle meshes. A triangle mesh is a generic representation of a 3D surface and has two components. The first component, called the mesh vertices, is simply the coordinates of the 3D points reconstructed onto the surface, i.e., the coordinates of the point cloud. The second component, the mesh faces, encodes the connectivity between points on the object and is an efficient way to interpolate between discrete sample points on a continuous surface. Each face is a triangle defined by three vertices, resulting in a surface that can be represented as a set of small triangular face patches.

[0008] The techniques described herein use previously acquired 3D scans of the patient's mandible and maxilla. In some cases, these scans are full-arch, meaning they include all teeth within the arch; in other cases, they may be quadrant scans containing only four to six teeth. Similarly, in some cases, the scans of the two arches may be interdigitated, meaning the arches are positioned relative to one another in 3D space so that the teeth are in occlusion. However, in general, the arches can be positioned in any 3D position relative to one another. In either case, the technique estimates a 3D transformation, including rigid body rotations and translations, that moves the arches to a position in 3D space such that the arches are interdigitated with one another so that the teeth are at least approximately in occlusion, the bite planes are horizontal and parallel to the XZ plane, the arches are canonically oriented and opposed along the Z axis, and the arches are centered on the origin.

[0009] The coordinate systems used herein are for illustrative purposes only, and the techniques described can be used to align and orient the bow pairs between other coordinate systems and in other perspectives. For example, an interlocked bow pair can be shown in a side view, with the front of the interlocked bow pair facing the X-axis and the side of the interlocked bow pair facing the Z-axis from the observer's perspective. Additionally, once the bow pairs are interlocked, the observer can optionally interact with the interlocked bow pair by rotating the bow pair to view it from a desired perspective.

[0010] To achieve these alignment and orientation techniques, method 22 in the flowchart of Figure 2 can be used to process received cases that include digital 3D scans of the mandibular arch and maxillary arch from the same person. Method 22 can be implemented in a software or firmware module, for example, for execution by processor 20, or it can be implemented in a hardware module or a combination of software and hardware.

[0011] Method 22 includes detecting whether a received scan, such as scan 12, is a full arch or a quadrant (step 24) and determining whether the scans are aligned (step 26). If the scans are already aligned, method 22 includes translating (e.g., rotating) both arches so that their representative planes (e.g., occlusal planes) are horizontal (step 28), determining an in-plane rotation about the Y axis to orient the arches along the Z axis (step 30), and centering the arches at the origin (step 42).

[0012] If the scans are not aligned (step 26), method 22 includes translating (e.g., rotating) the mandible so that its representative plane (e.g., the occlusal plane) is horizontal (step 32), translating (e.g., rotating) the mandible so that its representative plane (e.g., the occlusal plane) is horizontal (step 34), determining a translation (e.g., rotation) about the Y axis to orient the mandible along the Z axis (step 36), determining a translation (e.g., rotation) about the Y axis to orient the maxilla along the Z axis (step 38), bringing the two arches into alignment so that they are at least approximately aligned (step 40), and centering the arches at the origin (step 42).

[0013] Transforming (and transformation) may include, but is not limited to, rotation, translation, or a combination of both rotation and translation.

[0014] Each of these steps, along with the other processing steps contained therein, is described below, and as also described below, steps 30, 36, and 38 differ depending on whether the scan is a full arc or a quadrant.

[0015] Several steps in method 22 require parameterization of the arch curve, which is essentially a smooth one-dimensional (1D) curve that traverses the arch and follows the crests of the teeth. This curve can be found by computing a 1D manifold embedding of the top part of the scan (i.e., the part near the occlusal surface) using a technique such as Locally Linear Embedding (LLE). An example of arch curve parameterization is shown in Figure 3, with the 1D curve 44 superimposed on the image of the arch. The arrow pointing out from the arch indicates the normal direction of the arch.

[0016] The 3D model acquired can be either a full-arch scan, covering all teeth present in the arch, or a quadrant scan, covering only 4-6 teeth. Examples of both types of scans are shown in Figure 4, which shows a full-arch scan 46 and a quadrant scan 48.

[0017] Whether a scan is a full arch or a quadrant can be determined automatically by parameterizing the arch as shown in FIG. 3 and measuring the geodesic length of the curve, e.g., curve 44. The arch curve of a full arch scan will be longer than the arch curve of a quadrant scan, and a suitable length threshold distinguishes between the two. In practice, a suitable threshold may be, for example, 70 mm, with arch curves above this threshold considered to represent a full arch and bow curves below the threshold considered to be quadrants. Other thresholds may also be used.

[0018] For embodiments of the present invention, a pair of arch (i.e., mandibular and maxillary arch) scans are considered in occlusion if they are close to each other perpendicular (vertical) to their representative planes and have significant overlap between them when projected onto their representative planes. Close to each other with respect to occlusion indicates that the arch pair is in occlusion, meaning that they are touching each other, or at a particular distance, such as within 20 mm of occlusion, meaning that the occluded arch pair is 20 mm or less apart.

[0019] The representative plane can be estimated using various techniques, such as those described in the section entitled "Alignment Method 3 - Regression or Plane Fitting" of U.S. Patent Application Publication No. 2016 / 0070821, specifically Table 3 therein. A support vector regression (SVR) approach has been found to robustly estimate the representative plane of a single arch, regardless of whether the single arch is a quadrant of the full arch scan. Table 1 provides exemplary pseudocode for implementing this SVR approach to estimate the representative plane.

[0020] [Table 1]

[0021] This representative surface estimation results in a surface of the following form: ax+by+cz+d=0 Here, this surface normal vector is given by the following equation: n=[abc] T

[0022] Figure 5 shows an example of a pair of uninterlocked bows 50 and 54, along with estimated representative planes 52 and 56, respectively. An example of an interlocked bow pair is shown in Figure 6. In addition to being interlocked, the bow pair shown in Figure 6 is also oriented in the direction of the desired viewpoint, in this case a frontal viewing direction, where the bows face the Z axis from the observer's viewpoint and are substantially horizontal to the XZ plane.

[0023] As mentioned above, one criterion for determining whether two arches are interdigitated is based on the normal distance between the two representative planes of the arches, which can be calculated for the mandibular arch according to the method in Table 2 (steps 2a, 2b, and 2c).

[0024] [Table 2]

[0025] The method in Table 2 is repeated for the maxillary arch. If these distances are large, this indicates that the mandibular and maxillary representative surfaces are far apart, as shown in Figure 5.

[0026] The second criterion for determining whether two bows are interdigitated is based on how much overlap there is between them when projected onto a surface. This is accomplished by first projecting both bows onto the smaller bow's representative surface, then measuring how much of the area occupied by one bow in this two-dimensional (2D) projection is occupied by the other bow. Denoting the smaller bow's representative surface as n and d, a coordinate system is formed with the Y axis as the normal vector and the X and Z axes as orthogonal. The vertices of both meshes are projected onto the X and Z axes, yielding the 2D coordinates of each point. This 2D space is then quantized, and the percentage of overlapping cells (occupied by both bows) is tabulated.

[0027] As identified above and listed in Table 1, an approximate representative surface can be fitted to each bow using one of the techniques described in U.S. Patent Application Publication No. 2016 / 0070821, an example of which is shown in Figure 5. The goal here is to rotate the bow in 3D space so that this representative surface is horizontal, i.e., parallel to the XZ plane of the coordinate system used herein.

[0028] The procedure for calculating this rotation is as follows: T Given x+d=0, the goal is to calculate a new coordinate system in which this plane is horizontal. This new reference can be expressed as [a1a2a3], where: 1. The new Y axis is a2 = n / ||n||, which is parallel to the normal vector of the face. For consistency, in this procedure, we will compare a2 with the original Y axis

[0010] . T Check that the dot product with a2 is positive, and if not, cancel a2. 2. The new X axis is a1 = a2 ×

[0001] Tis chosen to be orthogonal to both the previous Z axis and the new Y axis, according to . As before, for consistency, this procedure checks that the dot product of a1 with the original X axis is positive, and cancels a1 if it is not. 3. Finally, a new Z axis is chosen to be orthogonal to the new X and Y axes: a3 = a1 × a2. This also ensures that it has a positive dot product with the original Z axis.

[0029] We can then form a rotation matrix R=[a1a2a3], which rotates each vertex in 3D according to x'=Rx.

[0030] If the scans are found to be already aligned, then both arches are rotated according to the mandibular representative plane; otherwise, if the scans are not aligned, each arch is rotated separately according to its own representative plane.

[0031] After ensuring that the representation plane is horizontal, the bows are rotated in-plane about the Y axis to align them to a standard orientation along the Z axis. This rotation is estimated differently based on whether the scan is a full-arch or quadrant scan, as explained below. If the scans are not interdigitated, this rotation about the Y axis is estimated independently for each bow. Otherwise, if the scans are interdigitated, the average rotation estimated for the two bows is used.

[0032] The above parameterization of the bow curve produces a smooth 1D curve that traverses the bow, such as curve 44 shown in Figure 3. Given this curve, its midpoint can be estimated by finding the points along the curve that lie half its geodesic distance on either side, such as point 45 on curve 44 in Figure 3.

[0033] We have the normal vector of the bow curve at the midpoint location. The direction of this vector is simply given by the arctangent of its X and Z components. This angle then indicates the amount by which the bow should be rotated around the Y axis so that this midpoint is directly opposed along the Z axis.

[0034] A different approach is taken with quadrant scans, as the midpoint of the arch is no longer a fixed position along the arch but is highly dependent on which teeth are included in the scan. Instead, the goal with quadrant scans is simply to align the arch as closely as possible along the Z axis. This is achieved by using principal component analysis (PCA) to calculate the first principal component of the vertices in 3D and then rotating the mesh so that this component is aligned with the Z axis.

[0035] If the arches are not in occlusion, they are brought into approximate alignment as follows: First, the maxilla is shifted along the Y axis so that its representative plane is the desired distance from that of the mandibular plane; for example, the planes should be at least approximately in occlusion by being no more than 20 mm apart. Then, the maxilla is shifted in the XZ plane so that its center of gravity coincides with that of the mandibular one.

[0036] An example of a pair of nearly interlocking front and rear arches is shown in Figures 7 and 8, respectively.

[0037] Another embodiment includes the following process: Automatic alignment of arch pair scans can be achieved through a combination of 3D mesh processing and optimization. First, the upper and lower arches are approximately aligned in the superior-subject direction (which is substantially parallel to the Z axis), defined as the principal component with the smallest corresponding eigenvalue when applying PCA to the mesh vertices. The arches are then oriented to align in the XY plane. This can be achieved by aligning 1D parameterized arch morphologies or by identifying the locations of landmarks on the teeth, such as canines, and setting the landmarks so that they are aligned along the XY plane; for example, by setting the tips of each pair of upper and lower canines so that the distance between each pair of canines is below a threshold. Another approach to anatomical landmarks is to use geometric features (e.g., spin image descriptors) around each vertex of the mesh. Next, vertices of the lower and upper arches with similar spin image descriptors are matched to each other, and a robust rigid transformation that aligns the majority of the matched vertices is estimated and applied to the upper arch. The arch is then transformed in 3D in an iterative manner until all directional distances are positive and minimized. For example, the 3D transformation can be determined using a constrained Iterative Closest Point (ICP) algorithm. This constrained / penalized ICP algorithm estimates a rigid transformation between two meshes such that negative distances between the meshes are penalized more aggressively compared to positive distances, and vice versa. This alternative embodiment can be used to present an occluded arch with teeth in occlusion. In addition to the above-described embodiments, the following aspects will be noted. (Appendix 1) 1. A method for aligning a pair of dental arches, comprising: receiving a first digital 3D model of at least a portion of a mandible of the person and a second digital 3D model of at least a portion of an upper jaw of the person; estimating a first representative surface in the first digital 3D model of the mandible and a second representative surface in the second digital 3D model of the maxilla; a first transformation step of transforming the first digital 3D model and the second digital 3D model such that the first representative surface and the second representative surface are aligned with respective coordinate systems; a second transformation step of transforming the first digital 3D model and the second digital 3D model such that the mandible and the maxilla in the digital 3D models are respectively aligned to the same coordinate system; after the first and second transforming steps, producing an occlusion between the first digital 3D model and the second digital 3D model such that in the digital 3D model the lower jaw is occluded with the upper jaw. (Appendix 2) 2. The method of claim 1, further comprising converting the first digital 3D model and the second digital 3D model so that the occluded mandible and maxilla are oriented in a frontal view. (Appendix 3) 2. The method of claim 1, wherein the first transformation step includes transforming the first digital 3D model and the second digital 3D model so that the first representative surface and the second representative surface are horizontal from an observer's perspective. (Appendix 4) 2. The method of claim 1, wherein the second transforming step includes transforming the first digital 3D model and the second digital 3D model so that the mandible and the maxilla in the digital 3D models are oriented in the same specific view. (Appendix 5) 2. The method of claim 1, further comprising detecting whether the first digital 3D model represents a full arch or a quadrant arch of the mandible and whether the second digital 3D model represents a full arch or a quadrant arch of the maxilla. (Appendix 6) The detecting step includes: superimposing a first curve onto the first digital 3D model across the mandible and determining whether a length of the first curve is greater than a threshold; and superimposing a second curve onto the second digital 3D model that crosses the upper jaw and determining whether the length of the second curve is greater than the threshold. (Appendix 7) The step of providing comprises: a first shifting step of shifting the second digital 3D model so that the second representative surface is at a specific distance from the first representative surface; and a second shifting step of shifting the first digital 3D model so that the center of gravity of the mandible is aligned with the center of gravity of the maxilla in the digital 3D model. (Appendix 8) 8. The method of claim 7, wherein the first shifting step includes shifting the second digital 3D model so that the second representative surface is within 20 mm of the first representative surface. (Appendix 9) 2. The method of claim 1, wherein the resulting step includes transforming the first digital 3D model and the second digital 3D model in an iterative manner until a directional distance between the mandible and the maxilla in the digital 3D models is positive and minimized. (Appendix 10) 1. A system for aligning a pair of dental arches, comprising: a module for receiving a first digital 3D model of at least a portion of a mandible of the person and a second digital 3D model of at least a portion of an upper jaw of the person; a module for estimating a first representative surface in the first digital 3D model of the mandible and a second representative surface in the second digital 3D model of the maxilla; a first transformation module for transforming the first digital 3D model and the second digital 3D model such that the first representative surface and the second representative surface are aligned with respective coordinate systems; a second transformation module for transforming the first digital 3D model and the second digital 3D model such that the mandible and the maxilla in the digital 3D models are respectively aligned to the same coordinate system; and a module for effecting an occlusion of the first digital 3D model and the second digital 3D model such that after the conversion, the lower jaw in the digital 3D model is occluded with the upper jaw. (Appendix 11) 11. The system of claim 10, further comprising a module for converting the first digital 3D model and the second digital 3D model so that the occluded mandible and maxilla are oriented in a frontal view. (Appendix 12) 11. The system of claim 10, wherein the first transformation module includes a module for transforming the first digital 3D model and the second digital 3D model so that the first representative plane and the second representative plane are horizontal from a viewer's perspective. (Appendix 13) 11. The system of claim 10, wherein the second transformation module includes a module for transforming the first digital 3D model and the second digital 3D model so that the mandible and the maxilla in the digital 3D models are oriented in the same specific view. (Appendix 14) 11. The system of claim 10, further comprising a module for detecting whether the first digital 3D model represents a full arch or a quadrant arch of the mandible and whether the second digital 3D model represents a full arch or a quadrant arch of the maxilla. (Appendix 15) The module for detecting a module for superimposing a first curve onto the first digital 3D model across the mandible and determining whether a length of the first curve is greater than a threshold; and a module for overlaying a second curve onto the second digital 3D model that intersects the upper jaw and determining whether a length of the second curve is greater than the threshold. (Appendix 16) The resulting module: a first shifting module for shifting the second digital 3D model so that the second representative surface is a specific distance from the first representative surface; and a second shift module for shifting the first digital 3D model such that a center of gravity of the mandible in the digital 3D model is aligned with a center of gravity of the maxilla. (Appendix 17) 17. The system of claim 16, wherein the first shifting module includes a module for shifting the second digital 3D model so that the second representative surface is within 20 mm of the first representative surface. (Appendix 18) 11. The system of claim 10, wherein the resulting module includes a module for transforming the first digital 3D model and the second digital 3D model in an iterative manner until a directional distance between the mandible and the maxilla in the digital 3D models is positive and minimized. (Appendix 19) 1. A method for aligning a pair of dental arches, comprising: receiving a first digital 3D model of at least a portion of a mandible of the person and a second digital 3D model of at least a portion of an upper jaw of the person; estimating a first representative surface in the first digital 3D model of the mandible and a second representative surface in the second digital 3D model of the maxilla; a first transformation step of transforming the first digital 3D model and the second digital 3D model so that the first representative surface and the second representative surface are aligned with respective coordinate systems, the first transformation step including transforming the first digital 3D model and the second digital 3D model so that the first representative surface and the second representative surface are horizontal from a viewpoint of a viewer; a second transformation step of transforming the first digital 3D model and the second digital 3D model so that the mandible and the maxilla in the digital 3D models are respectively aligned to the same coordinate system, the second transformation step including transforming the first digital 3D model and the second digital 3D model so that the mandible and the maxilla in the digital 3D models are respectively oriented to the same specific view; after the first and second transforming steps, bringing about an occlusion of the first digital 3D model and the second digital 3D model such that the lower jaw in the digital 3D model is occluded with the upper jaw, The step of providing comprises: a first shifting step of shifting the second digital 3D model so that the second representative surface is at a specific distance from the first representative surface; and a second shifting step of shifting the first digital 3D model so that the center of gravity of the mandible in the digital 3D model is aligned with the center of gravity of the maxilla. (Appendix 20) detecting whether the first digital 3D model represents a full arch or a quadrant arch of the mandible and whether the second digital 3D model represents a full arch or a quadrant arch of the maxilla; The detecting step includes: superimposing a first curve onto the first digital 3D model across the mandible and determining whether a length of the first curve is greater than a threshold; 20. The method of claim 19, comprising the steps of: superimposing a second curve onto the second digital 3D model that intersects the upper jaw; and determining whether a length of the second curve is greater than the threshold.

Claims

1. 1. A computer-implemented method for aligning a pair of dental arches, comprising: a processor included in the computer receiving a first digital 3D model of at least a portion of a mandible of the person and a second digital 3D model of at least a portion of an upper jaw of the person; the processor estimating a first occlusal surface in the first digital 3D model of the mandible and a second occlusal surface in the second digital 3D model of the maxilla; a first transformation step in which the processor transforms the first digital 3D model and the second digital 3D model so that the first occlusal surface and the second occlusal surface are aligned in their respective coordinate systems, the first transformation step including transforming the first digital 3D model and the second digital 3D model so that the first occlusal surface and the second occlusal surface are horizontal in their respective coordinate systems; a second transformation step in which the processor transforms the first digital 3D model and the second digital 3D model such that the mandible and the maxilla in the digital 3D models are respectively aligned to the same coordinate system; the processor producing an occlusion of the first digital 3D model and the second digital 3D model such that, after the first and second transforming steps, the lower jaw is occluded with the upper jaw in the digital 3D model.

2. The method of claim 1, further comprising, after the receiving step and before the first transformation step and the second transformation step, a step in which the processor detects whether the first digital 3D model represents the full arch or quadrant arch of the mandible, and whether the second digital 3D model represents the full arch or quadrant arch of the maxilla.

3. 1. A system for aligning a pair of dental arches, comprising: a module for receiving a first digital 3D model of at least a portion of a mandible of the person and a second digital 3D model of at least a portion of an upper jaw of the person; a module for estimating a first occlusal surface in the first digital 3D model of the mandible and a second occlusal surface in the second digital 3D model of the maxilla; a first transformation module for transforming the first digital 3D model and the second digital 3D model such that the first occlusal surface and the second occlusal surface, respectively, are aligned to respective coordinate systems; a second transformation module for transforming the first digital 3D model and the second digital 3D model such that the mandible and the maxilla in the digital 3D models are respectively aligned to the same coordinate system; a module for providing an occlusion between the first digital 3D model and the second digital 3D model such that after the transformation, the lower jaw in the digital 3D model is occluded with the upper jaw; the first transformation module includes modules for transforming the first digital 3D model and the second digital 3D model, respectively, so that the first occlusal plane and the second occlusal plane are horizontal in the respective coordinate systems.

4. 4. The system of claim 3, further comprising a module for detecting whether the first digital 3D model represents a full arch or a quadrant arch of the mandible and whether the second digital 3D model represents a full arch or a quadrant arch of the maxilla.

Images