Intraoral scanning and tracking for diagnosis

Abstract

To provide methods and apparatuses for taking, using and displaying dental information including information extracted from three-dimensional (3D) volumetric models of a patient's dental arch.SOLUTION: A 3D volumetric model may include surface (e.g., color) information as well as information on internal structures (e.g., near-infrared transparency values for internal structures including enamel and dentin). Described herein are methods and user interfaces for displaying and manipulating (e.g., sectioning, marking, selecting sub-regions of, etc.) 3D volumetric models.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This patent application claims priority to U.S. Provisional Patent Application No. 62 / 622,798, filed January 26, 2018, entitled "DIAGNOSTIC INTRAORAL SCANNERS," and to U.S. Provisional Patent Application No. 62 / 758,503, filed November 9, 2018, entitled "DIAGNOSTIC INTRAORAL SCANNERS," each of which is incorporated herein by reference in its entirety.

[0002] This patent application is a continuation of U.S. patent application Ser. No. 15 / 662,234, filed July 27, 2017, entitled "INTRAORAL SCANNER WITH DENTAL DIAGNOSTICS CAPABILITIES," which claims priority to U.S. Provisional Patent Application Ser. No. 62 / 367,607, filed July 27, 2016, and U.S. Provisional Patent Application Ser. No. 62 / 477,387, filed March 27, 2017, and U.S. patent application Ser. No. 15 / 662,250, filed July 27, 2017, entitled "METHODS AND APPARATUSES FOR FORMING A THREE-DIMENSIONAL VOLUMETRIC MODEL OF A SUBJECT'S TEETH." This application may be related to one or more of U.S. patent application Ser. No. 15 / 672,248, filed Aug. 8, 2017, entitled "METHODS FOR DENTAL DIAGNOSTICS," which claims priority to U.S. provisional patent application Ser. Nos. 62 / 367,607, filed July 27, 2016, and 62 / 477,387, filed March 27, 2017. Each of these applications is incorporated herein by reference in its entirety.

[0003] Literature citations All publications and patent applications mentioned in this specification are herein incorporated by reference in their entirety to the same extent as if each individual publication or patent application was specifically and individually indicated to be incorporated by reference. [Background technology]

[0004] Various dental and orthodontic procedures benefit from accurate three-dimensional (3D) representations of a patient's dentition and the interior of their mouth. In particular, it would be useful to have a 3D representation of both the surface and internal structure of the teeth (including enamel and dentin, as well as caries), as well as the general internal composition of the tooth volume. While a surface representation of the 3D tooth surface alone has proven very useful in the design and fabrication of dental prostheses (e.g., crowns and bridges) and treatment planning, it would be extremely useful to be able to visualize the internal structure, including the manifestation of caries and cracks in the enamel and underlying dentin, especially when combined with surface topography mapping.

[0005] Historically, ionizing radiation (e.g., x-rays) has been used to image the interior of teeth. For example, bitewing radiographs are often used to obtain non-quantitative images of the interior of teeth. However, in addition to the risks of ionizing radiation, such images typically have limited characterization capabilities and can be time-consuming and costly to obtain. Some intraoral features, such as soft tissue, plaque, and calculus, are not easily visualized with x-rays due to their low density. Other techniques, such as cone-beam computed tomography (CBCT), can obtain tomographic images but also require ionizing radiation.

[0006] It would therefore be useful to provide methods and apparatus (including devices and systems (e.g., intraoral scanning systems)) that can be used to model one or more teeth of a subject using non-ionizing radiation, encompassing both the external (surface) and internal (within the enamel and dentin) structure and composition. The model of the subject's teeth can be a 3D volumetric model or a panoramic image. In particular, it would be useful to provide methods and apparatus that can provide this functionality in a single device. Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention has been made to solve the above-mentioned problems of the prior art. [Means for solving the problem]

[0008] Described herein are methods and apparatus for acquiring, using, and displaying dental information, including information extracted from a three-dimensional (3D) volumetric model of a patient's dental arch. The 3D volumetric model may include surface (e.g., color) information and internal structure information (e.g., near-infrared transparency values ​​of internal structures, including enamel and dentin). In some variations, the 3D volumetric scan may include or be derived from one or more other scanning modalities, such as, but not limited to, optical coherence tomography (OCT), ultrasound (US), magnetic resonance imaging (MRI), and X-ray.

[0009] In particular, described herein are methods and user interfaces for displaying and manipulating (e.g., sectioning, marking, subregion selection, etc.) 3D volumetric models. For example, methods and apparatus for displaying images from 3D volumetric models are provided, including methods for generating cross sections of 3D volumetric models, methods for displaying both surface and internal structures, and methods for generating easy-to-interpret images, e.g., pseudo-X-ray images, from 3D volumetric models.

[0010] Further described herein are methods and apparatus for marking and tracking regions of interest from a 3D volumetric model of a patient's dental arch. These methods may include automatically, manually, or semi-automatically (e.g., with user approval or input) identifying and marking one or more regions in the 3D volumetric model (including surface and / or internal features of the dental arch), which may be regions where caries, cracks, or other irregularities have occurred or are likely to occur. The marked regions may be analyzed in detail and tracked over time. Furthermore, the marked regions may alter the manner in which subsequent scans are performed, for example, the marked region may be scanned at a higher resolution. Each region of the volumetric model may correspond to one or more voxels, including adjacent voxel regions. These regions may be referred to herein as volumetric regions.

[0011] Further described herein are methods and apparatus for improving or modifying dental procedures using 3D volumetric models, including modifying treatment plans and / or modifying one or more dental appliances. For example, described herein are dental tools that include 3D volumetric scans or that are capable of interfacing with 3D volumetric models (including robotic or automated control using 3D volumetric models). Also described are methods for diagnosing one or more conditions (e.g., dental conditions) using 3D volumetric models, particularly using 3D volumetric models over time.

[0012] One method of displaying images from a three-dimensional (3D) volumetric model of a patient's dental arch includes acquiring a 3D volumetric model of the patient's dental arch, the 3D volumetric model including surface color values, shade values, and near-infrared transparency values ​​of the internal structures of the dental arch; a user selecting an orientation for displaying a view of the 3D volumetric model; generating a two-dimensional (2D) view of the 3D volumetric model using the selected orientation, the view including the patient's dental arch, and including a weighted portion of the surface color values ​​and a weighted portion of the near-infrared transparency of the internal structures; and displaying the 2D view.

[0013] For example, described herein is a method for displaying images from a three-dimensional (3D) volumetric model of a patient's dental arch. The method may include receiving a 3D volumetric model of the patient's dental arch, the 3D volumetric model including surface color values ​​and near-infrared transparency values ​​of the internal structures of the dental arch, and generating a two-dimensional (2D) view of the patient's dental arch within the 3D volumetric model, the 2D view including both the surface values ​​and the near-infrared transparency of the internal structures. In any of the methods and apparatus described herein, the 3D model (including the volumetric 3D model) may be displayed as a voxel view. Thus, the methods described herein may generate one or more voxel views, each of which may have a color (or hue) corresponding to its density and / or translucency. Thus, one of the methods and apparatus described herein can generate a 3D colored map of all or a portion of the voxels of the 3D model (and can display one or more 2D images derived from the 3D colored view, e.g., cross-sections, slices, projections, oblique views, perspective views in which all or a portion of the 3D model is transparent, etc.). In some variations, flagged regions (e.g., regions corresponding to one or more irregular regions and / or regions (e.g., voxels) that have changed over time, regions / voxels to be removed, regions / voxels that are suspected of problems, etc.) can be displayed as 3D and / or 2D views.

[0014] Generating a two-dimensional (2D) view within the 3D volumetric model may include including in the 2D view a weighted portion of a surface color value and a weighted portion of a near-infrared transparency of an interior structure, where the near-infrared transparency may be based on or calculated from near-infrared scattering or near-infrared absorption of a material. The weighted portion of the surface color value may include a percentage of the full value of the surface color value, and the weighted portion of the near-infrared transparency of the interior structure includes a percentage of the full value of the near-infrared transparency of the interior structure, where the percentage of the full value of the surface color value and the percentage of the full value of the near-infrared transparency of the interior structure sum to 100%.

[0015] In some variations, the method further includes adjusting the displayed color value and / or the weighted portion of the near-infrared transparency of the internal structure by a user or in response to user input.

[0016] Any of these methods may include scanning the patient's dental arch with an intraoral scanner.

[0017] Generating the 2D view may include sectioning the 3D volumetric model at a plane intersecting the 3D volumetric model, and a user may select a cross section of the 3D volumetric model to display and / or an orientation of the 2D view.

[0018] For example, one method of displaying images from a three-dimensional (3D) volumetric model of a patient's dental arch may include receiving a 3D volumetric model of the patient's dental arch, the 3D volumetric model including surface color values ​​and near-infrared transparency values ​​of the internal structures of the dental arch; selecting, by a user or in response to user input, a cross-section of the 3D volumetric model to display; using the selected cross-section to generate a two-dimensional (2D) view of the 3D volumetric model that includes the patient's dental arch and, optionally, includes a weighted portion of the surface color values ​​and a weighted portion of the near-infrared transparency of the internal structures; and displaying the 2D view.

[0019] One method of displaying images from a three-dimensional (3D) volumetric model of a patient's dental arch may include collecting a 3D volumetric model of the patient's dental arch, the 3D volumetric model including surface values ​​and near-infrared transparency values ​​of the internal structures of the dental arch; generating a two-dimensional (2D) view of the 3D volumetric model that includes the patient's dental arch and includes both the surface values ​​and near-infrared transparency values ​​of the internal structures; and displaying the 2D view.

[0020] A method for tracking a region of a patient's dental arch over time may include receiving a first three-dimensional (3D) volumetric model of the patient's dental arch, the 3D volumetric model including surface color values ​​and near-infrared transparency values ​​of the internal structure of the dental arch; identifying regions of the 3D volumetric model to be marked; flagging the identified regions; and displaying one or more images of the 3D volumetric model showing the marked regions.

[0021] For example, one method for tracking a region of a patient's dental arch over time includes collecting a first three-dimensional (3D) volumetric model of the patient's dental arch, the 3D volumetric model including surface values ​​and near-infrared transparency values ​​of the internal structure of the dental arch; identifying a region of the 3D volumetric model; flagging the identified region; collecting a second 3D volumetric model of the patient's dental arch; and displaying one or more images with any differences between the first and second 3D volumetric models in the flagged region marked on the one or more images.

[0022] Identifying the regions may include automatically identifying using a processor. For example, automatically identifying may include identifying regions likely to have defects, including cracks and caries. Identifying regions likely to have defects may include comparing a near-infrared transparency value of a region in the 3D model to a threshold. Automatically identifying may include identifying surface color values ​​that fall outside a threshold range. Automatically identifying may include segmenting the 3D volumetric model to identify enamel regions and identifying regions where the enamel thickness is below a threshold. Flagging the identified regions may include automatically flagging the identified regions. Flagging the identified regions may include manually reviewing the identified regions for flagging.

[0023] Any of these methods may include receiving a second 3D volumetric model of the patient's dental arch and displaying any differences between the first 3D volumetric model and the second 3D volumetric model in the marked areas.

[0024] Additionally, any of these methods may include pre-scanning or re-scanning the patient's dental arch, where the scanning of the flagged regions may be performed at a higher resolution than the non-flagged regions and with a different scanning modality than the non-flagged regions.

[0025] For example, one method of tracking a region of a patient's dental arch over time may include receiving a first three-dimensional (3D) volumetric model of the patient's dental arch, the 3D volumetric model including surface color values ​​and near-infrared transparency values ​​of the internal structures of the dental arch, identifying regions of the 3D volumetric model to be marked using an automated process, flagging the identified regions, receiving a second 3D volumetric model of the patient's dental arch, and displaying any differences between the first and second 3D volumetric models in the marked regions. In some cases, the second 3D volumetric model of the patient's dental arch may be from a scan of the patient when the patient returns to the dental practitioner at a later date.

[0026] Thus, one method of tracking a region of a patient's dental arch over time may include collecting a first three-dimensional (3D) volumetric model of the patient's dental arch taken at an initial time point, the 3D volumetric model including surface color values ​​and near-infrared transparency values ​​of the internal structure of the dental arch; identifying, using an automated process, regions of the 3D volumetric model that should be flagged; flagging the identified regions; collecting a second 3D volumetric model of the patient's dental arch taken at a different time point; and displaying any differences between the first and second 3D volumetric models in the flagged regions.

[0027] Further described herein is a method for displaying pseudo-X-ray images from a three-dimensional (3D) volumetric model of a patient's dental arch. For example, one method may include receiving a 3D volumetric model of the patient's dental arch, the 3D volumetric model including near-infrared transparency values ​​of internal structures of the dental arch; generating a two-dimensional (2D) view of the 3D volumetric model including the patient's dental arch and near-infrared transparency values ​​of the internal structures; mapping the near-infrared transparency values ​​of the internal structures in the 2D view to pseudo-X-ray density, where the near-infrared transparency values ​​are inverted; and displaying the mapped pseudo-X-ray density. Generating the 2D view may include cross-sectioning the 3D volumetric model at a plane that intersects the 3D volumetric model. The 3D volumetric model may include surface information.

[0028] For example, one method of displaying pseudo-x-ray images from a three-dimensional (3D) volumetric model of a patient's dental arch may include acquiring a 3D volumetric model of the patient's dental arch, the 3D volumetric model including near-infrared transparency values ​​of internal structures of the dental arch; generating a two-dimensional (2D) view within the 3D volumetric model that includes the patient's dental arch and includes near-infrared transparency values ​​of the internal structures; mapping the near-infrared transparency values ​​of the internal structures in the 2D view to pseudo-x-ray density, the pseudo-x-ray density values ​​in the 2D view being based on inverted near-infrared transparency values; and displaying the mapped pseudo-x-ray density.

[0029] Any of these methods may include identifying a subregion in the 3D volumetric model before generating the 2D view, where the 2D view includes a 2D view of the identified subregion. The method may further include segmenting the 3D volumetric model into teeth, where generating the 2D view may include a 2D view including only one of the identified teeth.

[0030] Mapping the near-infrared transparency may include inverting the near-infrared transparency values ​​so that enamel in the 2D view is brighter than dentin in the 2D view.

[0031] One method of displaying pseudo-x-ray images from a three-dimensional (3D) volumetric model of a patient's dental arch may include receiving a 3D volumetric model of the patient's dental arch, the 3D volumetric model including surface features and near-infrared transparency values ​​of internal structures of the dental arch, where enamel is more transparent than dentin; generating a two-dimensional (2D) view within the 3D volumetric model that includes the patient's dental arch and that includes near-infrared transparency values ​​of internal structures, including dentin and enamel; mapping the near-infrared transparency of the internal structures in the 2D view to pseudo-x-ray density, where the near-infrared transparency values ​​are inverted so that enamel is brighter than dentin; and displaying the mapped pseudo-x-ray density.

[0032] For example, one method of displaying pseudo-x-ray images from a three-dimensional (3D) volumetric model of a patient's dental arch may include the steps of: acquiring a 3D volumetric model of the patient's dental arch, the 3D volumetric model including surface features and near-infrared transparency values ​​of the internal structures of the dental arch, where enamel is more transparent than dentin; generating a two-dimensional (2D) view within the 3D volumetric model that includes the patient's dental arch and near-infrared transparency values ​​of the internal structures, including dentin and enamel; mapping the near-infrared transparency of the internal structures in the 2D view to pseudo-x-ray density, where the near-infrared transparency values ​​are inverted so that enamel is brighter than dentin; and displaying the mapped pseudo-x-ray density.

[0033] Further described herein are methods and devices for performing real-time virtual review (e.g., virtual sectioning, virtual scanning, virtual exams) of a volumetric model of a patient's dental arch. These devices may include non-transitory, machine-readable tangible media storing instructions that cause one or more machines to perform operations to implement any of the methods described herein. In particular, any of these methods and devices may operate on a dataset that includes both 3D models of the patient's dental arches, or in some variations, both of the patient's dental arches. The 3D models may be, but are not limited to, 3D volumetric models; in some variations, the 3D models are 3D surface models of the dental arches. The dataset may also include multiple images of the dental arches taken from various positions relative to the dental arches, for example, from various angles between the plane of the image of the dental arch and various subregions of the dental arch. Some of these images may be taken from the occlusal surface, some from the gingival side, and some from the lingual side. In some variations, these images may be the same as the images (or may be a subset of those images) used to generate the 3D model of the teeth. The dataset may include multiple images taken of the same (or approximately the same) region of the dental arch and the same (or approximately the same) angle relative to the dental arch. In some variations, the dataset may include two or more sets of images (e.g., image pairs), each taken of approximately the same region of the dental arch and at the same angle relative to the dental arch, but using different imaging modalities (e.g., various imaging modalities such as visible light, infrared / near-infrared light, fluorescence, x-ray, ultrasound, etc.).

[0034] For example, one method may include the steps of displaying a three-dimensional (3D) model of a patient's dental arch; displaying a view window over at least a portion of the 3D model of the patient's dental arch; allowing a user to change the relative position between the view window and the 3D model of the patient's dental arch; and, as the user changes the relative position between the view window and the 3D model of the patient's dental arch, successively identifying, from both the 3D model of the patient's dental arch and multiple images of the patient's dental arch acquired from various angles and positions relative to the patient's dental arch, images acquired at angles and positions that approximate the relative angle and relative position between the view window and the 3D model of the patient's dental arch, and displaying the identified images acquired at angles and positions that approximate the relative angle and relative position between the view window and the 3D model of the patient's dental arch.

[0035] In any of the methods described herein, the dataset may include both a 3D model of the patient's dental arch and multiple images of the patient's dental arch taken from various angles and positions relative to the patient's dental arch. The dataset may also, or alternatively, include metadata associated with each value (or each set of values) indicating the angle and / or region of the dental arch from which the image was taken. Additional metadata may be included (e.g., metadata indicating the distance from the dental arch, metadata indicating the exposure time, metadata indicating that the image is an average of multiple other images, metadata indicating a quality measure of the image, etc.).

[0036] For example, described herein are methods for displaying a 3D model (e.g., a surface 3D model) of a patient's teeth and / or a volumetric model of the patient's teeth that allows a user to perform a more detailed virtual scan by moving a view window over the 3D model of the dental arch. For example, the methods described herein include the steps of displaying a three-dimensional (3D) model of the patient's dental arch; displaying a view window over a portion of the 3D model of the patient's dental arch; and allowing a user to change the relative position between the view window and the 3D model of the patient's dental arch, the relative position including one or more of an angle between the plane of the view window and the patient's dental arch and a portion of the dental arch proximate to the view window; and displaying a continuous image of the 3D model of the patient's dental arch as the user changes the relative position between the view window and the 3D model of the patient's dental arch. and subsequently, from both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch (e.g., in some variations, from a dataset including both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch), where each image is taken from a different angle and position relative to the patient's dental arch, identifying images taken at angles and positions that approximate a relative angle and position between the view window and the 3D model of the patient's dental arch, and displaying the identified images taken at angles and positions that approximate an angle and position of the view window relative to the displayed 3D model of the patient's dental arch.

[0037] For example, one method includes the steps of displaying a three-dimensional (3D) model of a patient's dental arch; displaying a view window over a portion of the 3D model of the patient's dental arch; The method may include the steps of: allowing a user to change a relative position between the viewing window and the 3D model of the patient's dental arch, the relative position including one or more of a relative angle between the patient's dental arch and a plane of the viewing window, and a portion of the dental arch proximate to the viewing window; and, as the user changes the relative position between the viewing window and the 3D model of the patient's dental arch, continuously identifying, from both the 3D model of the patient's dental arch and a plurality of image pairs of the patient's dental arch (e.g., optionally from a dataset including both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch), where each pair of the plurality of pairs includes a first imaging wavelength and a second imaging wavelength, respectively, acquired at the same angle and position relative to the patient's dental arch, image pairs acquired at an angle and position that approximates the angle and position of the viewing window relative to the displayed 3D model of the patient's dental arch; and displaying at least one of the identified image pairs acquired at an angle and position that approximates the angle and position of the viewing window relative to the displayed 3D model of the patient's dental arch.

[0038] The methods and apparatus described herein may be used with a 3D model that is a surface model or any representation of a patient's dental arch. This may be, but is not limited to, a 3D volumetric model of the patient's teeth (e.g., constructed from images) (e.g., multiple images of the patient's dental arch taken from various angles and positions relative to the patient's dental arch). The model may represent the patient's actual dentition, may be extracted from the patient's dentition, or may be generic.

[0039] One method described herein includes the steps of displaying a three-dimensional (3D) model of a patient's dental arch; displaying a view window over a portion of the 3D model of the patient's dental arch; and allowing a user to change a relative position between the view window and the 3D model of the patient's dental arch, the relative position including one or more of an angle between the view window and the patient's dental arch and a portion of the dental arch proximate to the view window; and continuously as the user changes the relative position between the view window and the 3D model of the patient's dental arch: From both a 3D model of the patient's dental arch and a plurality of near-infrared images of the patient's dental arch (e.g., from a dataset including both a 3D model of the patient's dental arch and a plurality of images of the patient's dental arch), where each near-infrared image is taken from a different angle and position relative to the patient's dental arch, identifying a near-infrared image taken at an angle and position that approximates the relative angle and position between the view window and the 3D model of the patient's dental arch, and displaying the identified near-infrared image taken at an angle and position that approximates the angle and position of the view window relative to the displayed 3D model of the patient's dental arch.

[0040] In any of these examples, the image may be an image acquired in a penetrating modality (eg, near-infrared). For example, a method described herein includes the steps of displaying a three-dimensional (3D) model of a patient's dental arch; displaying a view window over a portion of the 3D model of the patient's dental arch; allowing a user to change a relative position between the view window and the 3D model of the patient's dental arch, the relative position including one or more of the angle between the view window and the patient's dental arch and a portion of the dental arch proximate to the view window; and, as the user changes the relative position between the view window and the 3D model of the patient's dental arch, continuously identifying from both the 3D model of the patient's dental arch and multiple near-infrared images of the patient's dental arch (where each near-infrared image is acquired from a different angle and position relative to the patient's dental arch), a near-infrared image acquired at an angle and position that approximates the relative angle and position between the view window and the 3D model of the patient's dental arch, and displaying the identified near-infrared image acquired at an angle and position that approximates the angle and position of the view window relative to the displayed 3D model of the patient's dental arch.

[0041] Any of these methods may include identifying and displaying multiple images taken at the same angle and position relative to the dental arch. For example, the images may be both visible light images and penetrating images (e.g., infrared / near-infrared images, etc.). For example, a method described herein may include displaying a three-dimensional (3D) model of a patient's dental arch; displaying a view window over a portion of the 3D model of the patient's dental arch; and allowing a user to change the relative position between the view window and the 3D model of the patient's dental arch, the relative position including one or more of the relative angle between the patient's dental arch and the plane of the view window and the portion of the dental arch proximate to the view window. and as the display progresses, successively identifying, from a dataset including both a 3D model of the patient's dental arch and a plurality of image pairs of the patient's dental arch, wherein each pair of the plurality of pairs includes a first imaging wavelength and a second imaging wavelength each acquired at the same angle and position relative to the patient's dental arch, image pairs acquired at an angle and position that approximates the angle and position of the viewing window relative to the displayed 3D model of the patient's dental arch, and displaying the identified image pairs acquired at an angle and position that approximates the angle and position of the viewing window relative to the displayed 3D model of the patient's dental arch.

[0042] In any of these methods, the identifying step may include determining multiple images that approximate the relative angle and relative position between the viewing window and the 3D model of the patient's dental arch and averaging these multiple images to form the identified image. For example, there may be multiple images in a data set taken at approximately the same angle (e.g., within ±0.1%, 0.5%, 1%, 2%, 3%, 4%, 5%, 7%, 10%, 15%, 20%, etc.) and approximately the same region of the dental arch (e.g., within ±0.1%, 0.5%, 1%, 2%, 3%, 4%, 5%, 7%, 10%, 15%, 20%, etc.), and these similar images can be combined to form an average image that may be better than the individual images.

[0043] Generally, the step of identifying one or more images acquired at angles and positions that approximate the relative angle and relative position between the view window and the 3D model of the patient's dental arch may be performed within an acceptable spatial range. For example, images acquired within ± a few degrees of the same angle as the plane of the viewing window (e.g., within + / - 0.1 degrees, 0.2 degrees, 0.3 degrees, 0.4 degrees, 0.5 degrees, 0.6 degrees, 1 degree, 1.2 degrees, 1.5 degrees, 1.7 degrees, 1.8 degrees, 2 degrees, 2.2 degrees, 2.5 degrees, 3 degrees, 3.2 degrees, 3.5 degrees, 4 degrees, 5 degrees, etc.), and within ± a distance range of the dental arch region over which the viewing window is positioned (e.g., within + / - 0.1 mm, 0.2 mm, 0.3 mm, 0.4 mm, 0.5 mm, 0.6 mm, 0.7 mm, 0.8 mm, 0.9 mm, 1 mm, 1.1 mm, 1.2 mm, 1.5 mm, 1.7 mm, 2.0 mm, 2.2 mm, 2.5 mm, etc.).

[0044] Any of these methods may include receiving the dataset at a processor. The dataset may be received directly from the intraoral scanner and / or may be stored and retrieved. In some variations, the dataset may be transmitted or received by the processor, and in some variations, the processor may read the dataset from a memory (e.g., a data store) connected to the processor.

[0045] Generally, any of these methods may include displaying a view window over a portion of a 3D model of the patient's dental arch. The view window may be any shape or size, such as a circle, oval, triangle, rectangle, or other polygon. For example, the view window may be a loop through which a portion of the 3D model of the patient's dental arch can be viewed. The view angle may allow the dental arch to be visualized through at least a portion of the view window. The view window may be smaller than the dental arch. In some variations, the view window may be made larger or smaller by the user.

[0046] Typically, these methods may include providing a display via a user interface. For example, the user interface may display, on one or more screens, a 3D model of the dental arch, a view window, and / or an image corresponding to a view of the dental arch through the view window. A user can independently move the view window and the dental arch (e.g., by manipulating the user interface (e.g., with a control such as a mouse, keyboard, touch screen, etc.)). This movement, as well as an image through the view window determined to correspond to the area and angle of the view window relative to the dental arch, may be displayed in real time as the user moves the view window and / or the dental arch relative to one another.

[0047] For example, allowing a user to change the relative position between the viewing window and the 3D model of the patient's dental arch may include independently controlling the angle and / or rotation of the 3D model of the patient's dental arch and the portion of the dental arch proximate the viewing window. In some variations, allowing a user to change the relative position between the viewing window and the 3D model of the patient's dental arch may include allowing a user to move the viewing window over the 3D model of the dental arch.

[0048] As described above, any images identified as having been acquired from angles and positions corresponding to the angles and positions of the viewing window as the viewing window is moved over and / or around the dental arch (or as the dental arch is moved relative to the viewing window) may be in any one or more modalities. Thus, for example, identifying images that approximate the relative angles and positions between the viewing window and the 3D model of the patient's dental arch may include identifying any of visible light images, infrared images, and fluorescent images.

[0049] Displaying the identified images that approximate the angle and position of the viewing window relative to the displayed 3D model may include displaying the identified images in a window adjacent to or partially overlaid on the display of the 3D model of the patient's dental arch. For example, the images may be displayed on a screen alongside the 3D model of the dental arch, and as a user moves the dental arch and / or the imaging window, the images may be displayed in one or more windows that change in real time or near real time to reflect the relative positions of the 3D model of the dental arch and the viewing window.

[0050] This specification further describes non-transitory machine-readable tangible media that store instructions that cause one or more machines to perform operations to implement any of the methods described herein, including virtually reviewing a patient's dental arch. For example, a non-transitory machine-readable tangible medium may store instructions that cause one or more machines to perform operations to virtually review a patient's dental arch, including displaying a three-dimensional (3D) model of the patient's dental arch; displaying a view window over a portion of the 3D model of the patient's dental arch; and enabling a user to change a relative position between the view window and the 3D model of the patient's dental arch, the relative position including one or more of an angle between the view window and the patient's dental arch and a portion of the dental arch proximate to the view window. and, as the user changes the relative position between the view window and the 3D model of the patient's dental arch, continuously identifying, from a dataset including both the 3D model of the patient's dental arch and multiple images of the patient's dental arch, where each image is taken from a different angle and position relative to the patient's dental arch, images taken at angles and positions that approximate the relative angle and position between the view window and the 3D model of the patient's dental arch, and displaying the identified images taken at angles and positions that approximate the angle and position of the view window relative to the displayed 3D model of the patient's dental arch.

[0051] For example, a non-transitory machine-readable tangible medium may store instructions that cause one or more machines to perform operations for virtually reviewing a patient's dental arch, the operations including displaying a three-dimensional (3D) model of the patient's dental arch; displaying a view window over a portion of the 3D model of the patient's dental arch; and enabling a user to change a relative position between the view window and the 3D model of the patient's dental arch, the relative position including one or more of an angle between the view window and the patient's dental arch and a portion of the dental arch proximate to the view window. and as the user changes the relative position between the view window and the 3D model of the patient's dental arch, continuously identifying, from a dataset including both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch, where each image is taken from a different angle and position relative to the patient's dental arch, images taken at angles and positions that approximate the relative angle and position between the view window and the 3D model of the patient's dental arch, and displaying the identified images taken at angles and positions that approximate the angle and position of the view window relative to the displayed 3D model of the patient's dental arch.

[0052] Further described herein are intraoral scanning systems configured to perform the methods described herein. For example, an intraoral scanning system may include a handheld wand having at least one image sensor and a light source configured to emit light in a spectral range within the near-infrared wavelength range, a display output (e.g., a visual output such as a monitor, screen, virtual reality interface / augmented reality interface, etc.), and a user input device (e.g., any control means for receiving and sending user input, such as, but not limited to, a keyboard, buttons, joystick, touch screen, etc.). The display output and the user input device may be the same touchscreen, and one or more processors are operatively connected to the handheld wand, the display output, and the user input device, and the one or more processors are configured to: display a three-dimensional (3D) model of the patient's dental arch on the display output; display a view window over a portion of the 3D model of the patient's dental arch on the display output; vary the relative position between the view window and the 3D model of the patient's dental arch based on input from the user input device; identify near-infrared images from both the 3D model of the patient's dental arch and multiple images of the patient's dental arch acquired from various angles and positions relative to the patient's dental arch, the near-infrared images being acquired at angles and positions that approximate the relative angle and relative position between the view window and the 3D model of the patient's dental arch; and display the identified near-infrared images being acquired at angles and positions that approximate the relative angle and relative position between the view window and the 3D model of the patient's dental arch.

[0053] One or more processors of the intraoral scanning system may be configured to receive multiple images of the patient's dental arch taken from various angles and positions relative to the patient's dental arch. For example, the images may be acquired by an image sensor in a handheld wand and sent to the one or more processors and / or stored in memory accessible by the one or more processors. The system may further include a controller that coordinates the activity of the one or more processors, the wand, and the display output (and user input device). The controller may display the images and / or a 3D model constructed from the images as a user manipulates the handheld wand to acquire images at various positions and / or angles relative to the patient's dental arch.

[0054] The one or more processors may be configured to continuously identify near-infrared images and display the identified near-infrared images as a user changes the relative position between the viewing window and the 3D model of the patient's dental arch. Thus, as a user adjusts (using user input) the position of the viewing window (e.g., loop) relative to the 3D model of the patient's dental arch on the display output (or, equivalently, as the user adjusts the position of the 3D model of the dental arch relative to the viewing window on the display output), the one or more processors may determine and display the near-infrared image of the patient's teeth that best approximates the relative position between the viewing window and the 3D model of the patient's dental arch.

[0055] The near-infrared image may be any of multiple images acquired with the handheld wand or an average of multiple images acquired with the handheld wand. Any of the devices (e.g., intraoral scanning systems) described herein may also identify and / or store the position and / or orientation of the handheld wand as it is being manipulated, and this information may be stored along with images acquired from this position. For example, the handheld wand may include one or more accelerometers. For example, the one or more processors may be configured to identify a near-infrared image acquired at an angle and position that approximates the relative angle and position between the viewing window and the 3D model of the patient's dental arch by determining multiple images that approximate the relative angle and position between the viewing window and the 3D model of the patient's dental arch and averaging the multiple images to form the identified near-infrared image.

[0056] As described above, the one or more processors may be configured to vary the relative position between the viewing window and the 3D model of the patient's dental arch on the display output based on input from a user input device. Specifically, the one or more processors may be configured to vary, based on user input to a user input device, one or more of the angle between the plane of the viewing window and the patient's dental arch and the portion of the dental arch proximate to (e.g., in some variations, visible through) the viewing window. As described above, the viewing window may be a loop (e.g., circular, oval, square, etc.) through which a 3D model is viewed. Accordingly, the one or more processors may be configured to display the viewing window over a portion of the 3D model of the patient's dental arch by displaying a loop through which the portion of the 3D model of the patient's dental arch can be viewed. The viewing window may be moved and positioned over the 3D model of the patient's teeth (this may include changing the side of the dental arch that the viewing window is positioned on (buccal, occlusal, lingual, or anything in between), including moving it in x, y, z, and / or rotating, e.g., pitch, roll, yaw), and / or the 3D model of the patient's teeth may be moved (e.g., rotated in pitch, roll, yaw, moved in x, y, z, etc.). Accordingly, the one or more processors may be configured to change the relative position between the view window and the 3D model of the patient's dental arch based on input from a user input device, by changing one or more of the angle of the 3D model of the patient's dental arch relative to the view window (which is equivalent to the angle of the view window relative to the 3D model of the patient's dental arch), the rotation of the 3D model of the patient's dental arch relative to the view window (which is equivalent to the angle of the view window relative to the 3D model of the patient's dental arch), or the portion of the dental arch proximate to the view window (e.g., the portion of the 3D model visible through the view window).For example, the one or more processors may be configured to change the relative position between the view window and the 3D model of the patient's dental arch based on input from a user input device by changing the position of the view window over the 3D model of the dental arch.

[0057] The one or more processors may be configured to identify a second image, which is one or more of a visible light image and a fluorescent image, from both the 3D model of the patient's dental arch and the multiple images of the patient's dental arch obtained from various angles and positions relative to the patient's dental arch, that approximates the relative angle and relative position between the view window and the 3D model of the patient's dental arch, and the one or more processors may be configured to display the second image simultaneously with the near-infrared image.

[0058] Further described herein are methods for automatically, semi-automatically / semi-manually, or manually identifying and grading features by coordinating multiple imaging modalities. For example, one dental diagnosis method may include identifying dental features in a first record including multiple images of a patient's dental arch acquired with a first imaging modality, correlating the first record with a model of the patient's dental arch, and using the model of the patient's dental arch to identify regions of the dental arch that correspond to the dental features in one or more different records, where each record of the one or more different records is acquired with an imaging modality different from the first imaging modality, and where each record of the one or more different records is correlated with the model of the patient's dental arch; determining a confidence score for the dental feature based on the identified regions that correspond to the dental feature in the one or more different records; and displaying the dental feature if the confidence score for the dental feature is above a threshold.

[0059] A dental diagnosis method may include identifying dental features in a first record including a plurality of images of the patient's dental arch acquired in a first imaging modality; correlating the first record with a three-dimensional (3D) volumetric model of the patient's dental arch; flagging the dental features on the 3D volumetric model; using the model of the patient's dental arch to identify regions of the dental arch that correspond to the dental features in one or more different records, each record of the one or more different records acquired with an imaging modality different from the first imaging modality, and each record of the one or more different records correlated with the model of the patient's dental arch; determining or adjusting a confidence score for the dental feature based on the identified regions that correspond to the dental features in the one or more different records; and displaying the dental feature and an indication of the confidence score for the dental feature if the confidence score for the dental feature is above a threshold.

[0060] In any of these methods (or systems implementing these methods), the tooth features may include one or more of cracks, gingival recesses, tartar, enamel thickness, pits, caries, pits, fissures, evidence of bruxism, and interproximal spaces.

[0061] The displaying step may include displaying the tooth feature and an indication of a confidence score for the tooth feature.

[0062] Correlating the first record with a model of the patient's dental arch may include correlating the first record with a three-dimensional (3D) volumetric model of the patient's dental arch. Any of these methods (or systems implementing these methods) may include flagging dental features on the model of the patient's dental arch and / or collecting dental features including a location of the dental feature and one or more of a type of the dental feature and a confidence score for the dental feature.

[0063] Determining the confidence score may include adjusting the confidence score of the dental feature based on the identified regions that correspond to the dental feature in one or more different records.

[0064] In any of these methods or systems, the step of identifying tooth features may include automating the identification of tooth features.

[0065] For example, one dental diagnosis method may include identifying one or more treatable dental features from one or more records of a plurality of records, each record including a plurality of images of the patient's dental arch, each image acquired using an imaging modality, and further wherein each record of the plurality of records is acquired using a different imaging modality; mapping the treatable dental features to corresponding regions of the one or more records; recording the one or more treatable dental features in the records, including recording the location of the treatable dental features; adjusting or determining a confidence score for the one or more treatable dental features based on the corresponding regions of the one or more records; and displaying the one or more treatable dental features if the confidence score for the one or more treatable dental features exceeds a threshold. As described above, the one or more treatable dental features may include one or more of a fissure, a gingival pit, a tartar, an enamel thickness, a pit, a caries, a pit, a fissure, evidence of bruxism, and an interproximal space.

[0066] The displaying step may include displaying the one or more treatable dental features and an indication of a confidence score for the dental features. The mapping of the treatable dental features to corresponding regions of the one or more records may include correlating the first record with a three-dimensional (3D) volumetric model of the patient's dental arch. The recording of the one or more treatable dental features in the record may include marking the dental features on the 3D volumetric model of the patient's dental arch. The identifying of the dental features may include automating the identification of the dental features.

[0067] This specification also describes a system for performing any of the methods described herein. For example, a system may include one or more processors and a memory coupled to the one or more processors, the memory configured to store computer program instructions that, when executed on the one or more processors, perform a computer-implemented method including: identifying dental features in a first record including a plurality of images of a patient's dental arch acquired in a first imaging modality, correlating the first record with a model of the patient's dental arch, identifying regions of the dental arch corresponding to the dental features in one or more different records using the model of the patient's dental arch, where each record of the one or more different records is acquired with an imaging modality different from the first imaging modality, and where each record of the one or more different records is correlated with the model of the patient's dental arch, determining a confidence score for the dental feature based on the identified regions corresponding to the dental features in the one or more different records, and displaying the dental feature if the confidence score for the dental feature is above a threshold.

[0068] This specification further describes methods and apparatus (e.g., systems) for tracking one or more regions (e.g., tagged or flagged regions) across different imaging modalities and / or over time. For example, one method of tracking dental features across different imaging modalities may include collecting a first three-dimensional (3D) volumetric model of a patient's dental arch, where the 3D volumetric model of the patient's dental arch includes surface values ​​and internal structures of the dental arch; identifying a region of the patient's dental arch from a first record of a plurality of records, where each record includes multiple images of the patient's dental arch each acquired using an imaging modality, and further, where each record of the plurality of records is acquired with a different imaging modality; flagging the identified region in a corresponding region of the 3D volumetric model of the patient's dental arch; correlating the flagged region with each record of the plurality of records by correlating the 3D volumetric model of the patient's dental arch with each record of the plurality of records; and storing, displaying, and / or transmitting an image including the region of the patient's dental arch. The region of the patient's dental arch may include dental features including one or more of fissures, gingival recesses, tartar, enamel thickness, pits, caries, pits, fissures, evidence of bruxism, and interproximal spaces.

[0069] The storing, displaying, and / or transmitting steps may include displaying the region of the patient's dental arch. Any of these methods may include flagging dental features on the 3D volumetric model. The identifying the region of the patient's dental arch may include automating the identification of the region of the patient's dental arch.

[0070] A system for tracking one or more regions (e.g., tagged or flagged regions) across different imaging modalities and / or over time may include one or more processors and a memory coupled to the one or more processors, the memory, when executed on the one or more processors, may include: collecting a first three-dimensional (3D) volumetric model of a patient's dental arch, the 3D volumetric model of the patient's dental arch including surface values ​​and an internal structure of the dental arch; and identifying a region of the patient's dental arch from a first record of a plurality of records, each record acquired using a respective imaging modality. and storing computer program instructions for implementing a computer-implemented method, the computer-implemented method including identifying a plurality of records including a plurality of images of the patient's dental arch obtained using a 3D volumetric model of the patient's dental arch, wherein each record of the plurality of records is acquired using a different imaging modality; flagging the identified regions in corresponding regions of a 3D volumetric model of the patient's dental arch; correlating the flagged regions with each record of the plurality of records by correlating the 3D volumetric model of the patient's dental arch with each record of the plurality of records; and saving, displaying, and / or transmitting the images including the regions of the patient's dental arch.

[0071] The novel features of the invention are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present invention will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the invention are utilized, and by reference to the accompanying drawings in which: [Brief explanation of the drawings]

[0072] [Figure 1A] FIG. 1 illustrates an example of a 3D (color) intraoral scanner that can be used as described herein and adapted to generate a model of a subject's teeth having both surface and internal features. [Figure 1B]FIG. 1 illustrates a schematic diagram of an example of an intraoral scanner configured to generate a model of a subject's teeth having both surface and internal features. [Figure 2] FIG. 1 shows a schematic diagram of an intraoral scanner configured to perform both surface scans (e.g., visible light, non-penetrating) and penetrating scans using near-infrared (IR) wavelengths. The scanner includes polarizers and filters that block near-infrared light reflected from tooth surfaces but collect near-infrared light reflected from internal structures. [Figure 3] FIG. 1 illustrates an example of how an intraoral scanner can scan teeth and identify internal structures using penetrating wavelengths (e.g., infrared and / or near-infrared wavelengths). [Figure 4] FIG. 1 illustrates one method for generating an image of internal structures (or pseudo-x-rays) from volumetric data. [Figure 5A] ~ [Figure 5B] 5A and 5B show virtual cross sections from a volumetric model of a tooth. These virtual cross sections may be annotated, colored / pseudo-colored, or textured to show internal features or characteristics of the tooth. In FIG. 5A, the virtual cross section is pseudo-colored to show enamel, and in FIG. 5B, the virtual cross section is pseudo-colored to show dentin. [Figure 6] 10A-10C illustrate one method of marking (e.g., flagging) a volumetric model of a patient's teeth and / or using both marks. [Figure 7] FIG. 1 illustrates a comparison of typical computer-aided design / computer-aided manufacturing (CAD / CAM) methods for density with the method described herein that performs 3D volumetric scanning and modeling. [Figure 8] Figure 8A shows an example of a display of tracking gingival recession over time using the 3D volumetric model described herein, and Figure 8B shows a subsequent enlargement of region B of Figure 8A. [Figure 9]9A and 9B illustrate one method of displaying volumetric information from a patient's teeth. FIG. 9A shows an example of a 3D volumetric model of a patient's upper jaw (with teeth and gums visible) from a top perspective. FIG. 9B shows the same 3D volumetric model, revealing internal features including highly transparent enamel and less transparent dentin. This 3D volumetric model may be manipulated to reveal more or less of the surface and / or internal structure. FIGS. 9C-9G show increasingly transparent views of a region ("C") of the 3D volumetric model of FIG. 9A. FIG. 9C shows a 2D image extracted from a region of the 3D volumetric model showing only the outer surface of the tooth (e.g., 100% color / outer surface image, 0% near-infrared / interior volume). FIG. 9D shows the same region as FIG. 9C, but with a combination of the outer surface (color) image and the interior (near-infrared) image (e.g., 75% color / outer surface image and 25% near-infrared / interior volume). FIG. 9E shows the same region as FIG. 9C, but with a combination of the outer surface (color) image and the interior (near-infrared) image (e.g., 50% color / outer surface image and 50% near-infrared / interior volume). FIG. 9F shows the same region as FIG. 9C, but with a combination of the outer surface (color) image and the interior (near-infrared) image (e.g., 25% color / outer surface image and 75% near-infrared / interior volume). FIG. 9G shows the same region as FIG. 9C, but with only the interior (near-infrared) image of the tooth (e.g., 0% color / outer surface image and 1000% near-infrared / interior volume). [Figure 10A] FIG. 10 illustrates an example of a user interface for analyzing and / or displaying a 3D volumetric model of a patient's teeth, showing a top view of the maxillary dental arch, tools that can be used to manipulate the view, and two enlarged views showing the outer surface of a magnified area of ​​the tooth (on the left) and the same view showing the internal features of the tooth (showing the dentin and enamel within the tooth). [Figure 10B] FIG. 10B illustrates the user interface of FIG. 10A with an area of ​​a tooth marked / flagged as described herein. [Figure 11]

[0033] Figures 11A and 11B show another example of how 3D volumetric image information can be blended with surface (non-penetrating) information for display. Figure 11A shows a visible light image of a region of a patient's dental arch acquired with a scanner further configured to acquire penetrating (near-infrared) scans. Figure 11B shows a reconstructed 3D volumetric model of a patient's tooth showing the interior dentin and enamel. Features not visible in the surface scan are clearly visible in the volumetric scan, including caries and porosity in the enamel. Figure 11C shows a hybrid image where the surface scan is combined with the 3D volumetric image, showing both surface structure and interior structure, including caries and porosity. [Figure 12] 1 illustrates an example of a method that allows a user to perform a virtual scan of a patient's dental arch, which may be performed in real time or near real time. [Figure 13] FIG. 1 shows a schematic diagram of a data structure containing a 3D model of a patient's dental arch and associated 2D images acquired (e.g., with an intraoral scanner) at multiple positions around the dental arch. [Figure 14A] 1 illustrates an example of a user interface that allows a user to virtually scan over a 3D model of a dental arch, detailing the corresponding optical and near-infrared (e.g., external and internal) regions as the user scans over the 3D dental arch; the user can use one or more tools to move (e.g., rotate, translate, etc.) the dental arch and / or the view window; and the corresponding optical and near-infrared images can be continuously or near-continuously updated as the view window and dental arch position change. A pair of imaging windows is shown adjacent to the view of the 3D model of the dental arch. [Figure 14B] 14A and 14B show alternative views in which a single large image window is shown above or next to the 3D image of the dental arch. In Fig. 14B, the image window shows a light image of the corresponding region of the dental arch. [Figure 14C]14A-14C show alternative views in which a single large image window is shown above or next to the 3D image of the dental arch. In FIG. 14C, the image window shows a near-infrared image of the corresponding region of the dental arch. [Figure 15A] 14A shows an example of a 3D model of the outer surface of a dental arch and a view window for the dental arch. A pair of image display windows are adjacent to the 3D model of the dental arch. A user can change the images shown in the two display windows by moving the view window over the dental arch (and / or by moving the dental arch relative to the view windows). The upper display window shows a near-infrared image corresponding to the dental arch at the position and angle in the plane of the view windows, and the lower display window shows a corresponding light image (which may be in color). [Figure 15B] FIG. 15B shows another image of the dental arch shown in FIG. 15A, where the dental arch has been rotated lingually relative to the view window, and the corresponding near-infrared image (top right) and visible light image (bottom right) adjacent to the 3D model of the dental arch have been updated to show a slightly rotated view, allowing the user to virtually scan the dental arch and display both external and internal views in real time (or near real time). [Figure 16A] FIG. 10 illustrates another example of how a 3D model of a dental arch may be displayed (in this example displayed as a mandibular dental arch (e.g., by selecting the mandibular dental arch display control in the top left of the user interface)), showing focused views of near-infrared and visible light images corresponding to a view window region that may be moved across and around (lingual-occlusal-buccal) the patient's dental arch model. [Figure 16B] FIG. 16B shows an example of a single window (magnified near-infrared view of the area within the tooth corresponding to the view window loop) similar to FIG. 16A. [Figure 16C] FIG. 16B shows an example of a single window (enlarged visible light view of the area within the tooth corresponding to the view window loop) similar to FIG. 16A. [Figure 17]FIG. 1 is a schematic diagram illustrating an example of an automated or semi-automated method for identifying, verifying, and / or characterizing one or more treatable dental features that may benefit from detection and / or treatment. DETAILED DESCRIPTION OF THE INVENTION

[0073] Described herein are methods and apparatus (e.g., devices and systems) for scanning both the external and / or internal structures of teeth. These methods and apparatus are capable of generating and manipulating a model of a subject's oral cavity (e.g., teeth, jaw, palate, gums, etc.), which can include both surface topographical features and internal features (e.g., dentin, dental fillings (including bases and linings), cracks, and / or caries). Apparatus for performing both surface and penetration scans of teeth can include an intraoral scanner for scanning the interior or periphery of a subject's oral cavity, the intraoral scanner comprising one or more light sources capable of illuminating in two or more spectral ranges: a spectral range that illuminates surface features (e.g., visible light) and a penetrating spectral range (e.g., the infrared range, and particularly "near infrared" (including, but not limited to, 850 nm)). The scanning device may further include one or more sensors that detect the emitted light and one or more processors that control the operation of the scan and analyze the received light in both the first and second spectral ranges to generate a model of the subject's teeth, including tooth surfaces and intra-tooth features (including within the enamel (and / or enamel-like restorations) and dentin). The generated model may be a 3D volumetric model or a panoramic image.

[0074] As used herein, a volumetric model may encompass a three-dimensional virtual representation of an object, in which internal regions (e.g., structures) are positioned within a physical three-dimensional volume in a proportional and relative relationship to other internal and surface features of the object being modeled. For example, a volumetric representation of a tooth may include the external surface and internal structures within the tooth (below the tooth surface) positioned proportionally relative to the tooth, such that a cross-section of the volumetric model approximately matches a cross-section of the tooth showing the location and size of the internal structures, and the volumetric model may be cross-sectional from any (e.g., any) direction and represent an equivalent cross-section of the object being modeled. A volumetric model may be electronic or physical. A physical volumetric model may be formed, for example, by 3D printing, etc. The volumetric models described herein may extend completely within a volume (e.g., extend throughout the entire volume (e.g., tooth volume)) or may extend partially within a volume (e.g., extend within the volume being modeled to a certain minimum depth (e.g., 2 mm, 3 mm, 4 mm, 5 mm, 6 mm, 7 mm, 8 mm, 9 mm, 10 mm, 12 mm, etc.)).

[0075] The methods described herein typically include methods for generating models of a subject's teeth, which typically generate a 3D model or rendering of the teeth, including both surface and internal features. Non-ionizing methods for imaging and / or detecting internal structures may be used, such as observing internal structures by illuminating the structures using one or more penetrating spectral ranges (wavelengths) and acquiring images at the penetrating wavelengths, including using trans-illumination (illuminating from one side and capturing light from the other side after passing through an object) and / or using low-angle penetration imaging (e.g., reflectance imaging, capturing light reflected / scattered from internal structures when illuminated with penetrating wavelengths). In particular, multiple penetration images may be acquired from the same relative position. While conventional penetration imaging techniques (e.g., trans-illumination) in which the angle between the illumination direction of the light emitter and the viewing angle of the detector (e.g., camera) is 90 degrees or 180 degrees may be used, methods and apparatuses are also described herein in which the angle is much smaller (e.g., 0-25 degrees, 0-20 degrees, 0-15 degrees, 0-10 degrees, etc.). Smaller angles (e.g., 0-15 degrees) can be particularly advantageous because they allow illumination (light source) and detection (detector (e.g., camera, etc.)) to be closer together and also allow the scanning wand for the intraoral scanner to be more easily positioned and moved around the patient's teeth. Such small-angle penetration images and imaging techniques are sometimes referred to herein as reflected illumination and / or imaging or reflected / scattered imaging. In general, penetration imaging may refer to any appropriate type of penetration imaging, including trans-illumination, small-angle penetration imaging, etc., unless otherwise specified. However, small angles can also cause direct reflections from the surface of an object (eg, a tooth), which can obscure internal structures.

[0076] The methods and apparatus described herein are particularly useful in combining a 3D surface model of one or more teeth with imaged internal features, such as damage (caries, cracks, etc.) detectable using penetration imaging, by using an intraoral scanner adapted to detect both surface and internal features separately but simultaneously (or nearly simultaneously). Combining surface scanning and penetration imaging can be achieved by alternating or switching between these different modalities to allow both to use the same coordinate system. Alternatively, both surface and penetration scans can be observed simultaneously, for example, by selectively filtering the imaging wavelengths to separate infrared (near-infrared) light from visible light. Thus, the 3D surface data can provide important reference and angular information for internal structures, enabling interpretation and analysis of penetration images that may otherwise be difficult or impossible to interpret.

[0077] The penetration scans described herein may be collected, for example, with an intraoral scanner (e.g., the intraoral scanner shown in FIGS. 1A-1B) that generates a three-dimensional (3D) model (which may include internal features of the teeth and may also include a surface model) of a subject's intraoral region (e.g., one or more teeth, gums, jaw, etc.), and methods for using such scanners are also described herein. While surface scans (including color scans) can be helpful and useful in various cases, a penetration (infrared) scan may be sufficient in some of the variations described herein.

[0078] In Figure 1A, an exemplary intraoral scanner 101 may be configured or adapted to generate a 3D model having both surface and internal features, or only internal (penetration) scans. As shown schematically in Figure 1B, one exemplary intraoral scanner may include a handle or wand 103 that can be handheld by an operator (e.g., dentist, dental hygienist, technician, etc.) and moved over one or more teeth of a subject to scan both surface and internal structures. The wand may include one or more sensors 105 (e.g., a camera such as a CMOS, a CCD, a detector, etc.) and one or more light sources 109, 110, 111. FIG. 1B illustrates three light sources: a first light source 109, a second light source (colored light source), and a third light source 111. The first light source 109 is configured to emit light in a first spectral range (e.g., visible light, monochromatic visible light, etc., although the light does not have to be visible light) for detecting surface features; the second light source (colored light source) is white light, for example, between 400 and 700 nm (e.g., approximately 400 and 600 nm); and the third light source 111 is configured to emit light in a second spectral range for detecting internal features of the tooth (e.g., by trans-illumination, small-angle penetrating imaging, laser fluorescence, etc., which may collectively be referred to as penetrating imaging (e.g., in the near-infrared)). While FIG. 1B illustrates multiple illumination sources separately, in some variations, switchable light sources may be used. The light sources may be any suitable light source, including LEDs, fiber optics, etc. The wand 103 may include one or more controls (buttons, switches, dials, touchscreen, etc.) to aid in control (e.g., turning the wand on and off, etc.), alternatively or in addition, one or more controls not shown may be present in another part of the intraoral scanner, such as a foot pedal, keyboard, console, touchscreen, etc.

[0079] In general, any suitable light source may be used, particularly one that is compatible with the mode to be detected. For example, any of these devices may include a visible light source or other (including non-visible) light source for surface detection (e.g., at or around 680 nm, or other suitable wavelength). A colored light source, typically a visible light source (e.g., a "white light" light source), may also be included for color imaging. Furthermore, a penetrating light source (e.g., an infrared light source, e.g., specifically, a near-infrared light source) may also be included for penetrating imaging.

[0080] The intraoral scanner 101 may include one or more processors, including linked or remote processors, that control the operation of the wand 103, including the coordination of scans, and the scanning and generation of 3D models, including surface and internal features, for review and processing. As shown in FIG. 1B , the one or more processors 113 may include or be coupled to memory 115 for storing scan data (surface data, internal feature data, etc.). Communications circuitry 117, including wireless or wired communications circuitry, may also be included for communicating with components of the system (including the wand) or external components (including external processors). For example, the system may be configured to receive and transmit scans or 3D models. One or more additional outputs 119 may also be included for outputting or presenting information, such as a display screen, printer, etc. As mentioned above, an input 121 (buttons, touchscreen, etc.) may be included to allow or require user input for controlling scanning and other operations.

[0081] FIG. 2 shows an example of a usable scanner. The illustrated scanner may be used as part of a system such as the one shown in FIGS. 1A-1B. For example, FIG. 2 shows a schematic of an intraoral scanner configured to perform both surface scans (e.g., visible light, non-penetrating) and penetrating scans using a near-infrared (NIR) wavelength (850 nm in this example). In FIG. 2, the scanner includes a near-infrared illumination source 289 and a first polarizer 281, as well as a second polarizer 283 before an image sensor 285, to block near-infrared light (P-polarized light) reflected from the surface of a tooth 290 while collecting near-infrared light (S-polarized light) scattered from internal tooth structures / regions. The near-infrared light illuminates the tooth with P-polarized light, and specular light reflected from the tooth surface (e.g., enamel) is specularly reflected, preserving its P-polarized state. Near-infrared light that penetrates internal tooth features, such as dentin, scatters, resulting in random polarization (S and P). The wavelength-selective quarter-wave plate 293 does not change the polarization of the near-infrared light (e.g., it leaves the polarization state of the delivered near-infrared light unchanged), but it changes the polarization of the returning scanning light from P to S so that only surface reflections are captured at the scanning wavelength. The returning near-infrared light is a mixture of S and P polarization and is first filtered through a polarizing beam splitter (PBS) 294 and a polarizing filter 283 to send only the S polarization to the image sensor. Thus, only the near-infrared S-polarized light coming from the internal tooth structures is captured by the image sensor, while the specular light with its original P polarization is blocked. Other intraoral scanner configurations may also be used as part of the probe, with or without a polarizing filter as shown in FIG. 2.

[0082] In FIG. 2, surface scanning may be performed by illuminating the surface (using scanner illumination unit 297) with P-polarized light, the polarization of which is reversed by wavelength-selective quarter-wave plate 293 (which transmits S-polarized light to the image sensor).

[0083] Various penetration scanning techniques (penetration imaging) may be used or incorporated into the devices described herein to perform scans using penetrating wavelengths or a spectral range of penetrating wavelengths to detect internal structures, including, but not limited to, transillumination and small-angle penetration imaging, both of which detect the passage of penetrating wavelength light from or through tissue (e.g., from or through one or more teeth). Thus, these devices and techniques may be used to scan intraoral components such as a tooth or teeth, gums, palate, etc., and to generate models of the scanned area. These models may be generated in real time or after scanning. These models may be referred to as 3D volumetric models of the teeth, but may also include the palate, gums, and other areas of the jaw that include the teeth. While the methods and devices described herein typically relate to 3D volumetric models, the methods and techniques described herein may also be used in any instance with a 3D surface model. The surface model information is typically part of the 3D volumetric model.

[0084] FIG. 3 shows an example of a data flow for scanning teeth with an intraoral scanner to build a 3D model including internal structures. In FIG. 3, the illustrated exemplary method may include three parts. First, teeth may be scanned with an intraoral scanner 1701 (or any other scanner) configured to perform a penetration scan of the interior of the tooth using an optical (e.g., infrared, near-infrared, etc.) wavelength or range of optical wavelengths. Both of these scanners can also simultaneously perform scans to identify surface features (e.g., at one or more non-penetrating wavelengths), color, etc., as described above. Multiple penetration scans 1703, 1703′ may be performed during scanning, and sensor (e.g., camera) positions 1705, 1705′ (e.g., x, y, z positions and / or pitch, roll, yaw angles) may be identified and / or recorded for each penetration image. In some variations, the tooth surface may also be simultaneously imaged, as described above, and a 3D surface model 1707 of the tooth may also be simultaneously determined. In this example, a scan of the patient's teeth may be performed, for example, with an intraoral 3D scanner 1702 capable of imaging the internal tooth structure (e.g., by near-infrared imaging). The position and orientation of the camera may be determined, in part, from the 3D scan data and / or the 3D tooth surface model 1707.

[0085] The penetration image may then be segmented (1711). In this example, segmentation may be performed in one of two ways. On the internal tooth structure image, the image may be segmented by contour finding 1713, 1713'. Machine learning methods may be applied to further automate this process. Alternatively or additionally, to accurately locate segments such as enamel, a proximity image (close to the camera position) may be used to identify nearby features and then back-project features from the 3D model onto the image. The method may further include back-projecting pixels of the internal tooth image onto the tooth and calculating a density map of internal tooth reflectance coefficients. The enclosing surfaces of the various segments can be found or estimated by using isosurfaces or thresholds of the density map and / or by machine learning methods. Furthermore, by segmenting the image and back-projecting the segments onto a model (e.g., a 3D surface model (e.g., back-projected onto the world)), it is possible to find the segments from the intersection of the segment's projection with the tooth surface.

[0086] The results may be displayed (1717), transmitted, and / or stored. For example, the results may be displayed by the scanning system during the intraoral scanning procedure. The results may be shown in an image, such as an outline of the various segments, a 3D density map, etc. In the example shown in FIG. 3, a density map 1715 is shown, which represents the dentin beneath the outer surface enamel. The image may be color-coded to indicate different segments. In this example, the interior segments (structures) are shown within the 3D surface model (shown transparent), but only some of the teeth are shown because this is an infiltration image in which not all teeth were scanned. Alternative views, cross sections, slices, projections, etc. may be provided. The example image of FIG. 3 includes artifacts 1716 present on the outside of the teeth, which may be removed or trimmed based on the surface model 1718.

[0087] Each pixel of the image may be marked by a segment. Internal structures such as dentin, enamel, cracks, damage, etc. may be automatically identified by segmentation and may be identified manually or automatically (e.g., based on machine learning of 3D structures, etc.). Each segment may be displayed separately or together (e.g., in different colors, shading, etc.), with or without a surface model (e.g., a 3D surface model).

[0088] Thus, in Figure 3, the patient is first scanned with a 3D scanner capable of both surface and penetrating scanning (e.g., near-infrared imaging), and the orientation and / or position of the camera is known (based on the position and / or orientation of the wand and / or surface scan). This position and orientation may be relative to the tooth surface. Thus, the method and apparatus may include an estimation of the camera position (where the camera is, e.g., the x, y, z position of the camera, and its rotational position).

[0089] In general, a penetrant image (eg, a near-infrared image or an infrared image) may be automatically segmented.

[0090] User Interface and Volumetric Information Display By collecting and analyzing volumetric data from within the oral cavity, it is possible to identify dental features and information that were previously difficult or impossible to discern from non-volumetric scans. However, analyzing three-dimensional volumetric information can be difficult or counterintuitive for dental practitioners (and / or patients). Described herein are methods and devices for viewing and interpreting 3D volumetric data of a patient's oral cavity.

[0091] For example, Figures 9A-9G illustrate one example of how 3D volumetric data can be displayed. Figure 9A shows a surface model (which may be a surface model portion of a volumetric model) of the maxillary dental arch from a top perspective, where external features (e.g., surface features) are visible. This view closely resembles a surface scan view, which may be colored (e.g., acquired with visible light). Internal structures exist in the model below the external surface of the scan and are less visible in Figure 9A. Figure 9B illustrates internal structures based on their relative transparency to near-infrared light. In Figure 9B, enamel is more transparent than dentin (and therefore appears to be more transparent), which appears to be less transparent. Figures 9B-9F illustrate the transition between the surface view of Figure 9A and the penetrating internal 3D view of Figure 9B for the indicated subregion (circled region "C"). For example, the relative surface to the internal view may be changed to provide a user with a visual representation of the internal structures within the dental arch relative to the surface structures. For example, an animated view may be obtained that cycles through a series of images, as shown in FIGS. 9C-9G. Alternatively, the user may slide a slider 903 to switch between the surface and internal views. Transitioning between these two views, which can be created from any angle, may help the user and / or patient see beneath the surface of the tooth to visually assess the rich internal data. The 3D volumetric model may be manipulated to display any view (including cross-sectional views) showing the internal and / or surface structures. A top view is shown in FIGS. 9A-9G. 9C-9G show increasingly transparent views of a region (“C”) of the 3D volumetric model of FIG. 9A, showing how region C has a progressively larger percentage of interior view (0% to 100%) and a progressively smaller percentage of surface view (100% to 0%) as in FIG. 9B.

[0092] FIGS. 11A-11C show another example, showing a hybrid image that combines and blends both a surface image scan (e.g., a visible light scan, as shown in FIG. 11A) and a volumetric model acquired using penetrating (e.g., near-infrared) wavelengths (as in FIG. 9E) with a volumetric model acquired using penetrating (e.g., near-infrared) wavelengths, as shown in FIG. 11B. Features present within the tooth enamel and dentin are visible in the volumetric reconstruction (the image shown in FIG. 11B) that are not clearly visible in the surface-only image (which may also be a reconstruction) shown in FIG. 11A. For example, carious region 1103 is clearly visible in FIG. 11B, but is not visible in FIG. 11A. Similarly, enamel void region 1105 is visible in FIG. 11B, but is not visible in FIG. 11A. FIG. 11C shows a hybrid image of a 3D volumetric model and a surface model (surface image), where both these structures, the carious region and the void region, are visible.

[0093] Generally, this specification describes methods and apparatus for displaying volumetric data acquired from a patient's oral cavity (e.g., teeth, gums, palate, etc.) in a simplified manner that is easily understandable to a user (e.g., a dental practitioner) and / or patient. Further, this specification describes methods for displaying volumetric data acquired from a patient's oral cavity in a manner that will be familiar to a user and / or patient. In a first example, the data may be provided as one or a series of X-ray-type images, much like those produced by a dental X-ray. FIG. 4 illustrates one method for generating X-ray (or pseudo-X-ray) images from a volumetric data set acquired as described above (e.g., using a penetrating wavelength of light (e.g., infrared light)).

[0094] As shown in FIG. 4 , one method for displaying a 3D volumetric image of a patient's oral cavity may include receiving 3D volumetric data 401, which may be received, for example, directly from a scan as described above or from a stored digital scan. In some variations, individual teeth or groups of teeth may be identified from the volumetric data (403). Alternatively, the entire volume may be used. Pseudo-x-ray images may then be generated from the volume or a subset of the volume corresponding to the individual teeth (405). For example, an image of the volume may be acquired from the “front” of one or more teeth, and the transparency of enamel (and / or enamel-like restorations), dentin, or other features may be obtained from the volumetric data. This volumetric data may be based on the absorption coefficients of materials in the oral cavity at the penetrating wavelengths of light used. Thus, projections from the volumetric data can be generated in a fixed orientation to obtain images that closely resemble X-rays, but in some variations, the projections can be inverted to show dentin density (which is more absorbent) as "darker" than enamel density (which is less absorbent and therefore more transparent), highlighting caries as more absorbent (darker) areas. Thus, the image can be inverted to resemble an X-ray image, with denser areas being brighter (e.g., brighter). These pseudo-X-ray images can be generated from the same locations as standard dental X-rays and presented to a user. For example, a panel of pseudo-X-ray images can be generated from a volumetric model of each of the patient's teeth. While the wavelength of light (e.g., near-infrared light) may not penetrate as deeply as traditional X-rays, the images generated in this manner can adequately substitute for X-rays, particularly in the coronal and mid-tooth regions above the gumline.

[0095] Other simplified or modified displays may be presented to the user or customized by the user for display to the patient. For example, in some variations, a simplified tooth image may be generated from the volumetric data by false-coloring the volumetric data to highlight specific regions. For example, areas that have already been marked or flagged (discussed in more detail below) may be colored red, while enamel may be shown in a more natural white or slightly off-white color. In some variations, enamel-like materials (e.g., materials such as fillings) may be represented separately and / or marked with colors, patterns, etc.

[0096] In some variations, the method and / or apparatus may display cross sections of the teeth of a dental arch. Similarly, individual teeth or groups of teeth may be individually displayed and / or labeled (e.g., by standard naming / numbering conventions). This display may be in addition to or instead of other displays. In some variations, teeth and / or internal structures may be false-colored or projected onto a color image.

[0097] For example, FIGS. 5A and 5B show acquired virtual cross-sections of the volumetric model of a patient's teeth, as described above, including near-infrared information generated from an intraoral scan. The cross-section in FIG. 5A may be automatically generated to display regions of interest within the tooth, including the enamel, or may be generated manually (e.g., by a user). The cross-section can display both density and / or surface cross-sections. These images may be false-colored to show distinct regions, including the outer surface, enamel, dentin, etc. Internal structures (e.g., those within the enamel and / or dentin) can reflect the behavior of near-infrared light within the tooth, e.g., the absorbance and / or reflectance of light at one or more near-infrared / visible wavelengths within the tooth. In FIG. 5B, the cross-section is false-colored with a heat map to show internal features, which may be keyed as shown. In any of these variations, 2D projections of the teeth may be generated from the volumetric information, showing one or more features of one and / or more teeth. As will be described in more detail below, additional features may be displayed as well, including damage (e.g., caries / pits, cracks, wear, plaque buildup, etc.), which may be indicated by color, texture, etc. Although these are illustrated as cross sections of the 3D volumetric model, in another embodiment, the 2D cross sections themselves may be displayed, thereby providing a cross-sectional view of one or more teeth similar to the view provided by a 2D x-ray image.

[0098] Any of the methods and apparatuses described herein for implementing them may include displaying one or more (or successive) cross sections of a 3D model, preferably a 3D volumetric model, of the patient's dental arch. For example, one method for displaying an image from a three-dimensional (3D) volumetric model of the patient's dental arch may include acquiring a 3D volumetric model of the patient's dental arch, including near-infrared transparency values ​​of the internal structures of the dental arch; generating a two-dimensional (2D) view of the 3D volumetric model, including the patient's dental arch and including near-infrared transparency values ​​of the internal structures; and displaying the 2D view. Any of these methods may optionally (but not necessarily) include scanning the patient's dental arch with an intraoral scanner.

[0099] Generating the 2D views may include cross-sectioning the 3D volumetric model with a plane that intersects the 3D volumetric model. A user may select the location and / or orientation of the plane, and this may be done sequentially. For example, any of these methods may include a user selecting cross sections of the 3D volumetric model to display, where the selecting step includes the user scanning the 3D model and sequentially displaying 2D views corresponding to each cross section. Generating the 2D views may include a user selecting an orientation for the 2D views.

[0100] In any of these methods, surfaces may be included. For example, as described and illustrated above, one method of displaying images from a three-dimensional (3D) volumetric model of a patient's dental arch may include acquiring a 3D volumetric model of the patient's dental arch, the 3D volumetric model including surface values ​​and near-infrared transparency values ​​of the internal structures of the dental arch; generating a two-dimensional (2D) view of the 3D volumetric model that includes the patient's dental arch and includes both the surface values ​​and the near-infrared transparency values ​​of the internal structures; and displaying the 2D view. The surface values ​​may include surface color values. The surfaces may be weighted relative to the internal (volumetric) structures. For example, generating the two-dimensional (2D) view of the 3D volumetric model may further include including a weighted portion of the surface values ​​and a weighted portion of the near-infrared transparency values ​​of the internal structures in the 2D view. The weighted portion of the surface value may include a percentage of the full value of the surface value, and the weighted portion of the near-infrared transparency of the internal structure may include a percentage of the full value of the near-infrared transparency of the internal structure, where the percentage of the full value of the surface value and the percentage of the full value of the near-infrared transparency of the internal structure sum to 100%. For example, a user may adjust the weighted portions of one or more of the surface value and the near-infrared transparency of the internal structure.

[0101] For example, one method of displaying images from a three-dimensional (3D) volumetric model of a patient's dental arch may include collecting a 3D volumetric model of the patient's dental arch, the 3D volumetric model including surface color values ​​and near-infrared transparency values ​​of the internal structures of the dental arch; a user selecting a view orientation of the 3D volumetric model to display; generating a two-dimensional (2D) view of the 3D volumetric model using the selected orientation, the 2D view including the patient's dental arch and including a weighted portion of the surface color values ​​and a weighted portion of the near-infrared transparency of the internal structures; and displaying the 2D view.

[0102] In addition to displaying qualitative images of the teeth, the methods and apparatus described herein may perform quantification and provide quantitative information regarding internal and / or external features. For example, volumetric measurements, including dimensions (peak or average length, depth, width, etc.), volume, etc., of one or more defects may be provided (selectably or automatically). This may be accomplished by manually or automatically segmenting the volumetric model to define regions of interest that include one or both of tooth features (enamel, dentin, etc.) and / or irregularities (e.g., caries, cracks, etc.). Any suitable segmentation technique may be used, such as, but not limited to, mesh segmentation (mesh decomposition), polyhedron segmentation, skeletonization, etc. After the volume is segmented, these regions may be displayed and / or measured individually or collectively. As described below, they may be marked or flagged and used for future analysis, display, and modification of scanning methods and systems.

[0103] In some variations of the user interfaces described herein, a summary report may be generated or created from the volumetric data and displayed to the user and / or patient. For example, the summary data may be projected onto a model of the patient's teeth. This model may be simplified so that the enamel is opaque and marked or selected internal features (including automatically selected internal features) are displayed in red or some other contrasting color (and / or flashing, blinking, etc.) within the tooth. For example, caries may be displayed in this manner. The summary report may be automatically entered into the patient's medical record.

[0104] Any image, including a volumetric image, may be animated. For example, a virtual cross-section of a patient's teeth showing a scanned or moved cross-section of the patient's dentition, possibly with a 3D model showing one or more cutting axes through the volume, may be displayed. The user interface may allow a user to show both external and internal features based on the volumetric scan in cross-section through one or more planes.

[0105] Generally, the devices described herein may generate separate views for the user (e.g., physician, dentist, orthodontist, etc.) than for the patient. The user may be provided with a "clinical view" that may include information not displayed in the separate "patient view." The clinical view may be more technical and, in some cases, closer to the raw images from the volumetric data. The user may select which layers of information to include in the patient view, which may be presented to the user during or after the scan, or during or after review of the dental scan. Patient education materials may be added to the patient view.

[0106] For example, in some variations, a user-facing display of volumetric data may include an overlay of the volumetric data, which displays 3D components within the volumetric data using false coloring. As described in more detail below, any of these displays / images may display marked or highlighted regions to draw attention to potential problem areas (e.g., caries, thin enamel, cracks, etc.). Two-dimensional (2D) colored data and 3D near-infrared data (e.g., surface and volumetric regions) may be displayed, including transitions between the two.

[0107] Generally, annotating (e.g., marking, flagging, etc.) the volumetric information may be done automatically, manually, or semi-automatically, and the annotations may be displayed. Furthermore, the annotations may be used both to annotate subsequent scans and to modify how subsequent scans for the same patient are performed and displayed. The annotations may be, for example, markers or flags on a region of interest. The region of interest may correspond to a particular area in which one or more features (such as cracks, caries, thinning of the enamel, plaque or calculus accumulation, etc.) are observed. Alternatively or additionally, the region of interest may be an area that has changed over time (e.g., from one scan to a further scan).

[0108] As discussed above, any of these methods may include placing one or more markers on a volumetric model of the patient's teeth. The markers (e.g., flags, pins, etc.) may be placed manually by a user, automatically by a device, or semi-automatically (e.g., suggested by the system, configured by the user, etc.), as described in more detail below.

[0109] Markers may be used to focus system attention and / or processing on one or more particular regions of the volumetric model for display and / or future follow-up (e.g., in subsequent scans). Markers may modify how subsequent scans are performed, for example, by more fine-tuning how subsequent scans of the marked region are performed (e.g., increasing resolution, changing wavelength, increasing frequency or repetition of scans, etc.). Marked regions may be displayed over time to show changes therein.

[0110] For example, a user may mark a digital representation of a patient's tooth (or the patient's actual tooth) with a marker (e.g., a pin, a flag, etc.) that allows for annotation (e.g., the marker may have notes associated with the tooth). This marker may then be used to track different scans over time. Subsequent scans may be marked at corresponding locations, and subsequent marked scans may be modified based on the marked areas. These marked areas may be scanned in detail, and analysis may be performed and / or displayed automatically, measuring and / or displaying changes compared to one or more previous scans. Thus, any of the systems described herein may track one or more marked areas from previous scans and provide feedback during and / or after a new scan to provide further details. This may be done for both surface and / or volumetric information, particularly with respect to enamel properties and / or by comparison with enamel, one or more outer tooth surfaces, and / or dentin.

[0111] For example, a subsequent scan of the same patient may be modified by one or more annotation markers from a previous scan. Prior to the scan, the user may enter an identifier for the patient being scanned (or the system may automatically identify the patient based on a database of previous scans). The system may automatically annotate the new scan based on the annotations from the previous scan.

[0112] In some variations, the system may automatically annotate subsequent scans by identifying differences between previous and current scans. For example, areas that show changes above a threshold compared to previous scans may be flagged and presented to the user. Annotation may be performed without user supervision (fully automatic) or with some degree of user supervision, such as by flagging annotations, indicating to the user why the annotations were flagged, and then allowing the user to keep, modify, or discard the markings. Reasons for automatically marking teeth include changes in enamel thickness, changes in surface smoothness, changes in the relative proportions of tooth enamel and dentin, changes in the position of the teeth (e.g., occlusion), etc. Additionally, changes in the outer structure of natural teeth (e.g., increased or decreased plaque or tartar accumulation) or changes in the gum structure surrounding the teeth may also occur. Thus, if the system detects one or more of these conditions, it may automatically flag that region in the volumetric model.

[0113] Modifications to subsequent scans may be made dynamically by flags in previous scans or by detecting changes in areas (which may be unmarked areas) compared to previous scans. For example, changes may be made to scan parameters, such as increasing the resolution of the scan (e.g., changing the scan dwell time, requesting the user to perform multiple scans of this area, etc.), changing the wavelength used for the scan, etc.

[0114] For example, FIG. 6 illustrates a method for automatically selecting regions to mark and / or use. A first volumetric model of the patient's teeth is generated (601) from a scan of the patient's teeth. The generation of the volumetric model may be performed in any suitable manner, such as the methods described above or discussed in U.S. Patent Application No. 15 / 662,234, filed July 27, 2017, entitled "INTRAORAL SCANNER WITH DENTAL DIAGNOSTICS CAPABILITIES," which is incorporated herein by reference in its entirety. The first volumetric model may be stored (digitally) as part of the patient's dental record. The first volumetric model may be analyzed (603) simultaneously or sequentially (immediately or over some time) (e.g., by a device including a device having a processor configured to operate as described herein) to identify any regions that should be flagged. This analysis may thus be performed automatically, and one or more characteristics of the patient's teeth may be examined from the scan. The automated analysis (or semi-automated analysis, etc., automated but with manual assistance for verification / confirmation) may be performed by a microprocessor, including a system trained (e.g., by machine learning) to identify areas of irregularities in the outer and / or inner volumes of the teeth. For example, the device may identify possible defects in the patient's teeth by examining a digital model, including, but not limited to, cracks, caries, cavities, changes in occlusion, malocclusion, etc. This may include identifying areas near the tooth surface with optical (near-infrared) contrast (605) that are indicative of possible cracks, caries, occlusion, etc. Areas closer to the surface that are less transparent (e.g., more absorbent) at near-infrared wavelengths than other parts of the enamel and are more likely to represent defects (either generally or within a specific range of near-infrared wavelengths). Alternatively or additionally, the surface characteristics of the teeth may be examined and flagged if they fall outside of thresholds (607).For example, regions of the tooth surface that are rough (e.g., smoothness below a set threshold, as may be determined from the outer surface of the enamel) may be flagged. Other surface characteristics may be analyzed and used to determine whether to mark the region or prompt the user to confirm marking, such as discoloration (based on colored or white light / surface scan) and gingival position (relative to the outer surface of the tooth). The distribution and size of the patient's enamel may also be examined (609). Enamel thickness can be determined from optical properties (e.g., comparison of absorption / reflection properties). Regions of enamel estimated to be below a threshold thickness or where the ratio of thickness to tooth dimension (e.g., diameter, width, etc.) is below a threshold may be marked or a prompt may be given to the user to confirm marking.

[0115] In some variations, during and / or after the automated analysis of the volumetric model, a user may manually flag one or more regions of the volumetric model of the patient's teeth (615). Regions manually added by the user may be added if the automated analysis of the volumetric model automatically flags identified volumetric models. In some variations, the user may be prompted to flag regions identified and suggested by the automated analysis. These regions may be marked and an indication may be provided as to why they were identified (e.g., irregular enamel, potential cracks, potential caries, potential thinning of the enamel, etc.). Generally, internal boundaries (e.g., boundaries within the tooth volume) may be defined in any of the methods and apparatus described herein. For example, in variations where an area of ​​enamel is thinned, the methods and apparatus described herein may be used for the entire hierarchy (e.g., layers of enamel, areas of the tooth's internal structure, etc.) that may be identified and used for qualitative and / or quantitative information.

[0116] In some variations, the regions flagged in this manner may then be displayed (617) on the digital model of the patient's teeth. This display may highlight the flagged regions, for example by applying color, animation (e.g., flashing lights), icons (e.g., arrows, flags, etc.), including combinations thereof. The display may also show a magnified view of any of them. The user may modify this display, for example, to rotate, segment, or magnify the flagged regions. Alternatively or additionally, the flagged regions may be magnified on the display device by default. An index or key of the flagged region may be provided and may be displayed and / or stored with the digital volumetric model of the patient's teeth.

[0117] In some variations, as shown on the right side of Figure 6, the method may include modifying future scans using the flagged regions, as described above. For example, the method may include performing a scan using the flagged regions (a "rescan") after some interim period between the first period (e.g., about 1 day, 1 week, 1 month, 2 months, 3 months, 4 months, 5 months, 6 months, 7 months, 8 months, 9 months, 10 months, 11 months, 1 year, 1.2 years, 1.5 years, 2 years, etc.) and the second period (e.g., about 1 month, 2 months, 3 months, 4 months, 5 months, 6 months, 7 months, 8 months, 9 months, 10 months, 11 months, 1 year, 1.5 years, 2 years, 3 years, 4 years, 5 years, 6 years, 7 years, 8 years, 9 years, 10 years, 11 years, 12 years, 13 years, 14 years, 15 years or more, etc.), or after an interim period longer than the second period. Using flagged regions to modify a scan may be done by increasing the resolution of the scan region during scanning, for example, by increasing the scan rate, increasing the dwell time of the region, scanning the region with a different wavelength, scanning the region multiple times, etc. The scanning device may prompt the user to adjust the scan (e.g., by moving the intraoral scanner wand more slowly through these regions, retracting the scanner multiple times over these regions, etc.) and / or may automatically adjust scan parameters during operation. Thus, the scanning device may receive a key or index of the flagged regions and / or marked (flagged) previous intraoral scans of the patient. Prior to scanning with the scanning device, the user may provide the identity of the patient to be scanned, which may be used to look up previous scans. Alternatively or additionally, the device may identify the patient based on the current scan, identify the patient (or confirm the patient's identity), and then review or recall previous annotated (flagged) scans. Alternatively, a second or subsequent scan may be performed without using the previous areas that were flagged.

[0118] After one or more subsequent (e.g., second, subsequent, or follow-up) scans, the method, or an apparatus configured to perform the method, may compare the flagged regions of the subsequent scan with corresponding regions of the previous scan (621). Additionally, the volumetric model from the subsequent scan may be automatically analyzed (e.g., the previously described automated or semi-automated analysis steps 603-615 may be repeated) to identify any regions that can or need to be marked or flagged in the new (subsequent) scan (621). The newly identified regions of the subsequent scan may be compared with corresponding previously unflagged regions of the previous volumetric model.

[0119] The flagged regions may be analyzed over time (623). Specific sub-regions from the volumetric model containing the flagged regions may be generated for display and analysis. The results may be output (625). For example, these regions may be displayed to a user along with descriptive analytical information about the scanned region. These regions may be marked to show change over time. The data may be displayed in an animated format, e.g., to show change over time. In some variations, the images may be displayed as time-lapse images (video, loop, etc.) showing change. The time-lapse images may show changes in internal and / or external structures over time. Cross sections (pseudo-sections generated from the volumetric model) may be used to show change. Color, texture, pattern, and any other highlight visualization technique may be used. As an alternative or in addition to displaying the flagged areas, these areas (and any accompanying analysis results) may be output in any other suitable manner, such as digitally (e.g., to the patient's dental record), by printing a description of the flagged areas, etc.

[0120] Any of the methods described herein for tracking a region of a patient's dental arch may include tracking over time and / or across different imaging modalities (e.g., records), as described in more detail below. Additionally, any of these methods may be automated and / or include automated agents, such as for identifying (including scoring) one or more regions of interest (e.g., features, defects (including treatable dental features)) and / or automating the identification, scoring, and / or display of such regions. Any of these methods may include any display methods or agents (e.g., for displaying cross sections, displaying internal structures, displaying virtual x-rays, displaying across multiple imaging modalities, etc.).

[0121] For example, one method of tracking a region of a patient's dental arch over time may include collecting a first three-dimensional (3D) volumetric model of the patient's dental arch, the 3D volumetric model including surface values ​​and near-infrared transparency values ​​of the internal structure of the dental arch; identifying a region of the 3D volumetric model; flagging the identified region; collecting a second 3D volumetric model of the patient's dental arch; and displaying one or more images with any differences between the first and second 3D volumetric models in the flagged region marked on the one or more images.

[0122] Any of these methods may include tracking and / or comparing across different records (e.g., different imaging modalities), whereby the identifying step includes identifying a region of the patient's dental arch from a first record of a plurality of records, each record including a plurality of images of the patient's dental arch each acquired using an imaging modality, and further, each record of the plurality of records acquired with a different imaging modality, and further, flagging includes flagging the identified region in a corresponding region of a 3D volumetric model of the patient's dental arch. Methods and apparatuses implementing them may also include correlating the flagged region with each record of the plurality of records by correlating each record of the plurality of records with the 3D volumetric model of the patient's dental arch. In some variations, the correlating may be performed to weight or rank the identified region to determine whether the identified region corresponds to a region of interest (e.g., a feature, a defect (including a treatable dental feature), etc.). For example, a region of the patient's dental arch may include dental features including one or more of fissures, gingival recesses, tartar, enamel thickness, pits, caries, pits, fissures, evidence of bruxism, and interproximal spaces. Identifying the region may include comparing a near-infrared transparency value of a region in the 3D model to a threshold value.

[0123] When surface values ​​are used, the surface values ​​may include surface color values. These methods may be used with stored data and / or data collected in real time (thus, for example, the methods may optionally, but not necessarily, collect a three-dimensional (3D) volumetric model by scanning a patient's dental arch to generate a 3D volumetric model).

[0124] Identifying the regions may include automatically identifying using a processor. For example, automatically identifying may include identifying surface color values ​​outside a threshold range. Automatically identifying may include segmenting the 3D volumetric model to identify enamel regions and identifying regions where the enamel thickness is below a threshold.

[0125] Flagging the identified regions may include automatically flagging the identified regions or manually reviewing the identified regions for flagging.

[0126] In any of these methods where regions are flagged, a step of rescanning the patient's dental arch may be included, and the scan of the flagged region may be performed at a higher resolution than the unflagged region.

[0127] One method of tracking a region of a patient's dental arch over time includes collecting a first three-dimensional (3D) volumetric model of the patient's dental arch acquired at a first time point, the 3D volumetric model including surface color values ​​and near-infrared transparency values ​​of the internal structures of the dental arch; and using an automated process to identify regions within the 3D volumetric model to be flagged from a first record of a plurality of records, each record including a plurality of images of the patient's dental arch, each acquired using an imaging modality, and further wherein each record of the plurality of records includes a different image. The method may include identifying the identified regions acquired in an imaging modality, flagging the identified regions, correlating the flagged regions with each record of the plurality of records by correlating the 3D volumetric model of the patient's dental arch with each record of the plurality of records, acquiring a second 3D volumetric model of the patient's dental arch acquired at a different time, and identifying any differences between the first 3D volumetric model and the second 3D volumetric model in the flagged regions.

[0128] Similarly, as previously summarized and described, one method for tracking dental features across different imaging modalities includes collecting a first three-dimensional (3D) volumetric model of a patient's dental arch, where the 3D volumetric model of the patient's dental arch includes surface values ​​and internal structure of the dental arch; identifying a region of the patient's dental arch from a first record of a plurality of records, where each record includes multiple images of the patient's dental arch acquired using an imaging modality, and further, where each record of the plurality of records is acquired with a different imaging modality; flagging the identified region in a corresponding region of the 3D volumetric model of the patient's dental arch; correlating the flagged region with each record of the plurality of records by correlating the 3D volumetric model of the patient's dental arch with each record of the plurality of records; and storing, displaying, and / or transmitting an image including the region of the patient's dental arch. All of these methods may also involve tracking over time, for example by comparing 3D volumetric models of the same region at different time points.

[0129] FIGS. 10A and 10B show user interfaces illustrating marking regions of interest in a 3D volumetric scan of a patient's oral cavity. In FIG. 10A, the user interface includes an image 1001 of internal features (e.g., based on near-infrared absorption of teeth) similar to FIG. 9B above. This view can be manipulated by user controls 1015, which include tools for sectioning, rotating, moving, etc. In FIG. 10A, the two top windows show a surface view 1003 and a volumetric (internal) view 1005 corresponding to the same region. This region is selectable. FIG. 10B shows the same features as FIG. 10A, but with a marked or flagged region 1011. As noted above, identifying a region to be flagged can be automatic, manual, or semi-automatic (e.g., user confirmation), and can be performed to select a region for later monitoring. In FIG. 10B, the region may correspond to, for example, a possible caries lesion.

[0130] Monitoring one or more internal regions of a patient's teeth over time using a volumetric model of the teeth obtained with the devices described herein can be particularly useful in predicting dental problems, including caries, cracks, missing teeth, gum recession, and the like. Specifically, these methods and devices can help users (e.g., dentists, dental technicians, orthodontists, etc.) educate and educate patients so that they can implement recommended treatments before more serious problems develop. Effective methods are needed to display changes in teeth over time and provide patients with the information they need to take early action to prevent more complex and potentially more painful problems from developing. In the absence of such methods, many patients are reluctant to undergo preventative treatment, especially if they are not currently experiencing any associated pain or discomfort. For example, pre-cavity caries can be particularly difficult to convince patients to treat because it is difficult to identify with current imaging technology and, even when identified, is typically painless. However, early treatment can be crucial to avoid more complex and risky treatment later.

[0131] Caries is one type of problem that can be identified using the methods and devices described herein. As previously shown and described, caries can be identified from 3D volumetric models (e.g., the 3D volumetric models described herein) that use light (e.g., near-infrared light), a type of non-ionizing radiation, to penetrate the tooth. In 3D volumetric models generated as described herein (e.g., using near-infrared light, typically in combination with a surface scan (e.g., white light)), the absorption coefficients of the interior regions of the tooth can show the distinction between dentin and enamel, revealing internal structure and flaws, including cracks, caries, etc. For example, areas of enamel that are less transparent than expected at near-infrared wavelengths may have different infrared optical properties; in particular, areas of enamel that appear to extend to the surface of the tooth in the volumetric model can be manually or automatically identified as cavities or caries. Other irregularities in the enamel and / or dentin may be identified (e.g., based on internal features of the tooth from the volumetric model) and may be characteristic of other dental problems. Thus, the techniques described herein may be used for the prognosis of dental problems such as caries.

[0132] As mentioned above, any of the devices and methods described herein may include improved methods for displaying internal tooth features using volumetric models of one or more of a patient's teeth. For example, the methods and devices described herein may be used to generate estimates of enamel thickness for one or more of a patient's teeth. These estimates may be visually displayed to show the exterior surfaces of multiple teeth or a particular tooth, and may also show the internal structure, including showing cross-sectional or 3D internal views (e.g., of the enamel, including enamel thickness). This information may be used clinically to determine the need for, assist in the design, and assist in the application of dental prostheses, including veneers, crowns, and the like. Any of the methods and devices described herein may be used, for example, to assist in the preparation of a dental implant design for a particular tooth or teeth.

[0133] Plaque and tartar detection and visualization The methods and devices described herein may be used to detect and visualize (including quantitate) plaque and calculus on a patient's teeth. Any of the intraoral scanners described herein may be used to detect plaque or calculus on a patient's teeth by using fluorescence imaging in conjunction with other imaging / scanning modalities, including penetrant (e.g., near-infrared) imaging. For example, the intraoral scanner may be cycled between different imaging modalities, such as white light and near-infrared light, which may include additional modalities such as fluorescence (e.g., laser fluorescence, etc.).

[0134] Using the fluorescence feature (and / or existing features) in the intraoral scanner may enable the detection of plaque and tartar on the tooth surface. In combination with 3D modeling using data from the intraoral scanner, the plaque / tartar condition can be modeled and visualized on a 3D model of the teeth, including 3D volumetric modeling of the teeth. Plaque and / or tartar may be detected, displayed, and highlighted as described above, which may occur before, during, or after treatment. For example, a dental technician (e.g., a dental hygienist) may detect and monitor a patient's condition with the intraoral scanner and perform a cleaning procedure. Any of the devices described herein may also use plaque and tartar data to determine and provide a predictive model that can indicate the incidence and / or location of plaque and tartar.

[0135] In some variations, identification of plaque and tartar may be performed at least in part using fluorescence information. It has been observed that plaque can fluoresce under blue light (e.g., around 405 nm). Any of the intraoral scanners described herein may include fluorescence information, and this information regarding plaque and tartar may be used to incorporate into a 3D model of the patient's teeth. For example, plaque and / or tartar may be visually represented as color and / or texture on the 3D model of the patient's teeth.

[0136] For example, a dichroic filter with a large aperture magnification of the fluorescence signal can be used to acquire the fluorescence signal from the intraoral scanner. This magnification can enhance the fluorescence, thereby allowing plaque and calculus areas to be detected, visualized, and segmented using RGB illumination, sensors, and images. Alternatively or additionally, the device can include a fluorescent light source (e.g., an LED emitting at 405 nm) and corresponding filters for plaque and calculus detection. This can be integrated into the intraoral scanner or added (e.g., as a sleeve, accessory, etc.) for use with the scanner.

[0137] Alternatively or additionally, in some variations, the absorptivity / reflectivity of plaque and calculus can be made different from enamel, depending on the wavelength of near-infrared light used. This may allow for the volumetric model to distinguish between tartar and / or plaque and enamel. Furthermore, the volumetric model can be used to detect materials on the teeth, including tartar and plaque, based on surface smoothness and geometry. In variations where tartar and / or plaque are not transparent to the near-infrared frequencies used, the device can use the volumetric model to distinguish between tartar and / or plaque and enamel. Thus, tartar and / or plaque can be segmented and distinguished from enamel.

[0138] The use of intraoral scanners to detect plaque and / or calculus can provide quantitative information and digital modeling, which may allow for monitoring and comparison of plaque / calculus over time based on alignment with 3D models (including real-time alignment and / or display).

[0139] Acquiring both the fluorescence image and the 3D scan at the same time and in the same position on the intraoral scanner (e.g., the scanning wand) allows for highly accurate registration of the plaque / calculus area with the 3D model. Simultaneous scanning is described in detail, for example, in U.S. Patent Application No. 15 / 662,234, filed July 27, 2017, entitled "INTRAORAL SCANNER WITH DENTAL DIAGNOSTICS CAPABILITIES." Accurate registration between different scanning modalities (e.g., white / visible light, penetrating (near-infrared) light, and / or fluorescence) can enable the device to demarcate the calculus and / or plaque boundaries and determine the volume / thickness with high resolution, allowing both precise current status measurements and relative comparison / tracking with previous scans.

[0140] The methods and devices described herein can acquire RGB images of the teeth simultaneously / nearly simultaneously with acquiring 3D scans of the teeth. These scans can then be used to construct a 3D model of the teeth / jaw, which may include volumetric information (a 3D volumetric model). For example, the RGB images can show signals of fluorescent surfaces, particularly areas of plaque and calculus, enhanced by the inherent color and brightness characteristics of such surfaces, as described above. For example, an image of the outer tooth surface (and possibly a volumetric model) can show areas with optical properties (fluorescence, brightness, color, etc.) representative of calculus and / or plaque. In some variations, this enhanced signal is obtained by spectral illumination that does not cause plaque and calculus to reflect visible light but generates a significant fluorescent signal. For example, typical RGB illumination (using a common RGB sensor) can be modified to enhance the fluorescent signal (e.g., in the near-infrared region) at the outer tooth surface. This enhancement can be achieved, by way of a non-limiting example, with a large aperture that transmits infrared signals and a smaller aperture that transmits the common RGB (visible) spectrum. This combination allows for the production of colored images that specifically highlight fluorescent surfaces, which may appear in characteristic colors and intensities in desired areas that represent dental calculus and / or plaque.

[0141] In any of the described methods and apparatuses in which an RGB image containing a fluorescent signal can be acquired (e.g., at a wavelength at which plaque or calculus fluoresces), segmentation of the fluorescent regions can be performed on the image. For example, using the RGB and the camera position during acquisition of the 3D scan (e.g., from an intraoral scanner), the fluorescent regions can be registered with a 3D model (including a volumetric model and / or simply a surface model) of the patient's teeth. This can result in the definition of the plaque and calculus regions of interest on the final 3D model, which may further enable the definition of these regions, e.g., the boundaries of the calculus on the teeth and the 3D surface and thickness of the plaque.

[0142] As described above, areas on the 3D model may be compared with previous or future scans of the same patient, thereby showing the growth of tartar over time and its effect on the patient's teeth. The device may automatically or semi-automatically mark (e.g., flag) these areas for monitoring. Thus, the size and shape of tartar on each tooth may be monitored. Alternatively or additionally, the thickness / depth of tartar may be compared with previous scans. Both of these pieces of information may be provided quantitatively and / or qualitatively, as described above. The thickness / depth of tartar may be compared with previous scans of clean teeth (including one or more previous scans after cleaning by a dental professional). This allows for an estimation of tartar thickness in subsequent scans. As described above, changes in plaque, particularly tartar, over time may be measured, and this data may be used to monitor the progression of plaque and tartar on the patient's teeth, even visualizing growth.

[0143] Generally, monitoring and visualization of a patient's teeth using the methods and devices described herein may be performed as part of a dental and / or orthodontic treatment plan. As described above, monitoring of tartar and / or plaque may be performed for treatment, including tooth cleaning. Scans may be performed before, during, and / or after cleaning, which can provide guidance to the dental practitioner on areas to emphasize, focus on, or return to. Based on the progression of plaque and / or tartar over time, other treatments (e.g., coding, caps, etc.) may be suggested. Additionally, treatment planning information may be obtained from monitoring any other features or areas of interest (e.g., including caries, fissures, etc., as described above). As described above, fissure and / or caries information may be used to suggest treatments, such as restorations, before potential problems progress further. In some variations, digital models of the teeth (e.g., surface models and / or volumetric models) may be modified using the volumetric information, and the modified models may be used to design orthodontic appliances or treatment plans. For example, a user can digitally remove plaque and / or calculus from a volumetric scan taken before or during treatment, and this corrected scan can be used to recommend treatment, including further cleaning of the teeth, as needed, or to shape or modify appliances (e.g., dental aligners) to improve their fit.

[0144] Combination with dental tools The intraoral scanners and volumetric modeling described herein may be used in conjunction with and / or combined with other dental tools (drills, probes, etc.), which offers numerous advantages.

[0145] For example, the present specification describes a drill that can be used in conjunction with or in combination with an intraoral scanner, and the use of 3D volumetric models. In some variations, a dental drill and an intraoral scanner may be combined; for example, a laser drill or a laser-accelerated water drill may be incorporated into the intraoral scanner. This combination can allow a dental professional to use the tool to directly visualize the tooth during or before drilling and provide real-time feedback to the user. In one example, near-infrared light can be applied to the probe head of a drill (e.g., a laser drill) to image the inside of the tooth, allowing direct forward-looking imaging before and / or during drilling. The enamel and dentin in the path of the drill can be imaged. Density information can inform a clinician when the dentin layer of the tooth, or a certain depth within the dentin, has been reached or when a diseased area has been removed. For example, density information can be used to provide tactile feedback to the operator, as tactile feedback is significantly limited when using a dental laser with a conventional headpiece.

[0146] Methods and devices including intraoral scanners and volumetric modeling as described herein may be incorporated into dental computer-aided design / computer-aided manufacturing techniques, as shown in FIG. 7 . For example, dental implants such as crowns (e.g., ceramic crowns) can be custom-fabricated for individual patients using computer-aided design and computer-aided manufacturing (CAD / CAM) equipment and procedures. For example, traditionally, CAD / CAM lab fabrication (the “Current Workflow” in FIG. 7 ) may include a pre-treatment scan 701 of the patient's teeth, or an impression of the patient's teeth (e.g., a caries-free scan of the jaw). The teeth may then be prepared for the crown 703, and then re-scanned 705 and evaluated 707. Finally, the crown may be fabricated by CAD / CAM. CAD / CAM software may receive scan information from the scanner and process the scan information for use in creating the design and implementing manufacturing. While the use of CAD / CAM software allows for restorations that are comparable to traditional restorations in all aspects, including aesthetics, current methods may require repeated tooth evaluation and preparation steps, as shown in FIG. 7, and typically require the user to perform these steps manually.

[0147] As shown in the "New Workflow" section at the bottom of FIG. 7 , the method may incorporate the 3D volumetric modeling described herein to simplify and improve CAD / CAM of a patient's teeth. For example, pretreatment may be digitally designed, and this process may be automated (fully or semi-automated, allowing a user to approve and / or modify the process). For example, in FIG. 7 , the pretreatment scan 711 may be performed using an intraoral scanner in direct communication with a CAD / CAM device, or the intraoral scanner may include CAD / CAM functionality. In this example, tooth pretreatment 713 may be fully digitally designed based on the performed scan, and the scanner may guide the tooth pretreatment (715), which may be done in real time with direct feedback and / or guidance from the device in which the scanner may be integrated. The scanner may then be used for pretreatment evaluation 717; in some cases, this step may be fully integrated with the guided pretreatment step 715, in which case a post-pretreatment evaluation is not required. Finally, CAD / CAM may be used to prepare crowns (or other dental devices) for properly prepared teeth (719).

[0148] root canal Methods and devices for 3D volumetric modeling of a patient's oral cavity (e.g., 3D volumetric modeling of a tooth) may also be used to modify a root canal procedure. Typically, root canal procedures require numerous x-rays to obtain images of the inside of the tooth before, during, and / or after the procedure. The methods and devices described herein can eliminate or reduce the need for x-rays in certain instances of root canal procedures. Specifically, as described herein, intraoral scanners including penetrating wavelengths (e.g., near-infrared wavelengths) may be used to examine the inside of the tooth, including examining the inside of the root during the procedure. This may enable the identification and location of the root canal. For example, preparing a tooth for a root canal may be performed by, for example, drilling a hole through the crown and into the tooth. This hole may be drilled with the guidance of the volumetric imaging described herein during or between drillings. For example, an initial hole may be drilled into the interior of a tooth (e.g., a molar) to expose the tooth's internal cavity. Intraoral scanners including near-infrared wavelengths may be used to image teeth (including imaging through a drilled hole) to visualize the pulp cavity. The scanner can be automatically or manually oriented to image deep into the cavity, which may allow visualization of the root within the cavity. Initial drilling into the tooth may be limited to penetrating the enamel to expose the cavity and visualize the cavity, allowing regions with different optical properties (at any wavelength, including certain near-infrared wavelengths) to penetrate the cavity despite calcification and / or infection, thereby allowing imaging of the root from inside the tooth itself. The nerve cavity of the root can be identified as being more or less dense than the surrounding areas in the dentin and enamel. By removing the roof of the cavity and exposing the inner pulp region of the tooth, the intraoral scanner can visualize within the drilled opening to provide additional volumetric information, including the location, curvature, and path of the root. This additional visualization information can facilitate the detection of hidden root canals and accessory canals, which can then be used to guide treatment.

[0149] For example, in some variations, the method may include using an intraoral scanner as described herein to obtain a 3D volumetric model of the patient's tooth before or after drilling to create an opening into the target tooth (e.g., where a root canal procedure will be performed). Drilling may be performed as described above with or without guidance from the intraoral scanner. The tooth's interior cavity may be visualized with the intraoral scanner (e.g., through an opening drilled through the crown). The device may then identify the location of the pulp chamber corners. Any of the methods described herein may be used in combination with x-ray information. Treatment planning may be performed by the device identifying the shape and / or location of the pulp horns, pulp chamber, and root to carefully plan the drilling / tissue removal procedure without overthinning or breaching the sides of the tooth. This treatment plan may then be used to guide the user in drilling the tooth and / or to automate the drilling. In some variations, the drill may be directly guided by imaging (e.g., using a hybrid drill / intraoral scanner as described above). Alternatively or additionally, robotic assistance using treatment planning may be used. In some variations, the procedure may be performed manually, with drilling being performed in small, incremental steps, with visualization between drilling steps to confirm the treatment path, avoid over-drilling, and ensure all areas have been drilled and the infected pulp removed. Further visualization (including the use of contrast agents) may be performed.

[0150] Generally, any of the methods described herein, including the root canal method described above, may be used with one or more contrast agents during imaging. For example, the contrast agent may include a substance applied to the outside of the tooth (or within a cavity or opening in the tooth, including a drilled hole in the tooth). A contrast agent that absorbs or reflects near-infrared wavelengths or other wavelengths used in intraoral scanners may be used. Preferably, a contrast agent that is distinguishable at some, but not all, of the imaging wavelengths may be used to achieve differential imaging. For example, the contrast agent may be visible under white light but not near-infrared light, or may be visible under near-infrared light but not white light, or may be visible under some near-infrared wavelengths at which images are acquired (but not other near-infrared wavelengths at which images are acquired). Preferably, a contrast agent may be used that binds to and mixes with or is applied to one or more targets within the tooth or oral cavity. For example, a contrast agent that selectively binds to one or more of bacteria, plaque, tartar, gums, dental pulp, etc. may be used. When used, the contrast agent may be applied to the tooth / oral cavity, rinsed off, and visualized (or visualized without rinsing). For example, a contrast agent that absorbs infrared light may be used, for example, as part of or mixed with the material forming a dental implant (e.g., a dental implant for filling a cavity, capping a tooth, filling a root canal, etc.) to create an infrared contrast filling material that can be easily visualized when scanned as described herein.

[0151] Further described herein are methods for identifying improvements in the soft tissue surrounding a tooth using the devices and methods for generating a 3D volumetric model of the tooth as described herein. For example, gingival recession may be monitored and / or quantified and monitored over time using these methods and devices. In addition to direct visualization of plaque and / or calculus as described above, the methods and devices described herein can also, or alternatively, detect effects on the tooth, including bone recession caused by plaque and calculus. Diseased areas may be directly visualized. In some variations, a contrast agent may be used to enhance the contrast of the intraoral scanner to detect diseased areas. Scanning the gingival surface can identify areas of inflammation and / or discoloration, which may indicate periodontal disease. This information may be combined with 3D volumetric modeling of the tooth, including the location of plaque and / or calculus, as described above.

[0152] 8A and 8B show an example of monitoring gingival recession over time. In this example, the display may show a 3D model of the teeth and a comparison between an initial scan and a subsequent scan taken two to three years later. In FIG. 8A, the two scans are aligned and compared, with differences indicated by a color indicator (e.g., a heat map). In FIG. 8, darker colors (which may be indicated by a color, e.g., red) indicate greater gingival recession. The circled region B in FIG. 8A is shown in more detail in FIG. 8B with respect to the subsequent scan. While FIG. 8 primarily shows surface features (e.g., gingival position), volumetric information may be used to generate this information to indicate, for example, changes in gingival thickness and / or vascularization, enamel thickness, etc.

[0153] In addition to guiding a user and / or dental technician based on scans (e.g., particularly showing plaque, calculus, and / or inflammation), these methods and devices may also be used by dental professionals to evaluate, grade, or quantify plaque and / or calculus removal immediately after treatment or over time, which may provide a measure for evaluating treatment. Scan information may also be used to provide information to patients, including maps or guides for home treatment (e.g., where to focus when brushing, flossing, etc.). This guide may include, for example, one or more images from a 3D volumetric model. Guidance information can be provided to an electric toothbrush regarding which areas of the teeth or oral cavity to focus home dental care (e.g., brushing) on, which may help the electric toothbrush guide the patient's brushing based on the identified areas.

[0154] The methods and devices described herein may also be used when the patient already has dental appliances (including braces, bridges, etc.) on their teeth. For example, in some variations, the patient may include a 3D representation of the dental appliance, which can provide information to aid in the design or modification of future dental appliances (e.g., retainers, aligners, braces, etc.).

[0155] In particular, the methods and devices described herein can be used to provide highly accurate volumetric and surface information of a patient's teeth, which can be useful in treatment planning for any type of dental treatment. In some variations, the methods and devices described herein can be useful in treatment planning for appliances (e.g., aligners and retainers) that fit closely to the patient's teeth for optimal placement. For example, methods and / or devices that include a 3D volumetric scan of a patient's teeth can be used to clean or remove any plaque, tartar, and / or food debris from the 3D model of the teeth that may be present at the time of the 3D scan. By digitally removing any present plaque, tartar, and / or food debris, the volumetric information can be used with a virtual representation of an appliance, such as an aligner, retainer, or night guard, to improve fit before the appliance is fabricated, applied, or fitted.

[0156] The gingival tissue surrounding the tooth has a different density (or optical absorption / reflection properties) than enamel and can be more accurately identified and characterized, allowing the boundary between the internal contour and the tooth surface to be identified. By doing this, the shape of the tooth surface beneath the gingival tissue can be accurately characterized, thereby enabling a more accurate representation of the tooth in a tooth movement prediction model. This is because initially invisible portions of the tooth are gradually exposed as the teeth straighten. In other words, a portion of the tooth may initially be obscured by gingival tissue, but as the tooth straightens, the gingival tissue moves, exposing the previously covered area. By accurately detecting the tooth area beneath the gingival tissue, the future state of the tooth after gingival movement can be more accurately modeled.

[0157] The methods and devices described herein may also be used to detect, diagnose, and / or treat oral disorders.

[0158] For example, the 3D volumetric scanning and modeling methods and devices described herein may be used to detect and / or treat salivary stones (e.g., blocked salivary ducts). Such glands may be located under a patient's tongue near the molars and can be scanned with the intraoral scanner described herein. Such scans can penetrate soft tissue and detect hard, stone-like formations (i.e., salivary stones, salivary gland stones, or duct stones), which are calcified structures that form inside the salivary glands or ducts and can obstruct the flow of saliva into the mouth. The methods and devices described herein may be used to identify these structures and / or to guide and / or confirm the removal of these stones.

[0159] In addition to or instead of using the devices and methods described herein to identify, diagnose, and / or track regions containing pre-cavity caries, fissures, etc., the methods and devices described herein may also or alternatively be used to identify and manipulate regions that have already been modified. For example, filling materials, attachments (e.g., for attaching anchors, braces, etc.), braces, retainers, etc., as well as any other structures, can be identified and / or displayed within the volumetric model. For example, regions of enamel and / or enamel-like restorations can be displayed separately in the volumetric model. These regions typically have different optical properties, e.g., scattering / absorption properties of near-infrared light (and possibly visible light), compared to each other and / or to other regions of the oral cavity, including dentin. Such regions may be identified manually, automatically, or semi-automatically, segmented, and / or separately manipulated. For example, in some variations, those areas on the teeth (e.g., attachments / cement for aligners or other appliances, etc.) may be identified by the dental practitioner as being removed, and 3D volumetric models or data (images) obtained from the 3D volumetric models may be provided to guide such treatment. These may also or alternatively be digitally stripped to improve the fit of new appliances once removed. The stripped views may also or alternatively be provided to the patient.

[0160] In some variations, the volumetric models described herein can be used to examine the internal structural integrity of artificial dental structures or modifications (e.g., dental bonds, fillings, etc.). For example, the volumetric models may include internal details of the artificial dental structures (e.g., structural details within the fillings, bonds, etc.) or their interface with natural teeth (enamel, dentin, etc.), and this information may be presented or illustrated in detail to a user, thereby enabling assessment (or automatic assessment) of the condition of such artificial dental structures, which may facilitate their removal, restoration, and / or replacement.

[0161] 3D volumetric tooth models (and methods and devices for generating them) may also be used as a tool for diagnosing or detecting future tooth sensitivity. For example, abfractions are a form of caries-free tooth tissue loss that typically occurs along the gingival margin. Abfraction lesions can be mechanical defects in tooth structure that are not caused by tooth aging and can occur in both the dentin and enamel of the tooth. They are thought to be caused by repeated stress cycles caused by the patient's bite and exacerbated by aggressive brushing. 3D volumetric tooth models, enhanced with density analysis of the enamel and dentin near the gingival gland, can provide early indicators of these lesions. For example, a device can examine the volumetric model to identify the early stages of the formation of these crescent-shaped lesions. Multiple 3D volumetric models acquired over time can indicate the rate of progression of these lesions. A system can be configured to identify them automatically or manually, and as described above, they can be flagged automatically or semi-automatically.

[0162] The device and method can therefore alert the user to potential "hot spots" that could lead to future tooth sensitivity and provide a treatment plan to slow, stop, or reverse the progression of the damage. Tooth sensitivity can result from these small cracks and exposed dentin. While detection can be triggered by identifying the characteristic crescent shape that manifests in more advanced stages of damage, earlier detection is possible by identifying areas of enamel and / or dentin thinning (e.g., near the gum line), which can progress over time. The device and method can flag and / or assign risk based on actual thickness and / or progression of thickness change.

[0163] The methods and devices described herein may also be used to detect the onset of acid reflux based, in part, on characteristic wear patterns and / or changes (e.g., over time) in a patient's enamel thickness. For example, acid reflux while a patient sleeps may gradually erode a patient's teeth in a characteristic pattern (e.g., starting from the posterior side of the teeth on the lingual side). A similar pattern may develop in cases of bulimia. A volumetric model of a patient's teeth obtained (e.g., with near-infrared light) can provide an accurate mapping of the enamel density and thickness of all of the patient's teeth. Thus, one method of detecting acid reflux (or bulimia) may include detecting (including detecting over time) characteristic thinning of the enamel in the posterior lingual regions of a patient's teeth. The more proximal lingual regions of the teeth may have abnormally thin enamel thickness (thinning) compared to the more anterior (front) regions of the patient's teeth on the opposing buccal side.

[0164] The methods and devices described herein may also be used to detect thinning of enamel areas due to attrition caused by a patient's habitual bruxism, and / or to predict potential tooth sensitivity as a result of this bruxism. A 3D volumetric model of a patient's teeth may provide a snapshot of the patient's occlusal thickness of enamel and the proximity of dentin to the occlusal surfaces. Additionally, multiple scans taken over time may indicate enamel defects at the occlusal surfaces. This has already been mentioned as one indicator that can be marked or flagged automatically, manually, or by semi-automatic mapping. For example, 0.5mm of dentin in 0.5mm of dentin may be present. 2Any occurrence of larger areas may be flagged, and the area may be highlighted on the digital model. This allows visualization and / or monitoring of all areas that meet the flagging criteria. Given the patient's age, and possibly gender, as well as the change in enamel thickness over time, an estimate of wear rate over time, along with proximity to dentin areas, may be provided, which may allow estimation or prediction of tooth sensitivity or pain. Bruxism may also be an indicator of other problems, including sleep apnea. For example, sleep apnea may be detected from a 3D volumetric model of a patient's teeth, particularly over time. Many sleep apnea sufferers grind their teeth (e.g., front-to-back and / or side-to-side), which may cause erosion patterns on the teeth. Accordingly, the methods and devices described herein may be used to aid in the diagnosis or confirmation of sleep apnea.

[0165] Generally, any of the methods and devices described herein may be used with non-human patients, for example, any of the methods and devices described herein may be used with veterinary patients (e.g., animals) to examine, for example, the condition of their teeth (such as tooth wear).

[0166] The methods and devices described herein may also be used to estimate a patient's risk of developing cracks and / or tooth sensitivity. For example, the 3D volumetric tooth models described herein may be used to identify dental malocclusions and resulting tooth wear and / or cracks based on mechanical estimation of tooth thickness and wear patterns. Functional information, such as chewing patterns and bite forces, may also be incorporated into the assessment. Wear patterns may be identified and displayed as "hot spots," which may be displayed, for example, on an image generated from a 3D representation of the patient's teeth. This may be displayed to the patient for information, for example, to warn of potential risks. Areas of high risk may be identified to the patient along with an explanation of the potential risks.

[0167] In general, the present methods and devices, and in particular the monitoring and comparison over time of 3D volumetric models containing information about the internal structure of a tooth (e.g., the distribution of enamel and dentin within the tooth), may be used to identify, monitor, diagnose, and guide treatment of various disorders in addition to those mentioned above, such as dentin dysplasia, enamel dysplasia, etc. These methods also enable the identification of different types of enamel within a patient's teeth, including regions with different amounts of hydroxyapatite, amelogenin, and / or enamelin, or regions with different organizations (which may be homogeneous or heterogeneous, and may have different optical properties at near-infrared wavelengths used for imaging).

[0168] Interactive display of a 3D model of the patient's dental arch As previously mentioned (and illustrated), the methods and apparatus described herein may allow a user to virtually scan a patient's dental arch. Specifically, a 3D model of the patient's dental arch (which may be a volumetric model, a surface model, or both (or in some variations, an abstracted or generic model)) may be used along with images acquired (e.g., using an intraoral scanner) from various positions around the dental arch. These images may be the images used to generate the 3D model of the dental arch. These images may be tagged and / or arranged as a data structure to identify their corresponding positions, regions, or angles relative to the 3D dental arch model. In some variations, the 3D model and the acquired images may be maintained as a data structure, although it is not necessary for the 3D model to be attached to the images in a single data structure.

[0169] For example, FIG. 13 shows an example data structure that includes one or more dental arch models 1305 and one or more (e.g., multiple sets) multiple images (e.g., more than 50, more than 100, more than 200, more than 500, more than 750, more than 1000, more than 10,000, etc.) acquired from positions around the patient's dental arch. In some variations, both visible and near-infrared (or near-infrared and other modalities) images 1301 may be shown and may share positional information. The positional information typically includes the region of the dental arch from which the image was acquired (e.g., the coordinates (e.g., in x, y, z coordinates) of the image's center point relative to the dental arch) and the angle of the dental arch relative to a plane (e.g., roll, pitch, and / or yaw, or radial coordinates, etc.) ("positional information" 1301). In some variations, the scan may be a composite (e.g., average, blend, etc.) of multiple scans stored together as a data structure, and the 3D model may be formed by virtually "stitching" the scans together.

[0170] The data structure may be stored in a compressed form. Although the data structure may contain a large amount of data, compressing and organizing the data structure may enable it to be manipulated and displayed. For example, Figure 12 shows one way of interactively displaying a 3D model of a patient's dental arch using a data structure such as that shown diagrammatically in Figure 13.

[0171] In FIG. 12 , the method includes step 1201 of displaying a 3D model of the patient's dental arch and step 1203 of displaying a view window on the 3D model. The user may then be allowed to continuously move the two (e.g., one or both of the view window and the 3D model), allowing each tooth in the dental arch to be closely examined virtually as a close-up view "through" the view window (1205). The angle of the view window, as well as its position along the dental arch, may be varied by the user (e.g., by continuously moving it over and / or around the 3D model of the dental arch) (1207). As the view window / dental arch moves relative to each other, one or more corresponding images (e.g., near-infrared images) acquired at positions relative to the dental arch corresponding to the position of the view window may be identified from a data structure / dataset (e.g., FIG. 13 ) (1209). The corresponding images may then be displayed (1211). This process may be repeated iteratively as the view window moves over and along the 3D dental arch model.

[0172] In some variations, the data structure may be organized or arranged topographically or in an indexed topographical fashion, so that images of adjacent regions may be linked or ordered in the data structure, thereby simplifying the method.

[0173] 14A-16C show an example of one variation of a user interface that may allow a user to virtually scan a 3D model of a dental arch and display corresponding images (e.g., near-infrared images), as described in FIG. 12. As described above, the user may view the near-infrared images to manually (or, in some variations, automatically) identify one or more structures / defects and / or treatable tooth features (including dental caries, cracks, wear, etc.). Displaying the corresponding 3D dental arch model and visible light images of the same area, both of which may provide perspective and allow direct comparison with the patient's teeth, simplifies and powerfully enhances dental analysis.

[0174] For example, in FIG. 14A , the display shown as user interface 1400 includes a dental arch model 1403 (a 3D dental arch model) that has been reconstructed from scans of the patient's teeth and that is stored as a data structure along with many or all of these scans. As noted above, it is not necessary, but it may be useful, to include the 3D dental arch model in a data structure along with multiple images. Furthermore, while the 3D dental arch model in this example is constructed from scans of the patient's teeth, it will be apparent that the 3D dental arch model may not be representative and may still be used to select 2D views to display, as described herein. A view window 1401, shown as a loop or circle, can be moved over or along the 3D model of the dental arch, and as the view window moves, each of two image displays 1405, 1407 is updated with an image that corresponds to the position (both the area of ​​the dental arch and the angle of the dental arch) relative to the plane of the view window. In Figure 14A, the first (top) image 1405 is a near-infrared image, and the corresponding visible light (e.g., color) image (acquired at approximately the same time / location) is shown in the lower image 1407. Alternatively, the displays shown in Figures 14B and 14C each show only one image, with Figure 14B showing a magnified near-infrared display image and Figure 14C showing a magnified visible light display image of the area corresponding to the imaging window view.

[0175] The user interface may include tools 1409 for manipulating the display (e.g., tools for rotating and moving the dental arch and / or view window, tools for modifying and marking the image and / or 3D model, tools for saving, opening, and recalling images, etc.).

[0176] 15A-15B show an example of moving a view window over the teeth and changing / updating the corresponding image. FIG. 15A shows an image of a dental arch and the corresponding near-infrared and visible light images "seen" through a view window in the mid-arch region. In FIG. 15B, the dental arch has been rotated by the user (or alternatively, the view window has been rotated lingually relative to the dental arch), causing the view window to be positioned slightly more lingual than in FIG. 15A, and the corresponding views (near-infrared and visible light) are updated in real time to show this change in the relative position of the view window.

[0177] Similarly, Figures 16A-16C show an example of a 3D model of a patient's lower dental arch, similar to the views shown in Figures 14A-14C. In use, the displayed image can change nearly continuously as the user scans over and along the dental arch by moving the view window (and / or the dental arch relative to the view window), thereby updating the displayed image in real time or near real time. The user can identify features in the near-infrared image, including density variations in areas of the enamel that are normally transparent to infrared light, which may represent caries, cracks, or wear in the enamel.

[0178] The intraoral scanning systems shown in Figures 1A-1B may be configured as an intraoral scanning system. Returning to Figure 1A, intraoral scanning system 101 includes a handheld wand 103 having at least one image sensor and a light source configured to emit light in a spectral range within the near-infrared wavelength range, and a display output (screen 102). The screen may be a touchscreen that operates as a user input device, or the system may include a separate user input device (e.g., a keyboard, touchpad, joystick, mouse, trackball, etc.). As shown in Figure 1B, the system may include one or more processors operatively connected to the handheld wand, the display, and the user input device. The one or more processors may include circuitry and / or software and / or firmware configured to: display a three-dimensional (3D) model of the patient's dental arch on a display output; display a view window over a portion of the 3D model of the patient's dental arch on the display output; vary the relative position between the view window and the 3D model of the patient's dental arch based on input from a user input device; identify near-infrared images from both the 3D model of the patient's dental arch and multiple images of the patient's dental arch acquired from various angles and positions relative to the patient's dental arch, the near-infrared images acquired at angles and positions that approximate the relative angle and position between the view window and the 3D model of the patient's dental arch; and display the identified near-infrared images (as shown in FIGS. 14A-16C) acquired at angles and positions that approximate the relative angle and position between the view window and the 3D model of the patient's dental arch.

[0179] Automatic characterization of tooth features Further described herein are methods and devices (e.g., systems including software) configured to automatically or semi-automatically identify, verify, and / or characterize one or more dental features using a 3D model (including, but not limited to, a volumetric 3D model) of all or a portion of a patient's dental arch. Specifically, these methods and devices may be configured to identify, verify, and / or characterize one or more treatable dental features that may benefit from detection and / or treatment. Treatable dental features include, but are not limited to, cracks, gingival recession, tartar, oral hard and soft tissue conditions, and the like. Another treatable dental feature may be enamel thickness. For example, the methods and devices described herein may automatically map enamel thickness (e.g., apply color mapping where the enamel thickness is less than x microns, where x may be preset and / or user-adjustable). Areas of thin enamel are areas where caries may potentially be present. Other potentially treatable tooth features include discoloration (e.g., discontinuities in color), pits, fissures, evidence of bruxism (including thinning over time), interproximal spaces, etc., or any other similar feature that may represent or suggest a site where caries is likely to form.

[0180] All of the methods and devices described herein can detect, analyze, and / or characterize dental features (particularly treatable dental features) using multiple different images or sets of images of a patient's teeth acquired with different imaging modalities. Each of the multiple different images or sets of images of a patient's teeth acquired with different imaging modalities may be referred to as a "record." Each record may be of a different imaging modality, such as a dental cone-beam computed tomography (CBCT) scan, a three-dimensional (3D) intraoral scan, a color scan (one or more of a 3D color scan, a surface color scan, etc.), a two-dimensional (2D) color scan, a near-infrared scan (including, but not limited to, one or more of volumetric near-infrared imaging, trans-illumination, and / or reflectance scan), an X-ray (including, but not limited to, a cephalometric analysis X-ray scan, a panoramic X-ray scan, etc.), etc., and may include textual or graphical patient chart information.

[0181] For example, each record may initially be processed individually. This initial scan may identify one or more tooth features, particularly one or more treatable tooth features. A single record (e.g., a single imaging modality) may initially be used to identify one or more treatable tooth features, or all records, or a subset of the records, may initially be processed to identify one or more treatable tooth features. The initial identification of one or more treatable tooth features may be performed manually, automatically, or semi-automatically. For example, one or more treatable tooth features may be identified automatically, where a system described herein may review the records (including one or more images of the patient's teeth) and flag or identify areas having characteristics associated with the treatable tooth feature. Using machine learning techniques, such as supervised learning techniques (e.g., classification, regression, similarity, etc.), unsupervised learning techniques (e.g., density estimation, cluster analysis, etc.), reinforcement learning (e.g., Markov decision process techniques, etc.), representation learning techniques, and / or principal component analysis, the system may be trained to identify or flag regions of a particular scan in a specified modality that are associated (even if loosely associated) with treatable dental features. Alternatively or additionally, a user (dental care professional, dental technician, etc.) may manually review one or more records (each in a particular imaging modality) and flag or identify regions suspected of indicating treatable dental features. In a semi-automated configuration, the system may initially flag one or more regions in the record, which may then be reviewed and confirmed / rejected by the user.

[0182] Once the one or more regions are identified, they may be flagged and / or stored in a collection of potentially treatable dental features. Their location may be relative to the originating record (e.g., location on the record) or relative to a reference model (e.g., a 3D volumetric model (described in more detail below)). In some variations, the collection (e.g., array, data structure, file, etc.) may include one or more of the type of potentially treatable dental feature, the range of potentially treatable dental feature, the grade and / or degree of the potentially treatable dental feature, the originating record, and / or the imaging modality of the originating record, etc. In some variations, the data structure may be incorporated into the originating record (or a copy thereof) and may modify the image of the originating record. This modification may be done, for example, by including a flag or marker at the identified potentially treatable dental feature and / or any metatext location, such as the grade and / or degree, etc. The grade and / or degree may refer to a level or score of confidence regarding a potentially treatable dental feature, including a level or score of confidence that an identified potentially treatable dental feature is likely to be "true."

[0183] This initial identification process of identifying potentially treatable tooth features may be performed across multiple records, or, as described above, may be limited to a subset of the records (e.g., including only one of the records). In some variations, this process may be performed iteratively.

[0184] Once one or more potentially treatable dental features have been identified, they may be cross-referenced with one or more other records using other imaging modalities. Thus, the location of the one or more potentially treatable dental features may be specifically examined to determine whether the same potentially treatable dental features are identifiable in these one or more other records. In some variations, additional records may be examined throughout during this review portion of the procedure, and any additional potentially treatable dental features in these one or more additional records may likewise be flagged as potentially treatable dental features, and the same areas of the dental arch may be examined for these other potentially treatable dental features (this may include returning to the record already reviewed (e.g., the initial or originating record)).

[0185] Comparisons across other records may be guided by translating the locations of dental features (including, but not limited to, potentially treatable dental features) between different records. In particular, it may be useful to link the examined individual dental records with a model of the patient's dental arch (e.g., any of the 3D models (especially 3D volumetric models) described above). The 3D model of the dental arch may thus act as a key for translating the locations of one or more potentially treatable dental features, allowing for fast and efficient comparisons between various records, e.g., various imaging modalities.

[0186] Thus, before or after the initial scan for potentially treatable dental features, correlations between each of the different records, and in particular between all or a portion of the different records and the 3D model of the dental arch (e.g., a 3D volumetric model), can be established. Any method for correlating a record with other records and / or the 3D model of the patient's dental arch (or a portion of the dental arch) may be used. For example, one or more easily recognizable features (e.g., tooth edges, shapes, segmentations, etc.) may be used to determine landmarks that allow for translation between one or more records and / or between one or more records and the 3D model of the patient's dental arch. In some variations, a transformation data set may be created that includes transformations between the records and / or between each record and the 3D volumetric model of the patient's dental arch. For example, a 3D volumetric model of all or a portion of the dental arch may include transformation information for each of the one or more records, which allows for transformation of images of the one or more records (e.g., estimation of the distance and / or orientation of an imaging modality relative to the record images). This allows both forward and inverse transformation of positions between each record and a 3D model (e.g., a volumetric model).

[0187] Thus, the transformation dataset may include the 3D model and transformation information for each record, allowing a portion or region of one or more record images to be projected onto the 3D (transformed) model, and then the same region can be projected back onto a second (or subsequent) record acquired with a different imaging modality, thereby examining the same region. In some variations, this process may begin by collecting all records and / or automatically, manually, or semi-automatically aligning all records. For example, identification of individual regions such as teeth, palate, and gingiva may be used to cross-correlate between various imaging modalities and / or 3D models. In one example, records containing x-ray images may be correlated with a 3D volumetric model of the patient's teeth by resolving (manually or automatically) the position and / or orientation of the x-ray camera that acquired the x-ray images corresponding to that record. The volumetric model may be used to identify and / or confirm the position and orientation of the imaging source for each record. In some variations, the record includes explicit (e.g., recorded) information about the position and / or orientation and / or imaging parameters used to acquire the image; alternatively or additionally, this information may be derived. As described above, one pseudo X-ray image may be generated and compared to the actual X-ray image in the record.

[0188] Once a region corresponding to a region of a potentially treatable dental feature is identified in another record, the system or method may determine whether the same potentially treatable dental feature is present in the other record. If so, the score (e.g., a confidence score indicating the likelihood that the potentially treatable dental feature truly exists) may be adjusted, e.g., increased if the same or similar potentially treatable dental feature is present. Depending on the type of record and the type of potentially treatable dental feature, the absence of the potentially treatable dental feature may result in an adjustment of the confidence score. For example, the absence of surface features that are typically not detectable by x-ray (e.g., discoloration, plaque, gum recession, etc.) may not result in a decrease in the confidence score of the one or more potentially treatable dental features. The more often a potentially treatable dental feature is found in corresponding locations across different records (and thus across different modalities), the more likely the potentially treatable dental feature truly exists.

[0189] When comparing corresponding locations of one or more potentially treatable dental features, regions may be inspected manually, automatically, or semi-automatically, similar to the original identification techniques described above. For example, regions of a further record corresponding to the location of a potentially treatable dental feature in another record may be inspected automatically to identify features correlated with the type of potentially treatable dental feature. The system may be trained to recognize the further features and may provide a score representing the likelihood that a potentially treatable dental feature is present at this location. In some variations, a user (e.g., a technician, dental professional, etc.) may be presented with images from the further records and manually indicate the likelihood (yes / no, a gradient scale, a numeric scale, etc.) that a potentially treatable dental feature is present in one or more of the further records.

[0190] The final confidence value determined for each potentially treatable tooth feature may be used by the system, i.e., stored, transmitted, and / or displayed. For example, the potentially treatable tooth features may be presented to the dental practitioner in any suitable manner, such as as a list or on a display, such as on a marked 3D model of the dental arch (including the converted 3D tooth model). For example, the system may output a display highlight, such as color, shape, etc., at the location of any or all potentially treatable tooth features that have a confidence level above a threshold (i.e., are more likely to be "true"), and the display may further include one or more views (from one or more records) of those potentially treatable tooth features. A user may set or adjust the confidence level threshold, for example, on the fly, e.g., make the threshold stricter or looser, and accordingly display more or fewer potentially treatable tooth features.

[0191] FIG. 17 illustrates an example method 1700 for characterizing dental features across various imaging modalities as discussed above. In FIG. 17 , the method (or a system configured to implement the method) may identify one or more treatable dental features from one or more records (e.g., one or more images or sets of images of a patient's teeth acquired with various imaging modalities) (1701). For example, the one or more treatable dental features may be identified by an agent or engine configured to automatically detect the one or more treatable dental features. For example, a system implementing the method of FIG. 17 may include a treatable dental feature analysis engine or may include multiple treatable dental feature analysis engines, each configured to identify one or more types of treatable dental features or one or more types of imaging modalities. The engine (e.g., treatable dental feature analysis engine) may be part of a computer system. As used herein, an engine includes one or more processors, or portions thereof. A portion of one or more processors may include less than all of the hardware comprising any given processor(s), e.g., a subset of registers. This portion of a processor may be dedicated to one or more threads of a multithreaded processor, and during a time slice, the processor may be dedicated in whole or in part to performing a portion of the engine's functionality, etc. Thus, a first engine and a second engine may have one or more dedicated processors, or the first engine and the second engine may share one or more processors with each other or with other engines. Depending on implementation-specific or other considerations, an engine may be centralized or its functionality may be distributed. An engine may include software embodied in hardware, firmware, or a computer-readable medium and executed by a processor. The processor transforms data into other data using implemented data structures and methods, such as those described herein with reference to the figures.

[0192] The engines described herein, or engines used to implement the systems and devices described herein, may be cloud-based engines. As used herein, a cloud-based engine is an engine capable of executing applications and / or functionality using a cloud-based computer system. All or part of those applications and / or functionality may be distributed across multiple computing devices and need not be limited to just one computing device. In some embodiments, the cloud-based engine is capable of executing functionality and / or modules that end users access through a web browser or container application, and these functionality and / or modules are not installed locally on the end user's computing device.

[0193] 17, one or more treatable tooth features may be manually or semi-manually identified from one or more records. For example, a treatable tooth feature analysis engine may initially identify one or more treatable tooth features, which may then be verified or examined by a user (e.g., a dental technician).

[0194] Each identified treatable dental feature may then be flagged and / or recorded (1703) (e.g., in a collection of potentially treatable dental features). For example, the collection of potentially treatable dental features may be part of a data structure. Adding a potentially treatable dental feature to a collection (e.g., a data structure) may include recording the location of the treatable dental feature (e.g., on the originating record) and / or one or more of the type of treatable dental feature, the grade / degree of confidence of the treatable dental feature, etc. As used herein, a data structure (which may be included as part of a data store) is intended to include a repository having any applicable data organization, including a table, a comma-separated value (CSV) file, a conventional database (e.g., SQL), or other applicable known or convenient organizational format. A data store may be implemented, for example, as physical computer-readable media on a special-purpose machine, firmware, hardware, a combination thereof, or software embodied in an applicable known or convenient device or system. A data store-related component (e.g., a database interface) may be considered "part of" the data store, part of some other system component, or a combination thereof, although the physical location and other characteristics of a data store-related component are not important to understanding the techniques described herein.

[0195] A data structure may be associated with a particular way of storing and organizing data in a computer, thereby enabling efficient use in a given context. Data structures are generally based on a computer's ability to fetch and store data anywhere in the computer's memory, specified by an address, which is a string of bits that can itself be stored in memory and manipulated by a program. Thus, some data structures are based on arithmetically calculating the address of a data item, while others are based on storing the address of the data item in the structure itself. Many data structures use both principles, sometimes combining them in non-trivial ways. Implementing a data structure typically involves writing a set of procedures that instantiate and manipulate the structure. The data stores described herein may be cloud-based data stores. Cloud-based data stores are data stores that are compatible with cloud-based computing systems and engines.

[0196] The identified "probable" treatable tooth features (e.g., "potentially treatable tooth features") may be mapped to corresponding physical locations in one or more other records (1705). As mentioned above, in some variations, this may be done using a 3D volumetric model, which may be converted between a variety of different types of records (with different imaging modalities), including projecting a first record onto the 3D model and then back onto a second region.

[0197] Thus, the same corresponding regions in other regions can be reviewed to determine whether a potentially treatable tooth feature is present or suggested in further records. In some variations, the method may simply collect all of the various corresponding regions for storage, transmission, and / or presentation to a user (e.g., a dental professional), and may, for example, optionally stop here and allow the user to simultaneously review these flagged regions from different imaging modalities (records). For example, the potentially treatable tooth feature may be shown for all corresponding views in a side-by-side (e.g., tiled) or series of views.

[0198] Alternatively or additionally, the method and / or system may automatically or semi-automatically adjust the confidence score for each identified potentially treatable dental feature. Thus, the system may determine whether the further records indicate a higher or lower likelihood of the presence of the potentially treatable dental feature, and may adjust (or determine) the confidence score for each potentially treatable dental feature based on whether it appears in a corresponding position in the further records (1707).

[0199] The adjusted confidence levels may then be used to narrow down the potential treatable tooth features. For example, the method or system may then perform filtering and / or thresholding based on the adjusted confidence levels for each of the potential treatable tooth features (1709). In some variations, the threshold may be fixed (e.g., a confidence level greater than x, where x is a number halfway between zero confidence and one (absolute confidence), e.g., 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, etc.). In some variations, the threshold may be manually adjusted by a user and / or may be based on one or more characteristics of the records (e.g., a quality measure specific to each record, etc.).

[0200] Potentially treatable tooth features with a confidence level above a threshold may then be stored, presented, and / or transmitted 1711. For example, a final list and / or representation (e.g., using a 3D model) including the flagged potentially treatable tooth features may be presented to the user.

[0201] Any of the methods and apparatus (e.g., systems) described herein may be configured to build a data structure containing all or a portion of multiple records. For example, the data structure may include a 3D volumetric model and all or a portion of the associated 2D images used to build it, as described above. Additionally, the data structure may include additional records, such as images acquired with X-rays (e.g., panoramic X-rays) and / or CBCT. Metadata (e.g., information about the patient and / or images, such as texture information) may also be included, which may optionally include patient chart information from the patient's health / dental records. Alternatively or additionally, any identified potentially treatable tooth features (e.g., findings identified from the records) may also be included. The potentially treatable tooth features may be used to search / find / mark other records.

[0202] Typically, when images (e.g., 2D near-infrared images) are aggregated to build a 3D (e.g., volumetric) model, the informative 2D images may be marked to indicate their significance to the 3D model. For example, the 2D images may be marked to indicate low or high relevance.

[0203] As described above, when a collection of potentially treatable dental features includes a confidence level for the potentially treatable dental feature based on the presence of the potentially treatable dental feature in multiple records, this collection may be included as part of the same data structure that includes the 3D model, or may be separate. The 3D model containing the potentially treatable dental feature may be directly marked (flagged, coded, etc.). Thus, the data structure may be an aggregation of all of the various records. The combined / aggregated data structure may be referred to as the marked data structure or the treatable dental feature data structure.

[0204] Any record containing a near-infrared 2D image may be used / scanned to identify potentially treatable dental features. As described above, if a suspect area is identified in any record automatically, semi-automatically / semi-manually, or manually (e.g., by a user), the method or system may search for corresponding areas of the dental arch in all or some of the other records to conclude whether a finding has been made. In some variations, the method or apparatus may update the images in all or some of the records (and / or images in the combined data structure) based on the analysis described herein.

[0205] In any of the methods and systems described herein, tooth segmentation may be performed on all or some of the records and / or 3D models to improve performance and ease of use. Tooth segmentation may be added before volumetric modeling to support and improve volumetric results and model quality. For example, volumetric 3D models can use segmentation information to potentially enhance performance because additional surface 3D information is added. Segmentation information can also assist in the segmentation of enamel-dentin lesions to enhance automated detection and marking of suspicious regions (e.g., including but not limited to, when using automated agents that identify potentially treatable tooth features). Alternatively or additionally, tooth segmentation may be added post-volumetric modeling to support the segmentation of enamel-dentin lesions to enhance automated detection and marking of suspicious regions. For example, segmentation may also or alternatively help correlate structures across various imaging modalities. This includes aligning findings on the volumetric model with other modalities to enable cross-modality visualization. Tooth segmentation may be used to improve the recording and cross-modality visualization of clinical findings and annotations.

[0206] In any of the methods and devices described herein, the indicated confidence level may be a quantitative and / or qualitative index. For example, a quantitative confidence level "score" may be provided (e.g., using a number (e.g., between 0 and 100, 0 and 1.0, -100 and 100, or scaled to any numerical range)). Qualitative indices may include "high," "medium-high," "medium," "medium-low," "low," etc. Both qualitative and quantitative confidence levels may be used. The multiple-record-based confidence level rating system described herein may have significant implications for insurance claims and / or patient communications.

[0207] In any of the methods and systems described herein, the morphology of the dental arch may be used to assist in identifying possible areas of concern or potential problems. Thus, in general, a 3D model (e.g., a volumetric model) may be used and / or modified as described herein to include areas of potentially treatable dental features. The modified 3D model may act as a visual map of areas for risk assessment, which may be used, for example, to guide patient treatment, including promoting the use of sealants, orthodontic treatment, or night guards. In some variations, the modified 3D model may be used to guide a user when additional scans are needed (e.g., when there are few scans in risk areas). Herein, the modified 3D model may include a 3D model (e.g., a volumetric model and / or a surface model) marked to indicate the location and / or type and / or confidence level of potentially treatable dental features. Thus, in general, using additional data sources to guide the user in capturing potential regions of interest (e.g., when potential regions of interest appear in records other than near-infrared / near-infrared imaging scans) can help confirm the discovery of potentially treatable tooth features. As described above, results including a modified 3D model can help guide the user in scanning or rescanning (at a future point in time) the user's dentition. For example, historical scans may be taken as targeting maps during scanning (and to ensure sufficient coverage in those areas). Additionally or alternatively, one or more derived images / presentations may be used. For example, tooth segmentation may generate a tooth chart map (e.g., from the 3D volumetric model), which may be used for follow-up and automatic import into dental practice management software (DPMS). For example, individual records may be aligned to match specified problems to the tooth map.

[0208] Any of the methods (including user interfaces) described herein may be implemented as software, hardware, or firmware and may be described as a non-transitory computer-readable storage medium storing a set of instructions executable by a processor (e.g., a computer, a tablet, a smartphone, etc.) that, when executed by the processor, cause the processor to control the performance of any of the following steps, including but not limited to: displaying, communicating with a user, analyzing, modifying parameters (including timing, frequency, intensity, etc.), determining, alerting, etc.

[0209] As used herein, when a feature or element is referred to as being "on" another feature or element, the feature or element may be directly adjacent to the other feature or element, or there may be intervening features and / or elements. Conversely, when a feature or element is referred to as being "directly on" another feature or element, there are no intervening features and / or elements. It should also be understood that when a feature or element is referred to as being "connected," "attached," or "coupled" to another feature or element, the feature or element may be directly connected, attached, or coupled to the other feature or element, or there may be intervening features or elements. Conversely, when a feature or element is referred to as being "directly connected," "directly attached," or "directly coupled" to another feature or element, there are no intervening features or elements. The features and elements so described or illustrated may be described or illustrated with respect to one embodiment, but may also apply to other embodiments. Also, as will be understood by those skilled in the art, when a structure or feature is referred to as being "adjacent" to another feature, the reference may include portions that overlap or underlie the adjacent feature.

[0210] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. For example, as used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly contradicts otherwise. Furthermore, it should be understood that the words "comprises" and / or "comprising," when used herein, specify the presence of stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ."

[0211] Spatially relative terms such as "under," "below," "lower," "over," "upper," and the like may be used herein for ease of description when describing the relationship of one element or feature to another, as shown in the figures. It should be understood that these spatially relative terms are intended to encompass other orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures were inverted, an element described as being "under" or "beneath" another element or feature would then be oriented "over" that other element or feature. Thus, for example, the term "under" can encompass both an orientation of "over" and "under." The device may be otherwise oriented (rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein interpreted accordingly. Similarly, terms such as "upwardly," "downwardly," "vertical," and "horizontal" are used herein for descriptive purposes only, unless otherwise specified.

[0212] The terms "first" and "second" may be used herein to describe various features / elements (including steps), but these features / elements should not be limited by these terms unless the context indicates otherwise. These terms may be used to distinguish one feature / element from another. Thus, a first feature / element may be referred to as a second feature / element below, and similarly, a second feature / element may be referred to as a first feature / element below, without departing from the teachings of the present invention.

[0213] Throughout this specification and the claims that follow, unless otherwise stated, the word "comprise" and its variations, "comprises," "comprising," etc., mean that various components may be used in conjunction with one another in methods and articles (e.g., devices and method-containing compositions and apparatuses). For example, the word "comprising" should be understood to imply the inclusion of all stated elements or steps, but not the exclusion of any other elements or steps.

[0214] In general, any apparatus and methods described herein should be understood to be inclusive, although all or some of the components and / or steps may alternatively be exclusive, and may be expressed as "consisting of," or alternatively "consisting essentially of," various components, steps, subcomponents, or substeps.

[0215] As used in this specification and claims, including in the examples, and unless otherwise specified, any numerical value may be read as if preceded by the word "about" or "approximately," even if the word is not explicitly stated. The words "about" or "approximately," when describing a size and / or location, may be used to indicate that the stated value and / or location falls within a reasonable expected range of values ​​and / or locations. For example, a numerical value may be ±0.1% of the stated value (or range of values), ±1% of the stated value (or range of values), ±2% of the stated value (or range of values), ±5% of the stated value (or range of values), ±10% of the stated value (or range of values), or other such values. Any numerical value provided herein should be understood to encompass both approximate and approximate values ​​above and below that value, unless the context contradicts otherwise. For example, if the value "10" is disclosed, "about 10" is also disclosed. Any numerical range described herein is intended to encompass all subranges subsumed therein. It should also be understood that, as one of ordinary skill in the art would appreciate, when a value is disclosed, values ​​"less than" that value, values ​​"greater than" that value, and possible ranges therebetween are also disclosed. For example, when a value "X" is disclosed, values ​​"less than" X and values ​​"greater than or equal to X" (e.g., where X is a number) are also disclosed. It should also be understood that throughout this application, data is provided in several different formats, and that this data represents endpoints and starting points, with ranges covering any combination of these data points. For example, if a specific data point "10" and a specific data point "15" are disclosed, then values ​​between 10 and 15 are considered to be disclosed, as well as values ​​greater than, greater than, equal to, less than, less than, less than, and equal to 10 and 15. It is also understood that each number between the two specified numbers is also disclosed.For example, if 10 and 15 are disclosed, then 11, 12, 13, and 14 are also disclosed.

[0216] While various exemplary embodiments have been described above, any of several modifications may be made to the various embodiments without departing from the scope of the present invention, as set forth in the claims. For example, the order in which the various described method steps are performed may often be changed in alternative embodiments, and in some alternative embodiments, one or more method steps may be skipped altogether. Optional features of the various device and system embodiments may be included in some embodiments and not included in other embodiments. Accordingly, the foregoing description has been intended primarily for illustrative purposes and should not be construed as limiting the scope of the present invention, as set forth in the claims.

[0217] The examples and specific examples contained herein are illustrative, not limiting, of specific embodiments in which the present subject matter may be practiced. As noted, other embodiments may be utilized or derived, and structural or logical substitutions or changes may be made without departing from the scope of the present disclosure. Such embodiments of the present subject matter may be referred to individually herein or collectively in the language of "the present invention," which is merely for convenience and is not intended to intentionally limit the scope of the present application to any single invention or inventive concept, even if more than one is actually disclosed. Thus, while specific embodiments have been shown and described herein, any configuration designed to achieve the same purpose may be substituted for the specific embodiment shown. The present disclosure is intended to encompass all adaptations or variations of various embodiments. Combinations of the above-described embodiments, as well as other embodiments not specifically described herein, will become apparent to those skilled in the art upon reviewing the above description.

Claims

1. displaying a three-dimensional (3D) model of the patient's dental arch; displaying a view window over at least a portion of the 3D model of the patient's dental arch; receiving from a user a change in relative position between the view window and the 3D model of the patient's dental arch; identifying, from both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch acquired from various angles and positions relative to the patient's dental arch, a near-infrared image acquired at an angle and position that approximates a relative angle and position between the view window and the 3D model of the patient's dental arch; displaying the identified near-infrared images acquired at angles and positions that approximate the relative angle and relative position between the viewing window and the 3D model of the patient's dental arch; A method comprising:

2. 2. The method of claim 1, wherein the steps of identifying the near-infrared image and displaying the identified near-infrared image are performed continuously as the user changes the relative position between the view window and the 3D model of the patient's dental arch.

3. 2. The method of claim 1, wherein allowing the user to change the relative position between the viewing window and the 3D model of the patient's dental arch includes allowing the user to change one or more of an angle between a plane of the viewing window and the patient's dental arch and a portion of the dental arch proximate to the viewing window.

4. 2. The method of claim 1, wherein the identifying step includes determining a plurality of images that approximate a relative angle and relative position between the viewing window and the 3D model of the patient's dental arch, and averaging the plurality of images to form the identified near-infrared image.

5. 10. The method of claim 1, further comprising collecting in a processor a data set including both the 3D model of the patient's dental arch and the plurality of images of the patient's dental arch taken from various angles and positions relative to the patient's dental arch.

6. 2. The method of claim 1, wherein the step of displaying the view window over a portion of the 3D model of the patient's dental arch comprises displaying a loop through which the portion of the 3D model of the patient's dental arch can be viewed.

7. 2. The method of claim 1, wherein allowing the user to change the relative position between the view window and the 3D model of the patient's dental arch includes independently controlling one or more of an angle of the 3D model of the patient's dental arch, a rotation of the 3D model of the patient's dental arch, and the portion of the dental arch proximate to the view window.

8. 2. The method of claim 1, wherein allowing the user to change the relative position between the view window and the 3D model of the patient's dental arch comprises allowing the user to move the view window over the 3D model of the dental arch.

9. 2. The method of claim 1, further comprising: identifying a second image, which is one of a visible light image and a fluorescent image, from both the 3D model of the patient's dental arch and the plurality of images of the patient's dental arch acquired from various angles and positions relative to the patient's dental arch, that approximates the relative angle and relative position between the viewing window and the 3D model of the patient's dental arch; and displaying the second image simultaneously with the near-infrared image.

10. 2. The method of claim 1, wherein the step of displaying the identified near-infrared image that approximates the angle and position of the view window relative to the displayed 3D model comprises displaying the identified near-infrared image in a window adjacent to or partially overlaid on the display of the 3D model of the patient's dental arch.

11. displaying a three-dimensional (3D) model of the patient's dental arch; displaying a view window over a portion of the 3D model of the patient's dental arch; allowing a user to change a relative position between the viewing window and the 3D model of the patient's dental arch, the relative position including one or more of a relative angle between the patient's dental arch and a plane of the viewing window and a portion of the dental arch proximate to the viewing window; continuously as the user changes the relative position between the view window and the 3D model of the patient's dental arch; identifying, from both the 3D model of the patient's dental arch and a plurality of image pairs of the patient's dental arch, each pair of the plurality of pairs comprising a first imaging wavelength and a second imaging wavelength, each pair acquired at the same angle and position relative to the patient's dental arch, an image pair acquired at an angle and position that approximates a relative angle and position between the viewing window and the displayed 3D model of the patient's dental arch; displaying at least one image of the identified image pair acquired at an angle and position that approximates an angle and position of the viewing window relative to the displayed 3D model of the patient's dental arch; A method comprising:

12. The method of claim 11 , wherein the displaying step includes displaying the image pairs in a window adjacent to the 3D model of the patient's dental arch.

13. 12. The method of claim 11, wherein the identifying step includes determining a plurality of image pairs that approximate a relative angle and relative position between the viewing window and the 3D model of the patient's dental arch, and further comprising averaging the first imaging wavelength images and averaging the second imaging wavelength images of the plurality of image pairs to form the identified image pair.

14. 12. The method of claim 11, further comprising receiving at a processor a dataset comprising the 3D model of the patient's dental arch and the plurality of images of the patient's dental arch taken from various angles and positions relative to the patient's dental arch.

15. 12. The method of claim 11, wherein the step of displaying the view window over a portion of the 3D model of the patient's dental arch comprises displaying a loop through which the portion of the 3D model of the patient's dental arch can be viewed.

16. 12. The method of claim 11, wherein allowing the user to change the relative position between the view window and the 3D model of the patient's dental arch comprises independently controlling a position of the 3D model of the patient's dental arch relative to the view window and a position of the view window relative to the patient's dental arch.

17. 12. The method of claim 11, wherein allowing the user to change the relative position between the view window and the 3D model of the patient's dental arch comprises allowing the user to move the view window over the 3D model of the dental arch.

18. 12. The method of claim 11, wherein the first imaging wavelength comprises a near-infrared wavelength and the second imaging wavelength comprises one or more of a visible light wavelength, a fluorescent wavelength, and an X-ray wavelength.

19. 12. The method of claim 11, wherein displaying the identified image pairs comprises displaying the identified image pairs in windows adjacent to or overlapping the display of the 3D model of the patient's dental arch.

20. displaying a three-dimensional (3D) model of the patient's dental arch; displaying a view window over a portion of the 3D model of the patient's dental arch; allowing a user to change a relative position between the viewing window and the 3D model of the patient's dental arch, the relative position including one or more of an angle between the viewing window and the patient's dental arch and a portion of the dental arch proximate to the viewing window; continuously as the user changes the relative position between the view window and the 3D model of the patient's dental arch; identifying, from both the 3D model of the patient's dental arch and a plurality of near-infrared images of the patient's dental arch, each near-infrared image acquired from a different angle and position relative to the patient's dental arch, a near-infrared image acquired at an angle and position that approximates a relative angle and position between the viewing window and the 3D model of the patient's dental arch; displaying the identified near-infrared images acquired at angles and positions that approximate the angle and position of the viewing window relative to the displayed 3D model of the patient's dental arch; A method comprising:

21. a handheld wand having at least one image sensor and a light source configured to emit light in a spectral range within the near-infrared wavelength range; Display output, a user input device; one or more processors operatively connected to the handheld wand, the display output, and the user input device; displaying a three-dimensional (3D) model of the patient's dental arch on the display output; displaying a view window over a portion of the 3D model of the patient's dental arch on the display output; Varying the relative position between the view window and the 3D model of the patient's dental arch based on input from the user input device; identifying, from both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch acquired from various angles and positions relative to the patient's dental arch, a near-infrared image acquired at an angle and position that approximates a relative angle and position between the view window and the 3D model of the patient's dental arch; displaying the identified near-infrared images acquired at angles and positions that approximate the relative angle and relative position between the viewing window and the 3D model of the patient's dental arch; the one or more processors configured to implement Intraoral scanning system including:

22. 22. The intraoral scanning system of claim 21, wherein the one or more processors are configured to receive the plurality of images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch.

23. 22. The intraoral scanning system of claim 21, wherein the one or more processors are configured to continuously identify the near-infrared images and display the identified near-infrared images as the user changes a relative position between the view window and the 3D model of the patient's dental arch.

24. 22. The intraoral scanning system of claim 21, wherein the one or more processors are configured to vary a relative position between the viewing window and the 3D model of the patient's dental arch based on input from the user input device by varying one or more of an angle between a plane of the viewing window and the patient's dental arch and a portion of the dental arch proximate to the viewing window.

25. 22. The intraoral scanning system of claim 21, wherein the one or more processors are configured to identify the near-infrared image acquired at an angle and position that approximates a relative angle and position between the viewing window and the 3D model of the patient's dental arch by determining a plurality of images that approximate a relative angle and position between the viewing window and the 3D model of the patient's dental arch, and averaging the plurality of images to form the identified near-infrared image.

26. 22. The intraoral scanning system of claim 21, wherein the one or more processors are configured to display the view window over a portion of the 3D model of the patient's dental arch by displaying a loop through which the portion of the 3D model of the patient's dental arch can be viewed.

27. 22. The intraoral scanning system of claim 21, wherein the one or more processors are configured to vary the relative position between the viewing window and the 3D model of the patient's dental arch based on input from the user input device by varying one or more of: an angle of the 3D model of the patient's dental arch with respect to the viewing window; a rotation of the 3D model of the patient's dental arch with respect to the viewing window; and a portion of the dental arch proximate to the viewing window.

28. 22. The intraoral scanning system of claim 21, wherein the one or more processors are configured to vary a relative position between the viewing window and the 3D model of the patient's dental arch based on input from the user input device by varying a position of the viewing window over the 3D model of the dental arch.

29. 22. The intraoral scanning system of claim 21, wherein the one or more processors are configured to identify a second image, which is one or more of a visible light image and a fluorescent image, from both the 3D model of the patient's dental arch and the plurality of images of the patient's dental arch acquired from various angles and positions relative to the patient's dental arch, that approximates a relative angle and relative position between the viewing window and the 3D model of the patient's dental arch, and wherein the one or more processors are configured to display the second image simultaneously with the near-infrared image.

30. 1. A method of tracking a region of a patient's dental arch over time, comprising: acquiring a first three-dimensional (3D) volumetric model of the patient's dental arch, the 3D volumetric model including surface values ​​and near-infrared transparency values ​​of internal structures of the dental arch; identifying a region of the 3D volumetric model; flagging the identified region; acquiring a second 3D volumetric model of the patient's dental arch; displaying one or more images marking differences between the first and second 3D volumetric models in the flagged region; A method comprising:

31. 1. A method of tracking a region of a patient's dental arch over time, comprising: acquiring a first three-dimensional (3D) volumetric model of the patient's dental arch taken at a first time point, the 3D volumetric model including surface color values ​​and near-infrared transparency values ​​of the internal structures of the dental arch; using an automated process to identify a region in the 3D volumetric model to be flagged from a first record of a plurality of records, each record including a plurality of images of the patient's dental arch, each image acquired using an imaging modality, and further wherein each record of the plurality of records is acquired with a different imaging modality; flagging the identified region; correlating the flagged region with each record of the plurality of records by correlating the 3D volumetric model of the patient's dental arch with each record of the plurality of records; acquiring a second 3D volumetric model of the patient's dental arch taken at another time; displaying differences between the first and second 3D volumetric models in the flagged region; A method comprising:

32. 1. A method for tracking dental features across various imaging modalities, comprising: acquiring a first three-dimensional (3D) volumetric model of the patient's dental arch, the 3D volumetric model of the patient's dental arch including surface values ​​and internal structures of the dental arch; identifying a region of the patient's dental arch from a first record of a plurality of records, each record including a plurality of images of the patient's dental arch, each image being acquired using an imaging modality, and further wherein each record of the plurality of records is acquired with a different imaging modality; flagging the identified regions in corresponding regions of the 3D volumetric model of the patient's dental arch; correlating the flagged region with each record of the plurality of records by correlating the 3D volumetric model of the patient's dental arch with each record of the plurality of records; storing, displaying, and / or transmitting an image including the region of the patient's dental arch; A method comprising:

33. one or more processors; a memory coupled to the one or more processors, wherein when executed by the one or more processors, acquiring a first three-dimensional (3D) volumetric model of the patient's dental arch, the 3D volumetric model of the patient's dental arch including surface values ​​and internal structures of the dental arch; identifying a region of the patient's dental arch from a first record of a plurality of records, each record including a plurality of images of the patient's dental arch, each image being acquired using an imaging modality, and further wherein each record of the plurality of records is acquired with a different imaging modality; flagging the identified regions in corresponding regions of the 3D volumetric model of the patient's dental arch; correlating the flagged regions with each record of the plurality of records by correlating the 3D volumetric model of the patient's dental arch with each record of the plurality of records; storing, displaying, and / or transmitting an image including the region of the patient's dental arch; the memory configured to store computer program instructions embodying computer-implemented instructions including: A system including:

34. 1. A method for displaying images from a three-dimensional (3D) volumetric model of a patient's dental arch, comprising: acquiring the 3D volumetric model of the patient's dental arch, the 3D volumetric model including near-infrared transparency values ​​of internal structures of the dental arch; generating a two-dimensional (2D) view of the 3D volumetric model that includes the patient's dental arch and that includes near-infrared transparency of the internal structures; displaying the 2D view; A method comprising:

35. 1. A method for displaying images from a three-dimensional (3D) volumetric model of a patient's dental arch, comprising: acquiring the 3D volumetric model of the patient's dental arch, the 3D volumetric model including surface values ​​and near-infrared transparency values ​​of internal structures of the dental arch; generating a two-dimensional (2D) view including both surface values ​​and near-infrared transparency of the internal structures within the 3D volumetric model including the patient's dental arch; displaying the 2D view; A method comprising:

36. 1. A method for displaying images from a three-dimensional (3D) volumetric model of a patient's dental arch, comprising: acquiring the 3D volumetric model of the patient's dental arch, the 3D volumetric model including surface color values ​​and near-infrared transparency values ​​of internal structures of the dental arch; a user selecting an orientation for displaying a view of the 3D volumetric model; generating a two-dimensional (2D) view in the 3D volumetric model including the patient's dental arch using the selected orientation, the 2D view including a weighted portion of the surface color values ​​and a weighted portion of the near-infrared transparency of the internal structures; displaying the 2D view; A method comprising:

37. 1. A method for displaying pseudo-x-ray images from a three-dimensional (3D) volumetric model of a patient's dental arch, comprising: acquiring the 3D volumetric model of the patient's dental arch, the 3D volumetric model including near-infrared transparency values ​​of internal structures of the dental arch; generating a two-dimensional (2D) view, including near-infrared transparency of the internal structures, within the 3D volumetric model including the patient's dental arch; mapping near-infrared transparency of the internal structure in the 2D views to pseudo-x-ray density, wherein the pseudo-x-ray density values ​​in the 2D views are based on the near-infrared transparency values ​​with inverted values; displaying the mapped pseudo X-ray density; A method comprising:

38. 1. A method for displaying pseudo-x-ray images from a three-dimensional (3D) volumetric model of a patient's dental arch, comprising: acquiring the 3D volumetric model of the patient's dental arch, the 3D volumetric model including surface features and near-infrared transparency values ​​of the internal structure of the dental arch, where enamel is more transparent than dentin; generating a two-dimensional (2D) view, including near-infrared transparency of the internal structures, including dentin and enamel, within the 3D volumetric model including the patient's dental arch; mapping near-infrared transparency of the internal structures in the 2D views to pseudo-x-ray density, the near-infrared transparency values ​​being inverted so that the enamel is brighter than the dentin; displaying the mapped pseudo X-ray density; A method comprising:

39. identifying dental features in a first record comprising a plurality of images of the patient's dental arch acquired in a first imaging modality; correlating the first record with a model of the patient's dental arch; using the model of the patient's dental arch to identify regions of the dental arch that correspond to the tooth features in one or more different records, each record of the one or more different records being acquired with an imaging modality different from the first imaging modality, and each record of the one or more different records being correlated with the model of the patient's dental arch; determining a confidence score for the dental feature based on the identified regions corresponding to the dental feature in the one or more different records; displaying the tooth feature if the confidence score of the tooth feature is above a threshold; A dental diagnostic method, including:

40. identifying dental features in a first record comprising a plurality of images of the patient's dental arch acquired in a first imaging modality; correlating the first record with a three-dimensional (3D) volumetric model of the patient's dental arch; flagging the dental features on the 3D volumetric model; using the model of the patient's dental arch to identify regions of the dental arch that correspond to the tooth features in one or more different records, each record of the one or more different records being acquired with an imaging modality different from the first imaging modality, and each record of the one or more different records being correlated with the model of the patient's dental arch; determining or adjusting a confidence score for the dental feature based on the identified regions corresponding to the dental feature in the one or more different records; displaying the dental feature and an indication of the confidence score of the dental feature if the confidence score of the dental feature is above a threshold; A dental diagnostic method, including:

41. identifying one or more treatable dental features from one or more records of a plurality of records, each record including a plurality of images of the patient's dental arch, each image acquired using an imaging modality, and further wherein each record of the plurality of records is acquired with a different imaging modality; mapping the treatable tooth features to corresponding regions of the one or more records; recording the one or more treatable dental features in a record, the step of recording including recording a location of the treatable dental feature; adjusting or determining a confidence score for the one or more treatable dental features based on the corresponding region of the one or more records; displaying the one or more treatable dental features if the confidence score of the one or more treatable dental features is above a threshold; A dental diagnostic method, including:

42. one or more processors; a memory coupled to the one or more processors, wherein when executed by the one or more processors, identifying dental features in a first record comprising a plurality of images of the patient's dental arch acquired in a first imaging modality; correlating the first record with a model of the patient's dental arch; using the model of the patient's dental arch to identify regions of the dental arch that correspond to the tooth features in one or more different records, each record of the one or more different records being acquired with an imaging modality different from the first imaging modality, and each record of the one or more different records being correlated with the model of the patient's dental arch; determining a confidence score for the dental feature based on the identified regions corresponding to the dental feature in the one or more different records; displaying the tooth feature if the confidence score of the tooth feature is above a threshold; the memory configured to store computer program instructions embodying computer-implemented instructions including: A system including:

Images