Diagnostic intraoral scanning and tracking

Abstract

The present invention relates to diagnostic intraoral scanning and tracking. The present invention relates to methods and apparatus for acquiring, using, and displaying a three-dimensional (3D) volume model of a patient's dental arch. The 3D volume model may include surface information (e.g., color) as well as information about internal structures (e.g., near-infrared (Near-IR) transparency values ​​of internal structures including enamel and dentin).

Description

[0001] This application is a divisional application of the PCT national phase application with the application date of January 25, 2019, application number 201980010348.X, and invention name “Diagnostic Intraoral Scanning and Tracking”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This patent application claims priority to U.S. Provisional Patent Application Serial No. 62 / 622,798, filed on January 26, 2018, and entitled “DIAGNOSTIC INTRAORAL SCANNERS,” and U.S. Provisional Patent Application Serial No. 62 / 758,503, filed on November 9, 2018, and entitled “DIAGNOSTIC INTRAORAL SCANNERS,” the entire contents of which are incorporated herein by reference.

[0004] This patent application may be related to one or more of the following: U.S. Patent Application Serial No. 15 / 662,234, filed on July 27, 2017, entitled “INTRAORAL SCANNER WITH DENTAL DIAGNOSTICS CAPABILITIES,” which claims priority to U.S. Provisional Patent Applications Serial No. 62 / 367,607, filed on July 27, 2016, and Serial No. 62 / 477,387, filed on March 27, 2017; U.S. Patent Application Serial No. 15 / 662,234, filed on July 27, 2017, entitled “INTRAORAL SCANNER WITH DENTAL DIAGNOSTICS CAPABILITIES,” which claims priority to U.S. Provisional Patent Applications Serial No. 62 / 367,607, filed on July 27, 2016, and Serial No. 62 / 477,387, filed on March 27, 2017; U.S. Patent Application Serial No. 15 / 662,234, filed on July 27, 2017, entitled “METHODS AND APPARATUSES FOR FORMING A THREE-DIMENSIONAL VOLUMETRIC MODEL OF A SUBJECT'S TEETH (METHOD AND APPARATUS FOR FORMING A THREE-DIMENSIONAL VOLUME MODEL OF A SUBJECT'S TEETH)”) which claims priority to U.S. Provisional Patent Applications Serial No. 62 / 367,607 (filed on July 27, 2016) and Serial No. 62 / 477,387 (filed on March 27, 2017); and U.S. Patent Application Serial No. 15,672,248, filed on August 8, 2017 (entitled “METHODS FOR DENTAL DIAGNOSTICS”), which claims priority to U.S. Provisional Patent Applications Serial No. 62 / 367,607 (filed on July 27, 2016) and Serial No. 62 / 477,387 (filed on March 27, 2017). The entire contents of each of these applications are incorporated herein by reference.

[0005] Join by reference

[0006] All publications and patent applications mentioned in this specification are herein incorporated by reference in their entirety to the same extent as if each individual publication or patent application was specifically and individually indicated to be incorporated by reference. Technical Field

[0007] The present invention relates to intraoral scanning and tracking, and more particularly to a method for interactively displaying near-infrared images of a patient's dental arch and an intraoral scanning system for interactively displaying near-infrared images of a patient's dental arch. Background Art

[0008] Many dental and orthodontic procedures can benefit from an accurate three-dimensional (3D) description of the patient's dentition and oral cavity. In particular, it is helpful to provide a 3D description of the surface and internal structure of the teeth, including enamel and dentin, as well as caries and the general internal composition of the tooth volume. While pure surface representation of the 3D surface of a tooth has proven very useful in the design and manufacture of dental prostheses (e.g., crowns or bridges) and in treatment planning, the ability to image the internal structure, including caries and the development of cracks in the enamel and underlying dentin, would be extremely useful (especially when combined with surface topography).

[0009] Historically, ionizing radiation (e.g., X-rays) has been used to image the interior of teeth. For example, X-ray bitewing radiograms are commonly used to provide non-quantitative images of the interior of teeth. However, in addition to the risks of ionizing radiation, such images are often limited in their ability to display features and can involve lengthy and expensive filming procedures. Certain oral features (e.g., soft tissue, plaque, and soft tartar) may not be easily visualized via X-rays due to their low density. Other techniques, such as cone-beam computed tomography (CBCT), can provide tomographic images, but still require ionizing radiation.

[0010] Therefore, it would be beneficial to provide a method and apparatus (including devices and systems, such as intraoral scanning systems) that can be used to model a subject's teeth (including both the external surface and the internal structure and composition (of enamel and dentin)) using non-ionizing radiation. The model of the subject's teeth can be a 3D volume model or a panoramic image. In particular, it would be helpful to provide a method and apparatus that can provide this capability using a single device. Summary of the Invention

[0011] Methods and apparatus for acquiring, using, and displaying dental information, including information extracted from a three-dimensional (3D) volume model of a patient's dental arch, are described herein. The 3D volume model can include surface information (e.g., color) as well as information about internal structure (e.g., near-infrared (near-IR) transparency values ​​of internal structure including enamel and dentin). In some embodiments, the 3D volume scan can include or be derived from one or more other scan modalities, including, but not limited to, optical coherence tomography (OCT), ultrasound (US), magnetic resonance imaging (MRI), X-ray, and the like.

[0012] In particular, methods and user interfaces for displaying and manipulating 3D volume models (e.g., sectioning, labeling, selecting sub-regions, etc.) are described herein. For example, methods and apparatus for displaying images from 3D volume models are provided, including methods for generating cross-sections through 3D volume models, methods for displaying both surface and internal structures, and methods for generating easily interpretable images (e.g., pseudo-X-ray images) from 3D volume models.

[0013] Also described herein are methods and apparatus for marking and tracking regions of interest from a 3D volume model of a patient's dental arch. These methods may include automatically, manually, or semi-automatically (e.g., with user approval or input) identifying one or more regions within the 3D volume model for marking (including surface features and / or internal features of the dental arch); these regions may be areas where caries, cracks, or other irregularities have developed or may develop. The marked regions can be analyzed in more detail and tracked over time. In addition, the marked regions can modify the manner in which subsequent scans are performed, for example, by scanning the marked regions at a higher resolution. Regions of the volume model may correspond to one or more voxels, including continuous regions of voxels. These regions may be referred to herein as volumetric regions.

[0014] Also described herein are methods and apparatus for using 3D volumetric models to improve or modify dental procedures, including modifying treatment plans and / or modifying one or more dental devices. For example, dental tools are described herein that include 3D volumetric scanning or can operate in conjunction with 3D volumetric models (including robotic or automated control using 3D volumetric models). Also described are methods for diagnosing one or more conditions (e.g., dental conditions) using 3D volumetric models, particularly over time.

[0015] A method for displaying an image from a three-dimensional (3D) volume model of a patient's dental arch, the method comprising: collecting a 3D volume model of the patient's dental arch, wherein the 3D volume model includes surface color and shading values ​​and near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; a user selecting a direction of a view of the 3D volume model to be displayed; generating a two-dimensional (2D) view of the interior of the 3D volume using the selected direction, the two-dimensional (2D) view including the patient's dental arch, a weighted portion including surface color values ​​and a weighted portion of near-IR transparency of internal structures; and displaying the 2D view.

[0016] For example, a method for displaying an image from a three-dimensional (3D) volumetric model of a patient's dental arch is described herein. The method may include: receiving a 3D volumetric model of the patient's dental arch, wherein the 3D volumetric model includes surface color values ​​and near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; and generating a two-dimensional (2D) view from the 3D volumetric model, the two-dimensional (2D) view including the patient's dental arch and including both the surface color values ​​and the near-IR transparency of the internal structures. In any of the methods and devices described herein, the 3D model (including the volumetric 3D model) can be displayed as a voxel view. Thus, the methods described herein can generate one or more voxel views, wherein each voxel can have a color or hue corresponding to its density and / or be translucent. Thus, one of the methods and devices described herein can generate a 3D color map of all or some voxels of the 3D model (and display one or more 2D images derived from the 3D color map, e.g., a cross-section, slice, projection, perspective view, a transparent view in which all or part of the 3D model is rendered transparent, etc.). In some embodiments, marked regions (e.g., regions corresponding to one or more irregular regions, and / or regions where, for example, voxels change over time, regions / voxels that should be removed, regions / voxels suspected of being problematic, etc.) can be displayed as 3D and / or 2D views.

[0017] Generating a two-dimensional (2D) view from a 3D volume may include including in the 2D view a weighted portion of a surface color value and a weighted portion of a near-IR transparency of an internal structure. Note that the near-IR transparency may be based on or otherwise calculated from near-IR scattering or absorption of a material. The weighted portion of the surface color value may include a percentage of a full value of the surface color value, and the weighted portion of the near-IR transparency of the internal structure may include a percentage of a full value of the near-IR transparency of the internal structure, wherein the percentage of the full value of the surface color value and the percentage of the full value of the near-IR transparency of the internal structure total 100%.

[0018] In some variations, the method further includes adjusting, by a user or in response to user input, a weighted portion of the surface color value and / or the near-infrared transparency of the internal structure.

[0019] Any of these methods may include the step of scanning the patient's dental arch using an intraoral scanner.

[0020] Generating the 2D view may include slicing the 3D volume model in a plane passing through the 3D volume model.A user may select a section of the 3D volume model to be displayed and / or a direction of the 2D view.

[0021] For example, a method of displaying an image from a three-dimensional (3D) volume model of a patient's dental arch may include: receiving a 3D volume model of the patient's dental arch, wherein the 3D volume model includes surface color values ​​and near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; selecting, by a user or in response to user input, a section of the 3D volume model to be displayed; generating a two-dimensional (2D) view through the 3D volume using the selected section, the two-dimensional (2D) view including the patient's dental arch and possibly also including a weighted portion of the surface color values ​​and a weighted portion of the near-IR transparency of the internal structures; and displaying the 2D view.

[0022] A method for displaying an image from a three-dimensional (3D) volume model of a patient's dental arch may include: collecting a 3D volume model of the patient's dental arch, wherein the 3D volume model includes surface values ​​and near-infrared transparency values ​​of internal structures within the dental arch; generating a two-dimensional (2D) view of the interior of the 3D volume model, the two-dimensional (2D) view including the patient's dental arch, including both the surface values ​​and near-infrared transparency of the internal structures; and displaying the 2D view.

[0023] A method for tracking regions of a patient's dental arch over time may include: receiving a first three-dimensional (3D) volume model of the patient's dental arch, wherein the 3D volume model includes surface color values ​​and near-infrared transparency values ​​of internal structures within the dental arch; identifying regions to be marked in the 3D volume model; marking the identified regions; and displaying one or more images of the 3D volume model indicating the marked regions.

[0024] For example, a method for tracking a region of a patient's dental arch over time includes: collecting a first three-dimensional (3D) volume model of the patient's dental arch, wherein the 3D volume model includes surface values ​​and near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; identifying a region of the 3D volume model; marking the identified region; collecting a second 3D volume model of the patient's dental arch; and displaying one or more image markers on one or more images that are the differences between the first 3D volume model and the second 3D volume model at the marked region.

[0025] Identifying areas may include automatically identifying using a processor. For example, the automatic identification may include identifying areas with possible defects, including cracks and caries. Identifying areas with possible defects may include comparing near infrared transparency values ​​of areas within the 3D model to a threshold value. The automatic identification may include identifying surface color values ​​outside of a threshold range. The automatic identification may include segmenting the 3D volume model to identify enamel areas and identifying areas where the enamel thickness is below a threshold. Marking the identified areas may include automatically marking the identified areas. Marking the identified areas may include manually confirming the identified areas for marking.

[0026] Any of these methods can include receiving a second 3D volume model of the patient's dental arch and displaying the difference between the first 3D volume model and the second 3D volume model at the marked region.

[0027] Additionally, any of these methods may include pre-scanning or re-scanning the patient's dental arch, wherein the marked areas are scanned at a higher resolution or other scanning manner than unmarked areas.

[0028] For example, a method for tracking regions of a patient's dental arch over time may include: receiving a first three-dimensional (3D) volume model of the patient's dental arch, wherein the 3D volume model includes surface color values ​​and near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; identifying regions to be marked in the 3D volume model using an automated process; marking the identified regions; receiving a second 3D volume model of the patient's dental arch; and displaying differences between the first 3D volume model and the second 3D volume model at the marked regions. In some cases, the second 3D volume model of the patient's dental arch may be from a scan of the patient taken at a subsequent visit to the dentist's office.

[0029] Therefore, a method for tracking regions of a patient's dental arch over time may include: collecting a first three-dimensional (3D) volume model of the patient's dental arch acquired at a first time, wherein the 3D volume model includes surface color values ​​and near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; identifying regions to be marked in the 3D volume model using an automated process; marking the identified regions; collecting a second 3D volume model of the patient's dental arch acquired at a separate time; and displaying differences between the first 3D volume model and the second 3D volume model at the marked regions.

[0030] Also described herein is a method for displaying a pseudo-X-ray image of a three-dimensional (3D) volume model of a patient's dental arch. For example, a method may include: receiving a 3D volume model of a patient's dental arch, wherein the 3D volume model includes near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; generating a two-dimensional (2D) view through the 3D volume, the two-dimensional (2D) view including the patient's dental arch including near-IR transparency values ​​of the internal structures; mapping the near-IR transparency values ​​of the internal structures in the 2D view to pseudo-X-ray density values, wherein the near-IR transparency values ​​are converted; and displaying the mapped pseudo-X-ray density values. Generating the 2D view may include sectioning the 3D volume model in a plane through the 3D volume model. The 3D volume model may include surface information.

[0031] For example, a method for displaying a pseudo X-ray image from a three-dimensional (3D) volume model of a patient's dental arch may include: collecting a 3D volume model of the patient's dental arch, wherein the 3D volume model includes near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; generating a two-dimensional (2D) view of the interior of the 3D volume model, wherein the two-dimensional (2D) view includes the patient's dental arch, including near-IR transparency of the internal structures; mapping the near-IR transparency of the internal structures in the 2D view to pseudo X-ray density, wherein the pseudo X-ray density values ​​in the 2D view are based on the converted near-IR transparency values; and displaying the mapped pseudo X-ray density.

[0032] Any of these methods may include identifying a subregion from the 3D volume model before generating the 2D view, wherein the 2D view includes a 2D view of the identified subregion. The method may also include segmenting the 3D volume model into a plurality of teeth, wherein generating the 2D view may include a 2D view including only one of the identified teeth.

[0033] Mapping the near infrared transparency may include converting the near infrared transparency values ​​such that enamel within the 2D view is brighter than dentin within the 2D view.

[0034] A method for displaying a pseudo X-ray image from a three-dimensional (3D) volume model of a patient's dental arch may include: receiving a 3D volume model of the patient's dental arch, wherein the 3D volume model includes near-infrared (near-IR) transparency values ​​of surface features and internal structures within the dental arch, wherein enamel is more transparent than dentin; generating a two-dimensional (2D) view from the 3D volume, the two-dimensional (2D) view including the patient's dental arch, including near-IR transparency of internal structures (including dentin and enamel); mapping the near-IR transparency of the internal structures in the 2D view to pseudo X-ray density, wherein the near-IR transparency values ​​are converted so that enamel is brighter than dentin; and displaying the mapped pseudo X-ray density.

[0035] For example, a method for displaying a pseudo X-ray image from a three-dimensional (3D) volume model of a patient's dental arch may include: collecting a 3D volume model of the patient's dental arch, wherein the 3D volume model includes near-infrared (near-IR) transparency values ​​of surface features and internal structures within the dental arch, wherein enamel is more transparent than dentin; generating a two-dimensional (2D) view of the interior of the 3D volume, the two-dimensional (2D) view including the patient's dental arch, including near-IR transparency of internal structures (including dentin and enamel); mapping the near-IR transparency of the internal structures in the 2D view to pseudo X-ray density, wherein the near-IR transparency values ​​are converted so that enamel is brighter than dentin; and displaying the mapped pseudo X-ray density.

[0036] Also described herein are methods and apparatus for virtually inspecting (e.g., virtually sectioning, virtually scanning, virtually inspecting) a volumetric model of a patient's dental arch in real time. These apparatus may include a non-transitory, machine-readable tangible medium storing instructions for causing one or more machines to perform operations for performing any of the methods described herein. In particular, any of these methods and apparatus may operate on a dataset comprising both a 3D model of a patient's (single) dental arch and, in some variations, a 3D model of both dental arches. The 3D model may be, but is not limited to, a 3D volumetric model; in some variations, the 3D model is a 3D surface model of the dental arch. The dataset may also include multiple images of the dental arch taken from different positions relative to the dental arch (e.g., different angles between the image plane and the dental arch and different subregions of the dental arch). Some of these images may be taken from the occlusal plane, some from the gingival plane, and some from the lingual plane. In some variations, the images may be the same as (or a subset of) the images used to form the 3D model of the teeth. The dataset may include multiple images taken from the same or substantially the same area of ​​the dental arch and at angles relative to the dental arch. In some variations, the dataset may include a collection of two or more images (e.g., image pairs), each image taken from substantially the same area of ​​the dental arch and at the same angle relative to the dental arch, but using different imaging techniques (e.g., different imaging techniques such as visible light, IR / near infrared, fluorescence, X-ray, ultrasound, etc.).

[0037] For example, a method may include: displaying a three-dimensional (3D) model of a patient's dental arch; displaying a viewing window on at least a portion of the 3D model of the patient's dental arch; allowing a user to change a relative position between the viewing window and the 3D model of the patient's dental arch; and as the user changes the relative position between the viewing window and the 3D model of the patient's dental arch, continuously identifying, from the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch, images taken at angles and positions close to the relative angles and positions between the viewing window and the 3D model of the patient's dental arch; and displaying the identified images taken at angles and positions close to the angles and positions of the viewing window relative to the 3D model of the patient's dental arch.

[0038] In any of the methods described herein, a dataset may include a 3D model of a patient's dental arch and a plurality of images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch. The dataset may also or alternatively include metadata associated with each image (or set of each image) indicating the angle and / or region of the dental arch from which the image was taken. Additional metadata may be included (e.g., indicating the distance from the dental arch, indicating the exposure time, indicating that the image is an average of other images, a quality metric for the image, etc.).

[0039] For example, this document describes a method for displaying a 3D model (e.g., a surface 3D model of a patient's teeth and / or a volumetric model of the patient's teeth) that a user can virtually scan in more detail by moving a viewing window over the 3D model of the dental arch. For example, the method described herein includes: displaying a three-dimensional (3D) model of a patient's dental arch; displaying an observation window on a portion of the 3D model of the patient's dental arch; allowing a user to change the relative position between the observation window and the 3D model of the patient's dental arch, including changing one or more of: the angle between the plane of the observation window and the patient's dental arch and the portion of the dental arch adjacent to the observation window; as the user changes the relative position between the observation window and the 3D model of the patient's dental arch, continuously identifying images from the 3D model of the patient's dental arch and multiple images of the patient's dental arch (for example, in some embodiments, from a data set that includes both the 3D model of the patient's dental arch and multiple images of the patient's dental arch) that are taken at angles and positions that are close to the relative angle and position between the observation window and the 3D model of the patient's dental arch, wherein each image is taken from a different angle and position relative to the patient's dental arch; and displaying the identified images that are taken at angles and positions that are close to the angle and position of the observation window relative to the displayed 3D model of the patient's dental arch.

[0040] For example, a method may include: displaying a three-dimensional (3D) model of a patient's dental arch; displaying a viewing window on a portion of the 3D model of the patient's dental arch; allowing a user to change a relative position between the viewing window and the 3D model of the patient's dental arch, including changing one or more of: a relative angle between the plane of the patient's dental arch and the viewing window and a portion of the dental arch adjacent to the viewing window; as the user changes the relative position between the viewing window and the 3D model of the patient's dental arch, continuously identifying, from the 3D model of the patient's dental arch and multiple pairs of images of the patient's dental arch (for example, optionally, from a data set that includes both the 3D model of the patient's dental arch and multiple images of the patient's dental arch), image pairs taken at angles and positions approximate to the angle and position of the viewing window relative to the displayed 3D model of the patient's dental arch, wherein each of the multiple image pairs includes a first imaging wavelength and a second imaging wavelength, respectively, acquired at the same angle and position relative to the patient's dental arch; and displaying at least one of the identified image pairs taken at angles and positions approximate to the angle and position of the viewing window relative to the displayed 3D model of the patient's dental arch.

[0041] The methods and apparatus described herein can be used with a 3D model, which is a surface model or any representation of a patient's dental arch. It can be, but need not be, a 3D volumetric model of the patient's teeth, such as constructed from images (e.g., multiple images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch). The model can represent the patient's actual dentition, can be extracted from the patient's dentition, or can be generic.

[0042] As described herein, a method may include: displaying a three-dimensional (3D) model of a patient's dental arch; displaying an observation window on a portion of the 3D model of the patient's dental arch; allowing a user to change a relative position between the observation window and the 3D model of the patient's dental arch, including changing one or more of: an angle between the observation window and the patient's dental arch and a portion of the dental arch adjacent to the observation window; and as the user changes the relative position between the observation window and the 3D model of the patient's dental arch, continuously identifying from both the 3D model of the patient's dental arch and a plurality of near-infrared images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch (e.g., from a data set that includes both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch) the following images: a near-infrared image taken at an angle and position close to the relative angle and position between the observation window and the 3D model of the patient's dental arch; and displaying the identified near-infrared image taken at an angle and position close to the angle and position of the observation window relative to the displayed 3D model of the patient's dental arch.

[0043] In any of these examples, the image can be an image taken in a penetrating mode, such as near infrared. For example, the method described herein includes: displaying a three-dimensional (3D) model of a patient's dental arch; displaying a viewing window over a portion of the 3D model of the patient's dental arch; allowing a user to change a relative position between the viewing window and the 3D model of the patient's dental arch, including changing one or more of: an angle between the viewing window and the patient's dental arch and a portion of the dental arch adjacent to the viewing window; and as the user changes the relative position between the viewing window and the 3D model of the patient's dental arch, continuously identifying, from a data set including both the 3D model of the patient's dental arch and a plurality of near infrared images of the patient's dental arch, an image taken at an angle and position approximating the relative angle and position between the viewing window and the 3D model of the patient's dental arch, wherein each near infrared image is taken from a different angle and position relative to the patient's dental arch; and displaying the identified near infrared image taken at an angle and position approximating the angle and position of the viewing window relative to the displayed 3D model of the patient's dental arch.

[0044] Any of these methods may also include identifying and displaying multiple images taken at the same angle and position relative to the dental arch. For example, the images may be both visible light images and penetrating images (such as infrared / near infrared images, etc.). For example, described herein is: displaying a three-dimensional (3D) model of a patient's dental arch; displaying an observation window on a portion of the 3D model of the patient's dental arch; allowing a user to change the relative position between the observation window and the 3D model of the patient's dental arch, including changing one or more of: the relative angle between the planes of the patient's dental arch and the observation window and the portion of the dental arch adjacent to the observation window; as the user changes the relative position between the observation window and the 3D model of the patient's dental arch, continuously identifying the following images from both the 3D model of the patient's dental arch and multiple pairs of images of the patient's dental arch: an image pair taken at an angle and position close to the angle and position of the observation window relative to the displayed 3D model of the patient's dental arch, wherein each of the multiple image pairs includes a first imaging wavelength and a second imaging wavelength, respectively, acquired at the same angle and position relative to the patient's dental arch; and displaying the identified image pair taken at an angle and position close to the angle and position of the observation window relative to the displayed 3D model of the patient's dental arch.

[0045] In any of these methods, identifying can include determining a plurality of images that approximate the relative angle and position between the viewing window and the 3D model of the patient's dental arch; and averaging the plurality of images to form the identified image. For example, a dataset can include a plurality of images that are taken at approximately the same angle (e.g., within + / - 0.1%, 0.5%, 1%, 2%, 3%, 4%, 5%, 7%, 10%, 15%, 20%, etc.) and approximately the same area (e.g., within + / - 0.1%, 0.5%, 1%, 2%, 3%, 4%, 5%, 7%, 10%, 15%, 20%, etc.) of the dental arch; these similar images can be combined to form an average image that can be better than the individual images.

[0046] Generally, identifying one or more images taken at an angle and position that approximates the relative angle and position between the viewing window and the 3D model of the patient's dental arch may be within an acceptable spatial range. For example, images are taken at angles between + / - a few degrees (e.g., + / - 0.1 degrees, 0.2 degrees, 0.3 degrees, 0.4 degrees, 0.5 degrees, 0.6 degrees, 1 degree, 1.2 degrees, 1.5 degrees, 1.7 degrees, 1.8 degrees, 2 degrees, 2.2 degrees, 2.5 degrees, 3 degrees, 3.2 degrees, 3.5 degrees, 4 degrees, 5 degrees, etc.) of the same angle as the plane of the observation window and within a range of + / - 0.1 mm, 0.2 mm, 0.3 mm, 0.4 mm, 0.5 mm, 0.6 mm, 0.7 mm, 0.8 mm, 0.9 mm, 1 mm, 1.1 mm, 1.2 mm, 1.5 mm, 1.7 mm, 2.0 mm, 2.2 mm, 2.5 mm, etc.) of the distance of the dental arch area on which the observation window is positioned.

[0047] Any of these methods can include receiving a dataset in a processor. The dataset can be received directly from an intraoral scanner and / or stored and retrieved. In some variations, the dataset can be sent and received by the processor, and in some variations, the processor can read the dataset from a memory (e.g., a data store) connected to the processor.

[0048] Generally, any of these methods can include displaying a viewing window over a portion of a 3D model of a patient's dental arch. The viewing window can be any shape or size, such as circular, oval, triangular, rectangular, or other polygonal shape. For example, the viewing window can be annular through which a portion of the 3D model of the patient's dental arch can be viewed. The viewing angle can allow visualization of the dental arch through at least a portion of the viewing window. The viewing window can be smaller than the dental arch. In some variations, the user can make the viewing window larger or smaller.

[0049] Typically, these methods may include displaying via a user interface. For example, the user interface may display a 3D model of the dental arch, an observation window, and / or an image corresponding to a view through the observation window of the dental arch on a screen. A user may independently move the observation window and the dental arch (e.g., by manipulating the user interface, e.g., via a control such as a mouse, keyboard, touch screen, etc.). As the user moves the observation window and / or the dental arch relative to each other, the movement, the image determined to correspond to the image of the observation window through the area, and the angle of the observation window relative to the dental arch may be displayed in real time.

[0050] For example, allowing the user to change the relative position of the viewing window and the 3D model of the patient's dental arch may include controlling the angle and / or rotation of the 3D model of the patient's dental arch and the portion of the dental arch adjacent to the viewing window, respectively. In some variations, allowing the user to change the relative position of the viewing window and the 3D model of the patient's dental arch may include allowing the user to move the viewing window on the 3D model of the dental arch.

[0051] As mentioned, as the viewing window is moved over and / or around the dental arch (or as the dental arch is moved relative to the viewing window), any images identified as being taken from angles and positions corresponding to the angles and positions of the viewing window can be of any one or more modalities. Thus, for example, identifying an image proximate the relative angle and position between the viewing window and the 3D model of the patient's dental arch can include identifying one of: a visible light image, an infrared image, and a fluorescent image.

[0052] Displaying the identified image proximate to the angle and position of the viewing window relative to the displayed 3D model may include displaying the identified image in a window adjacent to or overlapping the display of the 3D model of the patient's dental arch. For example, the image may be displayed on a screen alongside the 3D model of the dental arch; as the user moves the dental arch and / or the imaging window, the image may be displayed in one or more windows that change in real time or near real time to reflect the relative position of the 3D model of the dental arch and the viewing window.

[0053] Also described herein is a non-transitory machine-readable tangible medium storing instructions that cause one or more machines to perform operations for performing any of the methods described herein, including virtually reviewing a patient's dental arch. For example, a non-transitory machine-readable tangible medium may store instructions that cause one or more machines to perform operations for virtually reviewing a patient's dental arch, including: displaying a three-dimensional (3D) model of the patient's dental arch; displaying a viewing window on a portion of the 3D model of the patient's dental arch; allowing a user to change the relative position between the viewing window and the 3D model of the patient's dental arch, including changing one or more of: the angle between the patient's dental arch and the plane of the viewing window and the portion of the dental arch adjacent to the viewing window; and as the user changes the relative position between the viewing window and the 3D model of the patient's dental arch, continuously identifying images taken at angles and positions approximating the relative angle and position between the viewing window and the 3D model of the patient's dental arch from a data set that includes both the 3D model of the patient's dental arch and multiple images of the patient's dental arch, wherein each image is taken from a different angle and position relative to the patient's dental arch; and displaying the identified images taken at angles and positions approximating the angle and position of the viewing window relative to the displayed 3D model of the patient's dental arch.

[0054] For example, a non-transitory machine-readable tangible medium storing instructions that cause one or more machines to perform operations for virtually reviewing a patient's dental arch, including: displaying a three-dimensional (3D) model of the patient's dental arch; displaying a viewing window on a portion of the 3D model of the patient's dental arch; allowing a user to change the relative position between the viewing window and the 3D model of the patient's dental arch, including changing one or more of: the angle between the patient's dental arch and the plane of the viewing window and the portion of the dental arch adjacent to the viewing window; and as the user changes the relative position between the viewing window and the 3D model of the patient's dental arch, continuously identifying images taken at angles and positions approximating the relative angle and position between the viewing window and the 3D model of the patient's dental arch from a data set that includes both the 3D model of the patient's dental arch and multiple images of the patient's dental arch, wherein each image is taken from a different angle and position relative to the patient's dental arch; and displaying the identified images taken at angles and positions approximating the angle and position of the viewing window relative to the displayed 3D model of the patient's dental arch.

[0055] Also described herein are intraoral scanning systems configured to perform the methods described herein. For example, the intraoral scanning system may include a handheld wand having at least one image sensor and a light source configured to emit light having a wavelength in the near infrared (near IR) range; a display output (e.g., a visual output such as a monitor, a screen, a virtual reality interface / augmented reality interface, etc.); a user input device (e.g., any control for receiving and sending user input, such as, but not limited to, a keyboard, buttons, a joystick, a touch screen, etc., where the display output and the user input device may be the same touch screen); and one or more processors operably connected to the handheld wand, the display, and the user input device, the one or more processors configured to: display a patient image on the display output; A three-dimensional (3D) model of a dental arch; displaying an observation window on a portion of the 3D model of a patient's dental arch on a display output; changing a relative position between the observation window and the 3D model of the patient's dental arch based on input from a user input device; identifying a near-infrared (near-IR) image taken at an angle and position approximating the relative angle and position between the observation window and the 3D model of the patient's dental arch from a data set including both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch; and displaying the identified near-IR image taken at an angle and position approximating the angle and position between the observation window and the 3D model of the patient's dental arch.

[0056] The one or more processors of the intraoral scanning system can be configured to receive multiple images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch. For example, the images can be taken by one or more image sensors on a handheld wand and sent to the one or more processors and / or stored in a memory accessed by the one or more processors. The system can also include a controller that coordinates the activities of the one or more processors, the wand, and the display output (and user input device). When the user operates the handheld wand to take images at different positions and / or angles relative to the patient's dental arch, the controller can display the images and / or a 3D model constructed from the images.

[0057] The one or more processors can be configured to continuously identify and display the near-infrared image as the user changes the relative position between the viewing window and the 3D model of the patient's dental arch. Thus, as the user (using user input) adjusts the position of the viewing window (e.g., annular) relative to the 3D model of the patient's dental arch on the display output (or equivalently adjusts the position of the 3D model of the dental arch on the display output relative to the viewing window), the one or more processors can determine and display a near-infrared image of the patient's teeth that best approximates the relative position between the viewing window and the 3D model of the patient's dental arch.

[0058] The near infrared image is one of the images captured by the handheld wand, or an average of the images captured by the handheld wand. Any device described herein (e.g., an intraoral scanning system) may also determine and / or store the position and / or orientation of the handheld wand as it operates, and this information may be stored along with the images captured from that position. For example, the handheld wand may include one or more accelerometers. For example, one or more processors may be configured to identify a near infrared image captured at an angle and position that approximates the relative angle and position between the observation window and the 3D model of the patient's dental arch by determining a plurality of images that approximate the relative angle and position between the observation window and the 3D model of the patient's dental arch, and averaging the plurality to form an identified near infrared image.

[0059] As mentioned, the one or more processors can be configured to change the relative position between the observation window on the display output and the 3D model of the patient's dental arch based on input from the user input device. Specifically, the one or more processors can be configured to change one or more of the following: the angle between the plane of the observation window and the patient's dental arch and the portion of the dental arch adjacent to the observation window (e.g., visible through the observation window in some variations) based on user input to the user input device. As described above, the observation window can be annular (e.g., circular, oval, square, etc.) through which the 3D model is visible. Therefore, the one or more processors can be configured to display the observation window on a portion of the 3D model of the patient's dental arch, including displaying it as an annular shape through which a portion of the 3D model of the patient's dental arch can be viewed. The viewing window may be moved or positioned (including changing which side of the dental arch (buccal, occlusal, lingual, or in between), including moving in x, y, z and / or rotating (e.g., pitch, roll, yaw)), the viewing window may be positioned and / or the 3D model of the patient's teeth may be moved (e.g., rotated in pitch, roll, yaw, moved in x, y, z). Thus, the one or more processors may be configured to change the relative position between the viewing window and the 3D model of the patient's dental arch based on input from a user input device by changing one or more of: an angle of the 3D model of the patient's dental arch relative to the viewing window (equivalent to an angle of the viewing window relative to the 3D model of the patient's dental arch), a rotation of the 3D model of the patient's dental arch relative to the viewing window (equivalent to a rotation of the viewing window relative to the 3D model of the patient's dental arch), and a portion of the dental arch adjacent to the viewing window (e.g., a portion of the 3D model visible through the viewing window). For example, the one or more processors may be configured to change the relative position between the viewing window and the 3D model of the patient's dental arch by changing the position of the viewing window on the 3D model of the dental arch based on input from the user input device.

[0060] The one or more processors can be configured to identify a second image from the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch, the second image approximating the relative angle and position between the observation window and the 3D model of the patient's dental arch, being one or more of: a visible light image and a fluorescent image; and wherein the one or more processors are configured to display the near infrared image and the second image simultaneously.

[0061] Also described herein are methods for automatically, semi-automatically / semi-manually, or manually identifying and grading features by coordinating between multiple imaging modalities. For example, a dental diagnostic method may include: identifying a dental feature in a first record comprising multiple images of a dental arch of a patient taken with a first imaging modality; associating the first record with a model of the patient's dental arch; using the model of the patient's dental arch, identifying an area of ​​the dental arch corresponding to the dental feature in one or more different records, wherein each of the one or more different records was taken with a different imaging modality than the first imaging modality, and wherein each of the one or more different records is associated with the model of the patient's dental arch; determining a confidence score for the dental feature based on the identified area corresponding to the dental feature in the one or more different records; and displaying the dental feature when the confidence score for the dental feature is above a threshold.

[0062] A dental diagnostic method may include: identifying a dental feature in a first record, the first record comprising multiple images of a patient's dental arch taken in a first imaging mode; associating the first record with a three-dimensional (3D) volume model of the patient's dental arch; marking the dental feature on the 3D volume model; using the model of the patient's dental arch, identifying an area of ​​the dental arch corresponding to the dental feature in one or more different records, wherein each of the one or more different records is taken in a different imaging mode different from the first imaging mode, and wherein each of the one or more different records is associated with the model of the patient's dental arch; determining or adjusting a confidence score for the dental feature based on the identified area corresponding to the dental feature in the one or more different records; and displaying the dental feature and an indicator of the confidence score for the dental feature when the confidence score of the dental feature is above a threshold.

[0063] In any of these methods (or systems for performing them), the tooth features may include one or more of: fissures, gum depressions, tartar, enamel thickness, pits, caries, pits, fissures, signs of grinding, and interproximal voids.

[0064] The display may include an indicator showing the dental feature and a confidence score for the dental feature.

[0065] Associating the first record with the model of the patient's dental arch can include associating the first record with a three-dimensional (3D) volumetric model of the patient's dental arch. Any of these methods (or systems performing the same) can include marking dental features on the model of the patient's dental arch, and / or collecting dental features, including a location of the dental features and one or more of: a type of the dental feature and a confidence score for the dental feature.

[0066] Determining the confidence score may include adjusting the confidence score for the dental feature based on the identified region corresponding to the dental feature in one or more different records.

[0067] In any of these methods or systems, identifying the dental features can include automatically identifying the dental features.

[0068] For example, a dental diagnostic method may include: identifying one or more actionable dental features from one or more records in a plurality of records, wherein each record includes multiple images of a patient's dental arch, each image being taken using an imaging mode, and wherein each record in the plurality of records is taken using a different imaging mode; mapping the actionable dental features to corresponding regions of the one or more records; recording the one or more actionable dental features, including recording a location of the actionable dental features; adjusting or determining a confidence score for the one or more actionable dental features based on the corresponding regions of the one or more records; and displaying the one or more actionable dental features when the confidence score for the one or more actionable dental features is above a threshold. As described above, the one or more actionable dental features include one or more of the following: fissures, gingival depressions, tartar, enamel thickness, pits, caries, pits, cracks, signs of grinding, and interproximal voids.

[0069] The display may include displaying one or more actionable dental features and an indicator of a confidence score for the dental feature. Mapping the actionable dental features to corresponding regions of the one or more records may include associating the first record with a three-dimensional (3D) volume model of the patient's dental arch. Recording the one or more actionable dental features may include marking the dental features on the 3D volume model of the patient's dental arch. Identifying the dental features may include automatically identifying the dental features.

[0070] Also described herein are systems for performing any of the methods described herein. For example, a system may include: one or more processors; and a memory coupled to the one or more processors, the memory being configured to store computer program instructions that, when executed by the one or more processors, perform a computer-implemented method, the method comprising: identifying a dental feature in a first record, the first record comprising a plurality of images of a dental arch of a patient taken in a first imaging mode; associating the first record with a model of the patient's dental arch; using the model of the patient's dental arch, identifying an area of ​​the dental arch corresponding to the dental feature in one or more different records, wherein each of the one or more different records is taken in a different imaging mode than the first imaging mode, and wherein each of the one or more different records is associated with the model of the patient's dental arch; determining a confidence score for the dental feature based on the identified area corresponding to the dental feature in the one or more different records; and displaying the dental feature when the confidence score for the dental feature is above a threshold.

[0071] Also described herein are methods and apparatus (e.g., systems) for tracking one or more regions (e.g., labeled or marked regions) across different imaging modalities and / or over time. For example, a method for tracking dental features across different imaging modalities may include: collecting a first three-dimensional (3D) volume model of a patient's dental arch, wherein the 3D volume model of the patient's dental arch includes surface values ​​and internal structures within the dental arch; identifying a region of the patient's dental arch from a first record in a plurality of records, wherein each record includes a plurality of images of the patient's dental arch, each image captured using an imaging modality, and further wherein each record in the plurality of records is captured using a different imaging modality; marking the identified region in a corresponding region of the 3D volume model of the patient's dental arch; associating the marked region with each record in the plurality of records by associating the 3D volume model of the patient's dental arch with each record in the plurality of records; and saving, displaying, and / or transmitting an image including the region of the patient's dental arch. The region of the patient's dental arch may include dental features including one or more of: fissures, gingival depressions, tartar, enamel thickness, pits, caries, pits, cracks, signs of grinding, and interdental spaces.

[0072] Saving, displaying and / or sending may include displaying the patient's dental arch region.Any of these methods may include marking dental features on the 3D volume model.Identifying the patient's dental arch region may include automatically identifying the patient's dental arch region.

[0073] A system for tracking one or more regions (e.g., tagged or marked regions) across different imaging modalities and / or over time may include: one or more processors; a memory coupled to the one or more processors, the memory configured to store computer program instructions that, when executed by the one or more processors, perform a computer-implemented method comprising: collecting a first three-dimensional (3D) volume model of a patient's dental arch, wherein the 3D volume model of the patient's dental arch includes surface values ​​and internal structures within the dental arch; identifying a region of the patient's dental arch from a first record of a plurality of records, wherein each record includes multiple images of the patient's dental arch, each image being taken using an imaging modality, and further, wherein each record of the plurality of records is taken using a different imaging modality; marking the identified region in a corresponding region of the 3D volume model of the patient's dental arch; associating the marked region with each of the plurality of records by associating the 3D volume model of the patient's dental arch with each of the plurality of records; and saving, displaying, and / or transmitting the image including the region of the patient's dental arch. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The novel features of the invention are particularly set forth in the appended claims.A better understanding of the features and advantages of the invention may be obtained by referring to the accompanying drawings and the following detailed description which sets forth illustrative embodiments in which the principles of the invention are utilized.

[0075] Figure 1A An example of a 3D (color) intraoral scanner is shown that can be used as described herein to generate a model of a subject's teeth having surface and internal features.

[0076] Figure 1B An example of an intraoral scanner configured to generate a model of a subject's teeth having both surface and internal features is schematically shown.

[0077] Figure 2 A schematic diagram of an intraoral scanner for both surface scanning (e.g., visible light, non-penetrating) and penetrating scanning using near-infrared (IR) wavelengths is shown. The scanner includes polarizers and filters for blocking near-IR light reflected from tooth surfaces while still collecting near-IR light reflected from internal structures.

[0078] Figure 3 is an example of a method of scanning teeth using an intraoral scanner to identify internal structures using penetrating wavelengths (e.g., infrared and / or near infrared).

[0079] Figure 4 A method of generating internal structure (or pseudo X-ray) images from volumetric data is shown.

[0080] Figure 5A and 5B Virtual cross-sections from a volume model of a tooth are shown. These virtual cross-sections can be annotated, colored / pseudo-colored, or textured to show the internal attributes or properties of the tooth. Figure 5A In , the virtual cross-section is pseudo-colored to show the enamel; in Figure 5B In the figure, the virtual cross-sections are pseudo-colored to show the dentin.

[0081] Figure 6 A method of marking (eg, annotating) a volumetric model of a patient's teeth and / or using marked regions is shown.

[0082] Figure 7 A comparison between typical computer-aided design / computer-aided manufacturing (CAD / CAM) methods used in dentistry and the method described in this paper to achieve 3D volume scanning and modeling.

[0083] Figure 8A is an example of a display using the 3D volumetric model described herein to track gum recession over time. Figure 8B Shows a later time Figure 8A Magnified view of area B in FIG.

[0084] Figures 9A-9G A method of displaying volumetric information from a patient's teeth is shown. Figure 9A An example of a 3D volume model of a patient's upper jaw (showing teeth and gums) is shown in top view. Figure 9B The same 3D volume model is shown, showing internal features, including more transparent enamel and less transparent dentin. The 3D volume model can be manipulated to show more or less of the surface and / or internal structure. Figures 9C-9G Gradually showed Figure 9A A more transparent view or area ("C") of the 3D volume model. Figure 9C A 2D image extracted from a region of the 3D volume model is shown, showing only the outer surface of the tooth (eg, 100% color / outer surface image, 0% near IR / inner volume). Figure 9D Shown with Figure 9C For the same area, an exterior surface (color) image and an interior (NIR-based) image were combined (e.g., 75% color / exterior surface image, 25% NIR / interior volume). Figure 9E Shown with Figure 9C For the same area, an exterior surface (color) image and an interior (NIR-based) image were combined (e.g., 50% color / exterior surface image, 50% NIR / interior volume). Figure 9F Shown with Figure 9CFor the same area, an exterior surface (color) image and an interior (NIR-based) image were combined (e.g., 25% color / exterior surface image, 75% NIR / interior volume). Figure 9G Shown with Figure 9C The same area, only the internal image of the tooth (near IR based) is shown (eg, 0% color / external surface image, 1000% near IR / internal volume).

[0085] Figure 10A An example of a user interface for analyzing and / or displaying a 3D volumetric model of a patient's teeth is shown, showing a top view of the upper dental arch, tools that can be used to manipulate the view, and a magnified view showing the exterior surface of a magnified area of ​​the tooth (on the left) and a magnified view showing internal features of the tooth (showing dentin and enamel within the tooth).

[0086] Figure 10B Shown Figure 10A A user interface of the present invention is provided, wherein regions of the tooth are marked / labeled as described herein.

[0087] Figures 11A-11C Another example of a method of displaying 3D volumetric image information by mixing the 3D volumetric image information with surface (non-penetrating) information is shown. Figure 11A Shown is a visible light image of the patient's dental arch area taken with a scanner that was also used to perform penetration (near infrared) scans. Figure 11B A volume model of a reconstructed 3D volume model of a patient's tooth is shown, showing the internal dentin and enamel. Features not visible in the surface scan are evident in the volume scan, including caries and blistering areas within the enamel. Figure 11C A hybrid image combining a 3D volume image with a surface scan is shown, showing both surface and internal structure, including caries and blister areas.

[0088] Figure 12 is an example of a method for allowing a user to virtually scan a patient's dental arch. The method can be performed in real time or near real time.

[0089] Figure 13 is a schematic illustration of a data structure comprising a 3D model of a patient's dental arch and associated 2D images of the dental arch taken (eg, by an intraoral scanner) at multiple locations around the dental arch.

[0090] Figure 14AThe present invention is an example of a user interface that allows a user to virtually scan a 3D model of a dental arch. As the user scans the 3D dental arch, the user interface details the corresponding optical and near-infrared (e.g., external and internal) regions. The user can use one or more tools to move (e.g., rotate, translate, etc.) the dental arch and / or viewing window. As the viewing window and dental arch position changes, the corresponding optical and near-infrared images can be continuously or nearly continuously updated. Paired imaging windows are shown adjacent to the view of the 3D model of the dental arch.

[0091] Figure 14B is an alternative display that shows a single large image window on or near the 3D image of the dental arch. Figure 14B , the image window shows a light image of the corresponding area of ​​the dental arch.

[0092] Figure 14C is an alternative display that shows a single large image window on or near the 3D image of the dental arch. Figure 14C , the image window shows a near-infrared image of the corresponding area of ​​the dental arch.

[0093] and Figure 14A resemblance, Figure 15A An example of a 3D model of the outer surface of a dental arch and an observation window relative to the dental arch are shown. An image pair display window is adjacent to the 3D model of the dental arch. The user can move the observation window over the dental arch (and / or the dental arch relative to the observation window), thereby changing the images displayed in both display windows. The upper display window shows a near-infrared image of the dental arch at a position and angle corresponding to the plane of the observation window; the lower display window shows a corresponding optical image (possibly in color).

[0094] Figure 15B Shown Figure 15A Another image of the dental arch shown in , where the arch is rotated lingually relative to the viewing window; the corresponding near-infrared image (upper right) and visible light (lower right) adjacent to the 3D model of the dental arch have been updated to show the slightly rotated view, allowing the user to virtually scan the dental arch and display external and internal views in real time (or near real time).

[0095] Figure 16A Another example of a method of showing a 3D model of a dental arch (in this example, the lower dental arch, for example by selecting the lower dental arch display control in the upper left corner of the user interface) and showing focused views of near infrared and visible light images corresponding to an observation window area that can be moved over and around (lingual-occlusal-buccal) the patient's dental arch model.

[0096] Figure 16B Shows something like Figure 16AExample of a single window (magnified near-infrared view of the tooth within the area corresponding to the viewing window ring).

[0097] Figure 16C Shows something like Figure 16A Example of a single window (magnified visible light view of the tooth within the area corresponding to the viewing window ring).

[0098] Figure 17 One example of a method for automatically or semi-automatically identifying, confirming, and / or characterizing one or more actionable dental features that may benefit from detection and / or treatment is schematically illustrated. DETAILED DESCRIPTION

[0099] Described herein are methods and apparatus (e.g., devices and systems) for applying scans of both the external and / or internal structures of teeth. These methods and apparatus can generate and / or manipulate a model of a subject's oral cavity (e.g., teeth, jaws, palate, gums, etc.), which can include surface topography and internal features (e.g., dentin, dental filling material (including base and lining), cracks and / or caries). An apparatus for performing both surface and penetration scanning of a tooth can include an intraoral scanner for scanning within or around the subject's oral cavity, and the intraoral scanner is equipped with one or more light sources that can illuminate in two or more spectral ranges: a surface feature illumination spectral range (e.g., visible light) and a penetration spectral range (e.g., IR range, particularly "near infrared," including but not limited to 850 nm). The scanning apparatus can also include: one or more sensors for detecting the emitted light; and one or more processors for controlling the operation of the scan and analyzing the light received from both the first spectral range and the second spectral range to generate a model of the subject's teeth, including features on the tooth surface and within the tooth (including features within the enamel (and / or enamel-like restoration) and dentin). The generated model can be a 3D volume model or a panoramic image.

[0100] As used herein, a volume model may comprise a three-dimensional virtual representation of an object in which internal regions (structures, etc.) are arranged within a physical three-dimensional volume in proportion and relative to other internal and surface features of the object being modeled. For example, a volume representation of a tooth may comprise an outer surface and the internal structure of the tooth (below the tooth surface) arranged in proportion to the tooth, such that a cross-section through the volume model will substantially correspond to a cross-section through the tooth, showing the location and size of the internal structure; the volume model may be a cross-section in any (e.g., arbitrary) direction and correspond to an equivalent cross-section through the object being modeled. The volume model may be electronic or physical. A physical volume model may be formed, for example, by 3D printing, etc. The volume models described herein may extend into the volume completely (e.g., through the entire volume, e.g., the volume of the tooth) or partially (e.g., to a volume modeled for a certain minimum depth (e.g., 2 mm, 3 mm, 4 mm, 5 mm, 6 mm, 7 mm, 8 mm, 9 mm, 10 mm, 12 mm, etc.)).

[0101] The methods described herein generally include methods for generating a model of a tooth of a subject, the methods generally generating a 3D model or perspective view of the tooth that includes both surface and internal features. Non-ionizing methods for imaging and / or detecting internal structures can be used, such as capturing images using a transmission wavelength to observe structures within the tooth by illuminating the tooth using one or more transmission spectral ranges (wavelengths), including using transillumination (e.g., illuminating from one side and capturing light from the other side after passing through the subject), and / or low-angle transmission imaging (e.g., reflectance imaging, capturing light reflected / scattered from internal structures when illuminating with a transmission wavelength). In particular, multiple transmission images can be captured from the same relative position. Although conventional transmission imaging techniques (e.g., transillumination) can be used, where the angle between the light emitter illumination direction and the detector (e.g., camera) viewing angle is 90 degrees or 180 degrees, methods and apparatus for smaller angles (e.g., between 0 and 25 degrees, between 0 and 20 degrees, between 0 and 15 degrees, between 0 and 10 degrees, etc.) are also described herein. Smaller angles (e.g., 0-15°) can be particularly advantageous because the illumination (light source) and sensing (detector (e.g., camera, etc.)) can be closer to each other and can provide a scanning wand of the intraoral scanner that can be more easily positioned and moved around the subject's teeth. These small-angle penetrating images and imaging techniques may also be referred to herein as reflected illumination and / or imaging, or as reflected / scattered imaging. Generally, unless otherwise specified, penetrating imaging may refer to any appropriate type of penetrating imaging, including transmitted illumination, small-angle penetrating imaging, etc. However, small angles may also result in direct reflections from the surface of the subject (e.g., a tooth), which may obscure internal structures.

[0102] The methods and apparatus described herein are particularly effective in combining a 3D surface model of a tooth with imaged internal features, such as lesions (caries, cracks, etc.), which can be detected using penetration imaging using an intraoral scanner adapted to independently but simultaneously (or nearly simultaneously) detect both surface and internal features. Surface scanning and penetration imaging can be combined by alternating or switching between these different modes in a manner that allows the different modes to use the same coordinate system. Alternatively, surface scanning and penetration scanning can be viewed simultaneously, for example, by selectively filtering the wavelength of imaging to separate IR (near IR) light from visible light. Thus, the 3D surface data can provide important reference and angular information for internal structures, and can allow interpretation and analysis of penetration images that might otherwise be difficult or impossible to interpret.

[0103] For example, one can Figures 1A-1B The intraoral scanners and methods of using such scanners are shown to collect penetration scans as described herein, which are used to generate a three-dimensional (3D) model of an intraoral region (e.g., teeth, gums, jaws, etc.) of a subject's mouth, which may include internal features of the teeth as well as a surface model. Although surface scans (including color scans) may be helpful and useful in many cases, in certain variations described herein, penetration (IR) scans may be sufficient.

[0104] exist Figure 1A In the example intraoral scanner 101 can be configured or adapted to generate a 3D model with both surface and internal features or only internal (through-the-air) scans. Figure 1B As schematically shown in FIG, an exemplary intraoral scanner may include a handle or wand 103 that may be held by an operator (e.g., a dentist, dental hygienist, technician, etc.) and moved over a subject's teeth to scan both the surface and internal structures. The wand may include one or more sensors 105 (e.g., cameras such as CMOS, CCD, detectors, etc.) and one or more light sources 109, 110, 111. Figure 1B In FIG, three light sources are shown: a first light source 109 configured to emit light in a first spectral range (e.g., visible light, monochromatic visible light, etc.; the light need not be visible light) for detecting surface features; a second color light source (e.g., white light between 400-700 nm, such as approximately 400-600 nm); and a third light source 111 configured to emit light in a second spectral range for detecting internal features within the tooth (e.g., by transillumination, small angle penetration imaging, laser fluorescence, etc., generally referred to as penetration imaging, such as in the near infrared). Although in Figure 1BA separate illumination source is shown in FIG, but in some variations, an alternative light source may be used. The light source may be any suitable light source, including LEDs, fiber optics, etc. The wand 103 may include one or more controls (buttons, switches, dials, touch screens, etc.) for auxiliary control (e.g., turning the wand on / off, etc.); alternatively or additionally, one or more controls not shown may be present on other portions of the intraoral scanner (e.g., a foot pedal, keyboard, console, touch screen, etc.).

[0105] In general, any suitable light source can be used, particularly one that matches the pattern being detected. For example, any of these devices can include a visible light source or other (including invisible) light source for surface detection (e.g., a light source at or around 680 nm, or other suitable wavelength). A color light source, typically a visible light source (e.g., a "white light" light source), for color imaging can also be included. Furthermore, a penetrating light source (e.g., an infrared light source, particularly a near-infrared light source) for penetrating imaging can also be included.

[0106] The intraoral scanner 101 may also include one or more processors, including linked processors or remote processors, for controlling the operation of the wand 103, including coordinating scans, reviewing and processing scans, and generating 3D models including surface and internal features. Figure 1B As shown, one or more processors 113 may include or be coupled to a memory 115 to store scanned data (surface data, internal feature data, etc.). Communication circuitry 117 (including wireless or wired communication circuitry) may also be included to communicate with components of the system (including a wand) or external components including an external processor. For example, the system can be configured to send and receive scans or 3D models. One or more additional outputs 119 for outputting or presenting information may also be included, including a display screen, a printer, etc. As described above, an input 121 (buttons, a touch screen, etc.) may be included, and the device may allow or request user input for controlling scanning and other operations.

[0107] Figure 2 An example of how a scanner can be used is shown. The scanner shown can be used as a system (e.g., Figures 1A-1B For example, Figure 2 A schematic diagram of an intraoral scanner configured for surface scanning (e.g., visible light, non-penetrating) and penetrating scanning using near infrared (NIR) wavelengths (850 nm wavelength in this example) is shown. Figure 2In the present invention, the scanner includes a near-infrared illumination light 289 and a first polarizer 281 and a second polarizer 283 located in front of the image sensor 285 to block the near-infrared light (P-polarized light) reflected from the surface of the tooth 290 while still collecting the near-infrared light (S-polarized light) scattered from the internal tooth structure / region. The NIR light illuminates the tooth with P polarization, and the specular light reflected from the tooth surface (e.g., enamel) is reflected by specular reflection, so its P polarization state is preserved. The near-infrared light that penetrates the internal tooth features (e.g., dentin) is scattered, resulting in random polarization (S and P). The wavelength selective quarter-wave plate 293 does not modify the polarization of the near-infrared light (e.g., it leaves the polarization state of the transmitted near-infrared light unchanged), but instead changes the polarization of the returned scanned light from P to S, so that only surface reflections are captured in the scanned wavelength. First, the mixed S-polarized and P-polarized returned near-infrared light is filtered by the polarization beam splitter (PBS) 294 and the polarization filter 283 so that only S-polarized light is transmitted to the image sensor. Therefore, only the near-infrared S-polarized light from the internal structure of the tooth is captured by the image sensor, while the specular light with the original P-polarization is blocked. Other intraoral scanner configurations with or without polarization filters (e.g. Figure 2 those shown) can be used as part of the probe.

[0108] exist Figure 2 Surface scanning can be performed by illuminating the surface with p-polarized light (using scanner illuminating unit 297), and the polarization can be inverted by wavelength selective quarter wave plate 293 (transmitting S-polarized light to the image sensor).

[0109] A variety of penetration scanning techniques (penetration imaging) can be used or incorporated into the devices described herein for scanning to detect internal structures using a penetration wavelength or a penetration wavelength within a spectral range, penetration scanning techniques including but not limited to transillumination and small angle penetration imaging, both of which detect the travel of the penetration wavelength of light from or through tissue (e.g., from or through teeth). Thus, these devices and techniques can be used to scan internal parts of the mouth such as teeth or one or more teeth, gums, palate, etc., and to generate models of the scanned area. These models can be generated in real time or after the scan. These models can be referred to as 3D volume models of the teeth, but can include other areas of the jaw, including the palate, gums, and teeth. Although the methods and devices described herein generally relate to 3D volume models, the techniques and methods described herein can also be used in conjunction with 3D surface models in some cases. The surface model information is typically part of the 3D volume model.

[0110] Figure 3 An example of data flow for scanning a tooth using an intraoral scanner to create a 3D model including the internal structure is shown. Figure 3In the example method shown, there are three parts. First, the teeth can be scanned using an intraoral scanner 1701 (or any other scanner) that is configured to provide a penetrating scan into the interior of the tooth using an optical wavelength (e.g., IR, near-IR, etc.) or wavelength range. As described above, any of these scanners can also scan simultaneously to determine surface features (e.g., via one or more non-penetrating wavelengths), color, etc. During the scan, multiple penetrating scans 1703, 1703' can be performed, and the position 1705, 1705' (e.g., x, y, z position and / or pitch, roll, yaw angles) of the sensor (e.g., camera) can be determined and / or recorded for each penetrating image. In some variations, as described above, the surface of the tooth can also be imaged simultaneously and a 3D tooth surface model 1707 determined. In this example, for example, the patient's teeth can be scanned using an intraoral 3D scanner 1702 that is capable of imaging internal tooth structures using, for example, near-infrared imaging. The position and orientation of the camera may be determined in part by the 3D scan data and / or the 3D tooth surface model 1707 .

[0111] Thereafter, the penetration image can be segmented 1711. In this example, the segmentation can be performed in one of two ways. On the internal tooth structure image, the image can be segmented using contour finding 1713, 1713'. Machine learning methods can be applied to further automate the process. Alternatively or additionally, nearby images (images with a close camera position) can be used to determine nearby features, and features can also be projected back from the 3D model to the image in order to correctly locate segments such as enamel. The method can also include projecting pixels from the internal tooth image back to the tooth, and calculating a density map of the internal tooth reflectance. By using isosurfaces or thresholds of the density map and / or by machine learning methods, closed surfaces of different segments can be found or estimated. Additionally, segmenting the image and projecting the segments back to a model (e.g., a 3D surface model, e.g., projected back to the real world) can be used to find segments by the intersection of the segment projection and the tooth surface.

[0112] The results can be displayed 1717, transmitted and / or stored. For example, during an intraoral scan, the results can be displayed by the scanning system. The results can be shown as an image with closed contours for different parts, a 3D density map, etc. Figure 3 In the example shown, a density map 1715 is shown that represents the dentin beneath the enamel on the outer surface. The image can be color coded to show different segments. In this example, the internal segments (structures) are shown within the 3D surface model (shown as transparent; not all teeth were scanned with through images, so only some are shown. Alternative views, sections, slices, projections, etc. can be provided. Figure 3 , the example image includes artifacts that exist on the exterior of the tooth 1716; based on the surface model 1718, these artifacts can be removed or trimmed.

[0113] Segments can be labeled for each pixel in the image. Internal structures such as dentin, enamel, cracks, lesions, etc. can be automatically determined by segmentation and can be identified manually or automatically (e.g., machine learning based on 3D structure, etc.). Segments can be displayed individually with or without a surface model (e.g., a 3D surface model), or together (e.g., with different colors, densities, etc.).

[0114] Therefore, in Figure 3 In the present invention, a patient is first scanned using a 3D scanner capable of surface scanning and penetration scanning (e.g., near-infrared imaging), and the orientation and / or position of a camera is known (based on the position and / or orientation of a wand and / or the surface scan). This position and orientation can be relative to the tooth surface. Thus, the method and apparatus can provide an estimate of the camera's position (e.g., its x, y, and z positions and its rotational position).

[0115] Typically, penetration images (eg, near-infrared images or infrared images) can be automatically segmented.

[0116] User interface and display of volume information

[0117] The collection and analysis of volumetric data from the oral cavity can identify dental features and information that were previously difficult or impossible to identify from non-volumetric scans. However, analyzing three-dimensional volumetric information can be difficult or unintuitive for dentists (and / or patients). This article describes methods and apparatus for viewing and interpreting 3D volumetric data of a patient's oral cavity.

[0118] For example, Figures 9A-9G An example of a method for displaying 3D volume data is shown. Figure 9A A surface model (which may be part of a surface model of a volume model) from a top view of the upper dental arch is shown, wherein external features are visible (e.g., surface features). This view is similar to a surface scan view, which may be in color (e.g., taken with visible light). Figure 9A It is not easy to see the internal structure of the model that exists beneath the scanned outer surface. Figure 9B In FIG, the internal structure is shown based on its transparency to near-infrared light. Figure 9B In the figure, enamel (appears to be more transparent) is more transparent than dentin (appears to be less transparent). Figures 9B to 9F For the sub-area shown (circled area "C"), Figure 9A Surface view of Figure 9B For example, a user display may be provided in which the relative surface versus internal views may be changed to provide a sense of the internal structure relative to the surface structure within the dental arch. Figures 9C-9G Alternatively, the user can slide slider 903 to switch between a surface view and an internal view. The transition between these two views (which can be done from any angle) can help the user and / or patient see beneath the tooth surface to visually assess the rich internal data. The 3D volume model can be manipulated to display any view, including cross-sections showing internal structure and / or surface features. Figures 9A-9G A top view is shown in FIG. Figures 9C-9G Shown Figure 9A Progressively more transparent views or regions ("C") of the 3D volume model. Figures 9C-9G For region C, Figure 9B The percentage of the interior view gradually increases (from 0% to 100%), while the percentage of the surface view gradually decreases (from 100% to 0%).

[0119] Figures 11A-11C Another example is shown, showing a mixed image (similar to Figure 9E ), which scans the surface image (e.g., Figure 11A The visible light scan shown in FIG) is compared to the volume model obtained using a penetrating wavelength (e.g., a near-infrared wavelength) (e.g., Figure 11B Features present in enamel and dentin are combined and mixed in volume reconstruction ( Figure 11B ), while Figure 11A This is not apparent in the surface-only images shown (which may also be reconstructions). Figure 11B In the figure, the carious area 1103 is obvious, but in Figure 11A Similarly, the bubbled area of ​​enamel 1105 is not visible in Figure 11B is visible in the Figure 11A It is not visible. Figure 11C A hybrid image of the 3D volume model and the surface model (surface image) is shown, wherein both structures (caries and blister areas) are visible.

[0120] In general, methods and apparatus are described herein for simplifying and displaying volumetric data from a patient's oral cavity (e.g., teeth, gums, palate, etc.) in a manner that is easily understood by a user (e.g., a dentist and / or a patient). Also described herein are methods for displaying volumetric data acquired from a patient's oral cavity in a manner that is familiar and understandable to the user and / or patient. In a first example, the data can be presented as one or a series of X-ray-type images, similar to those produced by dental X-rays. Figure 4 A method of generating an X-ray (or pseudo X-ray) image from a volumetric dataset acquired as described above (eg, using penetrating light (eg, near infrared) wavelengths) is shown.

[0121] like Figure 4 As shown, a method for displaying a 3D volumetric image of a patient's oral cavity may include receiving 3D volumetric data 401, for example, directly from a scan described above or from a stored digital scan. In some variations, individual teeth or groups of teeth may be identified from the volumetric data 403. The teeth may be identified automatically (e.g., by segmenting the volume, by machine learning, etc.) or manually. Alternatively, the entire volume may be used. A pseudo-X-ray image 405 may then be generated from the volume corresponding to each tooth or a subset of the volume. For example, a volumetric image may be acquired from the "front" of the tooth, where the transparency of enamel (and / or enamel-like restorations), dentin, and other features is preserved from the volumetric data. The volumetric data may be based on the absorption coefficients of intraoral materials for the wavelength of light being used. Thus, a fixed-direction projection through the volumetric data may be generated from the volumetric data to produce an X-ray-like image. However, in some variations, the density of dentin (highly absorbing) is inverted and appears "darker" than the density of enamel (less absorbing and therefore more transparent); caries may also appear as more absorbing (darker) areas. Thus, the image can be inverted to resemble an X-ray image where the dense areas are brighter (e.g., brighter). These pseudo X-ray images can be generated and presented to the user from the same location as a standard dental X-ray. For example, a set of pseudo X-ray images can be generated for each tooth of a patient from a volumetric model. Although the wavelength of light (e.g., near-infrared light) may not penetrate as deeply as traditional X-rays, images generated in this manner can provide a comparable alternative to X-rays, especially in the mid-tooth area above the crown and gums.

[0122] Other simplified or modified displays may be provided or customized to the user for display by the user to the patient. For example, in some variations, images of teeth may be generated from the volumetric data that are simplified by highlighting certain areas by pseudo-coloring the volumetric data. For example, areas that have been previously marked or labeled (as described in more detail below) may be colored red, while enamel may be displayed as a more natural white or slightly off-white. In some variations, enamel-like materials (e.g., from fillings, etc.) may be separately represented and / or labeled by color, pattern, etc.

[0123] In some variations, the method and / or apparatus may display the teeth in a cross-section through the dental arch. Similarly, individual teeth or groups of teeth may be shown and / or labeled separately (e.g., using a standard naming / numbering convention). This may be in addition to or in lieu of other displays. In some variations, the teeth and / or internal structures may be displayed in pseudo-color or projected onto a usable color image.

[0124] For example, Figure 5A and Figure 5B A virtual section is shown through a volume model of the patient's teeth, generated from an intraoral scan including near-IR information, as described above. Figure 5A In , cross-sectional images can be generated automatically or manually, for example, by a user, to show areas of interest within the tooth, including the enamel. The cross-sections can show both density cross-sections and / or surface cross-sections. These images can be pseudo-colored to show different areas, including the outer surface, enamel, dentin, etc. The internal structure (e.g., inside the enamel and / or dentin) can reflect the effects of near-infrared light within the tooth, such as absorption and / or reflection of one or more near-infrared / visible wavelengths of light within the tooth. In Figure 5B In , the cross-sections are pseudo-colored, using a heat map to show internal features, and a key can be provided, as shown. In any of these variations, a 2D projection of the tooth can be generated from the volumetric information, showing one or more features on the tooth. As will be described in more detail below, other features can also be displayed, including lesions (e.g., caries / cavities, cracks, wear, plaque accumulation, etc.), and these features can be indicated by color, texture, etc. Although illustrated as cross-sections of a 3D volumetric model, other embodiments may display the 2D cross-sections alone to provide a cross-sectional view of the tooth similar to the view provided by a 2D X-ray image.

[0125] Any of the methods and apparatus described herein for performing the same may include displaying one or more (or continuous) cross-sections through a 3D model (preferably a 3D volumetric model) of a patient's dental arch. For example, a method for displaying an image from a three-dimensional (3D) volumetric model of a patient's dental arch may include: collecting a 3D volumetric model of the patient's dental arch, wherein the 3D volumetric model includes near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; generating a two-dimensional (2D) view of the interior of the 3D volumetric model, including the patient's dental arch with near-IR transparency of the internal structures; and displaying the 2D view. In any of these methods, the method may optionally (but not necessarily) include scanning the patient's dental arch using an intraoral scanner.

[0126] Generating the 2D views may include sectioning the 3D volume model in a plane passing through the 3D volume model. A user may select a location and / or orientation of the plane, and the sectioning may then be performed in a continuous manner. For example, any of these methods may include selecting, by the user, sections through the 3D volume model to be displayed, wherein the selecting includes continuously selecting sections through the 3D volume model as the user scans the 3D model, and continuously displaying a 2D view corresponding to each section. Generating the 2D views may include selecting, by the user, an orientation of the 2D views.

[0127] In any of these methods, a surface may be included. For example, as described and illustrated above, a method for displaying an image from a three-dimensional (3D) volume model of a patient's dental arch may include: collecting a 3D volume model of the patient's dental arch, wherein the 3D volume model includes surface values ​​and near-infrared transparency values ​​of internal structures within the dental arch; generating a two-dimensional (2D) view of the interior of the 3D volume model, including the patient's dental arch and including both surface values ​​and near-infrared transparency values ​​of the internal structures; and displaying the 2D view. The surface value may include a surface color value. The surface may be weighted relative to the internal (volume) structures. For example, generating a two-dimensional (2D) view from the 3D volume may further include including a weighted portion of the surface value and a weighted portion of the near-infrared transparency of the internal structures in the 2D view. The weighted portion of the surface value may include a percentage of the full value of the surface value, and the weighted portion of the near-infrared transparency of the internal structures may include a percentage of the full value of the near-infrared transparency of the internal structures, wherein the sum of the percentage of the full value of the surface value and the percentage of the full value of the near-infrared transparency of the internal structures is 100%. For example, a user may adjust the weighting of one or more of the surface value and the near-infrared transparency of the internal structures.

[0128] For example, a method of displaying an image from a three-dimensional (3D) volume model of a patient's dental arch may include: collecting a 3D volume model of the patient's dental arch, wherein the 3D volume model includes surface color values ​​and near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; a user selecting a direction of a view of the 3D volume model to be displayed; generating a two-dimensional (2D) view of the interior of the 3D volume using the selected direction, the two-dimensional (2D) view including the patient's dental arch, including a weighted portion of the surface color values ​​and a weighted portion of the near-IR transparency of the internal structures; and displaying the 2D view.

[0129] In addition to displaying qualitative images of teeth, the methods and devices described herein can be quantitative and can provide quantitative information about internal and / or external features. For example, volume measurements of one or more lesions can be provided (selectively or automatically), including size (peak or average length, depth, width, etc.), volume, etc. This can be performed by manually or automatically segmenting the volume model to define regions of interest, including one or two of the tooth features (enamel, dentin, etc.) and / or irregular shapes (e.g., caries, cracks, etc.). Any suitable segmentation technique can be used, such as, but not limited to: mesh segmentation (mesh decomposition), polyhedron segmentation, skeletonization, etc. Once the volume is segmented, these areas can be displayed and / or measured separately or collectively. As described below, they can also be marked / labeled and used for further analysis, display, and modification of the scanning method and system.

[0130] In some variations of the user interfaces described herein, a summary report can be generated or created and displayed for a user and / or patient from the volumetric data. For example, the summary data can be projected onto a model of the patient's teeth. The model can also be simplified so that the enamel is opaque, but markers or selected internal features (including automatically selected internal features) are displayed in red or some other contrasting color (and / or flashing, glowing, etc.) within the tooth. For example, caries can be displayed in this manner. The summary report can be automatically entered into the patient's chart.

[0131] Any image, including volumetric images, can be animated. For example, a virtual cross-section through a patient's teeth can be displayed, showing a scanned or progressed cross-section through the patient's dentition, and in some cases, using a 3D model to display one or more cutting axes through the volume. The user interface can allow the user to slice through one or more planes, thereby displaying both external and internal features based on the volume scan.

[0132] Typically, the devices described herein can also generate separate views for users other than the patient (e.g., physicians, dentists, orthodontists, etc.). A "clinical view" can be provided to the user, which can include information not available in the separate "patient view." The clinical view can be more technical and, in some cases, closer to the original image in the volume data. The user can select which information layers to include in the patient view, which can then be presented to the user during or after the scan or examination of the dental scan. Patient education materials can be attached to the patient view.

[0133] For example, in some variations, user displays of volumetric data may include an overlay of the volumetric data showing pseudo-coloring of the 3D components within the volumetric data. As will be discussed in more detail below, in any of these displays / images, marked or highlighted areas may be shown to draw attention to potential problem areas (e.g., caries, thin enamel, cracks, etc.). Two-dimensional (2D) color data and 3D near-infrared data (e.g., surface and volumetric regions) may be shown, including transitions between the two.

[0134] Typically, the volume information can be annotated (e.g., labeled, marked, etc.) automatically, manually, or semi-automatically, and the annotations can be displayed. In addition, the annotations can be used to annotate additional scans in the future, as well as to modify how future scans of the same patient are acquired and displayed. For example, the annotation can be a mark or marker on a region of interest. The region of interest can correspond to a specific area where one or more features (cracks, caries, thinning of enamel, plaque or tartar buildup, etc.) have been observed. Alternatively or additionally, the region of interest can be a region that changes over time (e.g., from one scan to another).

[0135] As described above, any of these methods can include placing one or more markers on the volumetric model of the patient's teeth. The markers (e.g., flags, pins, etc.) can be placed manually by the user, automatically by the device, or semi-automatically (e.g., suggested by the system, configured by the user, etc.). This will be described in more detail below.

[0136] Markers can be used by the system to focus attention and / or processing on one or more specific regions of the volume model for display and / or for subsequent tracking (e.g., in future scans). Markers can modify the way subsequent scans are performed, for example, performing more detailed scans of the marked region in the future (e.g., higher resolution, different wavelength, greater scanning frequency, or reputations, etc.). Marked regions can be displayed over time to show changes in the marked region.

[0137] For example, a user can mark a digital representation of a patient's teeth (or the patient's actual teeth) with a marker (e.g., a pin, a marker, etc.) that can be annotated (e.g., can have a note associated with it). The marker can then be used to track between different scans over time. Later scans can be marked at corresponding locations, and later scans can be modified based on the marked areas. These marked areas can be scanned in more detail, and analysis, measurements, and / or indications of changes compared to one or more earlier scans can be automatically performed and / or displayed. Thus, any system described herein can track one or more marked areas of a previously scan and provide feedback during and / or after a new scan to provide additional detail. This can be done for surface and / or volume information (particularly information about the properties of the enamel), and / or by comparison to the enamel, the outer surface of the tooth, and / or the dentin.

[0138] For example, one or more annotation marks from an earlier scan can modify a subsequent scan of the same patient. Before scanning, the user can enter an identifier for the patient being scanned (or the system can automatically identify the patient based on a database of earlier scans). The system can automatically annotate new scans based on previous scan annotations.

[0139] In some variations, the system may automatically annotate subsequent scans by identifying differences between a previous scan and the current scan. For example, an area that shows a change above a threshold compared to an earlier scan may be marked and presented to the user. Annotation may be done without user supervision (fully automatically) or with some user supervision, for example, by marking it and indicating to the user the reason for the marking, and then allowing the user to keep, modify, or reject the marking. Reasons for automatically marking a tooth may include changes in enamel thickness, changes in surface smoothness, changes in the relative proportions of enamel to dentin, changes in tooth position (e.g., bite), etc. Reasons for automatically marking a tooth may also include changes in the external structure of the natural tooth, such as an increase or decrease in plaque or tartar, or changes in the gum structure surrounding the tooth. Thus, if the system detects one or more of these conditions, it may automatically mark the relevant area in the volume model.

[0140] Later scans can be dynamically modified by using markers from earlier scans or by detecting changes in the area (even unmarked areas) compared to earlier scans. For example, scan parameters can be modified to scan at a higher resolution (e.g., changing the scan dwell time, requiring the user to scan the area multiple times, changing the wavelength used for the scan, etc.).

[0141] For example, Figure 6A method for automatically selecting areas for marking and / or using selected areas is shown. A first volume model 601 of the patient's teeth is generated from a scan of the patient's teeth. The volume model can be generated using any suitable method, including those discussed above in U.S. patent application Ser. No. 15 / 662,234, filed on July 27, 2017, entitled "INTRAORAL SCANNER WITH DENTALDIAGNOSTICS CAPABILITIES," which is incorporated herein by reference in its entirety. The first volume model can be stored (digitally stored) as part of the patient's dental record. The first volume model can be analyzed simultaneously or subsequently (immediately or at some time thereafter) to identify any areas that should be marked 603 (e.g., by a device, including a device having a processor configured to operate as described herein). Thus, the analysis can be performed automatically, and one or more properties of the patient's teeth can be examined from the scan. This can be performed automatically (or semi-automatically, etc., automatically but requiring human assistance for verification / validation) by a microprocessor that has been trained (e.g., through machine learning) to identify irregularities in the external and / or internal volume of a tooth. For example, the device can examine a digital model to identify possible defects in the patient's teeth, such as (but not limited to): cracks, caries, spaces, changes in occlusal relationship, malocclusion, etc. This can include identifying areas 605 where optical (e.g., near-infrared) contrast near the tooth surface indicates possible cracks, caries, occlusion, etc. Areas closer to the surface that are more opaque (e.g., more absorptive) than the remainder of the tooth enamel at near-infrared wavelengths (generally or within a specific near-infrared wavelength range) may correspond to defects. Alternatively or additionally, if the surface properties of the tooth are outside a threshold, they can be examined and marked 607. For example, areas of the tooth surface that have a rough surface (e.g., a smoothness below a set threshold, where the smoothness can be determined based on the outer surface of the tooth enamel) can be marked. Other surface properties can also be analyzed and used to determine whether the area should be marked or presented to the user for confirmation, including discoloration (based on color or white light / surface scan), gum position (relative to the outer surface of the tooth), etc. The distribution and size of the patient's enamel can also be examined 609. The enamel thickness can be determined from optical properties (e.g., comparing absorption / reflection properties). Putative areas of enamel below a threshold thickness or areas with a ratio of thickness to tooth size (e.g., diameter, width, etc.) below a threshold can be marked or presented to the user for confirmation of the marking.

[0142] In some variations, during and / or after the automatic analysis of the volume model, the user may also manually mark one or more regions of the volume model of the patient's teeth 615. If the automatic analysis of the volume model automatically marks the identified volume model, the regions manually added by the user may be added. In some variations, the user may be prompted to mark regions identified and suggested by the automatic analysis. These regions may be marked and an indication of the reason for their identification may be provided (e.g., irregular enamel, potential cracks, potential caries, potential enamel thinning, etc.). Generally, internal boundaries (e.g., boundaries within the tooth volume) may be defined in any of the methods and apparatus described herein. For example, in variations where the enamel region is thinned, the methods and apparatus described herein may be applied to entire layers (e.g., the enamel layer, regions of the internal structure of the tooth), which may be identified and used for qualitative and / or quantitative information.

[0143] In some variations, the method may then display the marked area 617 on the digital model of the patient's teeth. The display may emphasize the marked area by, for example, providing color, animation (e.g., flashing), icons (e.g., arrows, symbols, etc.), or combinations thereof. The display may also show a magnified view of any of these. The user may modify the display, for example, by rotating, sectioning, zooming in on the marked area, etc. Alternatively or additionally, the marked area may be zoomed in on the display by default. An index or keyword for the marked area may be provided and may be displayed and / or stored with the digital volume model of the patient's teeth.

[0144] In some variations, such as Figure 6As shown to the right of , the method can include using the marked area to modify future scans, as described above. For example, the method can include scanning using the marked area after some intermediate time period between a first time (e.g., approximately 1 day, 1 week, 1 month, 2 months, 3 months, 4 months, 5 months, 6 months, 7 months, 8 months, 9 months, 10 months, 11 months, 1 year, 1.2 years, 1.5 years, 2 years, etc.) and a second time (e.g., approximately 1 month, 2 months, 3 months, 4 months, 5 months, 6 months, 7 months, 8 months, 9 months, 10 months, 11 months, 1 year, 1.5 years, 2 years, 3 years, 4 years, 5 years, 6 years, 7 years, 8 years, 9 years, 10 years, 11 years, 12 years, 13 years, 14 years, 15 years, or more), or after a time longer than the second time period. Marked regions can be used to modify the scan by increasing the resolution of the scanned region during the scan (e.g., by increasing the scan rate, increasing the dwell time in the region, scanning the region at a different wavelength, scanning the region multiple times, etc.). The scanning device can notify the user to adjust the scan (e.g., moving the intraoral scanner wand more slowly in these regions, moving the scanner back and forth over these regions multiple times, etc.) and / or can automatically adjust the scan parameters during operation. Thus, the scanning device can receive a keyword or index of the marked regions and / or a marked (annotated) version of an earlier intraoral scan of the patient. Before scanning the scanning device, the user can indicate the identity of the patient being scanned, which can be used to locate the earlier scan. Alternatively or additionally, the device can identify the patient based on the current scan to identify the patient (or confirm identity) to verify or retrieve the earlier annotated (marked) scan. Alternatively, a second or subsequent scan can be performed without using the earlier marked regions.

[0145] After a subsequent scan (e.g., a second, later, or subsequent scan), the method or a device configured to perform the method can compare the marked regions from the subsequent scan with corresponding regions from the previous scan 621. In addition, the volume model from the subsequent scan can be automatically analyzed to identify any regions of the new (subsequent) scan that can / should be marked / annotated 621 (e.g., repeating the previous automatic or semi-automatic analysis steps 603-615). The newly identified regions from the subsequent scan can be compared with corresponding regions in the earlier volume model that were not previously marked.

[0146] The marked areas can be analyzed over time 623. Specific sub-areas can be generated for display and analysis, which are derived from the volume model and include the marked areas. The results can be output 625. For example, these areas can be displayed to the user along with descriptive analysis information about the scanned areas. These areas can also be marked to show changes over time. The data can be displayed in an animated form, for example, to show changes over time. In some variations, the images can be displayed as time-lapse images (video, looping film footage, etc.) showing the changes. The time-lapse images can show changes in internal and / or external structures over time. Cross-sections (pseudo-sections generated from the volume model) can be used to show the changes. Color, texture, pattern, and any other highlighting visualization techniques can be used. As an alternative to or in addition to displaying the marked areas, these areas (and any accompanying analysis) can be output in other appropriate ways, including digital output (e.g., a patient's dental record), printed descriptions of the marked areas, and the like.

[0147] Any of the methods described herein for tracking regions of a patient's dental arch may include tracking over time and / or tracking across different imaging modalities (e.g., recordings) as described in detail below. In addition, any of these methods may be automated and / or may include, for example, automated agents for identifying one or more regions of interest (e.g., features, defects, including actionable dental features), including for scoring them and / or automatically identifying, scoring, and / or displaying such regions. Any of these methods may also include any display methods or agents (e.g., methods or agents for displaying cross-sections, displaying internal structures, displaying virtual X-rays, displaying across imaging modalities, etc.).

[0148] For example, a method for tracking a region of a patient's dental arch over time may include: collecting a first three-dimensional (3D) volume model of the patient's dental arch, wherein the 3D volume model includes surface values ​​and near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; identifying a region of the 3D volume model; marking the identified region; collecting a second 3D volume model of the patient's dental arch; and displaying one or more image markers on one or more images that are the differences between the first 3D volume model and the second 3D volume model at the marked region.

[0149] Any of these methods may also include tracking and / or comparing across different records (e.g., different imaging modalities) such that the identifying includes identifying a region of the patient's dental arch from a first record of a plurality of records, wherein each record includes multiple images of the patient's dental arch, each image being taken using an imaging modality, and each record of the plurality of records being taken using a different imaging modality. In addition, marking includes marking the identified region in a corresponding region of a 3D volume model of the patient's dental arch. The methods and apparatus for performing the same may also include associating the marked region with each of the plurality of records by associating the 3D volume model of the patient's dental arch with each of the plurality of records. In some variations, the identified region may be weighted or ranked using correlation to determine whether it corresponds to an area of ​​interest (e.g., a feature, a defect, including an actionable tooth feature, etc.). For example, the patient's dental arch region may include a dental feature including one or more of: a crack, a gingival depression, tartar, enamel thickness, a pit, caries, a pit, a crack, signs of grinding, and an interdental space. Identifying the region may include comparing a near-infrared transparency value of the region within the 3D model to a threshold.

[0150] Where surface values ​​are used, the surface values ​​may include surface colour values. The methods may be used with stored data and / or data collected in real time, and thus the method optionally but not necessarily generates a 3D volume model by scanning the patient's dental arch, thereby collecting a three-dimensional (3D) volume model.

[0151] Identifying the regions can include automatically identifying regions using a processor. For example, automatically identifying can include identifying surface color values ​​outside a threshold range. Automatically identifying can include segmenting the 3D volume model to identify enamel regions and identifying regions where the enamel thickness is below a threshold.

[0152] Marking the identified area may include automatically marking the identified area or manually confirming the identified area for marking.

[0153] In any of these methods where areas are marked, the method may include rescanning the patient's dental arch, wherein the marked areas are scanned at a higher resolution than unmarked areas.

[0154] A method for tracking regions of a patient's dental arch over time may include: collecting a first three-dimensional (3D) volume model of the patient's dental arch acquired at a first time, wherein the 3D volume model includes surface color values ​​and near-infrared (near-IR) transparency values ​​of internal structures within the dental arch; identifying regions to be marked in the 3D volume model from a first record of a plurality of records using an automated process; wherein each record includes a plurality of images of the patient's dental arch, each taken using an imaging mode, and each of the plurality of records is taken using a different imaging mode; marking the identified regions; associating the marked regions with each of the plurality of records by associating the 3D volume model of the patient's dental arch with each of the plurality of records; collecting a second 3D volume model of the patient's dental arch acquired at another time; and displaying differences between the first 3D volume model and the second 3D volume model at the marked regions.

[0155] Similarly, as outlined and described above, a method for tracking dental features across different imaging modalities includes: collecting a first three-dimensional (3D) volume model of a patient's dental arch, wherein the 3D volume model of the patient's dental arch includes surface values ​​and internal structures within the dental arch; identifying a region of the patient's dental arch from a first record in a plurality of records, wherein each record includes multiple images of the patient's dental arch, each image taken using an imaging modality, and each record of the plurality of records is taken using a different imaging modality; marking the identified region in a corresponding region of the 3D volume model of the patient's dental arch; associating the marked region with each record in the plurality of records by associating the 3D volume model of the patient's dental arch with each record in the plurality of records; and saving, displaying, and / or transmitting images including the region of the patient's dental arch. Any of these methods may also include tracking over time, for example, by comparing the same region to the 3D volume model at different times.

[0156] Figure 10A and 10B The user interface shown shows marking of regions of interest from a 3D volume scan of a patient's mouth. Figure 10A In the example, the user interface includes an image 1001 of an internal feature (e.g., based on near infrared absorption of teeth), similar to the above Figure 9B The view can be manipulated through user controls 1015. User controls 1015 include a sectioning tool, a rotation tool, a move tool, etc. Figure 10A , the two upper windows show a surface view 1003 and a volume (interior) view 1005 corresponding to the same area. The area can be selected. Figure 10B Shown with Figure 10AThe same features, but with marked or annotated areas 1011. As described above, the areas to be marked may be identified automatically or manually, or semi-automatically (e.g., by user confirmation), and may be selected to be monitored later. Figure 10B In the example, this area can correspond to possible dental caries, for example.

[0157] Monitoring one or more internal areas of a tooth over time using a volumetric model of a patient's teeth obtained using the apparatus described herein may be particularly helpful in predicting dental problems (including caries, cracks, tooth loss, gum recession, etc.). In particular, these methods and devices can help users (e.g., dentists, dental technicians, orthodontists, etc.) inform and educate patients so that they can take recommended treatments before developing more serious problems. There is a need for effective methods to display changes in teeth over time and to provide patients with the necessary information to take action early to prevent the development of more complex and potentially painful problems. Otherwise, many patients are reluctant to take preventative measures, especially when there is no associated pain or discomfort at the moment. For example, pre-cavitated caries are difficult to identify using current imaging technology, and even when identified, it is difficult to convince patients to undergo treatment because they are generally painless. However, early treatment is essential to avoid more complex and dangerous surgeries later.

[0158] Dental caries is a type of problem that can be identified using the methods and devices described herein. As shown and discussed above, dental caries can be identified from a 3D volume model (e.g., the 3D volume model described herein) that uses light (e.g., near-infrared) to penetrate (a non-ionizing radiation) into the tooth. In a 3D volume model generated as described herein, for example, using near-infrared light (typically used in combination with surface scanning (e.g., white light), the absorption coefficient of the internal area of ​​the tooth can indicate the difference between dentin and enamel, and internal structures and defects (including cracks, dental caries, etc.) can be found. For example, enamel areas that are less transparent than expected at near-infrared wavelengths (e.g., have different infrared optical properties), especially those enamel areas that extend to the tooth surface in the volume model, can be manually or automatically identified as cavities or dental caries. Other irregularities in the enamel and / or dentin (e.g., based on internal features of the tooth from the volume model) can be identified and may be characteristics of a dental problem. Therefore, the technology described herein can be used for prognostic diagnosis of dental problems such as dental caries.

[0159] As mentioned, any of the devices and methods described herein may include improved methods for displaying internal tooth features using one or more volumetric models of a patient's teeth. For example, the methods and devices described herein can be used to generate estimates of the thickness of the enamel of one or more of a patient's teeth. These estimates can be displayed visually, showing the outer surface of the tooth or a specific tooth, and can also show the internal structure (including showing, for example, the enamel, including the thickness of the enamel, in a cross-sectional view or a 3D internal view). This information can be used clinically to determine the need for dentures, including veneers, crowns, etc., to aid in their design, and to aid in their application. For example, any of the methods and devices described herein can be used to help prepare dental implants designed for specific teeth.

[0160] Plaque and tartar detection and visualization

[0161] The methods and apparatus described herein can also be used to detect and visualize (including quantify) dental plaque and tartar on a patient's teeth. Any of the intraoral scanners described herein can be used to detect dental plaque or tartar on a patient's teeth by using fluorescence imaging in addition to other imaging / scanning modes including penetrating (e.g., near infrared) imaging. For example, the intraoral scanner can cycle between different imaging modes (e.g., between white light and near infrared), including additional modes such as fluorescence (e.g., laser fluorescence, etc.).

[0162] The use of fluorescence capabilities of an intraoral scanner (and / or the use of the capabilities of a current one) can allow for the detection of plaque and tartar on tooth surfaces. In combination with 3D modeling using data from the intraoral scanner, the plaque / tartar condition can be modeled and visualized on a 3D model of the teeth (including 3D volumetric modeling of the teeth). As described above, plaque and / or tartar can be detected and can be displayed and highlighted, and can be used before, during, or after treatment. For example, a dental technician (e.g., a dental hygienist) can use an intraoral scanner to detect and monitor cleaning treatments and the condition of a patient. Plaque and tartar data can also be used by any of the devices described herein to determine and provide a predictive model that can indicate the rate and / or location of plaque and tartar (e.g., tartar) generation.

[0163] In some variations, dental plaque and tartar may be identified, at least in part, using fluorescence information. It has been observed that dental plaque may fluoresce under blue light (e.g., light having a wavelength of approximately 405 nm). Any intraoral scanner described herein may include fluorescence information, from which information about dental plaque and tartar may be used and incorporated into a 3D model of the patient's teeth. For example, dental plaque and / or tartar may be visually displayed on the 3D model of the patient's teeth in the form of color and / or texture.

[0164] For example, a fluorescence signal can be obtained from an intraoral scanner using a dichroic filter that has a large aperture expansion for the fluorescence signal. This expansion can enhance the fluorescence, enabling detection, visualization, and segmentation of plaque and tartar areas using RGB illumination, sensors, and images. Alternatively or in addition, the device can include a fluorescence source (e.g., an LED emitting light at a wavelength of 405 nm) and corresponding filters for detecting plaque and / or tartar. These components can be integrated into the intraoral scanner, or these components can be added (e.g., as a sleeve, attachment, etc.) for use with the scanner.

[0165] Alternatively or additionally, in some variations, plaque and tartar may have different absorption / reflection than enamel, depending on the wavelength of the near-infrared light used. This enables the plaque and / or plaque to be distinguished from the enamel in the volumetric model. Furthermore, the volumetric model can be used to detect materials on the teeth based on surface smoothness and geometry, including detecting plaque and tartar. In variations where the plaque and / or plaque are not transparent to the near-infrared frequencies used, the device can use the volumetric model to distinguish the plaque and / or plaque from the enamel. Thus, the plaque and / or plaque can be segmented and distinguished from the enamel.

[0166] Detecting dental plaque and / or tartar using an intraoral scanner can provide quantitative information and a digital model. This enables monitoring and comparing dental plaque / tartar over time based on registration (including real-time registration and / or display) of the 3D model.

[0167] An intraoral scanner (e.g., a scanning wand) acquires fluorescence images and 3D scans at the same time and in the same location, enabling very accurate registration of plaque / tartar areas and 3D models. For example, simultaneous scanning is described in more detail in U.S. patent application Ser. No. 15 / 662,234, filed on July 27, 2017, entitled "INTRAORAL SCANNER WITH DENTAL DIAGNOSTICS CAPABILITIES." Accurate registration between different scanning modes (e.g., white light / visible light, penetrating (near infrared) light, and / or fluorescence) enables the device to define the boundaries of tartar and / or plaque, and can allow the device to determine volume / thickness with high accuracy, thereby allowing accurate measurement of the current condition and comparison / tracking relative to previous scans.

[0168] The methods and devices described herein can take RGB images of teeth at the same time / similar time as a 3D scan of the teeth is being performed. These scans can then be used to construct a 3D model of the teeth / jaw, which can include volumetric information (3D volumetric model). For example, as described above, RGB images can show prominent signals of fluorescent surfaces (particularly plaque and tartar areas) because such surfaces have specific characteristic colors and brightness. For example, an image (or volumetric model in some cases) of the outer surface of a tooth may show areas with optical properties (fluorescence, brightness, color, etc.) that are indicative of tartar and / or plaque. In some variations, such prominent signals may be caused by spectral illumination that does not produce any reflections in visible light but produces significant fluorescence signals from plaque and tartar. For example, a typical RGB illumination (using an ordinary RGB sensor) can be modified to amplify the fluorescence signal on the outer surface of the tooth (e.g., in the near IR region). As a non-limiting example, this amplification can be achieved by a large aperture and a small aperture, the large aperture being able to pass IR signals and the small aperture being able to pass conventional RGB (visible) spectrum. This combination can produce a color image with additional emphasis on the fluorescent surface. This fluorescence can reveal the characteristic color and brightness of desired areas, indicating tartar and / or plaque on the teeth.

[0169] In any of the methods and devices described herein that can capture an RGB image that includes a fluorescence signal (e.g., a wavelength at which plaque or tartar fluoresces), segmentation of the fluorescent regions can be performed on the image. For example, using camera positions during acquisition of an RGB and 3D scan (e.g., from an intraoral scanner), the fluorescent regions can be registered with a 3D model of the patient's teeth (including a volumetric model and / or a surface-only model). This can result in the relevant plaque and tartar regions being defined on the final 3D model, thereby allowing further definition of these regions, such as the boundaries of the tartar on the teeth and the 3D surface and thickness of the plaque.

[0170] As described above, areas on the 3D model can be compared to previous / future scans of the same patient, which can show the development of tartar over time and the effects of tartar on the patient's teeth. The device can automatically or semi-automatically annotate (e.g., mark) these areas for monitoring. Thus, the size and shape of the tartar on each tooth can be monitored. Alternatively or in addition, the thickness / depth of the tartar can be compared to previous scans. As described above, any of this information can be provided quantitatively and / or qualitatively. The thickness / depth of the tartar can be compared to previous scans of cleaned teeth (including one or more earlier scans after cleaning by a dental professional). This can provide an estimate of the thickness of the tartar in later scans. As described above, changes in dental plaque (particularly tartar over time) can be measured and this data can be used to monitor the progression of plaque and tartar on a patient's teeth and can also provide a visualization of the development.

[0171] Typically, monitoring and visualization of a patient's teeth using the methods and devices described herein can be used as part of a dental and / or orthodontic treatment plan. As described above, monitoring of plaque and tartar can be used for treatments that include tooth cleaning. Scans can be performed before, during, and / or after cleaning to provide guidance to the dentist about which areas to emphasize, focus on, or return to. Based on the progression of plaque and / or tartar over time, other treatments (coatings, caps, etc.) can be proposed. In addition, as described above, monitoring of any other features or areas of concern (including, for example, caries, cracks, etc.) can also provide treatment plan information. As described above, information about cracks and / or caries can be used to suggest treatments that include restorations before potential problems develop further. In some variations, volume information can be used to modify a digital model of a tooth (e.g., a surface and / or volume model), and the modified model is used to design an orthodontic appliance or treatment plan. For example, a user can digitally remove plaque and / or tartar from a volume scan taken before or during treatment. If necessary, the revised scan can be used to guide treatment, including further cleaning of the teeth, and forming or modifying the appliance so that the appliance (eg, dental aligners) fits better.

[0172] Combined with dental tools

[0173] The intraoral scanners and volumetric models described herein may be used with and / or combined with other dental tools (drills, probes, etc.) The combined tools may provide numerous advantages.

[0174] For example, described herein is a drill that can be combined or used in conjunction with an intraoral scanner, as well as the use of a 3D volume model. In some variations, a dental drill and an intraoral scanner can be combined; for example, a laser drill or a laser-accelerated water drill can be incorporated into an intraoral scanner. This combination can allow dental professionals to use the tool to directly visualize the teeth before and during drilling, thereby providing real-time feedback to the user. In one example, near-infrared light can be applied to the probe of a drill (e.g., a laser drill) to image the teeth, which will allow direct prospective imaging before and / or during drilling. The enamel and dentin in the direct path of the drill can be imaged. When they reach the dentin layer of the tooth or a certain depth within the dentin, or when the lesion area has been removed, density information can be used to notify the clinician. For example, density information can be used to provide tactile feedback to the operator, because tactile feedback is more limited when using dental lasers compared to traditional headpieces.

[0175] The methods and apparatus described herein, including intraoral scanners and volume modeling, can also be integrated into computer-aided design / computer-aided manufacturing techniques for dentistry, such as Figure 7 For example, dental implants such as crowns (e.g., ceramic crowns) can be manufactured for individual patients using computer-aided design and computer-aided manufacturing (CAD / CAM) equipment and programs. For example, traditionally, CAD / CAM laboratories manufacture ( Figure 7 The "current workflow" in the figure may include taking a pre-treatment scan 701 of the patient's teeth or an impression of the patient's teeth, such as a caries-free scan of the jaw. The tooth may then be prepared for the crown 703 and then rescanned 705 and evaluated 707. Finally, the crown may be manufactured using CAD / CAM. CAD / CAM software may receive the scanned information from the scanner and may process it to form a design and perform manufacturing. The use of CAD / CAM software may provide a restoration that is comparable to conventional restorations in all respects, including aesthetics, but current methods may require repeated steps to evaluate and prepare the tooth, such as Figure 7 and usually require the user to perform these steps manually.

[0176] like Figure 7 As shown in the "New Workflow" at the bottom, this method can be integrated with the 3D volume modeling described herein to simplify and improve CAD / CAM of a patient's teeth. For example, the preparation can be designed digitally and the process can be automated (fully or semi-fully so that the user can approve and / or modify the process). For example, in Figure 7 In , a pre-treatment scan 711 can be performed using an intraoral scanner that communicates directly with the CAD / CAM device, or the intraoral scanner can include CAD / CAM functionality. In this example, the tooth preparation can be designed completely digitally based on the scan performed 713, and the scanner can guide the preparation of the tooth 715. This can be done in real time with direct feedback and / or guidance from the device, which can have an integrated scanner. The scanner can then be used to evaluate the preparation 717, and in some cases, this step can be fully integrated into the guided preparation step 715, eliminating the need for a post-preparation evaluation. Finally, CAD / CAM can be used to prepare a crown (or other dental appliance) 719 for the correctly prepared tooth.

[0177] root canal

[0178] Methods and apparatus for 3D volume modeling of a patient's oral cavity (e.g., 3D volume modeling of teeth) can also be used to modify root canal surgery. Typically, root canal surgery requires extensive X-rays to provide images of the interior of the tooth before, during, and / or after the procedure. The methods and apparatus described herein can eliminate or reduce the need for X-rays in specific instances of root canal surgery. Specifically, as described herein, an intraoral scanner comprising a penetrating wavelength (e.g., a near-infrared wavelength) can be used to examine the interior of a tooth (including the interior of a tooth root) during surgery. This can allow the root canal to be identified and located. For example, a tooth can be prepared for a root canal by, for example, drilling through the crown into the tooth. During drilling or during insertion, the drilling can be guided by the volume imaging described herein. For example, a tooth (e.g., a molar) can have an initial hole drilled into it to expose the camber within the tooth. An intraoral scanner comprising near-infrared can be used to image the tooth, including imaging through the hole drilled into the tooth to visualize the interior of the pulp chamber. The scanner can be automatically or manually oriented downward to image the interior of the chamber, which enables visualization of the root within the chamber. The initial drilling of the tooth can be limited to penetrating the enamel and exposing the internal cavity, and visualizing the interior of the cavity so that areas with different optical properties (at any wavelength, particularly including near-infrared wavelengths) can penetrate the cavity despite calcification and / or infection to allow imaging of the root from within the tooth itself. The neural cavity of the root can be identified as being more dense or less dense than the surrounding areas within the dentin and enamel. By removing the top of the cavity to expose the internal pulp region of the tooth, an intraoral scanner can visualize through the drilled opening to provide additional volumetric information, including the position, curvature, and trajectory of the tooth root. This additional visualization information can facilitate the detection of hidden root canals and accessory root canals. This information can then be used to guide treatment.

[0179] For example, in some variations, the method may include acquiring a 3D volume model of the patient's tooth using an intraoral scanner as described herein before or after drilling to form an opening into a target tooth (e.g., a tooth to be treated with root canal surgery). As described above, drilling can be performed with or without the guidance of an intraoral scanner. For example, an intraoral scanner can be used to visualize the internal cavity of a tooth through an opening drilled from the crown. The device can then determine the location of the horns of the pulp cavity of the tooth. Any of the methods described herein can be used in conjunction with X-ray information. The device can perform treatment planning to determine the shape and / or location of the pulp horns, pulp cavity, and root to develop a treatment plan for drilling / tissue removal, thereby avoiding over-consideration or damage to the sides of the tooth. The treatment plan can then be used to guide the user in drilling the tooth and / or to automate drilling. In some variations, the drill can be directly guided, for example, by using a hybrid drill bit / imaging using the intraoral scanner described above. Alternatively or additionally, the treatment plan can be used to provide robotic assistance. In some variations, the procedure can be performed manually and drilling can be performed in smaller increments, with visualization performed between drilling steps to confirm the treatment path and avoid overdrilling, while confirming that the entire area has been drilled and the infected pulp removed. Other visualization methods (including the use of contrast agents) can be used.

[0180] Typically, during imaging, any of the methods described herein (including the root canal methods described above) can be used with one or more contrast agents. For example, the contrast agent can include a material applied to the exterior of the tooth (or to a hole or opening in the tooth (including a hole drilled into the tooth)). Contrast agents that absorb or reflect near-infrared wavelengths or other wavelengths used by intraoral scanners can be used. Preferably, a contrast agent that is distinguishable at certain imaging wavelengths (but not all) can be used to provide differential imaging. For example, the contrast agent can be visible under white light but not under near-infrared light; alternatively, the contrast agent can be visualized under near-infrared light rather than white light, or the contrast agent can be visualized at certain wavelengths in the near-infrared other than those used to capture the image. Preferably, a contrast agent can be used that is admixed with or coated with one or more targets within the tooth or oral cavity. For example, a contrast agent can be used that selectively binds to one or more of the following: bacteria, plaque, tartar, gums, dental pulp, etc. When used, the contrast agent can be applied to the tooth / oral cavity, rinsed, and then visualized (or visualized without rinsing). For example, an infrared light absorbing contrast agent can be used as a partial inclusion or mixed with a material forming, for example, a dental implant (e.g., to fill a cavity, cover a tooth, fill a root canal, etc.) to create an IR contrast filling material that can be easily visualized when a scan as described herein is performed.

[0181] Also described herein are methods for determining improvements in the soft tissue surrounding a tooth using the apparatus and methods described herein for generating a 3D volumetric model of a tooth. For example, gum recession can be monitored and / or quantified, and gum recession can be observed over time using these methods and apparatus. In addition to directly visualizing plaque and / or tartar as described above, the methods and apparatus described herein can also or alternatively detect effects on the tooth (including bone atrophy due to plaque and tartar). The diseased area can be directly visualized. In some variations, a contrast agent can be used to provide additional contrast to the intraoral scanner to detect the diseased area. Scanning the gum surface can identify areas of inflammation and / or discoloration that may be indicative of gum disease. This information can be combined with a 3D volumetric model of the tooth, including the location of the plaque and / or tartar, as described above.

[0182] Figure 8A and 8B An example of monitoring gum recession over time is shown. In this example, the display can show a 3D model of the teeth and a comparison between the original scan and a follow-up scan taken 2-3 years later. Figure 8A In FIG8 , the two scans have been aligned and compared, and the differences are shown by color indicators (e.g., heat maps). In FIG8 , the darker color (which can be displayed in color, such as red) shows a greater degree of gum recession. Figure 8B This is shown in more detail in Figure 8A Although FIG8 primarily shows surface features (eg, gum position), volumetric information can be used to generate this information, for example, showing variations in gum thickness and / or vascularization, enamel thickness, etc.

[0183] In addition to guiding the user and / or dental technician based on the scan (e.g., in particular, showing plaque, tartar and / or inflammation), the dental professional can also use these methods and devices to rate, rank or quantify plaque and / or tartar removal immediately after treatment or over time. This can provide a metric that can be used to judge treatment. The scan information can also be used to provide information to the patient, including a map or guide for home treatment, including which areas to focus brushing, flossing, etc. For example, the guide can include one or more images from the 3D volume model. Guidance information about which teeth or areas of the mouth to focus on for home dental care (e.g., brushing) can be provided to the electronic toothbrush, which can also help guide the patient in brushing based on the identified areas.

[0184] The methods and apparatus described herein may also be used with patients who already have dental appliances installed on their teeth, including braces, bridges, etc. For example, in some variations, the patient may include a 3D representation of the dental appliance and may provide information to aid in the design or modification of future dental appliances (e.g., retainers, aligners, braces, etc.).

[0185] In particular, the methods and apparatus described herein can be used to provide very accurate volumetric and surface information about a patient's teeth, which can be used for treatment planning of any type of dental treatment. In certain variations, the methods and apparatus described herein can be used for treatment planning of an appliance (e.g., an aligner or retainer) that is optimally worn proximate to the patient's teeth. For example, a method and / or apparatus comprising a 3D volumetric scan of a patient's teeth can be used to subtract or remove any plaque, tartar, and / or food debris that may be present at the time of the 3D scan from a 3D model of the teeth. By digitally subtracting any plaque, tartar, and / or food debris that is present, the volumetric information can be used for a virtual rendering of an aligner, retainer, night guard, or other device, and the fit can be improved prior to manufacturing, applying, or wearing the device.

[0186] Gum tissue surrounding teeth that has a different density (or different optical absorption / reflection properties) than enamel can also be identified and characterized with greater accuracy, making it possible to identify the connection between the inner contour and the tooth surface. By doing so, the shape of the tooth surface beneath the gum tissue can be accurately characterized so that predictive models of tooth movement can more accurately represent the teeth as initially invisible portions of the teeth are gradually exposed as the teeth align. In other words, portions of the teeth may initially be obscured by the gum tissue, but as the teeth straighten, the gum tissue moves upward, exposing the previously covered areas. By accurately detecting the areas of the teeth beneath the gum tissue, the future state of the teeth after the gums have moved upward can be more accurately modeled.

[0187] The methods and devices described herein may also be used to detect, diagnose, and / or treat oral diseases.

[0188] For example, 3D volume scanning and modeling methods and equipment described herein can be used to detect and / or treat salivary stones (salivary stone) (e.g., salivary duct obstruction). These glands may be located near the molars and under the patient's tongue, and these glands can be scanned using an intraoral scanner as described herein. These scans can penetrate soft tissue and can detect hard stone-like structures (i.e., salivary stones, salivary gland stones, or duct stones), which are calcified structures that may form inside the salivary glands or ducts and prevent saliva from flowing into the mouth. Methods and equipment described herein can be used to identify these structures and / or can guide and / or confirm the removal of these stones.

[0189] In addition to or in lieu of using the devices and methods described herein to identify, diagnose and / or track areas including pre-cavitated caries, cracks, etc., the methods and devices described herein may also or alternatively be used to identify and manipulate areas that have been modified. For example, fillings, accessories (for connecting anglers, braces, etc.), braces, retainers, etc., and any other structures may be identified and / or displayed within a volume model. For example, areas of enamel and / or enamel-like restorations may be displayed differently in a volume model. These areas typically have different optical properties compared to each other and / or other areas of the oral cavity (including dentin), including different scattering / absorption for near-infrared light (and in some cases visible light). Such areas may be identified manually, automatically, or semi-automatically and may be segmented and / or manipulated individually. For example, in some variations, these areas on the teeth (e.g., attachments / adhesives for aligners or other appliances, etc.) may be identified for removal by the dentist, and a 3D volume model or data (image) obtained therefrom may be provided to guide such treatment. They may also or alternatively be digitally subtracted to provide a better fit for the new appliance once removed. The subtracted views may also or alternatively be provided to the patient.

[0190] In some variations, the volumetric models described herein can be used to determine the internal structural integrity of artificial tooth structures or modifications (e.g., dental adhesives, fillings, etc.). For example, the volumetric model can include internal details of the artificial tooth structure (e.g., internal structural details of a filling, adhesive, etc.), or the interface between natural teeth (enamel, dentin, etc.), and this information can be displayed or presented to the user in detail to allow for an assessment (or automatic assessment) of the condition of such artificial tooth structures. This can facilitate their removal, repair, and / or replacement.

[0191] A 3D volume model of a tooth (and a method and apparatus for generating a 3D volume model) can also be used as a diagnostic or detection tool for future tooth sensitivity. For example, abfraction is a form of non-caries tissue loss that typically occurs along the gingival margin. Abfraction lesion may be a mechanical loss of tooth structure that is not caused by tooth decay and may occur in the dentin and enamel of the tooth. It is believed that this is caused by repetitive pressure cycles from the patient's bite and is exacerbated by vigorous brushing. A 3D volume model of the tooth enhanced by density analysis of the enamel and dentin near the gum line can provide an early indication of these lesions. For example, the device can examine the volume model to identify the initial stages of the formation of these crescent-shaped lesions. Multiple 3D volume models acquired over time can indicate the rate of progression of these lesions. The system can be configured to identify them automatically or manually; as described above, they can be marked automatically or semi-automatically.

[0192] Thus, the device and method can identify and modify such "hot spots" that may occur in a user (leading to future tooth sensitivity) and can provide a treatment plan to slow, stop, or reverse the progression of the lesions. These small fractures and exposed dentin may cause tooth sensitivity. Detection can be triggered by identifying the characteristic crescent shape that develops in more mature lesions, but earlier detection can be performed by identifying areas in the enamel and / or dentin that may develop thinning over time (e.g., near the gum line). The device and method can flag and / or assign risk based on actual thickness and / or progression of thickness change.

[0193] The methods and apparatus described herein may also detect the development of acid reflux based in part on characteristic wear patterns and / or changes in the thickness of a patient's tooth enamel (e.g., over time). For example, acid reflux that occurs while a patient is asleep may cause the patient's teeth to gradually erode in a characteristic pattern (e.g., from the back of the teeth, on the lingual side). Similar patterns may occur due to bulimia. A volumetric model of a patient's teeth, acquired by, for example, near infrared, can provide an accurate mapping of the enamel density and thickness of all of the patient's teeth. Thus, a method of detecting acid reflux (or bulimia) may include detecting (including detecting over time) a characteristic thinning of the enamel of teeth in a lingual region of the patient's back. The enamel thickness of the near-lingual region of the teeth may be abnormally thin (or thinning) compared to the front (forward) region on the opposite, buccal side of the patient's teeth.

[0194] The methods and apparatus described herein can also be used to detect areas of thin enamel resulting from occlusal wear (due to chronic grinding of a patient's teeth) and / or to predict tooth sensitivity due to such grinding. A 3D volumetric model of the patient's teeth can show a snapshot of the occlusal thickness of the patient's enamel and the proximity of the dentin to the occlusal surface. In addition, multiple scans taken over time can show the loss of enamel in the occlusal surface. As described above, this is an indicator that can be marked or annotated automatically, manually, or semi-automatically. For example, as long as a 0.5 mm thick enamel greater than 0.5 mm is present within 0.5 mm of the dentin, the 3D volumetric model can be used to detect areas of thin enamel resulting from occlusal wear (due to chronic grinding of a patient's teeth) and / or to predict tooth sensitivity due to such grinding. 2An area of ​​the tooth that has been marked can be marked and that area of ​​the digital model highlighted. This allows any area that meets the marking criteria to be visualized and / or monitored. By giving the patient's age (and in some variations sex) and the change in enamel thickness over time, an estimate of the wear rate over time can be provided, as well as the proximity to the dentin area, so that tooth sensitivity or pain can be estimated or predicted. Teeth grinding may also be an indication of other symptoms (including sleep apnea). For example, sleep apnea can also be detected from a 3D volumetric model of a patient's teeth (especially over time). Many patients with sleep apnea grind their teeth (e.g., forward and / or side to side movements), which can result in a pattern of tooth erosion. Therefore, the methods and devices described herein can be used to help diagnose or confirm sleep apnea.

[0195] Generally, any method and apparatus described herein can be used for non-human patients.For example, any method and apparatus described herein can be used for veterinary patients (e.g., animals) to determine, for example, the state of the animal's teeth (including the wear of the teeth).

[0196] The methods and devices described herein can also be used to provide a patient with an estimate of the risk of tooth fracture and / or tooth sensitivity. For example, based on mechanical estimates of tooth thickness and wear patterns, the 3D volumetric model of the teeth described herein can be used to identify malocclusions in the teeth and the eventual wear and / or fracture of the teeth. Functional information (e.g., chewing patterns and bite forces) can also be integrated into the assessment. For example, wear patterns can be identified on an image generated from a 3D rendering of the patient's teeth and displayed as "hot spots." This can be displayed to the patient as information, including as a warning of potential risks. High-risk areas can be identified and the potential risks explained to the patient.

[0197] In general, these methods and apparatus, and in particular, monitoring and comparing 3D volume models including information about the internal structure of a tooth (e.g., the distribution of enamel and dentin within a tooth) over time, can be used to identify, monitor, diagnose, and guide treatment of a variety of conditions other than those described above, such as dentin dysplasia, enamel dysplasia, etc. These methods also allow for the identification of multiple different types of enamel within a patient's teeth, including regions with varying amounts of hydroxyapatite, amelogenin, and / or enamel, or regions of varying organization thereof, including regions that are homogeneous or non-homogeneous and that may have varying optical properties at the near-infrared wavelengths used for imaging.

[0198] Interactive display of a 3D model of the patient's dental arch

[0199] As already described (and as shown in the above figures), the methods and apparatus described herein can allow a user to virtually scan a patient's dental arch. In particular, a 3D model of the patient's dental arch (which can be a volumetric model, a surface model, or both, or in some variations an abstract or generic model) can be used in conjunction with captured images, which are captured, for example, from various positions around the dental arch using an intraoral scanner. These images can be the images used to generate the 3D model of the dental arch. These images can be labeled and / or arranged in a data structure to indicate their corresponding positions or areas or angles relative to the 3D dental arch model. In some variations, the 3D model and the captured images can be maintained as a data structure, however, it is not necessary to include the 3D model and the images together as a single data structure.

[0200] For example, Figure 13 is an example of a data structure that includes one or more dental arch models 1305 and multiple images (e.g., greater than 50, greater than 100, greater than 200, greater than 500, greater than 750, greater than 1000, greater than 10,000, etc.) from one or more (e.g., groups of) images taken from locations around the patient's dental arch. In some variations, visible light images and near infrared (or near infrared and other modalities) images 1301 may be shown and may share position information. The position information typically includes the area of ​​the dental arch from which the image was taken (e.g., in x, y, z coordinates, such as the coordinates of the center point of the image relative to the dental arch) and the angle relative to the plane of the dental arch (e.g., roll, pitch and / or yaw or radial coordinates, etc.) ("position information" 1301). In some variations, a scan may be a composite of multiple scans that are combined and stored in a data structure (e.g., averaged, blended, etc.). A 3D model may be formed by virtually "stitching" the scans together to form the 3D model.

[0201] Data structures can be stored in a compressed configuration; although the data structure may contain a large amount of data, the compression and organization of the data structure can allow it to be manipulated for display. For example, Figure 12 A method for using a data structure (e.g., Figure 13 A method for interactively displaying a 3D model of a patient's dental arch (data structure schematically shown in ).

[0202] exist Figure 12In the method, the method includes displaying a 3D model of a patient's dental arch 1201 and displaying a viewing window 1203 on the 3D model. The user may then be allowed to continuously move both (e.g., one or both of the viewing window and the 3D model) so that the teeth of the dental arch can be virtually viewed "through" the viewing window in more detail in a nearby view 1205. The user may change the angle of the viewing window and the position of the viewing window along the dental arch 1207, e.g., continuously moving over and / or around the 3D model of the dental arch. As the viewing window / dental arch are moved relative to each other, data structures / data sets (e.g., Figure 13 ) are identified 1209 in the image data, which are taken at positions relative to the dental arch corresponding to the position of the viewing window. These corresponding images can then be displayed 1211, and this process can be iteratively repeated as the viewing window is moved along the 3D dental arch model.

[0203] In some variations, the data structure may be configured or arranged topologically or in an indexed topology; thus, images of adjacent regions may be linked or ordered in the data structure, thereby simplifying the method.

[0204] Figures 14A-16C An example of a variation of the user interface is shown, which can allow the user to virtually scan a 3D model of a dental arch, and Figure 12 As described above, a user can view the near-infrared image to manually identify (or, in some variations, automatically identify) one or more structures / defects and / or actionable tooth features (including caries, cracks, wear, etc.). Displaying the corresponding 3D dental arch model and the visible light image of the same area both provides perspective and allows direct comparison with the patient's teeth, thereby simplifying and significantly enhancing dental analysis.

[0205] For example, in Figure 14A, the display is shown as a user interface 1400 including a dental arch model 1403 (3D dental arch model) that has been reconstructed from scans of a patient's teeth and stored in a data structure along with many or all of those scans. As already described above, it is not necessary but may be helpful to include the 3D dental arch model with the multiple images in the data structure. Again, the 3D dental arch model in this example is constructed from scans of the patient's teeth, but it should be clear that the 3D dental arch model may be non-representative but may still be used to select 2D views to be displayed as described herein. A viewing window 1401, shown as an annular or circular shape, is moveable over or along the 3D model of the dental arch; as the viewing window is moved, each of the two image displays 1405, 1407 is updated with an image corresponding to that position (both the area of ​​the dental arch and the angle of the dental arch relative to the plane of the viewing window). In Figure 14A , the first (upper) image 1405 is a near infrared image, and the corresponding visible light (e.g., color) image (taken at the same approximate time / location) is shown in the lower image 1407. Alternatively, Figure 14B and 14C The displays shown in each show only a single image. Figure 14B In the figure, an enlarged near-infrared display image is shown, while in Figure 14C , an enlarged visible light display image of a single area corresponding to the imaging window view is shown.

[0206] The user interface may also include tools 1409 for manipulating the display (eg, rotating, moving the dental arch and / or rotating, moving the viewing window; modifying, marking up images and / or 3D models; and saving, opening / calling images; etc.).

[0207] Figures 15A-15B An example of moving the viewing window on the teeth and changing / updating the corresponding image is shown. Figure 15A An image of a dental arch is shown with corresponding near infrared and visible light images "seen" through a viewing window at the middle region of the dental arch. Figure 15B In the embodiment, the user rotates the dental arch (or alternatively, rotates the viewing window lingually relative to the dental arch) so that the viewing window is relative to Figure 15A Positioned slightly lingually; the corresponding views (near infrared and visible light) have been updated in real time to show this change in the relative position of the viewing windows.

[0208] Similarly, Figures 16A-16C An example of a 3D model of a patient's lower dental arch is shown, which is similar to Figures 14A-14CIn use, as the user moves the viewing window over the dental arch (and / or the dental arch relative to the viewing window) and scans along the dental arch, the displayed image can change virtually continuously, so that the image can be updated in real time or near real time. The user can identify features in the near-infrared image, including density changes in normally infrared-transparent enamel areas that may indicate the presence of caries, cracks, or wear in the enamel.

[0209] Figures 1A-1B The intraoral scanning system shown can be configured as an intraoral scanning system. Figure 1A , the intraoral scanning system 101 includes a handheld wand 103 having at least one image sensor and a light source configured to emit light with wavelengths in the near infrared (near IR) range, and a display output (screen 102). The screen can be a touch screen used as a user input device, or the system can include a separate user input device (e.g., a keyboard, touchpad, joystick, mouse, trackball, etc.). Figure 1B As shown, the system may also include one or more processors operably connected to the handheld wand, the display, and the user input device. The one or more processors may include circuitry and / or software and / or firmware configured to: display a three-dimensional (3D) model of the patient's dental arch on a display output; display an observation window on a portion of the 3D model of the patient's dental arch on the display output; change a relative position between the observation window and the 3D model of the patient's dental arch based on input from the user input device; identify a near-infrared (near-IR) image taken at an angle and position close to the relative angle and position between the observation window and the 3D model of the patient's dental arch from the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch; and display the identified near-IR image (e.g., taken at an angle and position close to the angle and position of the observation window relative to the 3D model of the patient's dental arch). Figures 14A-16C shown).

[0210] Automatic characterization of tooth features

[0211] Also described herein are methods and apparatus (e.g., systems including software) configured to automatically or semi-automatically identify, confirm, and / or characterize one or more dental features using a 3D model (including, but not limited to, a volumetric 3D model) of all or a portion of a patient's dental arch. In particular, these methods and apparatus may be configured to identify, confirm, and / or characterize one or more actionable dental features that may benefit from detection and / or treatment. Actionable dental features may include, but are not limited to, cracks, gum recession, tartar, hard and soft tissue oral conditions, and the like. Enamel thickness may be another actionable dental feature. For example, the methods and apparatus described herein may automatically map enamel thickness (e.g., applying a color map where the enamel is less than x microns thick, where x may be pre-set and / or user-adjustable). Areas of thin enamel are potential areas where caries may be present. Other potential actionable dental features may include discoloration (e.g., color discontinuities), pits, cracks, signs of grinding (thinning, including over time), spaces between teeth, or any other similar features that indicate or suggest areas where caries may develop.

[0212] Any of the methods and devices described herein may use multiple different images or image sets of a patient's teeth taken in different imaging modes to detect, analyze and / or characterize dental features (particularly actionable dental features). Multiple different images or image sets of a patient's teeth taken in different imaging modes may be referred to as "recordings" respectively. Each record may be a different imaging mode, for example, a dental cone beam computed tomography (CBCT) scan, a three-dimensional (3D) intraoral scan, a color scan (one or more of a 3D color scan, a surface color scan, etc.), a two-dimensional (2D) color scan, a near infrared scan (including but not limited to one or more of the following: volumetric near infrared imaging, transmission irradiation and / or reflection scanning), an X-ray (including but not limited to: a cranial analysis X-ray scan, a panoramic X-ray scan, etc.), etc., and may include text or graphic chart information of the patient.

[0213] For example, initially, each recording can be processed independently. One or more dental features, particularly one or more actionable dental features, can be identified from this initial scan. A single recording (e.g., a single imaging modality) can be used to identify the one or more actionable dental features, or all or a subset of the recordings can be processed first to identify the one or more actionable dental features. The initial identification of the one or more actionable dental features can be performed manually, automatically, or semi-manually. For example, the one or more actionable dental features can be identified automatically. The system described herein can examine a recording (including one or more images of a patient's teeth) to mark or identify regions having features associated with actionable dental features. The system can be trained using machine learning techniques, such as supervised learning techniques (e.g., classification, regression, similarity, etc.), unsupervised learning techniques (e.g., density estimation, cluster analysis, etc.), reinforcement learning techniques (e.g., Markov decision process techniques, etc.), representation learning techniques, and / or principal component analysis, etc., to identify / mark regions of a particular scan in a particular modality that are associated (even loosely associated) with actionable dental features. Alternatively or additionally, a user (dental professional, technician, etc.) can manually review one or more records (each for a specific imaging mode) and can mark or identify areas suspected of displaying actionable dental features. In a semi-automatic configuration, initially, the system can mark one or more areas from the record, and the user can then review the one or more areas and confirm / reject them.

[0214] When one or more regions are identified, these regions may be marked and / or stored in a set of potentially actionable dental features. The location may be relative to the original record (e.g., a location on it) or relative to a reference model (e.g., a 3D volume model, as described in more detail below). In some variations, the set (e.g., an array, a data structure, a file, etc.) may also include one or more of the type of potentially actionable dental features, the degree of potentially actionable dental features, the grade and / or degree of potentially actionable dental features, the original record and / or the imaging mode of the original record, etc. In some variations, the data structure may be integrated into the original record (or a copy thereof), and the image of the original record may be modified, for example, by marking or indicia at the location of the identified potentially actionable dental features, and / or any metatext (e.g., grade and / or degree, etc.). The grade and / or degree may refer to a confidence level or score for the potentially actionable dental features, including a confidence level or score that the identified potentially actionable dental features are likely to be "real."

[0215] The initial identification process of identifying potentially actionable dental features may be performed across multiple records, or the initial identification process may be limited to a subset of records (eg, including only one of the records), as described above. In some variations, the process may be performed iteratively.

[0216] Once one or more potentially actionable dental features are identified, they can be cross-referenced to other one or more records using other imaging modalities. Thus, the location of the one or more potentially actionable dental features can be examined in particular detail to determine whether the same potentially actionable dental features are evident on these one or more additional records. In some variations, all of the additional records can be examined during this confirmation portion of the process, and any additional potentially actionable dental features from the additional one or more records can be similarly marked as potentially actionable dental features, and the same area of ​​the dental arch can be examined for these other potentially actionable dental features (including returning to an examined record, such as the first or original record).

[0217] By translating the positions of dental features (including but not limited to potentially actionable dental features) between different records, comparisons across other records can be guided. In particular, it may be helpful to coordinate the various dental records of the initial examination with a model of the patient's dental arch (e.g., any of the 3D models described above, particularly a 3D volumetric model). Thus, the 3D model of the dental arch can be used as a key to translate the positions of one or more potentially actionable dental features, and can allow for quick and efficient comparisons between different records (e.g., different imaging modalities).

[0218] Thus, before or after the initial scan of potentially actionable dental features, correlations can be established between the various records, and in particular, correlations between all or some of the various records and a 3D model of the dental arch (e.g., a 3D volume model). Any method for correlating records with other records and / or a 3D model of the patient's dental arch (or a portion of the dental arch) can be used. For example, one or more readily identifiable features (e.g., tooth edges, shapes, segmentations, etc.) can be used to determine landmarks that can be transformed between one or more records and / or between one or more records and a 3D model of the patient's dental arch. In certain variations, a transformation dataset can be created that includes transformations between the records and / or between each record and a 3D volume model of the patient's dental arch. For example, a 3D volume model of the entire dental arch or a portion of the dental arch can include transformation information for each of the one or more records, thereby allowing the image of one or more records to be transformed, such as an estimate of the distance and / or direction of the imaging mode relative to the recorded image. This allows forward and reverse transformations of positions between each record and the 3D model (e.g., a volume model).

[0219] Thus, the transformed dataset can include a 3D model and transformation information for each recording, so that a portion or region of the recorded image can be projected onto the 3D (transformed) model and then the same region can be projected back onto a second (or more) recording acquired in another imaging mode, thereby allowing the same region to be examined. In some variations, the process can begin by collecting all the recordings and / or automatically, manually, or semi-automatically performing registration between all the recordings. For example, the identification of individual regions such as teeth, palate, gums, etc. can be used to correlate between different imaging modes and / or 3D models. In one example, the recordings including these X-ray images can be associated with a 3D volume model of the patient's teeth by (manually or automatically) solving for the position and / or orientation of the X-ray camera used to capture the X-ray images corresponding to the recordings. The volume model can be used to determine and / or confirm the position and / or orientation of the imaging source for each recording. In some variations, the recordings include explicit (e.g., recorded) information about the position and / or orientation and / or imaging parameters used to capture the images; alternatively or additionally, this information can be derived. As described above, a pseudo X-ray image may be generated and compared with the recorded actual X-ray image.

[0220] Once an area corresponding to an area of ​​a potentially actionable dental feature is identified from another record, the system or method can determine whether the same potentially actionable dental feature exists in the other record. If so, the score (e.g., a confidence score showing the likelihood that the potentially actionable dental feature is true) can be adjusted. For example, if the same or similar potentially actionable dental feature exists, the score is increased. Depending on the type of record and the type of potentially actionable dental feature, the absence of a potentially actionable dental feature may result in an adjustment of the confidence score. For example, the absence of surface features that are not typically detectable by X-ray (e.g., discoloration, plaque, gum recession, etc.) may not result in a reduction in the confidence score of one or more potentially actionable dental features. The more times potentially actionable dental features are found in corresponding positions between different records (and therefore in different modes), the greater the likelihood that the potentially actionable dental feature actually exists.

[0221] When comparing the corresponding positions of one or more potentially actionable dental features, the area can be manually, automatically, or semi-automatically inspected, similar to the original recognition technique described above. For example, an area in an additional record corresponding to the location of a potentially actionable dental feature in another record can be automatically inspected to identify features associated with the type of potentially actionable dental feature. The system can be trained to identify potentially actionable dental features in the imaging mode of the additional record, and a score can be provided to indicate the likelihood that a potentially actionable dental feature is present at that location. In some variations, images from the additional record can be presented to a user (e.g., a technician, a dental professional, etc.), and the likelihood of the presence of a potentially actionable dental feature in one or more additional records can be manually indicated (yes / no, graded scale, numeric scale, etc.).

[0222] The final confidence value determined for each potentially actionable dental feature can be used by the system: stored, transmitted, and / or displayed. For example, the potentially actionable dental features can be presented to the dentist in any suitable manner, including in a list, on a display (e.g., on a 3D model of a marked dental arch), etc. For example, the system can output a display in which the locations of any or all potentially actionable dental features that exceed a threshold confidence level (are likely to be "true") are highlighted by color, shape, etc.; the display can also include one or more views (from one or more records) of the potentially actionable dental features. The user can set or adjust the threshold confidence level, including dynamically adjusting the threshold confidence level (e.g., making the threshold more or less stringent), and in response, the addition or removal of potentially actionable dental features is shown.

[0223] Figure 17 An example of a method 1700 for characterizing tooth features across different imaging modalities as just discussed is shown. Figure 17 In the method (or a system configured to perform the method) 1701, one or more actionable dental features may be identified from one or more records (e.g., one or more images or sets of images of a patient's teeth taken in different imaging modes). For example, the one or more actionable dental features may be identified by an agent or engine configured to automatically detect the one or more actionable dental features. For example, performing Figure 17The system of the method may include an operational dental feature analysis engine, or may include multiple operational dental feature analysis engines, each of which is configured to recognize one or more types of operational dental features or one or more types of imaging modes. The engine (e.g., an operational dental feature analysis engine) may be part of a computer system. As used herein, an engine may include one or more processors or a portion thereof. A portion of one or more processors may include some parts of hardware that are less than the entire hardware of any given one or more processors, such as a subset of registers, a portion of a processor of a multi-threaded processor dedicated to one or more threads, a time slice in which a processor is fully or partially dedicated to executing part of the functions of the engine, etc. In this way, the first engine and the second engine may have one or more dedicated processors, or the first engine and the second engine may share one or more processors with each other or with other engines. Depending on implementation-specific or other considerations, the engine may be centralized or its functions may be distributed. The engine may include hardware, firmware, or software contained in a computer-readable medium for execution by a processor. The processor converts data into new data using implemented data structures and methods, such as described with reference to the figures of this article.

[0224] The engine described herein or the engine by which the systems and devices described herein can be implemented can be a cloud-based engine. As used herein, a cloud-based engine is an engine that can run applications and / or functions using a cloud-based computing system. All or part of the applications and / or functions can be distributed among multiple computing devices and are not necessarily limited to only one computing device. In some embodiments, the cloud-based engine can execute functions and / or modules that an end user accesses through a web browser or container application without having to install the functions and / or modules locally on the end user's computing device.

[0225] return Figure 17 One or more actionable dental features can be manually or semi-manually identified from one or more records. For example, initially, an actionable dental feature analysis engine can identify one or more actionable dental features, which can then be verified or reviewed by a user (e.g., a dental technician).

[0226] Each identified operational dental feature 1703 can then be labeled and / or recorded, for example, in a set of potentially operational dental features. For example, the set of potentially operational dental features can be part of a data structure. Adding a potentially operational dental feature to a set (e.g., a data structure) can include recording the location of the operational dental feature (e.g., the location on the original record) and / or one or more of the following: the type of operational dental feature, the level / confidence of the operational dental feature, etc. As used herein, a data structure (which can be included as part of a data store) is intended to include a repository having any applicable data organization, including tables, comma-separated value (CSV) files, traditional databases (e.g., SQL), or other applicable known or convenient organizational formats. The data repository can be implemented, for example, as software embodied in a physical computer-readable medium on a dedicated machine, implemented in firmware, hardware, a combination thereof, or an applicable known or convenient device or system. Data repository-related components such as database interfaces may be considered "part of" the data repository, part of some other system component, or a combination thereof, although the physical location and other characteristics of the data repository-related components are not critical to understanding the techniques described herein.

[0227] A data structure can be associated with a specific way of storing and organizing data in a computer so that it can be used efficiently in a given context. Data structures are generally based on the ability of a computer to retrieve and store data at any location in its memory, where the location is specified by an address, which is a bit string that can itself be stored in memory and manipulated by a program. Thus, some data structures are based on calculating the addresses of data items using arithmetic operations; while other data structures are based on storing the addresses of data items within the structure itself. Many data structures use both principles, sometimes combined in non-general ways. The implementation of a data structure generally requires writing a set of procedures to create and manipulate instances of the structure. The data repository described herein can be a cloud-based data repository. A cloud-based data repository is a data repository that is compatible with cloud-based computing systems and engines.

[0228] The identified "putative" actionable dental features (e.g., "potentially actionable dental features") can be mapped to corresponding physical locations in one or more other records 1705. As described above, in some variations, this mapping can be accomplished using a 3D volume model, and conversion between various different types of records (with different imaging modalities) can be performed, including projecting a first record onto a 3D model and then onto a second area.

[0229] Thus, the same corresponding regions in other records can be examined to determine if potentially actionable dental features are present or suggested in the other records. In some variations, the method can simply collect all of the different corresponding regions for storage, transmission, and / or presentation to a user (e.g., a dental professional), for example, optionally stopping there and allowing the user to examine these marked regions from multiple different imaging modes (recordings) in parallel. For example, potentially actionable dental features can be shown for all corresponding views in a side-by-side (e.g., tiled) or sequential view format.

[0230] Alternatively or additionally, the method and / or system can automatically or semi-automatically adjust the confidence score for each potentially actionable dental feature identified. Thus, the system can determine whether other records indicate that the potentially actionable dental feature is more likely to be present or less likely to be present, and can adjust (or determine) the confidence score 1707 for each potentially actionable dental feature based on the corresponding positional occurrence in the other records.

[0231] The adjusted confidence level can then be used to narrow down the potentially actionable dental features. For example, the method or system can then filter and / or apply a threshold 1709 based on the adjusted confidence level for each potentially actionable dental feature. In some variations, the threshold can be fixed (e.g., a confidence level greater than x, where x is a number between 0 confidence and 1 (absolute confidence), e.g., 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, etc.). In some variations, the threshold can be manually adjusted by the user and / or can be based on one or more characteristics of the record (e.g., a quality metric specific to each record, etc.).

[0232] The potentially actionable dental features having a confidence level above the threshold may then be stored, presented, and / or transmitted 1711. For example, a final list and / or display (e.g., using a 3D model) of the marked potentially actionable dental features may be presented to the user.

[0233] Any of the methods and devices (e.g., systems) described herein can be configured to construct a data structure that includes all or a portion of a plurality of records. For example, as described above, the data structure can include a 3D volume model and all or some of the associated 2D images used to construct the model. In addition, the data structure can include additional records, such as images taken by X-ray (e.g., panoramic) and / or CBCT, etc. The data structure can also include metadata (e.g., information about the patient (including textual information)) and / or images, optionally including patient chart information from the patient's health / dental records. Alternatively or additionally, any identified potentially actionable dental features (e.g., findings identified from the records) can also be included. Potentially actionable dental features can be used to search / find / mark on the additional records.

[0234] Typically, when compiling images (e.g., 2D near-infrared images) to build a 3D (e.g., volumetric) model, the informative 2D images can be labeled to indicate their significance to the 3D model. For example, the 2D images can be labeled as irrelevant or more relevant.

[0235] As described above, the set of potentially actionable dental features (including confidence levels based on their presence in multiple records) can be included as part of the same data structure that includes the 3D model, or can be separate. The 3D model can be directly labeled (annotated, coded, etc.) to include potentially actionable dental features. Thus, the data structure can be a compilation of all the different records. The combined / compiled data structure can be referred to as a labeled data structure or an actionable dental feature data structure.

[0236] Any record (including near-infrared 2D images) can be used / scanned to identify potentially actionable dental features. As described above, when a suspicious area is automatically, semi-automatically / semi-manually, or manually (e.g., by a user) identified in one of the multiple records, the method or system can search for the corresponding area of ​​the dental arch across all or some of the other records and infer whether a finding exists. In some variations, the method or device can update the images across all or some of the records (and / or combined data structures) based on the analysis described herein.

[0237] In any of the methods and systems described herein, tooth segmentation can be used on all or some of the recordings and / or 3D models to enhance performance and usability. Tooth segmentation can be added prior to volume modeling to assist and improve volumetric results and model quality. For example, when additional surface 3D information is added, the volumetric 3D model can use the segmented information to potentially enhance performance. Segmentation information can also assist in segmenting enamel-dentin-lesions to improve automatic detection and suspicious area labeling (for example, including but not limited to when using automatic agents to identify potentially actionable tooth features). Alternatively or additionally, tooth segmentation can be added to volume modeling post-processing to assist in segmenting enamel-dentin-lesions to improve automatic detection and suspicious area labeling. For example, segmentation can also or alternatively help to associate structures between different imaging modes, including registering findings on the volume with other modes to provide cross-modal visualization. Tooth segmentation can be used to improve the recording and cross-modal visualization of clinical findings and annotations.

[0238] In any of the methods and devices described herein, the indicated confidence level can be a quantitative indicator and / or a qualitative indicator. For example, a quantitative confidence level "score" can be provided (e.g., using a number such as 0-100, 0 to 1.0, -100 to 100, or scaling to any numerical range). Qualitative indicators can include "high, medium-high, medium, medium-low, low" and the like. Both qualitative and quantitative confidence levels can be used. A grading system for confidence levels based on multiple records as described herein may have an impact on insurance claims and / or patient communication.

[0239] In any of the methods and systems described herein, the morphology of the dental arch can be used to help identify possible areas of concern or potential problems. Thus, generally, as described herein, a 3D model (volume model) can be used and / or modified to include areas of potential actionable dental features. The modified 3D model can be used as a map that visually indicates areas requiring risk assessment; for example, this can be used to guide a patient's treatment, including promoting the use of sealants, orthodontic treatment, or night guard, etc. In certain variations, a modified 3D model can be used to guide the user when additional scans are required (e.g., when the number of scans in the risk area is low). As used herein, a modified 3D model can include a 3D (e.g., volume and / or surface) model that has been marked to indicate the location and / or type and / or confidence level of potentially actionable dental features. Thus, generally, using additional data sources to guide the user to capture potential areas of concern (e.g., when they appear in a recording, particularly in a recording other than a near infrared / NIRI scan) can help confirm the discovery of potentially actionable dental features. As previously described, the results, including the modified 3D model, can help guide the user in scanning or rescanning (at a future time) the user's dentition. For example, historical scans can be used as a target map while scanning (and for confirming adequate coverage in those areas). Additionally or alternatively, one or more exported images / representations can be used. For example, the tooth segmentation can be used to generate a dental chart (e.g., from the 3D volume model) that can be used for follow-up and automatically imported into dental practice management software (DPMS). For example, individual records can be aligned to match a specified problem with a dental chart.

[0240] Any method described herein (including user interfaces) may be implemented as software, hardware, or firmware and may be described as a non-transitory computer-readable storage medium storing a set of instructions executable by a processor (e.g., a computer, tablet, smartphone, etc.) that, when executed by the processor, causes the processor to control the execution of any of the following steps, including but not limited to: displaying, communicating with a user, analyzing, modifying parameters (including timing, frequency, intensity, etc.), determining, issuing an alarm, etc.

[0241] When a feature or element is referred to as being "on" another feature or element in this article, it can be directly located on the other feature or element, and / or there may also be intermediate features and / or elements. On the contrary, when a feature or element is referred to as being "directly on" another feature or element, there are no intermediate features or elements. It should also be understood that when a feature or element is referred to as being "connected," "attached" or "coupled" to another feature or element, it can be directly connected, attached or coupled to the other feature or element, or there may be intermediate features or elements. On the contrary, when a feature or element is referred to as being "directly connected," "directly attached" or "directly coupled" to another feature or element, there are no intermediate features or elements. Although described or shown with respect to one embodiment, the features and elements described or shown in this way can be applied to other embodiments. Those skilled in the art will also recognize that the reference to the structure or feature provided "adjacent" another feature can have a portion overlapping with the adjacent features or located below the adjacent features.

[0242] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the present invention. For example, unless the context clearly indicates otherwise, as used herein, the singular forms "a", "an" and "the" are intended to also include the plural forms. It should be further understood that the terms "include" and / or "comprise" when used in this specification specify the presence of stated features, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components and / or their groups. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0243] For ease of description, spatially related terms such as "below", "below", "lower than", "above" and "upper" can be used in this article to describe the relationship between an element or feature and another (or multiple) element or feature as shown in the drawings. It will be understood that spatially related terms are intended to include different orientations of the device in use or operation in addition to the orientation depicted in the drawings. For example, if the device in the drawings is inverted, the elements described as "below" or "below" other elements or features will be oriented to be "above" other elements or features. Therefore, the exemplary term "below" can include both above and below orientations. The device can be oriented otherwise (rotated 90 degrees or in other orientations), and the spatially relative descriptors used herein are interpreted accordingly. Similarly, unless otherwise specifically stated, the terms "upward", "downward", "vertical", "horizontal" etc. are used herein for illustrative purposes only.

[0244] Although the terms "first" and "second" may be used herein to describe various features / elements (including steps), these features / elements should not be limited by these terms unless the context indicates otherwise. These terms can be used to distinguish one feature / element from another feature / element. Thus, without departing from the teachings of the present invention, a first feature / element discussed below may be referred to as a second feature / element, and similarly, a second feature element discussed below may be referred to as a first feature / element.

[0245] In this specification and the appended claims, unless the context requires otherwise, the word "comprise" and variations such as "include" and "have" mean that various components can be used together in methods and articles of manufacture (e.g., compositions and apparatus including devices and methods). For example, the term "comprising" will be understood to imply the inclusion of any stated elements or steps but not the exclusion of any other elements or steps.

[0246] In general, any apparatus and method described herein should be understood to be inclusive, but all or a subset of components and / or steps may optionally be exclusive and may be expressed as "consisting of," or alternatively, "consisting essentially of," the various components, steps, subcomponents, or sub-steps.

[0247] As used herein in the specification and claims, including in the examples, unless otherwise expressly stated, all numbers may be understood as if beginning with the word "about" or "approximately", even if the term does not explicitly appear. The phrases "about" or "approximately" may be used when describing an amplitude and / or position to indicate that the value and / or position described is within a reasonable expected range of values ​​and / or positions. For example, a numerical value may have a value of + / -0.1% of the value (or range of values), + / -1% of the value (or range of values), + / -2% of the value (or range of values), + / -5% of the value (or range of values), + / -10% of the value (or range of values), etc. Any numerical value given herein should also be understood to include approximately that value or approximately that value, unless the context indicates otherwise. For example, if the value "10" is disclosed, "approximately 10" is also disclosed. Any numerical range recited herein is intended to include all subranges contained therein. It should also be understood that when a value is disclosed, "less than or equal to" that value, "greater than or equal to that value," and possible ranges between values ​​are also disclosed, as appropriately understood by one skilled in the art. For example, if a value "X" is disclosed, then "less than or equal to X" and "greater than or equal to X" (e.g., where X is a numerical value) are also disclosed. It should also be understood that throughout this application, data is provided in a variety of different formats, and that the data represents ending points and starting points, as well as ranges for any combination of data points. For example, if a specific data point "10" and a specific data point "15" are disclosed, it should be understood that greater than, greater than or equal to, less than, less than or equal to, and equal to 10 and 15, as well as between 10 and 15, are considered disclosed. It should also be understood that every unit between two specific units is also disclosed. For example, if 10 and 15 are disclosed, then 11, 12, 13, and 14 are also disclosed.

[0248] Although various illustrative embodiments have been described above, any of several changes may be made to the various embodiments without departing from the scope of the invention as described in the claims. For example, in alternative embodiments, the order in which the various described method steps are performed may generally be changed, and in other alternative embodiments, one or more method steps may be skipped altogether. Optional features of the various apparatus and system embodiments may be included in some embodiments and not included in other embodiments. Therefore, the foregoing description is provided primarily for exemplary purposes and should not be construed as limiting the scope of the present disclosure as set forth in the claims.

[0249] The examples and descriptions included herein show, by way of illustration and not limitation, specific embodiments in which the subject matter can be practiced. As mentioned, other embodiments can be utilized and derived therefrom, so that structural and logical replacements and changes can be made without departing from the scope of the present disclosure. For convenience only, the term "invention" may be used herein, individually or collectively, to refer to these embodiments of the subject matter of the present invention, and it is not intended that the scope of the present application be actively limited to any single invention or inventive concept, if in fact more than one invention or inventive concept is disclosed. Therefore, although specific embodiments have been illustrated and described herein, it is expected that any arrangement for achieving the same purpose can replace the specific embodiments shown. The present disclosure is intended to cover any and all modifications or variations of the various embodiments. By reading the above description, the combination of the above embodiments and other embodiments not specifically described herein will be apparent to those skilled in the art.

Claims

1. An intraoral scanning system, comprising: a handheld wand having at least one image sensor and a light source configured to emit light having a spectral range in the near infrared wavelength range; display output device; user input devices; as well as One or more processors operably connected to the handheld wand, the display, and the user input device, the one or more processors being configured to: displaying a three-dimensional model of the patient's dental arch on the display output device; displaying a viewing window on the portion of the three-dimensional model of the patient's dental arch on the display output device, wherein the viewing window displayed on the portion of the three-dimensional model of the patient's dental arch includes a display ring through which the portion of the three-dimensional model of the patient's dental arch can be viewed; changing the relative position between the viewing window and the three-dimensional model of the patient's dental arch based on input from the user input device; and A near infrared image is displayed that is associated with the angle and position of the viewing window relative to the three-dimensional model of the patient's dental arch.

2. The intraoral scanning system according to claim 1, wherein: The one or more processors are configured to receive a plurality of images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch.

3. The intraoral scanning system according to claim 1, wherein: The one or more processors are configured to continuously identify the near infrared image and display the near infrared image as a user changes a relative position between the viewing window and the three-dimensional model of the patient's dental arch.

4. The intraoral scanning system according to claim 1, wherein: The one or more processors are configured to change the relative position between the observation window and the three-dimensional model of the patient's dental arch based on input from the user input device by changing one or more of the following: the angle between the plane of the observation window and the patient's dental arch, and the portion of the patient's dental arch adjacent to the observation window.

5. The intraoral scanning system according to claim 1, wherein: The one or more processors are configured to identify a near infrared image taken at an angle and position that approximates a relative angle and position of the viewing window relative to the three-dimensional model of the patient's dental arch.

6. The intraoral scanning system according to claim 1, wherein: The one or more processors are configured to change the relative position between the observation window and the three-dimensional model of the patient's dental arch based on input from the user input device by changing one or more of the following: the angle of the three-dimensional model of the patient's dental arch relative to the observation window, the rotation of the three-dimensional model of the patient's dental arch relative to the observation window, and the portion of the patient's dental arch adjacent to the observation window.

7. The intraoral scanning system according to claim 1, wherein: The one or more processors are configured to identify an image from the three-dimensional model of the patient's dental arch and a plurality of images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch, the image approximating the relative angle and position of the viewing window relative to the three-dimensional model of the patient's dental arch.

8. A non-transitory computer-readable medium storing a set of instructions executable by a processor, the set of instructions, when executed by the processor, causing the processor to perform a method comprising: displaying a three-dimensional model of the patient's dental arch on a display output device; displaying a viewing window on the portion of the three-dimensional model of the patient's dental arch on the display output device, wherein the viewing window displayed on the portion of the three-dimensional model of the patient's dental arch includes a display ring through which the portion of the three-dimensional model of the patient's dental arch can be viewed; changing the relative position between the viewing window and the three-dimensional model of the patient's dental arch based on input from a user input device; and A near infrared image is displayed that is associated with the angle and position of the viewing window relative to the three-dimensional model of the patient's dental arch.

9. The non-transitory computer-readable medium of claim 8, wherein: The near infrared image is continuously displayed when a user changes the relative position between the viewing window and the three-dimensional model of the patient's dental arch.

10. The non-transitory computer-readable medium of claim 8, wherein: Changing the relative position between the viewing window and the three-dimensional model of the patient's dental arch includes allowing a user to change one or more of: the angle between the plane of the viewing window and the patient's dental arch, and the portion of the patient's dental arch adjacent to the viewing window.

11. The non-transitory computer-readable medium of claim 8, wherein: The instruction set is configured to execute the method, which also includes: determining multiple images close to the relative angle and position of the observation window relative to the three-dimensional model of the patient's dental arch, and averaging the multiple images to form a near-infrared image, which is associated with the angle and position of the observation window relative to the three-dimensional model of the patient's dental arch.

12. The non-transitory computer-readable medium of claim 8, wherein: The instruction set is configured to perform the method, which also includes: collecting a data set in a processor, the data set including a three-dimensional model of the patient's dental arch and multiple images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch.

13. The non-transitory computer-readable medium of claim 8, wherein: Changing the relative position between the observation window and the three-dimensional model of the patient's dental arch includes controlling one or more of the following: the angle of the three-dimensional model of the patient's dental arch, the rotation of the three-dimensional model of the patient's dental arch, and the portion of the patient's dental arch adjacent to the observation window.

14. The non-transitory computer-readable medium of claim 8, wherein: The instruction set is configured to perform the method, which further includes: identifying a second image from the three-dimensional model of the patient's dental arch and a plurality of images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch, the second image approximating the relative angle and position between the viewing window and the three-dimensional model of the patient's dental arch, comprising one of: a visible light image and a fluorescent image; and The second image is displayed simultaneously with the near-infrared image.

15. The non-transitory computer-readable medium of claim 8, wherein: Displaying the near infrared image approximating the angle and position of the viewing window relative to the displayed three-dimensional model includes displaying the near infrared image in a window adjacent to or overlapping a display of the three-dimensional model of the patient's dental arch.

Images