Oral diagnostic scans and tracking

Abstract

This is a method and device for taking, using, and displaying a three-dimensional (3D) volumetric model of a patient's dental arch. The 3D volumetric model may include information about internal structures, such as near-infrared (near-IR) transparency values ​​for internal structures including enamel and dentin, as well as surface information (e.g., color).

Description

Technology Field

[0001] This patent application claims priority to U.S. Provisional Application No. 62 / 622,798, titled “DIAGNOSTIC INTRAORAL SCANNERS,” filed January 26, 2018, and U.S. Provisional Application No. 62 / 758,503, titled “DIAGNOSTIC INTRAORAL SCANNERS,” filed November 9, 2018, each of which is incorporated herein by reference in its entirety.

[0002] This patent application may be related to one or more of the following: U.S. patent application No. 15 / 662,234 titled “INTRAORAL SCANNER WITH DENTAL DIAGNOSTICS CAPABILITIES” filed July 27, 2017, claiming priority to U.S. provisional patent applications No. 62 / 367,607 (filed July 27, 2016) and 62 / 477,387 (filed March 27, 2017); U.S. Patent Application No. 15 / 662,250, titled “METHODS AND APPARATUSES FOR FORMING A THREE-DIMENSIONAL VOLUMETRIC MODEL OF A SUBJECT'S TEETH,” filed on July 27, 2017, claiming priority to U.S. Provisional Patent Application Nos. 62 / 367,607 (filed July 27, 2016) and 62 / 477,387 (filed March 27, 2017); and U.S. patent application No. 15 / 672,248, titled “METHODS FOR DENTAL DIAGNOSTICS,” filed August 8, 2017, claiming priority to U.S. provisional patent applications No. 62 / 367,607 (filed July 27, 2016) and 62 / 477,387 (filed March 27, 2017). Each of these applications is incorporated herein by reference in its entirety.

[0003] All publications and patent applications mentioned in this specification are incorporated herein by reference in their entirety to the same extent that each individual publication or patent application is specifically and individually included by reference. Background Technology

[0004] Many dental and orthodontic procedures can benefit from an accurate three-dimensional (3D) representation of the patient's dentition and oral cavity. In particular, providing a 3D representation of the internal structure of the tooth, including the tooth surface, enamel, and dentin, as well as caries and the general internal composition of the tooth volume, would be helpful. While a pure surface representation of the tooth's 3D surface has proven to be very useful for the design, fabrication, and treatment planning of dental prosthetics (e.g., crowns or bridges), the ability to image the internal structure, including the occurrence of caries and cracks in the enamel and underlying dentin, is particularly useful when combined with surface topography mapping.

[0005] Historically, ionizing radiation (e.g., X-rays) has been used to form images of teeth. For instance, X-ray occlusal radiograms are often used to provide non-quantitative images of teeth. However, in addition to the risks associated with ionizing radiation, these images generally have limited ability to display features and can involve time-consuming and costly procedures. Some intraoral features, such as soft tissues, plaque, and soft calculus, may not be easily visualized via X-rays due to their low density. Other techniques, such as cone-beam computed tomography (CBCT), can provide tomographic images but still require ionizing radiation.

[0006] Therefore, it would be beneficial to provide a method and apparatus including a device and a system, such as an intraoral scanning system that can be used to model a subject's teeth or teeth and includes both external (surface) and internal (within enamel and dentin) structures and configurations using non-ionizing radiation. Prior art literature

[65535] Patent Document 1: U.S. Patent Application Publication US2014 / 0272764

[0007] Methods and apparatus for obtaining, using, and displaying dental information including information extracted from a three-dimensional (3D) volumetric model of a patient's dental arch are described herein. The 3D volumetric model may include information regarding internal structures, such as near-infrared (near-IR) transparency values ​​for internal structures including enamel and dentin, as well as surface (e.g., color) information. In some variations, the 3D volumetric scan may include or be derived from one or more other scanning techniques, including but not limited to optical coherence tomography (OCT), ultrasound (US), magnetic resonance imaging (MRI), X-rays, etc.

[0008] In particular, methods and user interfaces for displaying and manipulating a 3D volumetric model (e.g., sectioning, marking, sub-region selection, etc.) are described in this specification. For example, methods and apparatus for displaying images from a 3D volumetric model are provided, including a method for generating a cross-section through a 3D volumetric model, a method for showing both surface and internal structures, and a method for generating images from a 3D volumetric model that are easy to interpret, such as pseudo-X-ray images.

[0009] Additionally, methods and apparatus for marking and tracking regions of interest from a 3D volumetric model of a patient's dental arch are described herein. Such methods may include methods for identifying and marking one or more regions within the 3D volumetric model automatically, manually, or semi-automatically (e.g., by user approval or input) (including surface features and / or internal features of the dental arch); these regions may be areas where caries, fissures, or other irregularities have occurred or may occur. The marked regions may be analyzed in more detail and tracked over time. Additionally, the marked regions may modify the way subsequent scans are performed, for example, by scanning the marked regions at a higher resolution. Regions of the volumetric model may correspond to one or more voxels containing continuous voxel regions. These regions may be referred to as volumetric regions herein.

[0010] Additionally, methods and apparatus for using a 3D volumetric model to improve or modify dental procedures, including modifying treatment plans and / or modifying one or more dental devices, are described herein. For example, what is described herein is a dental tool that may include 3D volumetric scanning or may work with a 3D volumetric model (including a robot or automatic control using the 3D volumetric model). Methods for diagnosing one or more conditions (e.g., dental conditions) using a 3D volumetric model, particularly using a 3D volumetric model over time, are also described.

[0011] A method for displaying an image from a three-dimensional (3D) volumetric model of a patient's dental arch, the method comprises: collecting a three-dimensional volumetric model of a patient's dental arch—the three-dimensional volumetric model including near-infrared (near-IR) transparency values ​​and shade values ​​for internal structures within the dental arch, and surface color values; selecting a viewing direction of the three-dimensional volumetric model to be displayed by a user; generating a two-dimensional (2D) view of the three-dimensional volume using the selected direction including a weighted portion of the surface color values ​​and a weighted portion of the near-infrared transparency of internal structures of the patient's dental arch; and displaying the 2D view.

[0012] For example, a method for displaying an image from a three-dimensional (3D) volumetric model of a patient's dental arch is described herein. This method may include: receiving a three-dimensional volumetric model of a patient's dental arch—said that the three-dimensional volumetric model includes near-infrared (near-IR) transparency values ​​and surface color values ​​for internal structures within the dental arch; and generating a two-dimensional (2D) view through the three-dimensional volumetric model including the patient's dental arch, which includes both surface color values ​​and near-infrared transparency of internal structures. In any of the methods and apparatus described herein, the 3D model (including the volumetric 3D model) may be displayed as a voxel view. Thus, the method described herein may generate one or more voxel views in which each voxel may have a color (or hue) corresponding to density and / or translucency. Accordingly, one of the methods and apparatus described herein may generate a 3D color map of all or part of the voxels of a 3D model (and display one or more 2D images derived from the 3D color view, such as a cross-section, slice, projection, perspective view, or a transparent view in which all or part of the 3D model is rendered transparently). In some variations, flagged areas (e.g., areas corresponding to one or more abnormal areas, and / or areas, e.g., voxels that have changed over time, areas / voxels that need to be removed, areas / voxels suspected of being problematic, etc.) may be displayed in 3D and / or 2D views.

[0013] The step of generating a two-dimensional (2D) view through a 3D volume may include, in the 2D view, a weighted portion of surface color values ​​and a weighted portion of near-infrared transparency of the internal structure. Near-infrared transparency may be based on near-infrared scattering or absorption by the material, or otherwise calculated from it. The weighted portion of surface color values ​​may include a percentage of the total value of surface color values, and the weighted portion of near-infrared transparency of the internal structure includes a percentage of the total value of near-infrared transparency of the internal structure, wherein the percentage of the total value of surface color values ​​and the percentage of the total value of near-infrared transparency of the internal structure are at most 100%.

[0014] In some variations, this method also includes a step of adjusting a weighted portion of the surface color value and / or the near-infrared transparency of the internal structure by or in response to user input.

[0015] Any of these methods may include a step of scanning the patient's dental arch with an oral scanner.

[0016] Generating a 2D view may involve cross-sectioning the 3D volumetric model in a plane through the 3D volumetric model. The user can select the orientation of the cross-section and / or 2D view to be displayed through the 3D volumetric model.

[0017] For example, a method for displaying an image from a three-dimensional (3D) volumetric model of a patient's dental arch may include: receiving a three-dimensional volumetric model of a patient's dental arch—said that the three-dimensional volumetric model includes near-infrared (near-IR) transparency values ​​and surface color values ​​for internal structures within the dental arch; selecting a cross-section to be displayed through the three-dimensional volumetric model by a user or in response to user input; generating a two-dimensional (2D) view through the three-dimensional volumetric model using the selected cross-section which includes the patient's dental arch and also includes a weighted portion of surface color values ​​and a weighted portion of near-infrared transparency of internal structures; and displaying the 2D view.

[0018] A method for displaying an image from a three-dimensional (3D) volumetric model of a patient's dental arch may include the following: collecting a three-dimensional volumetric model of a patient's dental arch—said that the three-dimensional volumetric model includes near-infrared (near-IR) transparency values ​​and surface values ​​for internal structures within the dental arch; generating a two-dimensional (2D) view of the three-dimensional volumetric model containing both near-infrared transparency and surface values ​​of the internal structures; and displaying the 2D view.

[0019] A method for tracking an area of ​​a patient's dental arch over time includes the following: receiving a first three-dimensional (3D) volumetric model of the patient's dental arch—the 3D volumetric model includes near-infrared (near-IR) transparency values ​​and surface color values ​​for internal structures within the dental arch—; identifying an area within the 3D volumetric model to be marked; flagging the identified area; and displaying one or more images of the 3D volumetric model representing the marked area.

[0020] For example, as a method for tracking a region of a patient's dental arch over time, the method comprises: collecting a first three-dimensional (3D) volumetric model of the patient's dental arch—said that the 3D volumetric model includes near-infrared (near-IR) transparency values ​​and surface values ​​for internal structures within the dental arch—; identifying a region of the 3D volumetric model; flagging the identified region; collecting a second 3D volumetric model of the patient's dental arch; and displaying one or more images on which the difference between the first 3D volumetric model and the second 3D volumetric model in the flagged region is marked.

[0021] Identifying regions may include identifying them automatically using a processor. For example, automatic identification may include identifying regions with potential defects, including cracks and cavities. Identifying regions with potential defects may include comparing the near-infrared transparency values ​​of regions within a 3D model to a threshold. Automatic identification may include identifying surface color values ​​outside the threshold range. Automatic identification may include segmenting a 3D volumetric model to identify enamel regions and identifying regions with enamel thickness below a threshold. Flagging identified regions may include automatically flagging the identified regions. Flagging identified regions may include manually verifying the identified regions for flagging.

[0022] Any of these methods may include receiving a second 3D volumetric model of the patient's dental arch and displaying the difference between the first 3D volumetric model and the second 3D volumetric model in a marked area.

[0023] Additionally, any of these methods may include pre-scanning or re-scanning the patient's dental arch, wherein the flagged area is scanned at a higher resolution or with a different scanning technique than the unflagged area.

[0024] For example, a method for tracking an area of ​​a patient's dental arch over time may include: collecting a first three-dimensional (3D) volumetric model of the patient's dental arch—said that the 3D volumetric model includes near-infrared (near-IR) transparency values ​​and surface color values ​​for internal structures within the dental arch—using an automated process to identify an area to be marked within the 3D volumetric model; flagging the identified area; receiving a second 3D volumetric model of the patient's dental arch; and displaying the difference between the first 3D volumetric model and the second 3D volumetric model in the marked area. In some examples, the second 3D volumetric model of the patient's dental arch may be derived from a scan of the patient during a subsequent visit to the dental clinic.

[0025] Accordingly, a method for tracking an area of ​​a patient's dental arch over time may include: a step of collecting a first three-dimensional (3D) volumetric model of the patient's dental arch acquired at a first time point—said that the 3D volumetric model includes near-infrared (near-IR) transparency values ​​and surface color values ​​for internal structures within the dental arch—; a step of identifying an area to be flagged within the 3D volumetric model using an automatic process; a step of flagging the identified area; a step of collecting a second 3D volumetric model of the patient's dental arch acquired at a different time point; and a step of displaying the difference between the first 3D volumetric model and the second 3D volumetric model in the flagged area.

[0026] Additionally, a method for displaying a pseudo-X-ray image from a three-dimensional (3D) volumetric model of a patient's dental arch is described herein. For example, the method may include: receiving a 3D volumetric model of a patient's dental arch—said that the 3D volumetric model includes near-infrared (near-IR) transparency values ​​for internal structures within the dental arch; generating a two-dimensional (2D) view through the 3D volumetric model including the patient's dental arch including near-infrared transparency of the internal structures; mapping the near-infrared transparency of the internal structures in the 2D view to a pseudo-X-ray density in which the near-infrared transparency values ​​are inverted; and displaying the mapped pseudo-X-ray density. Generating the 2D view may include cross-sectionally cutting the 3D volumetric model in a plane through the 3D volumetric model. The 3D volumetric model may include surface information.

[0027] For example, a method for displaying a pseudo-X-ray image from a three-dimensional (3D) volumetric model of a patient's dental arch may include: collecting a three-dimensional volumetric model of the patient's dental arch—said that the three-dimensional volumetric model includes near-infrared (near-IR) transparency values ​​for internal structures within the dental arch; generating a two-dimensional (2D) view of the three-dimensional volumetric model containing the patient's dental arch including the near-infrared transparency of the internal structures; mapping the near-infrared transparency of the internal structures in the 2D view to a pseudo-X-ray density—said that the pseudo-X-ray density values ​​in the 2D view are based on near-infrared transparency values ​​with inverted values; and displaying the mapped pseudo-X-ray density.

[0028] Any of these methods may include identifying sub-regions from a 3D volumetric model before generating a 2D view, wherein the 2D view includes a 2D view of the identified sub-region. The method may also include the step of segmenting the 3D volumetric model into multiple teeth, wherein generating the 2D view may include a 2D view containing only one of the identified teeth.

[0029] Mapping near-infrared transparency may involve inverting near-infrared transparency values ​​so that the enamel in the 2D view is brighter than the dentin in the 2D view.

[0030] A method for displaying pseudo-X-ray images from a three-dimensional (3D) volumetric model of a patient's dental arch may include: receiving a three-dimensional volumetric model of a patient's dental arch—said that the three-dimensional volumetric model includes near-infrared (near-IR) transparency values ​​and surface features for internal structures within the dental arch where enamel is more transparent than dentin; generating a two-dimensional (2D) view through the three-dimensional volume of the patient's dental arch including near-infrared transparency of internal structures including dentin and enamel; mapping the near-infrared transparency of internal structures in the two-dimensional view—said that the near-infrared transparency values ​​are inverted so that enamel is brighter than dentin—into a pseudo-X-ray density; and displaying the mapped pseudo-X-ray density.

[0031] For example, a method for displaying pseudo-X-ray images from a three-dimensional (3D) volumetric model of a patient's dental arch may include: collecting a three-dimensional volumetric model of the patient's dental arch—said that the three-dimensional volumetric model includes near-infrared (near-IR) transparency values ​​and surface features for internal structures within the dental arch where enamel is more transparent than dentin; generating a two-dimensional (2D) view of the patient's dental arch containing near-infrared transparency of internal structures including dentin and enamel; mapping the near-infrared transparency of internal structures in the 2D view—said that the near-infrared transparency values ​​are inverted so that enamel is brighter than dentin—to pseudo-X-ray density; and displaying the mapped pseudo-X-ray density.

[0032] Additionally, methods and apparatus for a real-time virtual review (e.g., virtual cross-section, virtual scanning, virtual inspection) of a volumetric model of a patient's dental arch(s) are described herein. These apparatus may include a non-transient machine-readable type medium that stores instructions for one or more machines to perform actions to carry out any of the methods described herein. In particular, any of these methods and apparatus may operate on a dataset containing a 3D model of the patient's dental arch or, in some variations, the patient's entire dental arch. The 3D model may be a 3D volumetric model, but is not limited thereto: in some variations, the 3D model is a 3D surface model of the arch. This dataset may also include multiple images of the dental arch taken from different positions relative to the arch, such as different angles between the plane of the image and the dental arch and different sub-regions of the dental arch. Some of these images may be taken from the occlusal plane, some from the gingival side, and some from the lingual side. In some variations, the images may be the same as (or a subset thereof) of the images used to form the 3D model of the teeth. The dataset may include multiple images taken from the same or nearly identical area and angle of the dental arch. In some variations, the dataset may include two or more sets of images (e.g., image pairs) each taken from approximately the same area of ​​the dental arch and the same angle of the dental arch but using different imaging techniques (e.g., various imaging techniques such as visible light, IR / near-infrared, fluorescence, X-ray, ultrasound, etc.).

[0033] For example, the method may include: displaying a three-dimensional (3D) model of a patient's dental arch; displaying a view window over at least a portion of the 3D model of the patient's dental arch; enabling a user to change the relative position between the view window and the 3D model of the patient's dental arch; subsequently, as the user changes the relative position between the view window and the 3D model of the patient's dental arch, identifying an image taken from both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken at different angles and positions relative to the 3D model of the patient's dental arch and the corresponding view window, an image taken at an angle and position close to the relative angle and position between the 3D model of the patient's dental arch and the relevant view window; and displaying the identified image taken at an angle and position close to the angle and position of the view window relative to the 3D model of the patient's dental arch.

[0034] In any of the methods described herein, the dataset may include a 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch. The dataset may also, or alternatively, include metadata associated with each (or each set) of drawings indicating the angle and / or area of ​​the dental arch from which the images were taken. Additional metadata may be included (e.g., indication of distance from the dental arch, indication of exposure time, indication that an image is an average of other images, quality metrics for the images, etc.).

[0035] For example, the method described herein is a method for displaying a 3D model of a patient's teeth (e.g., a surface 3D model and / or a volumetric model of the patient's teeth) that a user can virtually scan in more detail by moving a view window over a 3D model of the dental arch. For example, the method described herein may include: a step of displaying a three-dimensional (3D) model of the patient's dental arch; a step of displaying a view window over a portion of the 3D model of the patient's dental arch; a step of enabling the user to change the relative position between the view window and the 3D model of the patient's dental arch, including an angle between the plane of the view window and the patient's dental arch, and one or more portions of the dental arch adjacent to the view window. And continuously, as the user changes the relative position between the view window and the 3D model of the patient's dental arch, the method identifies an image taken from both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch—each image taken from a different angle and position relative to the patient's dental arch—(e.g., from a dataset containing both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch in some variation) that is close to the relative angle and position between the 3D model of the patient's dental arch and the relevant view window; and the method displays the identified image taken from an angle and position close to the angle and position of the view window relative to the displayed 3D model of the patient's dental arch.

[0036] For example, the method may include: displaying a three-dimensional (3D) model of a patient's dental arch; displaying a view window over at least a portion of the 3D model of the patient's dental arch; enabling a user to change the relative position between the view window and the 3D model of the patient's dental arch, including an angle between the plane of the view window and the patient's dental arch, and one or more portions of the dental arch adjacent to the view window; and subsequently, as the user changes the relative position between the view window and the 3D model of the patient's dental arch, identifying a pair of images taken from both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch—each pair of said plurality of pairs comprising a first imaging wavelength and a second imaging wavelength respectively taken at the same angle and position for the patient's dental arch—from both (e.g., optionally from a dataset including both the 3D model of the patient's dental arch and a plurality of images for the patient's dental arch), a pair of images taken at an angle and position close to the angle and position of the view window for the displayed 3D model of the patient's dental arch; and displaying at least one pair of identified images taken at an angle and position close to the angle and position of the view window for the displayed 3D model of the patient's dental arch.

[0037] The method and apparatus described herein may be used with a surface model or a 3D model which is any representation of the patient's dental arch(s). For example, it may be, but not necessarily, a 3D volumetric model of the patient's teeth constructed from images (e.g., multiple images of the patient's dental arch taken at different angles and positions relative to the patient's dental arch). The model may represent a generic or actual tooth of the patient drawn from the patient's teeth.

[0038] As described herein, the method may include: displaying a three-dimensional (3D) model of a patient's dental arch; displaying a view window over at least a portion of the 3D model of the patient's dental arch; enabling a user to change the relative position between the view window and the 3D model of the patient's dental arch, including an angle between the view window and the patient's dental arch and one or more portions of the dental arch adjacent to the view window; and subsequently, as the user changes the relative position between the view window and the 3D model of the patient's dental arch, identifying a near-infrared image taken from both the 3D model of the patient's dental arch and a plurality of near-infrared images of the patient's dental arch—each of which is taken from a different angle and position relative to the patient's dental arch (e.g., from a dataset containing both the 3D model of the patient's dental arch and the plurality of images of the patient's dental arch) that is close to the relative angle and position between the 3D model of the patient's dental arch and the relevant view window. and displaying identified near-infrared images taken at angles and positions close to the angle and position of the view window for the displayed 3D model of the patient's dental arch.

[0039] In any of these examples, the image may be an image taken by a transmission technique such as near-infrared. For example, the method described herein includes: displaying a three-dimensional (3D) model of a patient's dental arch; displaying a view window over at least a portion of the 3D model of the patient's dental arch; enabling a user to change the relative position between the view window and the 3D model of the patient's dental arch, including an angle between the view window and the patient's dental arch and one or more portions of the dental arch adjacent to the view window; and subsequently, as the user changes the relative position between the view window and the 3D model of the patient's dental arch, identifying a near-infrared image taken at an angle and position close to the relative angle and position between the 3D model of the patient's dental arch and the relevant view window from a dataset comprising both the 3D model of the patient's dental arch and a plurality of near-infrared images of the patient's dental arch—each near-infrared image taken at a different angle and position relative to the patient's dental arch. and displaying identified near-infrared images taken at angles and positions close to the angle and position of the view window for the displayed 3D model of the patient's dental arch.

[0040] Any of these methods may also include identifying and displaying a plurality of images taken at the same angle and position with respect to the dental arch. For example, the images may be visible light images and transmitted images (e.g., IR / near-infrared images, etc.). For example, the method described herein includes: displaying a three-dimensional (3D) model of the patient's dental arch; displaying a view window over at least a portion of the 3D model of the patient's dental arch; enabling a user to change the relative position between the view window and the 3D model of the patient's dental arch, including the angle between the plane of the view window and the patient's dental arch, and one or more portions of the dental arch adjacent to the view window. And, as the user continuously changes the relative position between the view window and the 3D model of the patient's dental arch, the method comprises: identifying a pair of images taken at an angle and position close to the angle and position of the view window for the displayed 3D model of the patient's dental arch from a dataset comprising both a 3D model of the patient's dental arch and an image of the patient's dental arch—each pair of said 3D pairs comprising a first imaging wavelength and a second imaging wavelength taken at the same angle and position for the patient's dental arch, respectively; and displaying the identified pair of images taken at an angle and position close to the angle and position of the view window for the displayed 3D model of the patient's dental arch.

[0041] In any of these methods, the identification step may include determining a plurality of images that are close to the relative angle and position between a 3D model of the patient's dental arch and a related view window, and averaging the plurality to form an identified image. For example, there may be multiple images in a dataset taken from approximately the same angle (e.g., + / - 0.1%, 0.5%, 1%, 2%, 3%, 4%, 5%, 7%, 10%, 15%, 20%, etc.) and approximately the same area (e.g., + / - 0.1%, 0.5%, 1%, 2%, 3%, 4%, 5%, 7%, 10%, 15%, 20%, etc.) of the dental arch. These similar images can be combined to form an average image that is better than individual images.

[0042] Generally, the step of identifying one or more images taken at angles and positions close to the relative angles and positions between the 3D model of the patient's dental arch and the related view window may be to identify them within an acceptable spatial range. For example, between the plane of the view window and an equal angle of + / - a few degrees (e.g., + / - 0.1°, 0.2°, 0.3°, 0.4°, 0.5°, 0.6°, 1°, 1.2°, 1.5°, 1.7°, 1.8°, 2°, 2.2°, 2.5°, 3°, 3.2°, 3.5°, 4°, 5°, etc.) and within a + / - range of distance to the dental arch area where the view window is located (e.g., + / - 0.1mm, 0.2mm, 0.3mm, 0.4mm, 0.5mm, 0.6mm, 0.7mm, 0.8mm, 0.9mm, 1mm, 1.1mm, 1.2mm, 1.5mm, 1.7mm, 2.0mm, It is an image taken at 2.2mm, 2.5mm, etc.

[0043] Any of these methods may include receiving a data set from a processor. The data set may be received directly from an intraoral scanner and / or stored and retrieved. In some variations, the data set may be transmitted and received by the processor, and in some variations, the processor may read the data set from memory connected to the processor (e.g., a data store).

[0044] Generally, any of these methods may involve displaying a view window over a portion of the 3D model of the patient's dental arch. The view window may be of any shape or size, such as a circle, ellipse, triangle, rectangle, or other polygon. For example, the view window may be a loop that allows viewing of a portion of the 3D model of the patient's dental arch. The viewing angle may allow the dental arch to be visualized through at least a portion of the view window. The view window may be smaller than the dental arch. In some variations, the view window may be made larger or smaller by the user.

[0045] Generally, this method may include a display through a user interface. For example, the user interface may display a 3D model of the dental arch, a view window, and / or image(s) corresponding to the view through the view window of the dental arch on a screen or on screens. The user may move the view window and the dental arch independently (by manipulating the user interface through controls such as a mouse, keyboard, touch screen, etc.). With such movement, and the image(s) determined to correspond to the image through the view window of the angle and area of ​​the view window of the dental arch, may be displayed in real time as the user moves the view window and / or the dental arch relative to each other.

[0046] For example, the step of allowing a user to change the relative position between the view window and the 3D model of the patient's dental arch may include the step of separately controlling the angle and / or rotation of the 3D model of the patient's dental arch and the portion of the dental arch adjacent to the view window. In some variations, the step of allowing a user to change the relative position between the view window and the 3D model of the patient's dental arch may include the step of allowing the user to move the view window over the 3D model of the dental arch.

[0047] As mentioned, any image identified as taken at an angle and position corresponding to the angle and position of the view window when moving over and / or around the dental arch (or when the dental arch moves relative to the view window) may be one or more techniques. Thus, for example, the step of identifying an image that approximates the relative angle and position between the 3D model of the patient's dental arch and the relevant view window may include identifying one of a visible light image, an infrared image, and a fluorescence image.

[0048] The step of displaying an identified image close to the angle and position of the view window for the displayed 3D model may include the step of displaying the identified image in a window adjacent to or overlapping with the display of the 3D model of the patient's dental arch. For example, the image may be displayed on the screen along with the 3D model of the dental arch; when the user moves the dental arch and / or imaging window, the image(s) may be displayed in one or more windows while changing in real-time or near real-time to reflect the relative position of the 3D model of the dental arch and the view window.

[0049] Additionally, a non-transient machine-readable type medium is described herein for storing instructions to cause one or more machines to perform any of the methods described herein, including virtually examining a patient's dental arch. For example, the non-transient machine-readable type medium may store instructions to cause one or more machines to perform an operation to virtually examine a patient's dental arch, including: displaying a three-dimensional (3D) model of the patient's dental arch; displaying a view window over at least a portion of the 3D model of the patient's dental arch; and enabling a user to change the relative position between the view window and the 3D model of the patient's dental arch, including an angle between the view window and the patient's dental arch and one or more portions of the dental arch adjacent to the view window. And continuously, as the user changes the relative position between the view window and the 3D model of the patient's dental arch, the method comprises the steps of: identifying an image taken at an angle and position close to the relative angle and position between the 3D model of the patient's dental arch and the relevant view window from a dataset containing both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch—each image taken at a different angle and position relative to the patient's dental arch; and displaying the identified image taken at an angle and position close to the angle and position of the view window relative to the displayed 3D model of the patient's dental arch.

[0050] For example, a non-transient machine-readable type medium storing a command to enable one or more machines to perform a task for virtually examining a patient's dental arch comprises: a step of displaying a three-dimensional (3D) model of the patient's dental arch; a step of displaying a view window over at least a portion of the 3D model of the patient's dental arch; a step of enabling a user to change the relative position between the view window and the 3D model of the patient's dental arch, including an angle between the view window and the patient's dental arch and one or more portions of the dental arch adjacent to the view window; and subsequently, as the user changes the relative position between the view window and the 3D model of the patient's dental arch, a step of identifying an image taken from a dataset containing both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch—each image taken from a different angle and position relative to the 3D model of the patient's dental arch and the relevant view window—that is close to the relative angle and position relative to the 3D model of the patient's dental arch and the relevant view window; and a step of displaying the identified image taken from an angle and position close to the angle and position relative to the view window of the displayed 3D model of the patient's dental arch.

[0051] In addition, an intraoral scanning system configured to perform the method described herein is described herein. For example, the intraoral scanning system comprises: a handheld wand having at least one image sensor and a light source configured to emit light in a spectral range within a near-infrared (near IR) wavelength range; a display output (a visual output such as, e.g., a monitor, a screen, a virtual reality interface / augmented reality interface, etc.); a user input device (any controller for receiving and transmitting user input such as, e.g., a keyboard, a button, a joystick, a touchscreen, etc.; the display output and the user input device may be the same touchscreen); and one or more processors operably connected to the handheld wand, the display, and the user input device, wherein the one or more processors display a three-dimensional (3D) model of the patient's dental arch on the display output; display a view window over a portion of the 3D model of the patient's dental arch on the display output; and change the relative position between the 3D model of the patient's dental arch and the view window based on input from the user input device; A processor configured to identify near-infrared (near-IR) images taken from both a 3D model of the patient's dental arch and multiple images of the patient's dental arch taken at different angles and positions relative to the 3D model of the patient's dental arch and a related view window, and to display the identified near-infrared images taken at an angle and position relative to the angle and position of the view window for the 3D model of the patient's dental arch.

[0052] One or more processors of an intraoral scanning system may be configured to receive multiple images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch. For example, images may be taken by image sensor(s) of a handheld wand and transmitted to one or more processors and / or stored in memory accessed by one or more processors. The system may also include a controller that coordinates the activities of one or more processors, wands, and display outputs (and user input devices). The controller may display images and / or 3D models constructed from images as the user manipulates the handheld to take images from different positions and / or angles relative to the patient's dental arch(s).

[0053] One or more processors may be configured to continuously identify near-infrared images and display the identified near-infrared images as the user changes the relative position between the view window and the 3D model of the patient's dental arch. Thus, when the user (using user input) adjusts the position of the view window (e.g., loop) relative to the 3D model of the patient's dental arch in the display output (or equivalently, adjusts the position of the 3D model of the dental arch in the display output relative to the view window), one or more processors may determine and display the near-infrared image of the patient's tooth closest to the relative position between the view window and the 3D model of the patient's dental arch.

[0054] The near-infrared image is one of the images taken with the handheld wand or the average of the images taken with the handheld wand. Any device described herein (e.g., an intraoral scanning system) may also determine and / or store the position and / or orientation of the handheld wand when operated, and this information may be stored along with the image(s) taken at this position. For example, the handheld wand may include one or more accelerometers. For example, one or more processors may be configured to identify near-infrared images taken at an angle and position close to the relative angle and position between the 3D model of the patient's dental arch and the associated view window by determining a plurality of images close to the relative angle and position between the 3D model of the patient's dental arch and the associated view window and averaging the plurality to form an identified near-infrared image.

[0055] As mentioned, one or more processors may be configured to change the relative position between the 3D model of the patient's dental arch and the view window in the display output based on input from a user input device. Specifically, one or more processors may be configured to change the angle between the view window plane and the patient's dental arch, and one or more parts of the dental arch adjacent to the view window (e.g., visible through the view window in some variations) based on input from the user input device. As discussed above, the view window may be a loop (e.g., circular, elliptical, square, etc.) over which the 3D model is visible. One or more processors may be configured to display the view window over a part of the 3D model of the patient's dental arch, including displaying a loop over which a part of the 3D model of the patient's dental arch is visible. The view window can be moved to position the 3D model of the patient's dental arch over the view window (including changes in the buccal, occlusal, lingual, or lateral aspects between these, x, y, z movement and / or rotation, e.g., pitch, roll, yaw), and / or move the 3D model of the patient's teeth (e.g., rotation by pitch, yaw, roll, movement by x, y, z, etc.). Accordingly, one or more processors may be configured to change the relative position between the 3D model of the patient's dental arch and the view window based on input from a user input device by changing one or more of the angle of the 3D model of the patient's dental arch relative to the view window (e.g., the angle of the view window relative to the 3D model of the patient's dental arch), the rotation of the 3D model of the patient's dental arch relative to the view window (e.g., the rotation of the view window relative to the 3D model of the patient's dental arch), and the part of the dental arch adjacent to the view window (e.g., the part of the 3D model visible through the view window).For example, one or more processors may be configured to change the relative position between the 3D model of the patient's dental arch and the view window based on input from a user input device by changing the position of the view window on the 3D model of the dental arch.

[0056] One or more processors identify a second image that is close to the relative angle and position between the 3D model of the patient's dental arch and the associated view window, which is one or more of a visible light image and a fluorescent image, from both a 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken at different angles and positions relative to the patient's dental arch; and one or more processors may be configured to display the second image simultaneously with the near-infrared image.

[0057] Also described herein is a method for identifying and grading features automatically, semi-automatically / semi-manually, or manually by adjusting across multiple imaging techniques. For example, a dental diagnostic method may include: identifying tooth features in a first record—the first record comprises multiple images of the patient’s dental arch taken using a first imaging technique; correlating the first record with a model of the patient’s dental arch; using the model of the patient’s dental arch to identify regions of the dental arch corresponding to the tooth features in one or more different records—each of the one or more different records is taken with an imaging technique different from the first imaging technique, and each of the one or more different records is correlated with a model of the patient’s dental arch—; determining a confidence score for the tooth features based on the identified regions corresponding to the tooth features in one or more different records; and displaying the tooth features when the confidence score for the tooth features exceeds a threshold.

[0058] A dental diagnostic method may include the following steps: identifying tooth features in a first record—the first record comprises a plurality of images of the patient’s dental arch taken with a first imaging technique; correlating the first record with a three-dimensional (3D) volumetric model of the patient’s dental arch; flagging tooth features in the 3D volumetric model; using a model of the patient’s dental arch to identify regions of the dental arch corresponding to tooth features in one or more different records—each of the one or more different records is taken with an imaging technique different from the first imaging technique, and each of the one or more different records is correlated with a model of the patient’s dental arch—; determining or adjusting a confidence score for tooth features based on the identified regions corresponding to tooth features in one or more different records; and displaying an indicator of the confidence score for tooth features and tooth features when the confidence score for tooth features exceeds a threshold.

[0059] In any of these methods (or systems for performing them), tooth features may include one or more of fissures, gum recess, tartar, enamel thickness, pits, caries, grooves, fissures, evidence of grinding, and interproximal voids.

[0060] Displaying may include displaying indicators of tooth characteristics and confidence scores for tooth characteristics.

[0061] Correlating the model of the patient's dental arch with the first record may include correlating the three-dimensional (3D) volumetric model of the patient's dental arch with the first record. Any such method (or system for performing the same) may include flagging tooth features in the model of the patient's dental arch and / or collecting tooth features, including one or more of the location of the tooth features, the type of the tooth features, and a confidence score for the tooth features.

[0062] Determining the reliability score may involve adjusting the reliability score for the tooth feature based on identified regions corresponding to the tooth feature in one or more different records.

[0063] In any of these methods or systems, identifying tooth features may include automatically identifying tooth features.

[0064] For example, a dental diagnostic method may include: a step of identifying one or more actionable tooth features from one or more of a plurality of records—each of which includes a plurality of images of a patient’s dental arch taken using an imaging technique, and each of which is taken using a different imaging technique—; a step of mapping the actionable tooth feature to a corresponding area of ​​one or more of the records; a step of recording one or more actionable tooth features, including recording the location of the actionable tooth feature; a step of adjusting or determining a confidence score for one or more actionable tooth features based on the corresponding area of ​​one or more of the records; and a step of displaying one or more actionable tooth features when the confidence score of one or more actionable tooth features exceeds a threshold. As mentioned above, one or more actionable tooth features may include one or more of fissures, gum recession, calculus, enamel thickness, grooves, caries, grooves, fissures, evidence of grinding, and interdental spaces.

[0065] Displaying may include displaying one or more actionable tooth features and indicators of confidence scores for the tooth features. Mapping actionable tooth features to corresponding areas of one or more records may include correlating the first record with a three-dimensional (3D) volumetric model of the patient's dental arch. Recording one or more actionable tooth features may include marking tooth features on a 3D volumetric model of the patient's dental arch. Identifying tooth features may include automatically identifying tooth features.

[0066] In addition, a system for performing any of the methods described herein is described herein. For example, the system may include: one or more processors; and a memory coupled to one or more processors, wherein the memory is configured to store computer-program instructions that perform a computer-implemented method when executed by one or more processors, and the method comprises: identifying a tooth feature in a first record—the first record comprises a plurality of images of a patient’s dental arch taken with a first imaging technique; correlating the first record with a model of the patient’s dental arch; using the model of the patient’s dental arch, identifying regions of the dental arch corresponding to the tooth feature in one or more different records—each of the one or more different records is taken with an imaging technique different from the first imaging technique, and each of the one or more different records is correlated with a model of the patient’s dental arch; determining a confidence score for the tooth feature based on the identified regions corresponding to the tooth feature in one or more different records; and displaying the tooth feature when the confidence score for the tooth feature exceeds a threshold.

[0067] Additionally, methods and apparatus (e.g., systems) for tracking one or more regions (e.g., tagged or flagged regions) across different imaging techniques and / or over time are described herein. For example, a method for tracking dental features across different imaging techniques comprises: collecting a first three-dimensional (3D) volumetric model of a patient’s dental arch—the 3D volumetric model of the patient’s dental arch includes internal structures and surface values ​​within the dental arch; identifying regions of the patient’s dental arch in a first record of a plurality of records—each record comprising a plurality of images of the patient’s dental arch taken using an imaging technique, and each record of the plurality of records being taken using a different imaging technique; flagging the identified regions in the corresponding regions of the 3D volumetric model of the patient’s dental arch; correlating the flagged regions with each record of the plurality of records by correlating the 3D volumetric model of the patient’s dental arch with each record of the plurality of records; and storing, displaying, and / or transmitting images including the regions of the patient’s dental arch. The area of ​​the patient's dental arch may include one or more of the following tooth features: fissures, gum recession, calculus, enamel thickness, grooves, caries, grooves, fissures, evidence of grinding, and interdental spaces.

[0068] Storage, display, and / or transmission may include displaying the area of ​​the patient's dental arch. Such methods may include flagging tooth features in a 3D volumetric model. Identifying the area of ​​the patient's dental arch may include automatically identifying the area of ​​the patient's dental arch.

[0069] A system for tracking one or more regions (e.g., tagged or flagged regions) across different imaging techniques and / or over time may include: one or more processors; a system comprising memory coupled to one or more processors, wherein the memory is configured to store computer-program instructions that perform a computer-implemented method when executed by one or more processors, and the method comprises: collecting a first three-dimensional (3D) volumetric model of a patient's dental arch—the 3D volumetric model of the patient's dental arch includes internal structure and surface values ​​within the dental arch; identifying regions of the patient's dental arch in a first record of a plurality of records—each record comprising a plurality of images of the patient's dental arch taken using an imaging technique, and each record of the plurality of records is taken using a different imaging technique; flagging the identified regions in the corresponding regions of the 3D volumetric model of the patient's dental arch; correlating the flagged regions with each record of the plurality of records by correlating the 3D volumetric model of the patient's dental arch with each record of the plurality of records; and storing, displaying, and / or transmitting an image containing the regions of the patient's dental arch. Brief explanation of the drawing

[0070] The novel features of the present invention are specifically described in the following claims. A better understanding of the features and benefits of the present invention can be obtained by referring to the following detailed description and accompanying drawings, which describe exemplary embodiments in which the principles of the present invention are used: FIG. 1a illustrates an example of a 3D (color) intraoral scanner that can be used as described herein to create a model of a subject's teeth having both surface and internal features. FIG. 1b schematically illustrates an example of an intraoral scanner configured to generate a model of a subject's teeth having both surface and internal features. FIG. 2 illustrates a schematic diagram of an intraoral scanner configured to perform both surface scanning (e.g., visible light, non-transmittance) and transmission scanning using near-infrared (IR) wavelengths. The scanner includes a polarizer and a filter that collect near-infrared light reflected from internal structures while blocking near-infrared light reflected from the tooth surfaces. Figure 3 is an example of a method for scanning teeth with an oral scanner to identify internal structures using transmitted wavelengths (e.g., IR and / or near-infrared). Figure 4 illustrates one method of generating an internal structure (or pseudo-X-ray) image from volumetric data. Figures 5a and 5b illustrate virtual cross-sections from a volumetric model of a tooth. These virtual cross-sections may display internal features or properties of the tooth using annotations, colors / pseudo-colors, or textures. In Figure 5a, the virtual cross-section is pseudo-colored to represent enamel. In Figure 5b, the virtual cross-section is pseudo-colored to represent dentin. Figure 6 illustrates one method of marking (e.g., flagging) a volumetric model of a patient's tooth and / or using the marked area. FIG. 7 is a comparison between a typical computer-aided design / computer-aided manufacturing (CAD / CAM) method for dental practice and a method for implementing 3D volumetric scanning and modeling as described herein. FIG. 8a is an example of a display tracking gingival recession over time using the 3D volumetric model described herein. FIG. 8b shows an enlarged view showing subsequent time for region B of FIG. 8a. Figures 9a–9g illustrate one method of displaying volumetric information from a patient's teeth. Figure 9a illustrates an example of a 3D volumetric model of a patient's upper jaw (representing the teeth and gingiva) in a plan view. Figure 9B illustrates the same 3D volumetric model showing internal features, including more transparent enamel and less transparent dentin. The 3D volumetric model can be manipulated to display more or less of the surface and / or internal structure. Figures 9c–9g illustrate progressively more transparent views or regions ("C") of the 3D volumetric model in Figure 9a. Figure 9c shows a 2D image extracted from a region of the 3D volumetric model showing only the external surface of the tooth (e.g., 100% of the color / external surface image, 0% of the near-infrared / internal volume). FIG. 9d shows the same area as FIG. 9c, combining an external surface (color) image and an internal (near-infrared based) image (e.g., 75% of the color / external surface image, 25% of the near-infrared / internal volume). FIG. 9E shows the same area as FIG. 9C, combining an external surface (color) image and an internal (near-infrared based) image (e.g., 50% of the color / external surface image, 50% of the near-infrared / internal volume). FIG. 9F shows the same area as FIG. 9C, combining an external surface (color) image and an internal (near-infrared based) image (e.g., 25% of the color / external surface image, 75% of the near-infrared / internal volume). Figure 9g shows the same area as Figure 9c, showing only the internal (near-infrared based) image of the tooth (e.g., 0% of the color / external surface image, near-infrared / internal volume 1000%). FIG. 10a illustrates an example of a user interface for the analysis and / or display of a 3D volumetric model of a patient's tooth, showing a plan view of the upper arch, tools that can be used to manipulate the view(s), two enlarged views showing the outer surface of an enlarged area of ​​the tooth (left), and the same view showing the internal features of the tooth (showing the dentin and enamel within the tooth). FIG. 10b illustrates the user interface of FIG. 10a with the area of ​​the tooth marked / flag as described in the present specification. Figures 11a–11c illustrate another example of a method for displaying 3D volumetric image information mixed with surface (non-transparent) information. Figure 11a also shows a visible light image of a patient's dental arch region taken with a scanner configured to perform a transmitted (near-infrared) scan. Figure 11b shows a volumetric model of a reconstructed 3D volumetric model of a patient's tooth showing internal dentin and enamel. Features not visible in the surface scan are evident in the volumetric scan, which includes caries and bubbling areas within the enamel. Figure 11c shows a hybrid image where the 3D volumetric image is combined with the surface scan, revealing both surface and internal structures, including caries and bubbling areas. Figure 12 is an example of a method that allows a user to virtually scan a patient's dental arch. This method can be performed in real-time or near real-time. Figure 13 is a schematic diagram of a data structure containing a 3D model of the patient's dental arch(s) and associated 2D images of the dental arch taken from many locations around the dental arch (e.g., via an oral scanner). FIG. 14a is an example of a user interface that allows a user to virtually scan a 3D model of the dental arch and displays the corresponding optical and near-infrared regions (e.g., external and internal) in detail when the user scans the 3D dental arch; the user can move the dental arch (e.g., rotation, translation, etc.) and / or the view window using one or more tools; the corresponding optical and near-infrared images can be continuously or nearly continuously updated as the position of the view window and the dental arch changes. A pair of image windows is displayed next to the 3D model view of the dental arch. Fig. 14b is an alternative display showing a single large image window over or adjacent to a 3D image of the dental arch. In Fig. 14b, the image window shows an optical image of the corresponding area of ​​the dental arch. FIG. 14c is an alternative display that shows a single large image window over or adjacent to a 3D image of the dental arch. In FIG. 14c, the image window shows a near-infrared image of the corresponding area of ​​the dental arch. Fig. 15a shows an example of a 3D model of the external surface of the dental arch and a view window for the dental arch, similar to Fig. 14a. A pair of image display windows are adjacent to the 3D model of the dental arch. The user can move the view window over the dental arch (and / or move the dental arch relative to the view window to change the images displayed in the two display windows). The upper display window shows a near-infrared image corresponding to the dental arch at a position and angle in the plane of the view window; the lower display window displays a corresponding light image (which may be in color). FIG. 15b shows another image of the dental arch depicted in FIG. 15a, in which the dental arch is rotated lingually relative to the view window; the corresponding near-infrared image (top right) and visible light image (bottom right) adjacent to the 3D model of the arch are updated to display a slightly rotated view, so that the user can virtually scan the dental arch and display both external and internal views in real-time (or near real-time). Figure 16a is another example of a method for displaying a 3D model of a dental arch (indicated as the mandibular arch in this example, e.g., by selecting the mandibular arch display control from the top left of the user interface) and showing a focused view of near-infrared and visible light images corresponding to a view window area that can be moved upward / laterally around the patient's arch model (lingual-occlusal-buccal). FIG. 16b illustrates an example of a single window similar to FIG. 16a (an enlarged near-infrared view of the teeth in the area corresponding to the view window loop). FIG. 16c illustrates an example of a single window similar to FIG. 16a (an enlarged visible light view of the teeth in the area corresponding to the view window loop). FIG. 17 schematically illustrates an example of a method for automatically or semi-automatically identifying, verifying, and / or characterizing one or more actionable tooth features that can benefit from detection and / or treatment. Specific details for implementing the invention

[0071] Methods and devices (e.g., devices and systems) for applying scans of both external and / or internal structures of a tooth are described herein. Such methods and devices may create and / or manipulate a model of a subject’s oral cavity (e.g., teeth, jaw, palate, gingiva, etc.) that may include surface topography and internal features (e.g., dentin, tooth filling material (including base and lining), cracks and / or caries). A device that performs both surface and transmission scans of a tooth may include an oral scanner for scanning inside or around a subject’s oral cavity and is equipped with a light source or light sources capable of illuminating in two or more spectral ranges: a surface feature illumination spectral range (e.g., visible light) and a transmission spectral range (e.g., an IR range, including but not limited to “near-infrared”). The scanning device may also include one or more sensors for detecting emitted light and one or more processors for controlling scanning operations and analyzing light received from a first spectral range and a second spectral range to generate a model of a subject's tooth including features of the tooth surface and inside the tooth, including the enamel (and / or enamel-like restoration) and the dentin. The generated model may be a 3D volumetric model or a panoramic image.

[0072] As used herein, a volumetric model may include a virtual representation of a three-dimensional object in which internal regions (structures, etc.) are arranged three-dimensionally within the volume in proportion to and relative to other internal and surface features of the object being modeled. For example, a volumetric representation of a tooth may include not only the external surface but also internal structures (below the tooth surface) within the tooth arranged in proportion to the tooth, so that a cross-section through the volumetric model substantially corresponds to a cross-section through the tooth to show the location and size of the internal structures; the volumetric model may be a cross-section in any (e.g., any) direction and corresponds to an equivalent cross-section through the object being modeled. The volumetric model may be electronic or physical. A physical volumetric model may be formed, for example, by 3D printing, etc. The volumetric model described in this specification may be extended entirely (e.g., through the total volume, e.g., the volume of the tooth) or partially (e.g., through a minimum depth, e.g., 2 mm, 3 mm, 4 mm, 5 mm, 6 mm, 7 mm, 8 mm, 9 mm, 10 mm, 12 mm, etc.).

[0073] The methods described herein generally include methods for generating a 3D model or rendering of a tooth that includes both surface and internal features, such as methods for generating a tooth model of a subject. Non-ionization methods for imaging and / or detecting internal structures may be used, such as taking images using a transmission wavelength to view structures within the tooth by illuminating with one or more transmission spectral ranges (wavelengths), including the use of trans-illumination (e.g., illuminating from one side and capturing light after it passes through an object on the opposite side) and / or small-angle transmission imaging (e.g., reflection imaging, capturing light reflected / scattered from internal structures when illuminated with a transmission wavelength). In particular, multiple transmission images may be taken at the same relative position. Traditional transmission imaging techniques (e.g., trans-illumination) in which the angle between the light-emitting illumination direction and the field of view of the detector (e.g., camera) is 90 or 180 degrees may be used, but methods and devices for which the angle is much smaller (e.g., 0 to 25 degrees, 0 to 20 degrees, 0 to 15 degrees, 0 to 10 degrees, etc.) are also described herein. Smaller angles (e.g., 0 to 15°) may be particularly advantageous because the illumination (light source) and the detection (detector(s), e.g., camera(s), etc.) can be closer to each other, and may provide a scanning wand for an oral scanner that can be more easily positioned and moved around the subject's teeth. These small-angle transmission imaging and imaging techniques may also be referred to herein as reflected illumination and / or imaging, or reflected / scattering imaging. Generally, transmission imaging may refer to any appropriate type of transmission imaging, including trans-illumination, small-angle transmission imaging, etc., unless otherwise specified. However, if the angle is small, it may be reflected directly from the surface of the object (e.g., teeth), making the internal structure unclear.

[0074] The method and apparatus described herein are particularly effective for combining imaged internal features, such as lesions (cavities, cracks, etc.) that can be detected by the use of transmission imaging with an oral scanner applied to detect both surface and internal features individually and simultaneously (or nearly simultaneously), with a 3D surface model of teeth or teeth. Combining surface scanning and transmission imaging can be performed by alternating or switching between different techniques in such a way that the same coordinate system can be used for both. Alternatively, both surface and transmission scanning can be viewed simultaneously, for example, by selectively filtering the imaged wavelengths to separate IR (near-IR) light from visible light. Thus, 3D surface data can provide important reference and angular information regarding internal structures and allow for the interpretation and analysis of transmission images that might otherwise be difficult or impossible to interpret.

[0075] The transmission scans described herein may be collected from an oral scanner, such as the one illustrated in FIG. 1a-1b, and a method of using such a scanner to create a three-dimensional (3D) model of a subject's oral region (e.g., teeth or teeth, gums, jaw, etc.) that may include, for example, internal features of the teeth and may include a model of the surface. In many cases, surface scanning (including color scanning) may be helpful and useful, but in some variations described herein, transmission (IR) scanning may be sufficient.

[0076] In FIG. 1a, the exemplary intraoral scanner (101) may be configured or applied to produce a 3D model having both surface and internal features, or only an internal (transparent) scan. As schematically illustrated in FIG. 1b, the exemplary intraoral scanner may include a handle or wand (103) that an operator (e.g., dentist, dental hygienist, technician, etc.) can hold in their hand and move over a subject's teeth or teeth to scan both surface and internal structures. The wand may include one or more sensors (105) (e.g., cameras such as CMOS, CCD, detectors, etc.) and one or more light sources (109, 110, 111). In FIG. 1b, three light sources are illustrated: a first light source (109) configured to emit light in a first spectral range (e.g., visible light, monochromatic visible light, etc.; this light does not necessarily have to be visible light) to detect surface features, a second color light source (e.g., white light between 400-700 nm, e.g., approximately 400-600 nm), and a third light source (111) configured to emit light in a second spectral range (e.g., in near-infrared light, generally referred to as transmission imaging, by transillumination, small-angle transmission imaging, laser fluorescence, etc.) to detect internal features within the tooth. Although separate light sources are illustrated in FIG. 1b, selectable light sources may be used in some variations. The light sources may be any suitable light sources, including LEDs, optical fibers, etc. The wand (103) may include one or more controls (buttons, switches, dials, touchscreens, etc.) to assist in control (e.g., turning the wand on / off, etc.); Alternatively or additionally, one or more unillustrated controls, such as a foot pedal, keyboard, console, touch screen, etc., may be present on other parts of the intraoral scanner.

[0077] Generally, any suitable light source, particularly one that matches the mode being detected, may be used. For example, any of these devices may include other (including non-visible) light sources or visible light sources for surface detection (e.g., near 680 nm or other suitable wavelengths). Color light sources, generally visible light sources for color imaging (e.g., "white light" sources), may also be included. Additionally, transmission light sources for transmission imaging (e.g., infrared, particularly near-infrared light sources) may also be included.

[0078] The intraoral scanner (101) may also include one or more processors, including a connected processor or a remote processor, for controlling the operation of the wand (103), including scanning adjustments and reviewing and processing the scanning and creation of a 3D model including surface and internal features. As illustrated in FIG. 1b, one or more processors (113) may include or be coupled with a memory (115) for storing scanned data (surface data, internal feature data, etc.). A communication circuit (117), including a wireless or wired communication circuit, may also be included to communicate with a component of the system (including the wand) or an external component including an external processor. For example, the system may be configured to send and receive scans or 3D models. One or more additional outputs (119) may also be included to output or present information, including a display screen, a printer, etc. As mentioned, an input (121) (button, touch screen, etc.) may be included, and the device may allow or request user input to control scanning and other operations.

[0079] FIG. 2 illustrates an example of a scanner that can be used. The illustrated scanner may be used as part of a system such as the system illustrated in FIG. 1a-1b. For example, FIG. 2 shows a schematic diagram of an intraoral scanner configured to perform both surface scanning (e.g., visible light, non-transmitting) and transmission scanning using a near-infrared (NIR) wavelength (850 nm in this example). In FIG. 2, the scanner includes a near-infrared illumination light (289) and a first polarizer (281) and a second polarizer (283) in front of an image sensor (285) to block near-infrared light (P-polarized light) reflected from the surface of the tooth (290) while collecting near-infrared light (S-polarized light) scattered from the internal tooth structure / region. Near-infrared light illuminates the tooth with P-polarization, and specular reflections from the tooth surface, such as enamel, are reflected specularly, preserving the P-polarization state. Near-infrared light passing through internal tooth features, such as dentin, is scattered, resulting in random polarization (S and P). A wavelength-selective quarter-wave plate (293) changes the polarization of the reflected scan light from P to S without changing the polarization of the near-infrared light (e.g., leaving the polarization state of the transmitted near-infrared light unchanged), so that only surface reflections are captured at the scan wavelength. The reflected near-infrared light, which is a mixture of S and P polarizations, is first filtered through a polarization beam splitter (PBS) (294) and a polarization filter (283) so that only the S polarization is transmitted to the image sensor. Thus, only the near-infrared S-polarized light emanating from the internal tooth structure is captured by the image sensor, and specular reflections with the original P-polarization are blocked. Other oral scanner configurations with or without a polarization filter, such as that shown in Fig. 2, can be used as part of the probe.

[0080] In FIG. 2, surface scanning can be performed by illuminating the surface (using a scanner illumination unit (297)) with P-polarized light, and the polarization can be reversed by a wavelength selection quarter-wave plate (293) (transmitting S-polarized light to an image sensor).

[0081] Various transmission scanning techniques (transmission imaging) may be used or integrated into the device described herein to perform a scan that detects internal structures using a transmission wavelength or a spectral range of transmission wavelengths, including but not limited to trans-illumination and small-angle transmission imaging that detect the transmission of a transmission wavelength of light from or through tissue (e.g., teeth or teeth or passing through them). Accordingly, such devices and techniques may be used to scan oral components such as teeth or one or more teeth, gingiva, palate, etc., and may be used to generate a model of the scanned area. Such a model may be generated in real time or after the scan. Such a model may be referred to as a 3D volumetric model of the teeth, but may include other areas of the jaw, including the palate, gingiva, and teeth. While the methods and devices described herein generally relate to 3D volumetric models, the techniques and methods described herein may also be used with 3D surface models in some cases. Surface model information is generally part of the 3D volumetric model.

[0082] FIG. 3 illustrates an example of a data flow for scanning teeth with an intraoral scanner to construct a 3D model including internal structures. The exemplary method illustrated in FIG. 3 may include three parts. First, the teeth may be scanned by an intraoral scanner (1701) (or any other scanner) configured to provide a transmission scan of the teeth using an optical (e.g., IR, near-infrared, etc.) wavelength or wavelength range. Any of these scanners may also scan simultaneously to determine surface features (e.g., through one or more non-transmitting wavelengths), color, etc., as described above. During scanning, multiple transmission scans (1703, 1703') may be taken, and the position of the sensor (1705, 1705') (e.g., camera) (e.g., x, y, z position and / or pitch, roll, yaw angle) may be determined and / or recorded for each transmission image. In some variations, the surface of the tooth may also be imaged simultaneously, and a 3D surface model of the tooth (1707) is determined as described above. In this example, the patient's tooth may be scanned with an intraoral 3D scanner (1702) capable of imaging the internal tooth structure, for example, using near-infrared imaging. The position and orientation of the camera may be partially determined from the 3D scanning data and / or the 3D tooth surface model (1707).

[0083] Subsequently, the transmitted image can be segmented (1711). In this example, segmentation can be performed in one of two ways. The image can be segmented using contour finding in the internal tooth structure image (1713, 1713'). This process can be further automated by applying machine learning methods. Alternatively, or additionally, a near image (where the camera position is close) can be used to determine nearby features and to re-project features from a 3D model back onto the image to correctly position segments such as enamel. The method may also include the step of projecting pixels back onto the tooth from the internal tooth image and calculating a density map of the internal tooth reflection coefficients. The enclosed surfaces of different segments can be found or estimated using equivalent surfaces or thresholds of the density map and / or machine learning methods. Additionally, segments can be found at the intersection of the segment projection and the tooth surface by segmenting the image and re-projecting the segments onto a model (e.g., a 3D surface model, e.g., actual re-projection).

[0084] The results may be displayed, transmitted, and / or stored (1717). For example, the results may be displayed by a scanning system during an intraoral scanning procedure. The results may be displayed as an image surrounding the contours of different segments, 3D density maps, etc. In the example illustrated in FIG. 3, a density map (1715) representing the dentin beneath the enamel on the outer surface is shown. This image may be color-coded to display different segments. In this example, the inner segment (structure) is displayed within a 3D surface model (displayed transparently); only a portion is displayed because not all teeth were scanned as a through-image. Alternative views, sections, slices, projections, etc. may be provided. In FIG. 3, the example image includes artifacts present outside the tooth (1716); these may be removed or trimmed based on the surface model (1718).

[0085] Segments can represent each pixel of an image. Internal structures such as dentin, enamel, cracks, and lesions can be automatically determined by segmentation and identified manually or automatically (e.g., based on machine learning of 3D structures). Segments can be displayed separately or together with or without a surface model (e.g., a 3D surface model) (e.g., different colors, densities, etc.).

[0086] Accordingly, in FIG. 3, the patient is initially scanned with a 3D scanner capable of surface scanning and transmission scanning (e.g., near-infrared imaging), and the orientation and / or position of the camera is known (based on the position and / or orientation of the wand and / or surface scan). These positions and orientations may be relative to the tooth surface. Thus, the method and device may have an estimate of the camera position (where it is located, e.g., the x, y, z positions and rotational positions of the camera).

[0087] Generally, transmitted images (e.g., near-infrared or IR images) can be automatically segmented.

[0088] User interface and volumetric information display

[0089] The collection and analysis of volumetric data from within the oral cavity can identify features and information of teeth that were difficult or impossible to identify with previous non-volumetric scanning. However, it may be difficult or counterintuitive for a dentist (and / or patient) to analyze three-dimensional volumetric information. A method and apparatus for viewing and interpreting 3D volumetric data of a patient's oral cavity are described herein.

[0090] For example, Figures 9a–9g illustrate an example of a method for displaying 3D volumetric data. Figure 9a illustrates a surface model (which may be a surface model portion of the volumetric model) from a top view of an upper arch where external features (e.g., surface features) are visible. This view is similar to a surface scan view that may be in color (e.g., taken in visible light). Internal structures existing within the model beneath the external surface of the scan are not easily visible in Figure 9a. In Figure 9b, internal structures are shown based on their relative transparency to near-infrared light. In Figure 9b, the enamel is more transparent (and appears more transparent) than the dentin, which appears less transparent. Figures 9b–9f illustrate the transition between the surface view of Figure 9a and the transmitted internal 3D view of Figure 9b for the illustrated sub-region (region "C") indicated by a circle. For example, a user display may be provided in which the relevant surface versus internal view can be switched to provide a perception of the internal structure within the dental arch relative to the surface structure. For example, an animated view that cycles through images, such as in FIGS. 9c-9g, may be provided. Alternatively, the user may slide a slider (903) to toggle between the surface and internal views. The transition between these two views (which can be made at any angle) may help the user and / or patient look beneath the tooth surface to visually evaluate rich internal data. The 3D volumetric model may be manipulated to display all views, including cross-sectional views showing internal structures and / or surface features. Plan views are shown in FIGS. 9a-9g. FIGS. 9c-9g illustrate a progressively more transparent view or area ("C") of the 3D volumetric model of FIG. 9a, where a progressively increasing percentage (from 0% to 100%) of the interior view of FIG. 9b is displayed for area C, while a progressively decreasing percentage (from 100% to 0%) of the surface view is displayed.

[0091] FIGS. 11a-11c illustrate another example showing a hybrid image that combines and blends both a volumetric model taken using a transmission (e.g., near-infrared) wavelength as shown in FIG. 11b and a surface image scanning (e.g., a visible light scan as shown in FIG. 11a) (as in FIG. 9e). Features present in the tooth enamel and dentin that are not clear in the surface-only image (which may be a reconstruction) shown in FIG. 11a can be seen in the volumetric reconstruction (the image shown in FIG. 11b). For example, in FIG. 11b, the carious area (1103) is evident, which is not visible in FIG. 11a. Similarly, the bubbling area (1105) of the enamel is visible in FIG. 11b but not in FIG. 11a. FIG. 11c shows a hybrid image of the 3D volumetric model and the surface model (surface image), where both of these structures, the carious area and the bubbling area, are visible.

[0092] In general, methods and apparatus for simplifying and displaying volumetric data from a patient's oral cavity (e.g., teeth, gums, palate, etc.) in a manner easily understood by a user (e.g., a dentist) and / or a patient are described herein. Additionally, methods for displaying volumetric data taken from a patient's oral cavity in a manner understandable by a user and / or a patient are described herein. In the first example, the data may be provided as one or a series of X-ray type images similar to those generated by dental X-rays. FIG. 4 illustrates one method of generating an X-ray (or pseudo-X-ray) image from a set of volumetric data taken as described above, using, for example, transmitted light (e.g., near-infrared) wavelength(s).

[0093] As illustrated in FIG. 4, a method for displaying a 3D volumetric image of a patient's oral cavity may include receiving (401) 3D volumetric data from, for example, the scan described above, either directly or from a stored digital scan. In some variations, individual teeth or groups of teeth may be identified (403) from the volumetric data. Teeth may be identified automatically or manually (e.g. by volumetric segmentation, machine learning, etc.), or the entire volume may be used. A pseudo-X-ray image may be generated from a volume or a subset of volumes corresponding to individual teeth (405). For example, a volumetric image may be taken from the 'front' of a tooth or teeth, and the transparency of the enamel (and / or enamel-like restoration), dentin, and other features is maintained in the volumetric data. This volumetric data may be based on the absorption coefficient of the material in the oral cavity for the transmitted wavelength of the light used. Therefore, projection through volumetric data is generated from the volumetric data in a fixed orientation to obtain an image similar to an X-ray, but in some variations, it is inverted so that the density of dentin (high absorption) appears "darker" than the density of enamel (low absorption and therefore more transparent); caries may also appear as areas with more absorption (dark). Thus, the image can be inverted to resemble an X-ray image where denser areas become brighter (e.g., luminous). These pseudo-X-ray images can be generated at the same location as standard dental X-rays and provided to the user. For example, a pseudo-X-ray image panel can be generated from a volumetric model for each of the patient's teeth. Although the penetration of light wavelengths (e.g., near-infrared) may not be as deep as conventional X-rays, images generated in this way can provide a similar proxy to X-rays, particularly for the crowns above the gingiva and the mid-tooth region.

[0094] Other simplified or modified displays may be provided to the user or customized for the user to display to the patient. For example, in some variations, an image of the tooth may be generated from simplified volumetric data by pseudo-coloring the volumetric data to highlight specific areas. For instance, previously marked or flagged areas (described in detail below) may be displayed in red, and the enamel may be displayed in a more natural white or slightly off-white. In some variations, enamel-like materials (e.g., fillings, etc.) may be represented individually or indicated by color, pattern, etc.

[0095] In some variations, the method and / or device may display the teeth in cross-section through the dental arch. Similarly, individual teeth or groups of teeth (e.g., according to standard naming / numbering rules) may be displayed or labeled individually. This may be displayed in addition to or instead of another display. In some variations, the teeth and / or internal structures may be pseudo-colored or projected onto a color image.

[0096] For example, Figures 5a and 5b illustrate virtual cross-sections taken through a volumetric model of a patient's tooth generated from an intraoral scan, as described above, which includes near-infrared information. In Figure 5a, the cross-section may be generated automatically or manually, for example, by a user, to display a region of interest within the tooth containing enamel. The cross-section may show both a density cross-section and / or a surface cross-section. These images may be pseudo-colored to represent different regions including the outer surface, enamel, dentin, etc. For example, internal structures within the enamel and / or dentin may reflect the effects of near-infrared light within the tooth, such as the absorption and / or reflection of light at one or more near-infrared / visible wavelengths within the tooth. In Figure 5b, the cross-section is pseudo-colored with a heat map to show internal features and may provide keys as indicated. In any of these variations, a 2D projection of the tooth may be generated from volumetric information showing the tooth and / or one or more features of the teeth. As described in more detail below, additional features including lesions (e.g., caries / cavities, cracks, wear, plaque accumulation, etc.) may also be displayed and may be indicated by color, texture, etc. Although illustrated as a cross-section of a 3D volumetric model, other embodiments may display the 2D cross-section itself to provide a cross-sectional view of the tooth / teeth similar to the view provided by a 2D X-ray image.

[0097] Any method and apparatus for carrying out the things described herein may include displaying one or more (or consecutive) cross-sections through a 3D model of a patient's dental arch, preferably a 3D volumetric model. For example, a method for displaying images from a 3D volumetric model of a patient's dental arch may include: collecting a 3D volumetric model of the patient's dental arch—said that the 3D volumetric model includes near-infrared (near-IR) transparency values ​​for internal structures within the dental arch; generating a 2D view of the 3D volumetric model containing the patient's dental arch including the near-infrared transparency of the internal structures; and displaying the 2D view. In any of these methods, the method may optionally (but not necessarily) include scanning the patient's dental arch with an oral scanner.

[0098] Generating a 2D view may involve cross-sectioning a 3D volumetric model with a plane passing through the 3D volumetric model. The user may select the position and / or orientation of the plane and may perform this operation sequentially. For example, any of these methods may include a step in which the user selects a cross section to display through the 3D volumetric model, wherein the selection step includes the user continuously selecting cross sections through the 3D volumetric model as the user scans through the 3D model and continuously displaying a 2D view corresponding to each cross section. Generating a 2D view may include the user selecting the orientation of the 2D view.

[0099] In any of these methods, a surface may be included. For example, as described and illustrated above, a method for displaying an image from a three-dimensional (3D) volumetric model of a patient's dental arch may include: collecting a three-dimensional (3D) volumetric model of the patient's dental arch—said that the three-dimensional (3D) volumetric model includes surface values ​​and near-infrared (near-IR) transparency values ​​for internal structures within the dental arch; generating a two-dimensional (2D) view of the 3D volumetric model containing both the surface values ​​and near-infrared transparency of the internal structures; and displaying the 2D view. Surface values ​​may include surface color values. Weights may be assigned to surfaces associated with internal (volumetric) structures. For example, the step of generating a two-dimensional (2D) view through the 3D volumetric may also include including a weighted portion of the surface values ​​and a weighted portion of the near-infrared transparency of the internal structures in the 2D view. The weighted portion of the surface value may include a percentage of the total value of the surface value, and the weighted portion of the near-infrared transparency of the internal structure includes a percentage of the total value of the near-infrared transparency of the internal structure, wherein the percentage of the total value of the surface value and the percentage of the total value of the near-infrared transparency of the internal structure are at most 100%. For example, the user may adjust one or more weighted portions of the near-infrared transparency of the internal structure and the surface value.

[0100] For example, a method for displaying an image from a three-dimensional (3D) volumetric model of a patient's dental arch may include: collecting a three-dimensional volumetric model of the patient's dental arch—said that the three-dimensional volumetric model includes near-infrared (near-IR) transparency values ​​and surface color values ​​for internal structures within the dental arch; selecting a viewing direction of the three-dimensional volumetric model to be displayed by a user; generating a two-dimensional (2D) view of the three-dimensional volume using the selected direction containing the patient's dental arch, which includes a weighted portion of the surface color values ​​and a weighted portion of the near-infrared transparency of the internal structures; and displaying the 2D view.

[0101] In addition to displaying qualitative images of the teeth, the methods and devices described herein may quantify and provide quantitative information regarding internal and / or external features. For example, volumetric measurements of one or more lesions, including dimensions (peak or average length, depth, width, etc.) and volume, may be provided (optionally or automatically). This may be accomplished by manually or automatically segmenting a volumetric model to define regions of interest that include tooth features (enamel, dentin, etc.) and / or irregularities (e.g., caries, cracks, etc.). Suitable segmentation techniques, such as mesh segmentation (mesh decomposition), polyhedral segmentation, skeletonization, etc., may be used. Once the volume is segmented, these regions may be displayed and / or measured individually or collectively. As described below, they may also be marked / flagged and used for further analysis, display, and modification of the scanning method and system.

[0102] In some variations of the user interface described herein, a summary report may be generated or created from volumetric data and displayed for the user and / or patient. For example, the summary data may be projected onto a model of the patient's teeth. The model may be simplified so that the enamel is opaque but marked, or selected internal features (including automatically selected internal features) are displayed within the teeth in red or other contrasting colors (and / or flash, blinking, etc.). For example, caries may be displayed in this manner. The summary report may be automatically entered into the patient's chart.

[0103] Any image containing a volumetric image can be animated. For example, a virtual cross-section through the patient's teeth showing a scanning or moving cross-section through the patient's dentition may be displayed, and in some cases, a 3D model showing one or more cutting axes through the volume may be displayed together. The user interface may allow the user to create cross-sections in one or more planes showing both external and internal features based on the volumetric scan.

[0104] Generally, the device described herein may generate a view separate from the patient for a user (e.g., physician, dentist, orthodontist, etc.). A separate 'clinical view' may be provided to the user, which may contain information not present in the 'patient view'. The clinical view may be more technical and, in some cases, closer to a raw image of volumetric data. The user may select an information layer to include in the patient view, which may be provided to the user during or after the scanning or review of the dental scan. Patient education materials may be added to the patient view.

[0105] For example, in some variations, the user display of volumetric data may include an overlay of volumetric data in which pseudo-colors of 3D components within the volumetric data are displayed. As discussed in more detail below, marked or highlighted areas in such displays / images may be displayed to draw attention to potential problem areas (e.g., cavities, thin enamel, cracks, etc.). Two-dimensional (2D) color data and 3D near-infrared data (e.g., surface and volumetric areas) may be displayed, including a transition between the two.

[0106] Generally, volumetric information can be annotated (e.g., marking, labeling, etc.) automatically, manually, or semi-automatically, and these annotations can be displayed. Additionally, annotations can be used to annotate future scans and to modify how future scans of the same patient are taken and displayed. Annotations may be, for example, markers or flags for regions of interest. A region of interest may correspond to a specific area where one or more features (such as fissures, caries, enamel thinning, or the accumulation of plaque or calculus) are observed. Alternatively, or additionally, a region of interest may be, for example, an area that changes over time from one scan to another.

[0107] As mentioned above, any of these methods may involve placing one or more markers on a volumetric model of the patient's teeth. The markers (e.g., flags, pins, etc.) may be placed manually by the user, automatically by the device, or semi-automatically (e.g., suggested by the system, configured by the user, etc.). This will be explained in more detail later.

[0108] Markers can be used to focus attention and / or processing by the system on one or more specific areas of the volumetric model for display and / or subsequent follow-up (e.g., in future scans). Markers can modify how subsequent scans are performed, for example, by performing future scans of the marked area in greater detail (e.g., higher resolution, different wavelengths, higher scan frequencies, or reputation). The marked area can be displayed over time to show changes in the marked area.

[0109] For example, the user may mark a digital representation of the patient's tooth (or the patient's actual tooth) with an annotable marker (e.g., a pin, flag, etc.) that can have relevant notes. This marker may be used to track between different scans over time. Later scans may be marked at corresponding locations, and later scans may be modified based on the marked areas. These marked areas may be scanned in more detail, and analysis may be performed automatically and / or displayed to measure and / or indicate changes compared to one or more previous scans. Thus, any system described herein may track one or more marked areas from previous scans and provide feedback during and / or after new scans to provide additional details. This may be performed with respect to surface and / or volumetric information, particularly the characteristics of the enamel, and / or comparison with the enamel, the outer surface of the tooth / teeth, and / or dentin.

[0110] For example, one or more annotation markers from a previous scan can modify a subsequent scan of the same patient. Before scanning, the user can enter the identifier of the patient being scanned (or the system can automatically identify the patient based on the database of previous scans). The system can automatically annotate the new scan based on the annotations from the previous scan.

[0111] In some variations, subsequent scans can be automatically annotated by the system by identifying differences between the previous scan(s) and the current scan. For example, areas showing a change exceeding a threshold compared to the previous scan may be flagged and displayed to the user. Annotation may be performed without user intervention (fully automatic) or with some user intervention, for example, by flagging, displaying the reason for the flagging to the user, and allowing the user to retain, modify, or reject the marking. Reasons for automatically marking teeth may include changes in enamel thickness, changes in surface smoothness, changes in the relative ratio of enamel to dentin, and changes in tooth position (e.g., occlusion). It may also include changes in the external structure of the natural tooth or changes in the gingival structure surrounding the tooth, such as an increase or decrease in plaque or calculus accumulation. Therefore, if the system detects one or more of these conditions, it automatically flags the relevant area of ​​the volumetric model.

[0112] Subsequent scans can be dynamically modified by flags from the previous scan or by detecting changes in areas (even unmarked areas) compared to the previous scan. For example, scanning parameters can be modified to scan at a higher resolution (e.g., changing the scan dwell time, requiring the user to scan these areas multiple times, etc.) or to change the wavelength used for scanning.

[0113] For example, FIG. 6 illustrates a method for automatically selecting and / or using a selected area to be marked. A first volumetric model of the patient's tooth is generated from a scan of the patient's tooth (601). The volumetric model may be generated using any suitable method, including that described above and discussed in U.S. Patent Application No. 15 / 662234, titled “INTRAORAL SCANNER WITH DENTAL DIAGNOSTICS CAPABILITIES,” filed July 27, 2017, the whole of which is incorporated by reference. The first volumetric model may be stored (digitally stored) as part of the patient's dental record. The first volumetric model may be analyzed simultaneously or sequentially (immediately or after some time) (e.g., by a device comprising a device having a processor configured to operate as described herein) to identify areas to be flagged (603). Thus, such analysis may be performed automatically and may examine one or more attributes of the patient's tooth in the scan. Automatic (or semi-automatic, etc., automatic but including manual support for verification / confirmation) can be performed by a microprocessor including a system (e.g., machine learning) trained to identify abnormal areas of the external and / or internal volume of the tooth. For example, the device can inspect a digital model to identify possible defects in the patient's tooth, such as cracks, cavities, voids, changes in occlusal relationships, malocclusion, etc. This may include identifying areas with optical (e.g., near-infrared) contrast near the tooth surface indicating possible cracks, cavities, occlusion, etc. (605). Areas that are less transparent (e.g., more absorbent) at near-infrared wavelengths than the rest of the enamel and are closer to the surface (generally or within a specific near-infrared wavelength) may correspond to defects. Alternatively or additionally, the surface characteristics of the tooth may be inspected and flagged if they fall outside a threshold (607).For example, areas of the tooth surface that are rough (e.g., having a smoothness below a set threshold, where the smoothness can be determined from the outer surface of the enamel) may be flagged. Other surface characteristics may also be analyzed and used to determine whether the area should be marked or displayed or presented to the user for verification of the marking, including discoloration (based on color or white light / surface scan), gingival position (relative to the outer surface of the tooth), etc. The distribution and size of the patient's enamel may also be examined (609). Enamel thickness may be determined from optical characteristics (e.g., comparison of absorption / reflection characteristics). Estimated enamel areas that are below the threshold thickness or have a ratio of thickness to tooth dimensions (e.g., diameter, width, etc.) below the threshold may be presented to the user or marked for verification of the marking.

[0114] In some variations, during and / or after the automatic analysis of the volumetric model, the user may also manually flag one or more regions of the volumetric model of the patient's tooth (615). When the automatic analysis of the volumetric model automatically flags the identified volumetric model, regions manually added by the user may be added. In some variations, the user may be prompted to mark regions identified and suggested by the automatic analysis. These regions may be marked, and an indication of the reason for identification may be provided (e.g., abnormal enamel, potential cracks, potential caries, potential thinning of the enamel, etc.). Generally, internal boundaries (e.g., boundaries within the volumetric of the tooth) may be defined in one of the methods and apparatus described herein. For example, in a variation where the enamel region is thinning, the methods and apparatus described herein may be used for entire layers (e.g., the enamel layer, internal structural regions of the tooth) that can be identified and used for qualitative and / or quantitative information.

[0115] In some variations, flagged areas may be displayed on a digital model of the patient's teeth (617). The display may highlight the flagged areas by providing, for example, colors, animations (e.g., flash), icons (e.g., arrows, flags, etc.) and combinations thereof. The display may also display an enlarged view of one of these. The user may modify the display, for example, rotate, section, or enlarge the flagged areas. Alternatively or additionally, the flagged areas may be enlarged in the display by default. An index or key of the flagged areas may be provided and may be displayed and / or stored along with the digital volumetric model of the patient's teeth.

[0116] In some variations, as illustrated on the right side of FIG. 6, the method may include using a flagged area to modify a future scan as described above. For example, the method scans ("re-scans") the flagged area after some intermediate period between the first time period (e.g., about 1 day, 1 week, 1 month, 2 months, 3 months, 4 months, 5 months, 6 months, 7 months, 8 months, 9 months, 10 months, 11 months, 1 year, 1.2 years, 1.5 years, 2 years, etc.) and the second time period (e.g., about 1 month, 2 months, 3 months, 4 months, 5 months, 6 months, 7 months, 8 months, 9 months, 10 months, 11 months, 1 year, 1.5 years, 2 years, 3 years, 4 years, 5 years, 6 years, 7 years, 8 years, 9 years, 10 years, 11 years, 12 years, 13 years, 14 years, 15 years or more, etc.) or after a period longer than the second time period. It may include. Flagged areas may be used to modify the scan during scanning by increasing the resolution of the scanned area, for example, by increasing the scan speed, increasing the dwell time in the area, scanning the area at additional wavelengths, or scanning the area multiple times. The scanning device may notify the user to adjust the scanning (e.g., move the wand of the intraoral scanner more slowly in these areas, move the scanner over these areas multiple times, etc.) and / or automatically adjust scanning parameters during operation. Thus, the scanning device may receive the key or index of the flagged area and / or the marked (flagded) version of the patient's previous intraoral scan(s). Before scanning the device, the user may indicate the identity of the patient being scanned, which can be used to look up previous scan(s).Alternatively or additionally, the device may identify the patient based on the current scan to identify (or verify) the patient, and may check or retrieve a previously annotated (flagped) scan. Alternatively, a second or subsequent scan may be performed without using the previously flagged area.

[0117] A method or device configured to perform a method following a subsequent operation (e.g., a second, subsequent, or subsequent scan or scan) may compare a flagged area of ​​the subsequent scan with a corresponding area of ​​the previous scan(s) (621). Additionally, the volumetric model of the subsequent scan may be automatically analyzed to identify any area of ​​the new (subsequent) scan that can or should be marked / flagged (e.g., repeating the previous automatic or semi-automatic analysis steps 603-615) (621). The newly identified area in the subsequent scan may be compared with a corresponding area that was not previously flagged in the previous volumetric model(s).

[0118] Flagged regions can be analyzed over time (623). Specific sub-regions of the volumetric model containing the flagged regions may be generated for display and analysis. The results may be output (625). For example, these regions may be displayed to the user along with descriptive and analytical information about the scanned regions. These regions may also be marked to indicate changes over time. Data may be displayed, for example, as an animation showing changes over time. In some variations, images may be displayed as time-lapse images (videos, loops, etc.) showing changes. Time-lapse images may show changes in internal and / or external structures over time. Changes may be indicated using sections (pseudo-sections generated from the volumetric model). Color, texture, pattern, and other highlighting visualization techniques may be used. Alternatively or in addition to the display of flagged regions, these regions (and any accompanying analysis) may be output in other appropriate ways, including digital output (e.g., patient dental records), printing descriptions of the flagged region(s), etc.

[0119] Any method of tracking regions of a patient's dental arch described herein may include tracking over time and / or across different imaging techniques (e.g., recordings) as described in more detail below. Additionally, any of these methods may be automated and / or may include an automated agent comprising, for example, the identification, scoring, and / or automation of the identification, scoring, and / or display of one or more regions of interest (e.g., features including actionable dental features, defects). Any of these methods may also include any display method or agent (e.g., cross-sectional display, internal structure display, pseudo-X-ray display, display across imaging techniques, etc.).

[0120] For example, a method for tracking a region of a patient's dental arch over time may include: collecting a first three-dimensional (3D) volumetric model of the patient's dental arch—said that the 3D volumetric model includes near-infrared (near-IR) transparency values ​​and surface values ​​for internal structures within the dental arch—; identifying a region of the 3D volumetric model; flagging the identified region; collecting a second 3D volumetric model of the patient's dental arch; and displaying one or more images on which the difference between the first 3D volumetric model and the second 3D volumetric model in the flagged region is marked.

[0121] Any of these methods may also include a step of tracking and / or comparing across different records (e.g., different imaging techniques), and the identifying step includes identifying a region of the patient's dental arch from a first record of a plurality of records, wherein each record comprises a plurality of images of the patient's dental arch taken using an imaging technique, and each record of the plurality of records is taken using a different imaging technique, and additionally, the flagging step includes flagging the identified region in a corresponding region of a 3D volumetric model of the patient's dental arch. A method and apparatus for performing these may also include a step of correlating the flagged region with each record of the plurality of records by correlating the 3D volumetric model of the patient's dental arch with each record of the plurality of records. In some variations, the correlation may be used to weight or grade the identified region to determine whether the identified region corresponds to a region of interest (e.g., features, defects including actionable dental features, etc.). For example, the area of ​​the patient's dental arch may include one or more dental features, such as fissures, gingival recession, calculus, enamel thickness, grooves, caries, grooves, fissures, evidence of grinding, and interdental spaces. Identifying the area may involve comparing the near-infrared transparency value of the area within the 3D model to a threshold value.

[0122] When surface values ​​are used, the surface values ​​may include surface color values. This method can be used with stored data and / or data collected in real time (for example, this method may optionally, but not necessarily, collect a three-dimensional (3D) volumetric model by scanning the patient's dental arch to generate a 3D volumetric model).

[0123] Identifying regions may include identifying automatically using a processor. For example, automatic identification may include identifying surface color values ​​outside a threshold range. Automatic identification may include segmenting a 3D volumetric model to identify enamel regions and identifying regions having an enamel thickness below a threshold.

[0124] Flagging identified regions may include automatically flagging identified regions or manually checking identified regions for flagging.

[0125] In any of these methods in which a region is flagged, the method may include a step of rescanning the patient's dental arch, wherein the flagged region is scanned at a higher resolution than the unflagged region.

[0126] A method for tracking an area of ​​a patient's dental arch over time may include the following: collecting a first three-dimensional (3D) volumetric model of the patient's dental arch acquired at a first time point—the 3D volumetric model includes near-infrared (near-IR) transparency values ​​and surface color values ​​for internal structures within the dental arch—using an automatic process to identify an area within the 3D volumetric model to be flagged from a first record of a plurality of records—each record comprising a plurality of images of the patient's dental arch taken using an imaging technique, and each record of the plurality of records being taken using a different imaging technique—; flagging the identified area; correlating the flagged area with each record of the plurality of records by correlating the 3D volumetric model of the patient's dental arch with each record of the plurality of records; collecting a second 3D volumetric model of the patient's dental arch acquired at a different time point; and displaying the difference between the first 3D volumetric model and the second 3D volumetric model in the flagged area.

[0127] Similarly, as summarized and described above, as a method for tracking tooth features across different imaging techniques, the method comprises: collecting a first three-dimensional (3D) volumetric model of a patient’s dental arch—said that the 3D volumetric model includes surface values ​​and internal structures within the dental arch; identifying a region of the patient’s dental arch in a first record of a plurality of records—said that each record includes a plurality of images of the patient’s dental arch taken using an imaging technique, and that each record of the plurality of records is taken using a different imaging technique; flagging the identified region in the corresponding region of the 3D volumetric model of the patient’s dental arch; correlating the flagged region with each record of the plurality of records by correlating the 3D volumetric model of the patient’s dental arch with each record of the plurality of records; and storing, displaying, and / or transmitting an image containing the region of the patient’s dental arch. Any of these methods may also include, for example, the step of tracking over time by comparing the same region(s) with 3D volumetric models at different times.

[0128] FIGS. 10a and 10b illustrate a user interface showing the marking of a region of interest from a 3D volumetric scan of a patient's oral cavity. In FIG. 10a, the user interface includes an image (1001) of internal features similar to FIG. 9b discussed above (e.g., based on near-infrared absorption of the teeth). This view can be manipulated by user controls (1015) including a sectioning tool, a rotation tool, a translation tool, etc. In FIG. 10a, two upper windows show a surface view (1003) and a volumetric (internal) view (1005) corresponding to the same region. These regions can be selected. FIG. 10b illustrates the same features of FIG. 10a but has a marked or flagged region (1011). As discussed above, the identification of the region to be flagged can be automatic, manual, or semi-automatic (e.g., verified by the user) and can be selected to choose the region to be monitored later. In Fig. 10b, the area may correspond, for example, to a place where tooth decay is possible.

[0129] Monitoring one or more internal regions of a tooth over time using a volumetric model of a patient's tooth taken with the device described herein can be particularly helpful in predicting dental problems, including caries, cracks, tooth loss, and gum recession. In particular, these methods and devices can assist users (e.g., dentists, dental technicians, orthodontists, etc.) in informing and educating patients so that they can receive recommended treatment before more serious problems develop. There is a need for an effective method to show changes in the tooth over time and provide patients with the information necessary to take early action to prevent more complex and potentially painful problems from occurring. Many patients are reluctant to undergo preventive procedures, especially if they are not currently experiencing any related pain or discomfort. For example, pre-caries are difficult to identify with current imaging technology and typically appear painlessly, making it particularly difficult to persuade patients to undergo treatment even when identified. However, early-stage treatment can be crucial to avoid more complex and risky procedures later on.

[0130] Dental caries is a type of problem that can be identified by the methods and apparatus described herein. As shown and discussed above, caries can be identified from a 3D volumetric model (e.g., the model described herein) transmitted through the tooth using light (e.g., near-infrared), which is a type of non-ionizing radiation. For example, using near-infrared light combined with surface scanning (e.g., white light), in the 3D volumetric model generated as described herein, the absorption coefficients of the internal regions of the tooth may indicate differences between dentin and enamel and may indicate internal structures and defects, including cracks, caries, etc. For example, enamel regions that are less transparent than expected at near-infrared wavelengths (e.g., having different IR optical properties), particularly regions that appear to extend to the tooth surface in the volumetric model, can be identified as caries or caries, either manually or automatically. Other abnormalities of the enamel and / or dentin (e.g., based on internal features of the tooth in a volumetric model) may be identified and may be characteristics of the tooth problem. Accordingly, the technique described herein may be used for the prognosis of tooth problems such as caries.

[0131] As mentioned, any device and method described herein may include an improved method for displaying internal tooth features using one or more volumetric models of a patient's teeth. For example, the method and device described herein may be used to generate an estimate of the enamel thickness for one or more of a patient's teeth. Such an estimate may be visually displayed showing the external surface of the teeth or a specific tooth, and may also show internal structures including a cross-sectional view showing the enamel, such as the thickness of the enamel, or a 3D internal view. This information may be used clinically to determine the need for dental prostheses, including veneers, crowns, etc., and to assist in the design and application of such prostheses. For example, any of the method and device described herein may be used to assist in preparing a dental implant design for a specific tooth or teeth.

[0132] Plaque and tartar detection and visualization

[0133] The methods and apparatus described herein may also be used for the detection and visualization (including quantification) of plaque and calculus on a patient's teeth. Any oral scanner described herein may be used to detect plaque or calculus on a patient's teeth by using fluorescence imaging in addition to other imaging / scanning techniques, including transmission (e.g., near-infrared) imaging. For example, an intraoral scanner may cycle between different imaging techniques, such as between white light and near-infrared, including additional techniques such as fluorescence (e.g., laser fluorescence).

[0134] The use of a fluorescence function (and / or current function) by an intraoral scanner can enable the detection of plaque and tartar on the tooth surface. Combined with 3D modeling using intraoral scanner data, the condition of plaque / tartar can be modeled and visualized in a 3D model of the tooth, including 3D volumetric modeling of the tooth. Plaque and / or tartar can be detected, displayed and highlighted as described above, and used before, during, or after treatment. For example, a dental technician (e.g., a dental hygienist) may use the intraoral scanner to detect and monitor the patient's condition and rinsing treatment. Data regarding plaque and tartar may also be used by any device described herein to determine and provide a predictive model that can indicate the rate and / or location of plaque and tartar (e.g., tartar) formation.

[0135] In some variations, plaque and tartar can be identified at least partially using fluorescence information. Plaque has been observed to emit fluorescence under blue light (e.g., about 405 nm). Any oral scanner described herein may include fluorescence information for which information regarding plaque and tartar can be used and which can be integrated into a 3D model of the patient's teeth. For example, plaque and / or tartar can be visually displayed as color and / or texture in a 3D model of the patient's teeth.

[0136] For example, a fluorescence signal can be obtained from an oral scanner using a dichroic filter with a large aperture amplification of the fluorescence signal. Since this amplification can enhance the fluorescence, areas of plaque and tartar can be detected, visualized, and segmented using RGB lighting, sensors, and images. Alternatively or additionally, the device may include a fluorescence source (e.g., an LED emitting at 405 nm) and corresponding filter(s) for the detection of plaque and / or tartar. This can be integrated into the oral scanner or added (e.g., a sleeve, accessory, etc.) to be used with the scanner.

[0137] Alternatively or additionally, in some variations, depending on the wavelength of the near-infrared light used, plaque and calculus may have different absorption / reflection from enamel. This allows calculus and / or plaque to be distinguished from the enamel of the volumetric model. Furthermore, the volumetric model can be used to detect tooth materials, including calculus and plaque, based on surface smoothness and geometry. In variations where calculus and / or plaque are not transparent to the near-infrared frequency used, the device can use the volumetric model to distinguish between enamel and calculus and / or plaque. Thus, calculus and / or plaque can be segmented and distinguished from enamel.

[0138] Using an intraoral scanner to detect plaque and / or calculus can provide quantitative information and digital modeling. This can enable monitoring and comparison of plaque / calculus over time based on registration of a 3D model, including real-time registration and / or display.

[0139] Acquiring both fluorescent images and 3D scans at the same time and location with an intraoral scanner (e.g., a scanning wand) allows for highly accurate registration of plaque / calculus regions and 3D models. Simultaneous scanning is described in more detail, for example, in U.S. Patent Application No. 15 / 662234, titled “INTRAORAL SCANNER WITH DENTAL DIAGNOSTICS CAPABILITIES,” filed July 27, 2017. Accurate registration between different scanning techniques, such as white / visible light, transmitted light (near-infrared), and / or fluorescence, enables the device to define the boundaries of calculus and / or plaque and to determine volume / thickness at high resolution, thereby allowing for accurate measurement of the current situation and comparison / tracking against previous scans.

[0140] The method and apparatus described herein may take RGB images of a tooth at the same or similar time as taking a 3D scan of the tooth. Such scans may be used to construct a 3D model of the tooth / jaw that may contain volumetric information (a 3D volumetric model). For example, the RGB images may display a highlighted signal on fluorescent surfaces, particularly in plaque and tartar areas, due to specific characteristics of the color and brightness of these surfaces as mentioned. For example, an image of the outer surface of the tooth (and in some cases, the volumetric model) may show areas having optical characteristics (fluorescence, brightness, color, etc.) indicating tartar and / or plaque. In some variations, this highlighted signal may arise from spectral illumination that does not produce reflection in visible light but generates a significant fluorescent signal in plaque and tartar. For example, standard RGB illumination (using a standard RGB sensor) may be modified to provide amplification of the fluorescent signal (e.g., in the near-infrared region) on the outer surface of the tooth. This amplification can be achieved, as a non-limiting example, by using a large aperture capable of passing IR signals and a small aperture capable of passing the general RGB (visible) spectrum. This combination can produce color images that further highlight the fluorescent surface. This fluorescence can manifest as the characteristic color and brightness of desired areas representing dental calculus and / or plaque.

[0141] In any method and device capable of taking an RGB image containing a fluorescence signal (e.g., at a wavelength where plaque or calculus exhibits fluorescence), segmentation of the fluorescence region can be performed on the image. For example, by using camera positioning while acquiring RGB and 3D scans (e.g., in an oral scanner), the fluorescence region can be registered in a 3D model of the patient's tooth (including a volumetric and / or surface model). This allows the associated plaque and calculus regions to be defined in the final 3D model, thereby defining regions such as the 3D surface and thickness of the plaque as well as the calculus boundaries of the tooth.

[0142] As previously discussed, areas on the 3D model can be compared with previous and future scans of the same patient, which can demonstrate the development of calculus over time and its impact on the patient's teeth. The device can automatically or semi-automatically mark (e.g., flag) these areas for monitoring. Thus, the size and shape of calculus on each tooth can be monitored. Alternatively, or additionally, the thickness / depth of the calculus can be compared with previous scans. This information can be provided quantitatively and / or qualitatively, as discussed above. The thickness / depth of the calculus can be compared with previous scans of clean teeth (including one or more previous scans followed by professional dental cleaning). This can provide an estimate of the calculus thickness in subsequent scans. As mentioned earlier, changes in plaque, particularly calculus over time, can be measured, and this data can be used to monitor the progression of plaque and calculus on the patient's teeth and provide a visualization of its development.

[0143] Generally, monitoring and visualizing a patient's teeth using the methods and devices described herein may be used as part of a dental and / or orthodontic treatment plan. As already mentioned above, monitoring of tartar and plaque may be used in treatments including tooth cleaning. Scans may be performed before, during, and / or after cleaning to provide the dentist with guidance on areas to highlight, focus on, or return to. Different treatments (coatings, caps, etc.) may be suggested based on the progression of plaque and / or tartar over time. Additionally, as described above, monitoring any other features or areas of interest, such as cavities, cracks, etc., may also provide information for treatment planning. As discussed above, information regarding cracks and / or cavities may be used to suggest treatments, including restorations, before potential problems develop further. In some variations, digital models of the teeth (e.g., surface and / or volumetric models) may be modified using volumetric information, and the modified model(s) are used to design orthodontic appliances or treatment plans. For example, users can digitally remove plaque and / or tartar from volumetric scans taken before or during treatment. Modified scans can be used to guide treatment, including additional tooth cleaning as needed, and to shape or modify devices (e.g., tooth aligners) to ensure a better fit.

[0144] Combination with dental tools

[0145] The intraoral scanner and volumetric modeling described herein may be used with and / or combined with other dental tools (drills, probes, etc.). Combined tools can provide many advantages.

[0146] For example, what is described herein is the use of a drill and a 3D volumetric model that can be used with or combined with an intraoral scanner. In some variations, a dental drill and an intraoral scanner may be combined. For example, a laser drill or a laser-accelerated water drill may be combined with an intraoral scanner. This combination allows a dental professional to provide real-time feedback to the user by visualizing directly when and before drilling the tooth using this tool. In one example, near-infrared light may be applied to the probe head of the drill (e.g., a laser drill) to provide imaging into the tooth, which allows for direct forward imaging before and / or during drilling. The enamel and dentin in the direct path of the drill may be imaged. Density information can be used to notify the doctor when they have reached a specific depth within the dentin layer or dentin of the tooth, or when a diseased area has been removed. For example, since tactile feedback is much more limited when using a dental laser compared to a conventional headpiece, density information can be used to provide haptic feedback to the operator.

[0147] As described in this specification, methods and devices including an intraoral scanner and volumetric modeling can also be integrated into dental computer-aided design / computer-aided manufacturing technology as described in FIG. 7. For example, dental implants, such as crowns (e.g., ceramic crowns), can be fabricated for individual patients using computer-aided design and computer-aided manufacturing (CAD / CAM) devices and procedures. For example, traditional CAD / CAM laboratory manufacturing (the "current workflow" in FIG. 7) may include impressions of the patient's teeth, such as a pre-processing scan (701) of the patient's teeth or a scan of the cavity-free jaw. The teeth can then be prepared for a crown (703), then re-scanned (705) and evaluated (707). Finally, the crown can be made using CAD / CAM. Software for CAD / CAM can receive scanned information from the scanner and process it to form a design and perform manufacturing. While the use of CAD / CAM software can provide restorations comparable to conventional restorations in all aspects, including aesthetics, current methods may require repetitive steps to evaluate and prepare the tooth, as shown in Fig. 7, and generally, the user must perform these steps manually.

[0148] As shown in the “new workflow” at the bottom of FIG. 7, this method may incorporate the 3D volumetric modeling described herein to simplify and improve CAD / CAM of the patient’s teeth. For example, the preparation may be designed digitally, and this process may be fully or semi-fully automated (so that the user can approve and / or modify the process). For example, in FIG. 7, the pre-processing scan (711) may be performed using an intraoral scanner that communicates directly with the CAD / CAM device, or the intraoral scanner may include CAD / CAM capabilities. In this example, the tooth preparation (713) may be designed entirely digitally based on the performed scan, and the scanner may guide (715) the preparation of the tooth. This may be performed in real time through direct feedback and / or guidance from a device capable of integrating the scanner. The scanner may be used to evaluate (717) the preparation, and in some cases, this step may be fully integrated into the guided preparation step (715), thereby eliminating the need for post-prep evaluation. Finally, CAD / CAM can be used to prepare a crown (or other dental instrument) for a properly prepared tooth (719).

[0149] root canal

[0150] Methods and devices for 3D volumetric modeling of a patient's oral cavity (e.g., 3D volumetric modeling of a tooth) may also be used to modify root canal procedures. Generally, root canal procedures require numerous X-rays to provide images of the inside of the tooth before, during, and / or after the procedure. The methods and devices described herein may eliminate or reduce the requirement for X-rays in specific examples of root canal procedures. Specifically, as described herein, an intraoral scanner including a transmitted wavelength (e.g., near-infrared) may be used to inspect the inside of the tooth, including the inside of the tooth root, during the procedure. This may enable identification and localization of the canal. For example, the tooth may be prepared for a root canal by drilling a hole in the tooth through the crown. The hole may be drilled during or between drillings, guided by the volumetric imaging described herein. For example, an initial hole may be drilled in a tooth (e.g., a molar) to expose the camber within the tooth. Oral scanners containing near-infrared light can be used to image teeth, including imaging through a hole drilled in the tooth to visualize the pulp chamber. The scanner can be oriented automatically or manually to image into the chamber and visualize the root within it. Initial drilling into the tooth can be restricted to expose the inner chamber by penetrating the enamel, allowing visualization into the chamber to image the root within the tooth itself, and to allow regions with different optical properties (particularly at any wavelength including near-infrared wavelengths) to penetrate the chamber despite calcification and / or infection. The nerve chamber of the root can be identified as being somewhat denser than the surrounding areas within the dentin and enamel.By removing the chamber cover to expose the internal pulp region of the tooth, the oral scanner can visualize through a drilled hole to provide additional volumetric information, including the position, curvature, and trajectory of the tooth root. This additional visualization information allows for the easy detection of hidden canals and accessory canals. This information can be used to guide treatment.

[0151] For example, in some variations, the method may include forming an opening in the target tooth by taking a 3D volumetric model of the patient's tooth before or after drilling using the oral scanner described herein (e.g., for root canal to be performed). Drilling may be performed with or without guidance from the intraoral scanner as described above. The internal chamber of the tooth may be visualized using the oral scanner, for example, through a hole drilled in the crown of the tooth. Then, the device may determine the location of the horn of the pulp chamber for the tooth. Any of the methods described herein may be used in conjunction with X-ray information. Treatment planning may be performed by the device to determine the shape and / or location of the pulp horn, pulp chamber, and root for establishing a treatment plan for drilling / tissue removal that prevents overthinking or rupture of the side of the tooth. This treatment plan may be used to guide the user in tooth drilling and / or drilling automation. In some variations, the drill may be guided directly by imaging, for example, using the aforementioned hybrid drill / intraoral scanner. Alternatively or additionally, robot-assisted support may be provided using a treatment plan. In some variations, the procedure may be performed manually, and drilling can be carried out in small increments through visualization between drilling steps to verify the treatment path and prevent over-drilling, as well as to ensure that the entire area is drilled and the infected pulp has been removed. Additional visualization (including the use of contrast agents) may be used.

[0152] Generally, any method described herein, including the aforementioned root canal method, may be used with one or more contrast agents during imaging. For example, the contrast agent may comprise a material applied to the outside of the tooth (or a hole or opening in the tooth, including a hole perforated in the tooth). A contrast agent that absorbs or reflects near-infrared light or other wavelengths used by an oral scanner may be used. Preferably, a contrast agent that is distinguishable in some, but not all, of the imaged wavelengths may be used to provide a difference image. For example, the contrast agent may be visualized under white light but not under near-infrared light; or, the contrast agent may be visualized under near-infrared light but not under white light, or may be visualized at some near-infrared wavelengths but not at other wavelengths where the image is taken. Preferably, a contrast agent that mixes or coats one or more targets within the tooth or oral cavity may be used. For example, a contrast agent that selectively binds to one or more of bacteria, plaque, calculus, gingiva, pulp, etc. may be used. When used, the contrast agent can be applied to the tooth / oral cavity and visualized after rinsing (or visualized without rinsing). For example, an IR light-absorbing contrast agent may be included as part of or mixed with a material forming a dental implant (e.g., cavity filling, crowning, root canal filling, etc.) to create an IR contrast filling material that can be easily visualized when scanned as described herein.

[0153] Additionally, methods for determining the improvement of soft tissues surrounding a tooth using an apparatus and method for generating a 3D volumetric model of a tooth as described herein are described herein. For example, gingival recession can be monitored and / or quantified and can be observed over time using such methods and apparatus. In addition to the direct visualization of plaque and / or tartar as described above, the methods and apparatus described herein can also, or alternatively, detect the effects on the tooth, including bone recession caused by plaque and tartar. Diseased areas can be directly visualized. In some variations, a contrast agent may be used to provide additional contrast for an intraoral scanner to detect diseased areas. Scanning the gingival surface can identify inflamed and / or discolored areas that may indicate gum disease. This information can be combined with a 3D volumetric model of the tooth, including the location of plaque and / or tartar, as discussed above.

[0154] Figures 8a and 8b illustrate an example of monitoring gum (gingival) recession over time. In this example, the display may show a 3D model of the tooth and a comparison between the original scan and a subsequent scan taken 2-3 years later. In Figure 8a, two scans are aligned and compared, and the difference is shown by a color indicator, for example, a heat map. In Figure 8, darker colors (e.g., may be shown in red) indicate greater gingival recession. Area B, illustrated by a circle in Figure 8a, is shown in more detail in Figure 8b for the subsequent scan. While Figure 8 primarily illustrates surface features (e.g., gingival position), volumetric information can be used to generate information showing changes in, for example, gingival thickness and / or angiogenesis, enamel thickness, etc.

[0155] In addition to guiding users and / or dental technicians based on scans (e.g., showing plaque, tartar, and / or inflammation), these methods and devices may be used by dental professionals to evaluate, grade, or quantify plaque and / or tartar removal immediately after treatment or over time. This may provide indicators for evaluating treatment. Scan information may also be used to provide information to patients, including maps or guides for home care, such as brushing and flossing, including areas to focus on. The guides may include, for example, one or more images of a 3D volumetric model. Guidance information regarding the teeth or oral areas to focus on for home dental care (e.g., brushing) may be provided to an electronic toothbrush that can assist the patient in brushing based on the identified areas.

[0156] The method and apparatus described herein may also be used for patients who already have dental devices installed on their teeth, including braces, bridges, etc. For example, in some variations, the patient may include a 3D representation of the dental device and may be provided with information to help design or modify future dental devices (e.g., retainers, aligners, braces, etc.).

[0157] In particular, the methods and devices described herein may be used to provide highly accurate volumetric and surface information about a patient's teeth, which may be useful for the treatment planning of any type of dental treatment. In some variations, the methods and devices described herein may be useful for the treatment planning of devices, such as aligners or retainers, that are optimally fitted in close proximity to the patient's teeth. For example, a method and / or device comprising a 3D volumetric scan of the patient's teeth may be used to subtract or remove plaque, tartar, and / or food debris that may be present at the time of the 3D scan from a 3D model of the teeth. By digitally removing plaque, tartar, and / or present food debris, the volumetric information can be used in conjunction with a virtual representation of the aligner, retainer, night guard, or other device, and the fit is improved before the device is fabricated, applied, or worn.

[0158] Gingival tissue surrounding the tooth, which has a density (or different optical absorption / reflection properties) different from that of enamel, can also be identified and characterized more accurately, thereby allowing the junction between the internal contour and the tooth surface to be identified. Through this, the shape of the tooth surface beneath the gingival tissue can be accurately characterized, enabling prediction models of tooth movement to represent the tooth more accurately as the initially invisible portion of the tooth is gradually exposed as the tooth aligns. In other words, while parts of the tooth may initially be obscured by gingival tissue, as the tooth straightens, the tissue shifts, exposing the previously covered area. Accurately detecting the tooth region beneath the gingival tissue allows for more precise modeling of the tooth's future state after gingival shift.

[0159] The methods and devices described in this specification may also be used to detect, diagnose, and / or treat oral disorders.

[0160] For example, the 3D volumetric scanning and modeling methods and devices described herein may be used to detect and / or treat salivary stones (e.g., salivary duct blockage). These salivary glands, which may be located near the molars and under the patient's tongue, may be scanned using the oral scanner described herein. Such scans can penetrate soft tissue and detect hard, stone-like structures (e.g., sialoliths, salivary gland stones, or duct stones) that are calcified structures formed inside the salivary glands or ducts and may block the flow of saliva into the mouth. The methods and devices described herein may be used to identify such structures and / or to guide and / or confirm the removal of such stones.

[0161] In addition to or instead of using the apparatus and methods described herein to identify, diagnose, and / or track areas including pre-cavity caries, fissures, etc., the methods and apparatus described herein may also or alternatively be used to identify and manipulate areas that have already been modified. For example, fillings, attachments (for attaching anglers, braces, etc.), braces, retainers, etc., and other structures may be identified and / or marked within the volumetric model. For example, areas of enamel and / or enamel-like restorations may be marked differently in the volumetric model. These areas will generally have different optical properties, including different scattering / absorption of near-infrared light (and in some cases visible light) compared to other areas of the oral cavity containing dentin and / or each other. These areas may be identified manually, automatically, or semi-automatically and may be segmented and / or manipulated individually. For example, in some variations, these areas of the tooth (e.g., attachments / cement for aligners or other instruments) can be identified by the dentist for removal, and a 3D volumetric model or data derived therefrom may be provided to guide this treatment. Additionally, once removed, the area may be digitally removed to better fit into a new device. The removed view may also, or alternatively, be provided to the patient.

[0162] In some variations, the internal structural integrity of an artificial tooth structure or modification (e.g., tooth bond, filling, etc.) may be determined using the volumetric model(s) described herein. For example, the volumetric model may include internal details of the artificial tooth structure, such as internal structural details of the filling, bond, etc., or boundaries between the artificial tooth and natural teeth (enamel, dentin, etc.), and this information may be presented or shown in detail to the user so that the condition of such artificial tooth structure can be evaluated (or automatically evaluated). This may facilitate removal, repair, and / or replacement.

[0163] 3D volumetric models of teeth (and methods and devices for generating them) can also be used as diagnostic or detection tools for future tooth sensitivity. For example, abfraction is a form of non-cariogenic tooth tissue loss that typically occurs along the gingival margin. Abfraction lesions may be mechanical damage to the tooth structure, not caused by caries, and can occur in both the dentin and enamel. This is believed to be caused by repetitive stress cycles resulting from the patient's occlusion and exacerbated by aggressive brushing. A 3D volumetric model of the tooth, enhanced by density analysis of the enamel and dentin near the gingival line, can provide early indicators of such lesions. For example, a device can examine the volumetric model to identify the early stages of formation for these crescent-shaped lesions. Using multiple 3D volumetric models over time can indicate the progression of these lesions. The system can be configured to identify these automatically or manually; These can be flagged automatically or semi-automatically as explained above.

[0164] Therefore, the device and method can identify and treat users who may develop such "hot spots" leading to future tooth sensitivity, and can provide a treatment plan to slow, stop, or reverse the progression of the lesion. Tooth sensitivity can be caused by these small fractures and exposed dentin. Detection can be triggered by identifying the characteristic crescent shape occurring in more advanced lesions, but early detection can be achieved by identifying areas of thinning in the enamel and / or dentin (e.g., near the gingival line) that may progress over time. The device and method can flag and / or assign risk based on the actual thickness and / or progression of thickness changes.

[0165] The methods and apparatus described herein may also be used to detect the occurrence of acid reflux based in part on characteristic wear patterns and / or changes in the patient's enamel thickness (e.g., over time). For example, acid reflux while the patient is sleeping may cause progressive erosion of the patient's teeth in a characteristic pattern (e.g., from the back of the teeth on the lingual side). A pattern similar to hyperphagia may occur. For example, a volumetric model of the patient's teeth taken with near-infrared light may provide an accurate mapping of the enamel density and thickness of all of the patient's teeth. Thus, a method for detecting acid reflux (or hyperphagia) may include detecting characteristic thinning of the enamel of the patient's teeth in the posterior lingual region (including detection over time). The more proximal, lingual region of the tooth may have an abnormally thin (or thinning) enamel thickness compared to the more anterior (outer) region of the opposite, buccal, and lateral sides of the patient's teeth.

[0166] The methods and apparatus described herein may also be used to detect thin enamel areas resulting from occlusal wear caused by chronic grinding of a patient's teeth and / or to predict tooth sensitivity that may result from such grinding. A 3D volumetric model of the patient's teeth may provide snapshots showing the occlusal thickness of the patient's enamel and the appearance of the dentin near the occlusal surface. Additionally, enamel loss on the occlusal surface may be revealed through multiple scans over time. This was mentioned above as an indicator that can be marked or flagged automatically, manually, or semi-automatically. For example, an area > 0.5 mm within 0.5 mm of the dentin 2Whenever an event occurs, a flag can be set, and the area of ​​the digital model is highlighted. This allows for the visualization and / or monitoring of all areas that meet the flag criteria. Given the patient's age and, in some variations, gender as well as changes in enamel thickness over time, an estimate of the rate of wear over time can be provided in conjunction with proximity to the dentin area, thereby enabling the estimation or prediction of tooth sensitivity or pain. Grinding of teeth may also be an indicator of other problems, including sleep apnea. For example, sleep apnea may be detected in a 3D volumetric model of the patient's teeth, particularly over time. Many patients with sleep apnea may develop tooth erosion patterns due to grinding their teeth (e.g., forward and / or lateral movements). Therefore, the method and apparatus described herein may aid in diagnosing or confirming sleep apnea.

[0167] In general, any method and apparatus described herein may be used on non-human patients. For example, any method and apparatus described herein may be used on veterinary patients (e.g., animals) to determine the condition of animal teeth, including, for example, tooth wear.

[0168] The methods and devices described herein may also be used to provide a patient with an estimate of the risk of tooth fracture and / or tooth sensitivity. For example, the 3D volumetric model of the tooth described herein may be used to identify malocclusion and resulting tooth wear and / or cracking based on mechanical estimations of tooth thickness and wear patterns. Functional information, such as chewing patterns and joint forces, may also be incorporated into the evaluation. Wear patterns may be identified and marked as 'hot spots' in images generated from, for example, a 3D representation of the patient's tooth. This may be displayed to the patient as information, including warnings about potential risks. High-risk areas may be identified to the patient along with a description of the potential risks.

[0169] In general, methods and devices for a 3D volumetric model containing information about the internal structure of a tooth (e.g., the distribution of enamel and dentin within the tooth), particularly for monitoring and comparison over time, can be used to identify, monitor, diagnose, and guide the treatment of various disorders in addition to those mentioned above. Examples include dentin dysplasia and enamel dysplasia. Using this method, it is possible to identify different types of enamel in a patient's tooth, including regions having different amounts of hydroxyapatite, amelogenin, and / or enamelins, or regions in which they are organized differently, including regions that may be homogeneous or heterogeneous and have different optical properties for the near-infrared wavelengths used for imaging.

[0170] Interactive display of the patient's dental arch 3D model

[0171] As already described (and illustrated in the drawings above), the method and apparatus described herein may enable a user to virtually scan a patient's dental arch. In particular, a 3D model of the patient's dental arch(s), which may be volumetric, surface, or both (or, in some variations, an abstract or general model), may be used together with images taken at various locations around the dental arch, for example, using an oral scanner. These images may be the images used to generate the 3D model of the dental arch. The images may be tagged and / or configured in a data structure to indicate the corresponding location, area, or angle for the 3D dental arch model. In some variations, the 3D model and the taken images may be maintained as a data structure, but the 3D model does not need to be included together with the images as a single data structure.

[0172] For example, FIG. 13 is an example of a data structure comprising one or more (e.g., sets) images taken from a location around the patient's dental arch as well as one or more dental arch models (1305) (e.g., greater than 50, greater than 100, greater than 200, greater than 500, greater than 750, greater than 1000, greater than 10,000, etc.). In some variations, visible light and near-infrared (or near-infrared and other modes) images (1301) may be shown and may share positional information. Positional information generally includes the area of ​​the dental arch from which the image was taken (e.g., x, y, z coordinates such as the center point coordinates of the image relative to the dental arch) as well as angles relative to the plane of the dental arch (e.g., roll, pitch and / or yaw, or radial coordinates, etc.) ("positional information" (1301)). In some variations, the scan may be a composite of multiple scans (e.g., average, blend, etc.) that are combined and stored in the data structure. A 3D model can be formed by virtually "stitching" scans to form a 3D model.

[0173] Data structures can be stored in a compressed configuration; although they may contain a large amount of data, the compression and configuration of the data structure can be manipulated to display it. For example, FIG. 12 illustrates one method of interactively displaying a 3D model of a patient's dental arch using a data structure such as that schematically illustrated in FIG. 13.

[0174] In FIG. 12, the method includes displaying a 3D model of the patient's dental arch (1201) and displaying a view window (1203) on the 3D model. The user can then move the two (e.g., one or both of the view window and the 3D model) so that the teeth of the dental arch can be virtually viewed in more detail through the view window in a close view (1205). The angle of the view window as well as the position of the view window along the dental arch can be changed by the user, for example, by moving continuously over and / or around the 3D model of the dental arch (1207). As the view window / dental arch moves relative to each other, a corresponding image (or images), such as a near-infrared image taken at a position on the dental arch corresponding to the position of the view window, can be identified (1209) from a data structure / data set (e.g., FIG. 13). Next, the corresponding image(s) can be displayed (1211), and this process can be repeated as the view window moves along the 3D dental arch model.

[0175] In some variations, the data structure can be organized or arranged topographically or indexed topographically; thus, images of adjacent areas are linked or aligned within the data structure, simplifying the method.

[0176] FIGS. 14a-16c illustrate an example of a variation of a user interface that allows a user to virtually scan a 3D model of the dental arch and display a corresponding image (e.g., a near-infrared image) as described in FIG. 12. As mentioned above, the near-infrared image can be viewed by the user to manually (or automatically in some variations) identify one or more structural / defect and / or actionable tooth features, including caries, cracks, wear, etc. Displaying a visible light image of the same area as the corresponding 3D dental arch model provides a sense of perspective and enables immediate comparison with the patient's teeth, thereby simplifying and strongly enhancing dental analysis.

[0177] For example, in FIG. 14a, the display is illustrated as a user interface (1400) containing a dental arch model (3D dental arch model) (1403) that reconstructs a scan of the patient's teeth and is stored in a data structure along with many or all of these scans. As mentioned above, the 3D dental arch model does not necessarily need to be included in the data structure along with multiple images, but it may be helpful. Also, it is clear that in this example, the 3D dental arch model is composed of the patient's tooth scan, but the 3D dental arch model may not be representative and may be used to select a 2D view to be displayed as described herein. A view window (1401), illustrated as a loop or circle, may be moved over or along the 3D model of the dental arch; as the view window moves, each of the two image displays (1405, 1407) is updated with an image corresponding to the position (both the area of ​​the dental arch and the angle of the dental arch relative to the plane of the view window). In FIG. 14a, the first (upper) image (1405) is a near-infrared image, and a corresponding visible light (e.g., color) image (taken at the same approximate time / location) is shown in the lower image (1407). Alternative displays are shown in FIG. 14b and 14c, each showing only one image; FIG. 14b shows an enlarged near-infrared display image, and FIG. 14c shows a single enlarged visible light display image of an area corresponding to the imaging window view.

[0178] The user interface may also include tools (1409) for manipulating the display (e.g., rotating, moving, modifying, marking, etc., images and / or 3D models, saving images, opening / calling, etc.).

[0179] FIGS. 15a-15b illustrate an example of moving a view window over the teeth and changing / updating the corresponding image. FIG. 15a illustrates an image of the dental arch having corresponding near-infrared and light images as seen through the view window in the middle region of the dental arch. In FIG. 15b, the dental arch is rotated by the user (or alternatively, the view window is rotated relative to the dental arch and relative to the tongue) so that the view window is positioned slightly lingually relative to FIG. 15a; the corresponding view (near-infrared and visible light) is updated in real time to show the change in the relative position of the view window.

[0180] Similarly, FIGS. 16a-16c illustrate examples of 3D models of a patient's mandibular arch similar to the views illustrated in FIGS. 14a-14c. When in use, as the user moves the view window (and / or the dental arch relative to the view window) to scan along the dental arch, the display images can change virtually continuously, thereby being updated in real-time or near real-time. The user can identify features of the near-infrared image(s), including changes in density of typical infrared-transmitting enamel areas, which may indicate caries, cracks, or wear of the enamel.

[0181] The intraoral scanning system illustrated in FIGS. 1a-1b may be configured as an intraoral scanning system. Returning to FIG. 1a, the intraoral scanning system (101) includes a handheld wand (103) having at least one image sensor and a light source configured to emit light in a spectral range within the near-infrared (near-IR) wavelength range, and a display output (screen 102). The screen may be a touchscreen operating as a user input device, or the system may include a separate user input device (e.g., keyboard, touchpad, joystick, mouse, trackball, etc.). As shown in FIG. 1b, the system may also include one or more processors operably connected to the handheld wand, the display, and the user input device. The one or more processors: display a three-dimensional (3D) model of the patient's dental arch on the display output; display a view window over a portion of the 3D model of the patient's dental arch on the display output; change the relative position between the 3D model of the patient's dental arch and the view window based on input from the user input device; The circuit and / or software and / or firmware may be configured to identify near-infrared (near-IR) images taken from both a 3D model of the patient's dental arch and multiple images of the patient's dental arch taken at different angles and positions relative to the 3D model of the patient's dental arch and a related view window, and to display the identified near-infrared images taken at angles and positions relative to the 3D model of the patient's dental arch and the view window (as illustrated in FIGS. 14A-16C).

[0182] Automatic characterization of tooth features

[0183] Also described herein are methods and apparatus (e.g., systems including software) configured to automatically or semi-automatically identify, identify, and / or characterize one or more dental features using a 3D model, including but not limited to a volumetric 3D model of all or part of a patient's dental arch. In particular, such methods and apparatus may be configured to identify, identify, and / or characterize one or more actionable dental features that can benefit from detection and / or treatment. Actionable dental features may include, but are not limited to, cracks, gingival recession, calculus, hard and soft tissue oral conditions, etc. Enamel thickness may be another actionable dental feature. For example, the methods and apparatus described herein may automatically map enamel thickness (e.g., apply a color map when the enamel is less than x microns thick, where x is preset and / or user adjustable). Areas of thin enamel are potential areas where caries may be present. Other potentially actionable tooth features may include discoloration (e.g., discontinuity of color), grooves, cracks, evidence of grinding, thinning (including thinning over time), interdental spaces, or other similar features that may indicate or suggest locations where caries may form.

[0184] Any of the methods and apparatus described herein may use multiple different images or sets of images of a patient's teeth used to detect, analyze, and / or characterize tooth features, particularly actionable tooth features. Multiple different images or sets of images of a patient's teeth taken by different imaging techniques may each be referred to as a "record." Each record may be a different imaging technique, such as dental cone beam computed tomography (CBCT) scanning, three-dimensional (3D) intraoral scanning, color scanning (one or more of 3D color scanning, surface color scanning, etc.), two-dimensional (2D) color scanning, near-infrared scanning (including but not limited to volumetric near-infrared imaging, trans-illumination and / or reflection scanning), X-ray (including but not limited to cephalometric analysis X-ray scanning, panoramic X-ray scanning, etc.), and may include text or graphic chart information of the patient.

[0185] For example, each record may be processed independently at first. One or more tooth features, in particular one or more actionable tooth features, may be identified by this initial scan. To identify one or more actionable tooth features, a single record (e.g., a single imaging technique) may be used first, or all records or a subset of records may be processed first to identify one or more actionable tooth features. The initial identification of one or more actionable tooth features may be performed manually, automatically, or semi-manually. For example, one or more actionable tooth features may be identified automatically; the system described herein may review the record (including one or more images of the patient's teeth) to flag or identify regions having characteristics associated with actionable tooth features. The system may be trained using machine learning techniques such as supervised learning techniques (e.g., classification, regression, similarity, etc.), unsupervised learning techniques (e.g., density estimation, cluster analysis, etc.), reinforcement learning (e.g., Markov Decision Process techniques, etc.), representation learning techniques, and / or principal component analysis to identify / flag areas of specific scans in specific techniques associated with (or even loosely associated with) actionable dental characteristics. Alternatively or additionally, a user (dentist, technician, etc.) may manually review one or more records (each representing a specific imaging technique) and flag or identify areas suspected of showing actionable dental characteristics. In a semi-automatic configuration, the system may initially flag one or more areas in a record, which the user can subsequently review and confirm / reject.

[0186] As one or more regions are identified, they may be flagged and / or stored in a collection of potential actionable tooth features. The location may be relative to the original record (e.g., location) or relative to a reference model (such as a 3D volumetric model described in more detail below). In some variations, the collection (e.g., array, data structure, file, etc.) may also include one or more of the type of potential actionable tooth feature, the range of potential actionable tooth features, the grade and / or degree of potential actionable tooth features, the original record, and / or the imaging technique of the original record. In some variations, the data structure may be integrated into the original record (or a copy thereof) and may modify the image(s) of the original record, for example, by including flags or markers at the locations of identified potential actionable tooth features and / or by including metatext such as grade and / or degree. The grade and / or degree may refer to the confidence level or score for the potential actionable tooth feature, including the confidence level or score indicating the likelihood that the identified potential actionable tooth feature is 'real'.

[0187] This initial identification process to identify potential actionable dental features can be performed across multiple records or, as mentioned above, limited to a subset of records (e.g., including only one of the records). In some variations, the process can be performed iteratively.

[0188] If one or more potential responsive dental features are identified, they may be cross-referenced with one or more other records using different imaging techniques. Accordingly, the location of one or more potential responsive dental features may be investigated in particular detail to determine whether the same potential responsive dental features are evident in these one or more other records. In some variations, the entire additional record(s) may be examined during this verification part of the procedure, any additional potential responsive dental features from one or more additional records may likewise be flagged as potential responsive dental features, and the same area of ​​the dental arch may be examined for these other potential responsive dental features (including a return to a record already reviewed, such as the first or original record).

[0189] Comparison between different records can be guided by translating the location of tooth features (including, but not limited to, potentially addressable tooth features) between different records. In particular, it may be helpful to coordinate the individual dental record(s) that begin the examination against a 3D model, specifically a model of the patient's dental arch such as the 3D volumetric model described above. Thus, the 3D model of the dental arch can serve as a key element in translating the location of one or more potentially addressable tooth features and can allow for rapid and efficient comparison between different records, for example, between different imaging techniques.

[0190] Accordingly, correlations between each of the different records, particularly between all or part of the different records and a 3D model of the dental arch (e.g., a 3D volumetric model), may be established before or after the initial scan for potential actionable tooth features. Any method of correlating the records and other records and / or the 3D model of the patient's dental arch (or part of the dental arch) may be used. For example, one or more easily recognizable features (e.g., tooth edges, shapes, segments, etc.) may be used to determine landmarks that can be transformed between one or more records and / or between one or more records and a 3D model of the patient's dental arch. In some variations, a transformation dataset may be generated that includes transformations between records and / or between each record and the 3D volumetric model of the patient's dental arch. For example, the 3D volumetric model of the entire or part of the dental arch may include transformation information for each of the one or more records that allows for transformations of the image(s) of one or more records, such as estimates of distance and / or orientation of the imaging technique for the record image(s). This allows the position between each record and a 3D model (e.g., a volumetric model) to be converted in the forward and reverse directions.

[0191] Accordingly, the transformed data set may include a 3D model and transformation information for each record, so that a part or region of the record image (or image) can be projected onto the 3D (transformed) model, and the same region can be re-projected onto a second (or more) record taken with a different imaging technique to examine the same region. In some variations, the process may begin with the collection of all records and / or automatic, manual, or semi-automatic registration between all records. For example, the identification of regions such as individual teeth, palate, and gingiva may be used for cross-correlation between different imaging techniques and / or 3D models. In one example, a record containing an X-ray image may be correlated with a 3D volumetric model of the patient's teeth by decoding (manually or automatically) the position and / or orientation of the X-ray camera acquiring the X-ray image corresponding to the record. The volumetric model may be used to determine and / or verify the position and / or orientation of the imaging source for each record. In some variations, the record includes explicit (e.g., recorded) information regarding the location and / or orientation and / or imaging parameters used to take the image(s); alternatively or additionally, this information may be derived. As described above, a pseudo-X-ray image is generated and can be compared with the actual X-ray image of the record.

[0192] When an area corresponding to a region of a potential actionable tooth feature is identified from another record, the system or method can determine whether the same potential actionable tooth feature is present in these other records. If present, the score (e.g., the confidence score indicating the likelihood that the potential actionable tooth feature is real) may be adjusted, and may increase, for example, if identical or similar potential actionable tooth features are present. Depending on the type of record and the type of potential actionable tooth feature, the confidence score may be adjusted due to the absence of the potential actionable tooth feature. For example, the absence of surface features that are generally undetectable by X-ray, such as discoloration, plaque, or gingival recession, may not lower the confidence score of one or more potential actionable tooth features. The more frequently potential actionable tooth features are found in corresponding locations across different records (and thus across different techniques), the higher the likelihood that the potential actionable tooth feature is actually present.

[0193] In comparing the corresponding locations of one or more potential actionable tooth features, the regions may be examined manually, automatically, or semi-automatically, similar to the original identification techniques discussed above. For example, regions in additional records corresponding to the locations of potential actionable tooth features in other records may be examined automatically to identify features correlated with the types of potential actionable tooth features. The system may be trained to recognize potential actionable tooth features in the imaging techniques of additional records and may provide a score indicating the likelihood of the potential actionable tooth feature being present at this location. In some variations, images of additional records may be displayed to a user (e.g., technician, dental professional, etc.), and the likelihood of the potential actionable tooth feature being present in one or more additional records (yes / no, grade scale, numeric scale, etc.) may be manually indicated.

[0194] The final confidence value determined for each potential actionable tooth feature may be used by a system that stores, transmits, and / or displays it. For example, the potential actionable tooth feature(s) may be presented to the dentist in any appropriate manner, including a list on a display such as a 3D model of the marked dental arch (including an interpretive 3D dental model). For example, the system may output a display highlighting, by color, shape, etc., the locations of some or all of the potential actionable tooth features that exceed a threshold confidence level (likely to be 'real'). The display may also include one or more views of the potential actionable tooth features (from one or more records). The user may adjust or set the threshold confidence level appropriately (e.g., by making the threshold somewhat stricter and showing the addition or removal of potential actionable tooth features in response).

[0195] FIG. 17 illustrates an example of a method (1700) for characterizing tooth features across different imaging techniques as just discussed. In FIG. 17, the method (or a system configured to perform the method) can identify one or more actionable tooth features from one or more records (e.g., one or more images or sets of images of a patient's teeth taken with different imaging techniques) (1701). For example, one or more actionable tooth features may be identified by an agent or engine configured to automatically detect one or more actionable tooth features. For example, a system performing the method of FIG. 17 may include an actionable tooth feature analysis engine, or may include a plurality of actionable tooth feature analysis engines, each configured to identify one or more types of actionable tooth features or one or more types of imaging techniques. An engine (e.g., an actionable tooth feature analysis engine) may be part of a computer system. As used herein, an engine includes one or more processors or parts thereof. One or more parts of a processor may include a portion of hardware less than all hardware comprising any given one or more processors, such as a subset of registers, a portion of a processor dedicated to one or more threads of a multithreaded processor, or a time slice to which the processor is wholly or partially dedicated to performing a portion of the engine's functions. As such, the first engine and the second engine may have one or more dedicated processors, and the first engine and the second engine may share one or more processors with each other or with other engines. Depending on the specific implementation or other considerations, the engines may be centralized or functions may be distributed. The engines may include hardware, firmware, or software implemented on a computer-readable medium for execution by the processor.The processor converts data into new data using a data structure and method implemented as described with reference to the drawings in this specification.

[0196] The engine described herein or the engine in which the system and device described herein may be implemented may be a cloud-based engine. As used herein, a cloud-based engine is an engine capable of executing applications and / or functions using a cloud-based computing system. All or part of the applications and / or functions may be distributed across multiple computing devices and are not limited to a single computing device. In some embodiments, the cloud-based engine may execute functions and / or modules accessed by the end user through a web browser or container application without functions and / or modules installed locally on the end user's computing device.

[0197] Returning to Fig. 17, one or more actionable tooth features can be identified manually or semi-manually in one or more records. For example, an actionable tooth feature analysis engine can initially identify one or more actionable tooth features, which can then be verified or inspected by a user (e.g., a dental technician).

[0198] Each identified actionable tooth feature may be flagged and / or recorded, for example, in a collection of potential actionable tooth features (1703). For example, a collection of potential actionable tooth features may be part of a data structure. Adding potential actionable tooth feature(s) to a collection (e.g., a data structure) may include recording one or more of the location of the actionable tooth feature (e.g., in the original record) and / or the type of the actionable tooth feature, the grade / reliability of the actionable tooth feature, etc. The data structure used herein (which may be included as part of a data store) is intended to include a store having an applicable data configuration, including a table, a comma-separated value (CSV) file, an existing database (e.g., SQL), or other applicable known or convenient structure format. For example, the data store may be implemented as a physical computer-readable medium of a machine for a specific purpose, firmware, hardware, a combination thereof, or software implemented on an applicable known or convenient device or system. Data store-associative components, such as database interfaces, may be considered as "part" of the data store, part of some other system components, or a combination thereof; however, the physical location and other characteristics of the data store-associative components are not important for understanding the technology described herein.

[0199] Data structures can be associated with specific ways of storing and organizing data in a computer so that it can be used efficiently within a given context. Data structures are generally based on the computer's ability to retrieve and store data at any addressable location in memory, which is a string of bits that is stored in memory and can be manipulated by a program. Thus, some data structures are based on calculating the address of a data item through arithmetic operations; on the other hand, other data structures are based on storing the address of a data item within the structure itself. Many data structures use both principles and are sometimes combined in a non-trivial manner. The implementation of a data structure generally requires writing a series of procedures to create and manipulate instances of that structure. The data store described in this specification may be a cloud-based data store. A cloud-based data store is a data store compatible with cloud-based computing systems and engines.

[0200] Identified "presumed" actionable tooth features (e.g., "potential actionable tooth features") can be mapped to corresponding physical locations in one or more other records (1705). As discussed above, in some variations this can be done using a 3D volumetric model, which can be converted between various different types of records (having different imaging techniques), including projecting the first record onto the 3D model and then projecting it back onto the second region.

[0201] Therefore, the same corresponding regions in other records may be reviewed to determine whether potential actionable tooth features exist or are implied in additional record(s). In some variations, the method may simply collect all different corresponding regions for storage, transmission, and / or provision to a user (e.g., a dental professional), for example, optionally pausing to allow the user to review these flagged regions in parallel across multiple different imaging techniques (records). For example, potential actionable tooth features may be displayed side-by-side (e.g., tiled) or sequentially for all corresponding views.

[0202] Alternatively or additionally, the method and / or system may automatically or semi-automatically adjust the confidence score for each identified potential actionable tooth feature. Thus, the system may determine whether additional records indicate a higher or lower probability of the existence of a potential actionable tooth feature, and may adjust (or determine) the confidence score for each potential actionable tooth feature based on the appearance at the corresponding location of the additional records(s) (1707).

[0203] Adjusted confidence levels may be used to narrow down potential actionable tooth features. For example, a method or system may apply filtering and / or thresholds based on an adjusted confidence level for each potential actionable tooth feature (1709). In some variations, the threshold may be fixed (e.g., a confidence level greater than x, where x is an intermediate numeric value between 0 confidence and 1 (absolute confidence), e.g., 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, etc.). In some variations, the threshold may be manually adjusted by the user and / or based on one or more features of the record, such as quality metrics specific to each record.

[0204] Potentially actionable tooth features having a confidence level exceeding a threshold may be stored, presented, and / or transmitted (1711). For example, the user may be provided with a final list and / or display containing flagged potentially actionable tooth features (e.g., using a 3D model).

[0205] Any method and apparatus (e.g., system) described herein may be configured to construct a data structure comprising all or part of multiple records. For example, the data structure may include all or part of a 3D volumetric model and the associated 2D images used to construct it, as described above. Additionally, the data structure may include additional records, such as images taken by X-rays (e.g., panoramic) and / or CBCT, etc. Optionally, information including metadata, such as text information about the patient and / or images, may be included, including patient chart information from the patient's health / dental records. Alternatively or additionally, identified potential actionable dental features (e.g., results identified in the records) may also be included. Other records may be searched / located / marked using potential actionable dental features.

[0206] Generally, when constructing a 3D (e.g., volumetric) model by compiling an image (e.g., a 2D near-infrared image), the 2D image that provided the information may be marked to indicate its importance to the 3D model. For example, the 2D image may be marked as having low relevance or high relevance.

[0207] As mentioned above, a collection of potential actionable tooth features, including confidence levels based on their presence in multiple records, may be included as part of the same data structure containing a 3D model or separated from it. The 3D model may be directly marked (flags, codes, etc.) to include potential actionable tooth features. Thus, the data structure may be a compilation of all different records. The combined / compiled data structure may be referred to as the marked data structure or the actionable tooth feature data structure.

[0208] Any record, including near-infrared 2D images, can be used / scanned to identify potential actionable dental features. If, as described above, a suspicious area is identified in one of the records automatically, semi-automatically / semi-manually, or manually (e.g., by a user), the method or system may search the corresponding area of ​​the dental arch in all or part of the other records and conclude whether a finding exists. In some variations, the method or device may update images in all or part of the records (and / or in a combined data structure) based on the analysis described herein.

[0209] In any method and system described herein, tooth segmentation may be used in all or part of the record and / or 3D model to improve performance and utility. Tooth segmentation may be added prior to volumetric modeling to support and improve volumetric results and model quality. For example, the volumetric 3D model may use segment information to potentially improve performance as additional surface 3D information is added. Segment information may also help improve automatic detection and marking of suspicious areas by segmenting the enamel-dentin-lesion (including, but not limited to, when using an automated agent to identify potentially actionable tooth features). Alternatively or additionally, tooth segmentation may be added to post-processing of volumetric modeling to support enamel-dentin-lesion segmentation to improve automatic detection and marking of suspicious areas. For example, segmentation may help correlate structures between different imaging techniques, including registering results for volumetric measurements with other modalities to provide cross-modal visualization. Tooth segmentation can be used to improve the recording of clinical outcomes and annotations, as well as cross-modal visualization.

[0210] The confidence level indicated in any method and device described herein may be a quantitative and / or qualitative indicator. For example, a quantitative confidence level "score" may be provided (e.g., using numbers between 0 and 100, 0 to 1.0, -100 to 100, or adjusted to a numeric value within any range). Qualitative indicators may include "high, upper-middle, medium, lower-middle, low," etc. Both qualitative and quantitative confidence levels may be used. A confidence level assessment system based on multiple records described herein may affect insurance claims and / or patient communication.

[0211] In any of the methods and systems described herein, the shape of the dental arch may be helpful in identifying areas of interest or potential problems. Accordingly, generally, a 3D model (volumetric model) may be used and / or modified as described herein to include areas of potential actionable tooth features. The modified 3D model may serve as a map visually representing the area of ​​the risk assessment; it may be used to guide patient treatment, including recommending the use of, for example, sealants, orthodontic treatment, or night protection. In some variations, the modified 3D model may be used to guide the user when additional scans are required (e.g., when there are few scans in the risk area). As used herein, the modified 3D model may include a 3D (e.g., volumetric and / or surface) model marked to indicate the location and / or type and / or confidence level of potential actionable tooth features. Therefore, generally, the use of additional data sources to guide the user in capturing potential areas of interest (e.g., records, particularly those appearing in records other than near-infrared / NIRI scans) can help identify the results of potentially actionable dental features. As previously mentioned, results including modified 3D models can help guide the user to scan or (future) rescan their dentition. For example, past scans can be used as target maps during the scan (and to verify appropriate coverage in those areas). One or more derived images / presentations may be used additionally or alternatively. For example, tooth segmentation can be used to generate tooth chart maps (e.g., from 3D volumetric models) that can be used for follow-up and automatic import into dental care management software (DPMS). For example, individual records can be side-by-side to align specific issues with the tooth map.

[0212] All methods described herein (including user interfaces) may be implemented in software, hardware, or firmware and may be described as a non-transient computer-readable storage medium storing a set of instructions that can be executed by a processor (e.g., computer, tablet, smartphone, etc.), which, when executed by a processor, controls the processor to perform all steps including but not limited to displaying, communicating with a user, analyzing, modifying parameters (including timing, frequency, intensity, etc.), determining, warning, etc.

[0213] In this specification, when a feature or element is referred to as being "on" another feature or element, it may be directly on the other feature or element, or intermediate features and / or elements may exist. In contrast, when a feature or element is referred to as being "directly on" another feature or element, there are no intermediate features or elements. Additionally, when a feature or element is referred to as being "connected," "attached," or "coupled" to another feature or element, it may be directly connected, attached, or coupled to the other feature or element, or intermediate features or elements may exist. In contrast, when a feature or element is referred to as being "directly connected," "directly attached," or "directly coupled" to another feature or element, there are no intermediate features or elements. Although described or illustrated in relation to one embodiment, the features and elements described or illustrated as such may apply to other embodiments. In addition, it will be recognized by those skilled in the art that a reference to a structure or feature placed "adjacent" to another feature may have a portion that overlaps or lies beneath the adjacent feature.

[0214] The terms used herein are for describing specific embodiments and are not intended to limit the invention. For example, the singular forms “a,” “an,” and “the” used herein are intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that when used herein, the terms “comprises” and / or “comprising” specify the presence of the mentioned features, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, actions, elements, components, and / or groups thereof. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items and may be abbreviated as “ / ”.

[0215] Spatially relative terms such as "under," "below," "lower," "over," and "upper" may be used to describe the relationship between one element or feature and another element(s) or feature(s) for ease of explanation. It should be understood that spatially relative terms are intended to include other directions of the device in use or operation in addition to the directions depicted in the drawings. For example, if the device in the drawing is inverted, an element described as "under" or "beneath" another element or feature will be oriented "over" of that other element or feature. Thus, the exemplary term "under" may include both upward and downward directions. The device may be rotated in other directions (90 degrees or other directions), and the spatially relative descriptors used herein are interpreted accordingly. Similarly, terms such as "upwardly," "downwardly," "vertical," and "horizontal" are used in this specification for descriptive purposes unless specifically stated otherwise.

[0216] Although the terms “first” and “second” may be used herein to describe various features / elements (including steps), such features / elements should not be limited by these terms unless the context otherwise indicates. These terms may be used to distinguish one feature / element from another. Accordingly, the first feature / element discussed below may be referred to as the second feature / element, and similarly, the second feature / element discussed below may be referred to as the first feature / element without departing from the teachings of the present invention.

[0217] Unless otherwise required by the context, throughout this specification and the following claims, the word “comprise” and variations such as “comprises” and “comprising” signify that various components may be used jointly in methods and provisions (e.g., configurations and devices including devices and methods). For example, the term “comprising” will be understood to mean the inclusion of any mentioned element or step, but not the exclusion of other elements or steps.

[0218] Generally, any device and method described herein should be understood as comprehensive, but all or a subset of components and / or steps may alternatively be exclusive, and various components, steps, subcomponents, or substeps may be expressed as "consisting of" or alternatively "consisting essentially of".

[0219] As used herein in the specification and claims, and unless explicitly specified otherwise than as used in the examples, all numbers may be read as if preceded by the words “about” or “approximately,” even if the term is not explicitly indicated. The phrases “about” or “approximately” may be used when describing a size and / or location to indicate that the described value and / or location is within a reasonable expected range of the value and / or location. For example, a numeric value may have a value of + / - 0.1% of the specified value (or value range), + / - 1% of the specified value (or value range), + / - 2% of the specified value (or value range), + / - 5% of the specified value (or value range), + / - 10% of the specified value (or value range), etc. Any numeric value given herein should be understood to include the value of about or approximately unless the context otherwise indicates. For example, if the value “10” is disclosed, “about 10” is also disclosed. Any numerical range mentioned in this specification is intended to include all sub-ranges contained therein. Additionally, it is understood that when a value is disclosed as "less than or equal to," a possible range between "greater than or equal to the value" and the value is also disclosed as appropriately understood by those skilled in the art. For example, when a value "X" is disclosed, not only "less than or equal to X" but also "greater than or equal to X" (e.g., where X is a numeric value) is disclosed. Furthermore, data is provided in various formats throughout the application, and this data represents endpoints, starting points, and ranges for all combinations of data points. For example, when a specific data point "10" and a specific data point "15" are disclosed, it is understood that greater than, greater than or equal to, less than, less than or equal to, and equal to 10 and 15, respectively.It is also understood that each unit between two specific units is also initiated. For example, if 10 and 15 are initiated, 11, 12, 13, and 14 are also initiated.

[0220] Although various exemplary embodiments have been described above, numerous modifications may be made to various embodiments without departing from the scope of the invention as defined by the claims. For example, the order in which various described method steps are performed may often be changed in alternative embodiments, and one or more method steps may all be omitted in other alternative embodiments. Optional features of various device and system embodiments may be included in some embodiments and not in others. Accordingly, the foregoing description is provided primarily for illustrative purposes and should not be interpreted as limiting the scope of the invention as defined by the claims.

[0221] The examples and illustrations included in this specification illustrate specific embodiments in which the subject matter may be practiced, as examples rather than limitations. As noted, other embodiments may be utilized and derived therefrom, so that structural and logical substitutions and modifications may be made without departing from the scope of this disclosure. Such embodiments of the subject matter of the invention may be referred to individually or collectively in this specification by the term “invention” merely for convenience, and there is no intention to voluntarily limit the scope of this application to any single invention or concept of invention where actually more than one is disclosed. Accordingly, while specific embodiments are illustrated and described herein, any arrangement calculated to achieve the same purpose may substitute for the specific embodiments illustrated. This disclosure is intended to include any and all adaptations or variations of various embodiments. Combinations of the above embodiments and other embodiments not specifically described in this specification will be apparent to those skilled in the art upon reviewing the above description.

Claims

Claim 1 A method performed by one or more processors of a system comprising: displaying a three-dimensional (3D) model of a patient's dental arch; displaying a view window over at least a portion of the 3D model of the patient's dental arch; receiving from a user a change in the relative position between the view window and the 3D model of the patient's dental arch; identifying a near-infrared (near-IR) image taken from both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken at different angles and positions relative to the 3D model of the patient's dental arch and the related view window, the near-infrared image taken at an angle and position close to the relative angle and position between the 3D model of the patient's dental arch and the related view window; and displaying the identified near-infrared image taken at an angle and position close to the angle and position between the 3D model of the patient's dental arch and the view window. Claim 2 A method according to claim 1, wherein the step of identifying the near-infrared image and displaying the identified near-infrared image is performed continuously as the user changes the relative position between the view window and the 3D model of the patient's dental arch. Claim 3 A method according to claim 1, wherein the step of allowing the user to change the relative position between the view window and the 3D model of the patient's dental arch comprises allowing the user to change the angle between the plane of the view window and the patient's dental arch, and one or more of the parts of the dental arch adjacent to the view window. Claim 4 A method according to claim 1, wherein the identifying step comprises determining a plurality of images that are close to the relative angle and position between a 3D model of the patient's dental arch and the associated view window, and averaging the plurality to form the identified near-infrared image. Claim 5 A method according to claim 1, further comprising the step of collecting a data set in the processor that includes a 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken at different angles and positions with respect to the patient's dental arch. Claim 6 A method according to claim 1, wherein the step of displaying a view window over a part of a 3D model of the patient's dental arch includes the step of displaying a loop—through which the part of the 3D model of the patient's dental arch can be viewed. Claim 7 A method according to claim 1, wherein the step of causing the user to change the relative position between the view window and the 3D model of the patient's dental arch comprises the step of separately controlling one or more of the 3D model angle of the patient's dental arch, the 3D model rotation of the patient's dental arch, and the dental arch portion adjacent to the view window. Claim 8 A method according to claim 1, wherein the step of allowing the user to change the relative position between the view window and the 3D model of the patient's dental arch includes the step of allowing the user to move the view window over the 3D model of the dental arch. Claim 9 The method of claim 1 further comprises the step of identifying a second image that is close to the relative angle and position between the 3D model of the patient's dental arch and the related view window from both the 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken at different angles and positions with respect to the patient's dental arch, wherein the step of identifying the second image includes the step of identifying one of a visible light image and a fluorescent image and the step of displaying the second image simultaneously with the near-infrared image. Claim 10 A method according to claim 1, wherein the step of displaying an identified near-infrared image close to the angle and position of a view window for the displayed 3D model comprises the step of displaying the identified near-infrared image in a window adjacent to or overlapping with the display of the 3D model of the patient's dental arch. Claim 11 delete Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete Claim 21 An intraoral scanning system comprises: a handheld wand having at least one image sensor and a light source configured to emit light in a spectral range within a near-infrared (near IR) wavelength range; a display output; a user input device; and one or more processors operably connected to the handheld wand, the display, and the user input device, wherein the one or more processors display a three-dimensional (3D) model of a patient's dental arch on the display output; display a view window on a portion of the 3D model of the patient's dental arch on the display output; and change the relative position between the 3D model of the patient's dental arch and the view window based on input from the user input device; A system configured to identify a near-infrared (near-IR) image taken at an angle and position close to the relative angle and position between the 3D model of the patient's dental arch and the relevant view window, from both a 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken at different angles and positions with respect to the patient's dental arch, and to display the identified near-infrared image taken at an angle and position close to the angle and position between the 3D model of the patient's dental arch and the view window. Claim 22 An intraoral scanning system according to claim 21, wherein one or more processors are configured to receive a plurality of images of the patient's dental arch taken from different angles and positions relative to the patient's dental arch. Claim 23 An intraoral scanning system according to claim 21, wherein one or more processors are configured to continuously identify the near-infrared image and display the identified near-infrared image as the user changes the relative position between the view window and the 3D model of the patient's dental arch. Claim 24 An intraoral scanning system according to claim 21, wherein one or more processors are configured to change the relative position between a 3D model of the patient's dental arch and the view window based on input from the user input device by changing the angle between the plane of the view window and the patient's dental arch and one or more of the parts of the dental arch adjacent to the view window. Claim 25 An intraoral scanning system according to claim 21, wherein one or more processors are configured to identify a near-infrared image taken at an angle and position close to the relative angle and position between the 3D model of the patient's dental arch and the related view window by determining a plurality of images close to the relative angle and position between the 3D model of the patient's dental arch and the related view window and averaging the plurality to form the identified near-infrared image. Claim 26 An intraoral scanning system according to claim 21, wherein one or more processors are configured to display a view window over a portion of a 3D model of the patient's dental arch, and to display a loop—through which the portion of the 3D model of the patient's dental arch can be viewed. Claim 27 An intraoral scanning system according to claim 21, wherein one or more processors are configured to change the relative position between the 3D model of the patient's dental arch and the view window based on input from the user input device by changing one or more of the angle of the 3D model of the patient's dental arch with respect to the view window, the rotation of the 3D model of the patient's dental arch with respect to the view window, and a portion of the dental arch adjacent to the view window. Claim 28 An intraoral scanning system according to claim 21, wherein one or more processors are configured to change the relative position between the view window of the patient's dental arch and the 3D model based on input from the user input device by changing the position of the view window on the 3D model of the dental arch. Claim 29 An intraoral scanning system according to claim 21, wherein one or more processors identify a second image that is close to the relative angle and position between the 3D model of the patient's dental arch, which is one or more of a visible light image and a fluorescent image, and the associated view window, from both a 3D model of the patient's dental arch and a plurality of images of the patient's dental arch taken at different angles and positions relative to the patient's dental arch; and the one or more processors configured to display the second image simultaneously with the near-infrared image. Claim 30 delete Claim 31 delete Claim 32 delete Claim 33 delete Claim 34 delete Claim 35 delete Claim 36 delete Claim 37 delete Claim 38 delete Claim 39 delete Claim 40 delete Claim 41 delete Claim 42 delete

Images