Intraoral scanner having dental diagnosis function

Abstract

To provide a method and a device for generating a model of teeth of a subject.SOLUTION: A scan device of the present invention may include an intraoral scanner for scanning the inside or the periphery of the oral cavity of a subject. The intraoral scanner includes one or more light sources for illumination in two or more spectrum ranges, that is, a spectrum range for illuminating a surface shape, and a permeable spectrum range. The scan device may further include one or more sensors for detecting the emitted light and one or more processors. The processors control scan operations, and generate a model of teeth of the subject including the surface of the teeth and the shape in the teeth (including inside of enamel and dentin) by analysing the light received both in the first spectrum range and in the second spectrum range. The generated model may be a 3D volumetric model or a panoramic image.SELECTED DRAWING: Figure 1A

Description

Technical Field

[0001] Cross - Reference to Related Applications This patent application claims priority to each of U.S. Provisional Patent Application No. 62 / 367,607, filed Jul. 27, 2016, entitled "INTRAORAL SCANNER WITH DENTAL DIAGNOSTICS CAPABILITIES"; U.S. Provisional Patent Application No. 62 / 477,387, filed Mar. 27, 2017, entitled "INTRAORAL SCANNER WITH DENTAL DIAGNOSTICS CAPABILITIES"; and U.S. Provisional Patent Application No. 62 / 517,467, filed Jun. 9, 2017, entitled "MINIMAL VALUE LIFTING TO FORM A VOLUMETRIC MODEL OF AN OBJECT". Each of these is hereby incorporated by reference in its entirety. Citation of References

[0002] All publications and patent applications mentioned in this specification are hereby incorporated by reference in their entirety to the same extent as if each individual publication or patent application were specifically and individually indicated to be incorporated by reference.

[0003] The methods and apparatuses described herein may relate to optical scanners, and in particular, to optical scanners that generate three - dimensional representations of objects. Specifically, this specification describes methods and apparatuses that may be useful for scanning (including 3D scanning) and analyzing the oral cavity for diagnosis, treatment, long - term follow - up, dental measurement, and detection of dental caries and cracks. These methods and apparatuses may generate volumetric models of the internal structure of teeth and / or may include color scans.

Background Art

[0004] For many dental and orthodontic procedures, an accurate three-dimensional (3D) depiction of a patient's dental structure and oral cavity can be useful. In particular, it would be useful to provide 3D renderings of both the surface and internal structure of teeth (including enamel and dentin, and including caries), as well as a rough internal composition of the volume of the teeth. It has been found that even just the surface representation of the 3D surface of teeth is very useful for the design and fabrication of dental prostheses (e.g., crowns and bridges) and treatment planning, but it would be very useful to be able to image internal structures including the growth of caries and cracks in enamel and underlying dentin, and particularly useful when combined with surface topography mapping.

[0005] Historically, ionizing radiation (e.g., X-rays) has been used for imaging the interior of teeth. For example, bitewing X-rays are often used when obtaining a non-quantitative image of the interior of a tooth. However, such images, in addition to the risk of ionizing radiation, typically have limitations in their ability to show shape and can be time-consuming and costly to obtain. Other techniques, such as cone beam computed tomography (CBCT), can provide tomographic images, but still require ionizing radiation.

[0006] Accordingly, it would be useful to provide methods and devices (including devices and systems such as intraoral scanning systems) that can be used to model one or more teeth of a subject using non-ionizing radiation and include both external (surface) and internal (within enamel and dentin) structures and compositions. The model of the subject's teeth may be a 3D volumetric model or a panoramic image. In particular, it would be useful to provide methods and devices that can provide this functionality in a single device. There is a need for improved methods and systems for scanning a patient's oral cavity and / or for automating the identification and analysis of tooth caries. SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] The present invention has been made to solve the above-described problems of the prior art.

Means for Solving the Problems

[0008] Generally, this specification describes methods and apparatuses (e.g., devices and systems) for scanning both the external structure and / or the internal structure of teeth. These methods and apparatuses are capable of generating a model of a subject's teeth, including both the surface topography shape and the internal shape (e.g., dentin, dental fillings, cracks, and / or caries). Any of these apparatuses may include an intraoral scanner that scans inside or around the subject's oral cavity, the intraoral scanner having one or more light sources capable of illuminating in two or more spectral ranges, namely, a spectral range for illuminating the surface shape (e.g., visible light), and a spectral range for penetration (e.g., infrared range, and particularly "near infrared" (including and not limited to 850 nm)). The scanning device may further include one or more sensors for detecting the emitted light and one or more processors, and these processors control the operation of the scan and analyze the light received in both the first spectral range and the second spectral range to generate a model of the subject's teeth, including the surface of the teeth and the shape within the teeth (including within enamel and dentin). The generated model may be a 3D volumetric model or a panoramic image.

[0009] As used herein, a volumetric model may include a three-dimensional virtual representation of an object, where the internal region (such as structure, etc.) is arranged in a proportional and relative relationship with other internal and surface shapes of the object being modeled within a physical three-dimensional volume. For example, a volumetric representation of a tooth may include an outer surface arranged proportionally to the tooth as well as internal structures (beneath the surface of the tooth) within the tooth, such that a cross-section of the volumetric model substantially coincides with a cross-section of the tooth showing the position and size of the internal structures. The volumetric model may be a cross-section from any (e.g., arbitrary) direction and may correspond to an equivalent cross-section of the object being modeled. The volumetric model may be electronic or physical. A physical volumetric model may be formed, for example, by 3D printing or the like. The volumetric models described herein may extend completely within the volume (e.g., may extend throughout the volume (such as the volume of a tooth)), or may extend partially within the volume (e.g., may extend within the volume being modeled over a certain minimum depth (e.g., 2 mm, 3 mm, 4 mm, 5 mm, 6 mm, 7 mm, 8 mm, 9 mm, 10 mm, 12 mm, etc.)).

[0010] The methods described herein typically include a method of generating a model of a subject's tooth, which typically generates a 3D model or rendering of the tooth, including both surface and internal shapes. Non-ionizing methods for imaging and / or detecting internal structures may be used. For example, structures within the tooth may be observed by illuminating the structures within the tooth using one or more penetrative spectral ranges (wavelengths) to obtain an image based on the penetrative wavelength, which may involve using transillumination (illuminating from one side and capturing light from the opposite side after passing through the object), and / or using small-angle penetrative imaging (e.g., reflection imaging, capturing light reflected / scattered from internal structures when illuminated with a penetrative wavelength). In particular, multiple penetrative images may be acquired from the same relative position. Conventional penetrative imaging techniques (e.g., transillumination) where the angle between the illumination direction of the emitter and the field of view angle of the detector (e.g., camera) is 90 degrees or 180 degrees may be used, but herein we also describe methods and apparatuses where that angle is significantly smaller (e.g., 0 to 25 degrees, 0 to 20 degrees, 0 to 15 degrees, 0 to 10 degrees, etc.). The smaller the angle (e.g., 0 to 15°), the more advantageous it may be in particular because the illumination (light source) and detection (detector (e.g., camera, etc.)) can be closer to each other, and the scan wand for an intraoral scanner can be more easily positioned and moved around the patient's teeth. Such small-angle penetrative images and imaging techniques may also be referred to herein as reflective illumination and / or imaging, or reflective / scattered imaging. Generally, penetrative imaging may mean any suitable type of penetrative imaging, including transillumination, small-angle penetrative imaging, etc., unless otherwise specified. However, small angles may also cause direct reflections from the surface of the object (e.g., tooth) that can obscure the internal structure.

[0011] The methods and apparatuses described herein are particularly effective in combining a 3D surface model of one or more teeth with an imaged internal shape, such as a detected lesion (caries, crack, etc.) detectable by using penetration imaging, by using an intraoral scanner adapted to detect both surface shape and internal shape separately, but simultaneously (or almost simultaneously). Combining surface scanning and penetration imaging can be implemented by alternating or switching these different modalities such that both use the same coordinate system. Alternatively, the surface scan and the penetration scan may be observed simultaneously, which may be done, for example, by selectively filtering the imaging wavelength to separate infrared light (near-infrared light) from visible light. The 3D surface data can thus enable the interpretation and analysis of penetration images that may be difficult or impossible to interpret by other means, providing important reference and angular information for the internal structure.

[0012] For example, this specification describes a method of generating a model of a subject's teeth, the method including capturing three-dimensional (3D) surface model data of at least a portion of the subject's teeth using an intraoral scanner, obtaining a plurality of images inside the teeth using a near-infrared wavelength with the intraoral scanner, and forming a 3D model of the teeth, including the internal shape, using the 3D surface model data and the plurality of images.

[0013] One way to generate a model of a subject's teeth is to capture 3D surface model data of at least a portion of the subject's teeth with an intraoral scanner operating in a first imaging modality, the 3D surface model data having a first coordinate system, the capturing step; and to acquire a plurality of images inside the teeth with an intraoral scanner operating in a second modality using an infiltration wavelength, the plurality of images being referenced to the first coordinate system, the acquiring step; and to form a 3D model of the teeth, including the internal structure, using the 3D surface model data and the plurality of images. Generally, by capturing the first wavelength, an image is not necessarily captured, but a 3D surface scan can be directly captured. The second infiltration modality may be captured as an image processed as described herein.

[0014] Generally, capturing the 3D surface model data may include identifying the 3D surface topology in any suitable way. For example, identifying the 3D surface topology may include using confocal focusing. Capturing the 3D surface model data may include using one or more of confocal scanning, stereoscopy, or structured light triangulation.

[0015] Any of the methods and apparatuses described herein may be used to model, image, and / or render 3D images of a single tooth or tooth region, multiple teeth, teeth and gums, or other intraoral structures, particularly from within the subject's mouth.

[0016] Generally, the methods and apparatuses described herein for performing these include 3D color intraoral scanners / scans. For example, these methods may include capturing color intraoral 3D data.

[0017] As will be described in detail later, the present method and apparatus are capable of controlling the switching between the collection of surface data and the collection of penetration imaging (penetrability) data. For example, these methods may each include obtaining an image using a penetration wavelength when 3D surface model data is being captured (e.g., by switching between a first imaging modality and a second (penetrability) imaging modality).

[0018] The same sensor may be used for collecting surface shape data and internal shape data, or different sensors may be used. For example, obtaining a plurality of images may include capturing 3D surface model data and a plurality of images at a penetration wavelength using the same sensor on an intraoral scanner. Alternatively, one or more separate sensors may be used.

[0019] As described above, obtaining an image of a tooth using a penetration wavelength (or a penetrability spectral range) may include obtaining a penetration image at any angle between an illumination source and a sensor (e.g., a detector or a camera). In particular, internal shape (e.g., reflection imaging) data may be imaged using a small angle configuration, in which case one, or preferably two or more, penetration images are obtained at various orientations relative to one or more teeth. For example, obtaining a plurality of images may include illuminating a tooth at an angle of 0° to 15° with respect to a sensor (e.g., a detector, a camera, etc.) that receives illumination from the tooth and is reflected from the internal composition of one or more teeth. Obtaining a plurality of images (e.g., penetration images such as these small angle penetration images) generally includes obtaining one or more (e.g., a plurality including two or more, three or more, etc.) penetration images at various angles of an intraoral scanner relative to the tooth over the same region of the tooth. Accordingly, the same internal region of the tooth is displayed in different scans from various angles.

[0020] Generally, an intraoral scanner (e.g., the wand of an intraoral scanner) may include any number of sensors. Any suitable sensor may be used to detect and record a desired spectral range (e.g., of light). The sensor may mean, and may include, a detector, a camera, etc. For example, acquiring multiple images may include using multiple sensors on the intraoral scanner and capturing multiple images using penetrating wavelengths.

[0021] The illumination used to acquire the penetration image is generally penetrative and thus can penetrate and / or pass through at least a portion of the tooth enamel and dentin. The light of the penetrating wavelength may mainly include infrared light (especially near-infrared light). For example, light in the range of 700 - 1090 nm (e.g., 850 nm) may be used. Other wavelengths or wavelength ranges including wavelengths shorter than the visible spectrum may also be used. Thus, acquiring multiple images may include illuminating the teeth with infrared light. Acquiring multiple images (e.g., penetration images) may include illuminating the teeth with one or more of white light (including but not limited to white light transillumination), UV / blue fluorescence, and red light fluorescence.

[0022] The illumination used to acquire the penetration image can be considered semi-penetrative in the sense that the internal regions of the tooth (e.g., points or voxels) may be visible only from a few camera positions and orientations. That is, that point may be blocked from view by other structures in some of the images that include the volume point within the field of view. In that sense, an image that includes a volume point within the field of view may not be able to image this volume point. Thus, the methods and apparatuses described herein can account for the high masking of volume points, unlike other penetration scanning techniques (e.g., CT) that use X-ray imaging without masking.

[0023] Generally, any suitable method may be used to form a 3D model of a tooth that includes a surface structure and an internal structure (a combination thereof) obtained by permeation imaging. These 3D models may be referred to as combined 3D surface / volume models, 3D volumetric surface models, or simply "3D models", etc. As described above, both the surface data and the permeation imaging data may generally be in the same coordinate system. The two may be combined by using a common coordinate system. Depending on the deformation form, the surface data may be represented as a surface model and an internal shape added to this model. Depending on the deformation form, this data may be reconstructed into a three-dimensional model simultaneously (after summation). One or both of the data sets may be modified separately (for example, filtering, subtraction, etc. may be performed). For example, forming a 3D model of a tooth that includes an internal structure may include combining 3D surface model data and internal structure data (including volumetric data). Forming a 3D model of a tooth that includes an internal structure may include combining a plurality of permeation images, and the plurality of permeation images may be acquired from various angles using an intraoral scanner.

[0024] In any of the methods and apparatuses configured to implement these methods described herein, the data may be analyzed automatically or manually by the system. In particular, the methods and apparatuses described herein may include examining the internal shape and / or identifying shapes of interest including cracks and caries. Shape recognition may be performed based on shape recognition conditions (for example, dark or bright regions in the permeation image), pattern recognition, machine learning, etc. Marking may be performed on the shape, including color coding, labeling, etc. The marking of the shape may be performed directly in the 3D model, on the permeation image, or in a data structure that is based on the 3D model of the tooth formed by the methods and apparatuses described herein (for example, sharing the coordinate system with the 3D model of the tooth).

[0025] This specification further describes an apparatus configured to perform any of the described methods. For example, an intraoral scanning system configured to generate a model of a subject's teeth described herein includes a handheld wand having at least one sensor and a plurality of light sources, the light sources being configured to emit light in a first spectral range and a second spectral range, the second spectral range being penetrative; a handheld wand; and one or more processors operatively connected to the handheld wand, the one or more processors being configured to generate a three-dimensional (3D) surface model of at least a portion of the subject's teeth using light in the first spectral range, and to generate a 3D model of the subject's teeth including the internal structure based on the 3D surface model and a plurality of images showing the internal structure obtained in the second spectral range.

[0026] An intraoral scanning system configured to generate a model of a subject's teeth may include a handheld wand having at least one sensor and a plurality of light sources, the light sources being configured to emit light in a first spectral range and a second spectral range, the second spectral range being penetrative; a handheld wand; and one or more processors operatively connected to the handheld wand, the one or more processors being configured to identify surface information by using light in the first spectral range detected by the handheld wand using a first coordinate system, to generate a three-dimensional (3D) surface model of at least a portion of the subject's teeth using the surface information, to obtain a plurality of images in the second spectral range with reference to the first coordinate system, and to generate a 3D model of the subject's teeth including the internal structure based on the 3D surface model and the plurality of images.

[0027] This specification further describes a method for generating a model of a subject's tooth that includes both a surface structure and an internal structure. In this method, the same intraoral scan cycles between a plurality of different modalities, for example, between a surface scan and an infiltration, and additional modalities (e.g., laser fluorescence, etc.) may be included as an alternative. Generally, the examples described in this specification focus on combining the surface and infiltration, but as an alternative or addition to the internal shape imaging described in this specification, another internal scan technique (e.g., laser fluorescence) may be used.

[0028] For example, this specification describes a method for generating a model of a subject's tooth that includes both a surface structure and an internal structure. This method includes scanning a portion of the subject's tooth with a hand-held intraoral scanner by a first modality that captures 3D surface model data of the tooth, scanning a portion of the subject's tooth with a hand-held intraoral scanner by a second modality that images the interior of the tooth using an infiltration wavelength to capture internal data of the tooth, and cycling between the first modality and the second modality such that the switching between the first modality and the second modality is rapidly performed by the cycling so that the image using the infiltration wavelength shares a coordinate system with the 3D surface model data captured by the first modality.

[0029] Any of the methods described in this specification may include automatically adjusting the duration spent on scanning in the first modality, the duration spent in the second modality, or the duration spent in the first and second modalities when cycling between the first and second modalities. For example, any of these methods may include automatically adjusting the duration spent on scanning in the first modality, the duration spent in the second modality, or the duration spent in the first and second modalities when cycling between the first and second modalities, based on the captured 3D surface model data, internal data, or both the 3D surface model data and internal data. Accordingly, one method of generating a model of a subject's teeth includes scanning a portion of the subject's teeth with a handheld intraoral scanner, performed by a first modality that captures 3D surface model data of the teeth, scanning the portion of the subject's teeth with a handheld intraoral scanner, performed by a second modality that images the interior of the teeth using a penetration wavelength to capture internal data of the teeth, cycling between the first and second modalities by a scanning method in which the first and second modalities are switched rapidly by cycling, such that the internal data uses the same coordinate system as the 3D surface model data captured in the first modality, and adjusting the scanning method based on the captured 3D surface model data, internal data, or both the 3D surface model data and internal data.

[0030] Adjusting the scanning method may include adjustment based on determination of the quality of the captured 3D surface model data. Adjusting the scanning method may include automatically adjusting the scanning method and / or adjusting the duration of scanning in the first modality and / or adjusting the duration of scanning in the second modality.

[0031] Any of these methods may include combining the 3D surface model data and the internal data of the teeth to form a 3D model of the teeth.

[0032] As described above, capturing 3D surface model data may include identifying a 3D surface topology using confocal focusing / confocal scanning, stereoscopy, or structured light triangulation.

[0033] Generally, traversing may include traversing a first modality, a second modality, and a third modality, and the first modality, the second modality, and the third modality are switched rapidly by traversing such that an image using the penetration wavelength shares a coordinate system with the 3D surface model captured in the first modality. The third modality may be another penetrative modality or a non-penetrative modality (e.g., color, visible image of the subject's teeth, etc.).

[0034] Performing, in a second modality, scanning a portion of a subject's teeth with a handheld intraoral scanner may include illuminating the teeth at an angle of 0° to 15° with respect to the observation direction of a sensor that receives the illumination (small angle illumination). The step of performing, by a second modality, scanning a portion of a subject's teeth with an intraoral scanner may include obtaining a plurality of penetration images at a plurality of different angles between an illumination source and a sensor and / or at a plurality of different positions or angles relative to the teeth such that the same internal region of the teeth is imaged from various angles relative to the teeth.

[0035] As described above, any suitable penetration wavelength may be used, including an infrared wavelength (e.g., a near-infrared wavelength). For example, performing, by a second modality, scanning a portion of a subject's teeth with an intraoral scanner may include illuminating the teeth with one or more of white light transillumination, UV / blue fluorescence, and red light fluorescence.

[0036] This specification further describes an intraoral scanning system configured to cycle through a plurality of scan modes to generate a model of a subject's teeth. For example, an intraoral scanning system described herein is a handheld intraoral wand having at least one sensor and a plurality of light sources, the light sources being configured to emit light in a first spectral range and a second spectral range, wherein the second spectral range is penetrative; a handheld intraoral wand; and one or more processors operatively connected to the handheld intraoral wand, the one or more processors being configured to cycle the wand through a first mode and a second mode, in the first mode, the wand emits light in the first spectral range for a first duration, and the one or more processors receive 3-dimensional (3D) surface data in response, and in the second mode, the wand emits light in the second spectral range for a second duration, and the one or more processors receive image data in response; and the one or more processors.

[0037] An intraoral scanning system for generating a model of a subject's teeth is a handheld intraoral wand having at least one sensor and a plurality of light sources, the light sources being configured to emit light in a first spectral range and a second spectral range, and further, the second spectral range being penetrative, the handheld intraoral wand, and one or more processors operatively connected to the wand, the one or more processors being configured to cycle the wand through a first mode and a second mode, in the first mode, the wand emits light in the first spectral range for a first duration, and the one or more processors receive 3D surface data as a response, in the second mode, the wand emits light in the second spectral range for a second duration, and the one or more processors receive image data as a response, the one or more processors, may include, and the one or more processors are configured to adjust the first duration and the second duration based on the received 3D surface data, the received image data, or both the 3D surface data and the image data. In any of the devices described herein, one mode may be a surface scan (3D surface), which may be, for example, 680 nm. Another mode may be a penetration scan, which may use, for example, near-infrared light (e.g., 850 nm). Another mode may be color imaging, which may use white light (e.g., generally 400 - 600 nm).

[0038] Also described is a method of diffuse imaging that uses a handheld intraoral scanner to visualize internal structures. Accordingly, any of the general methods and apparatuses described herein may be configured to model one or more teeth and detect internal features such as cracks and caries, in particular using diffuse imaging data. For example, one method of imaging the interior of a tooth to detect cracks and caries includes obtaining a plurality of diffuse images of the tooth in various orientations using a handheld intraoral scanner that emits light at a diffuse wavelength, identifying surface position information using the intraoral scanner in the first position, and generating a three-dimensional (3D) model of the tooth using the plurality of diffuse images and the surface position information.

[0039] Generating a 3D model of a tooth may include repeating the steps of obtaining a plurality of diffuse images and generating a 3D model corresponding to a plurality of different positions.

[0040] Obtaining a plurality of diffuse images of a tooth in various orientations may include obtaining each diffuse image using a different light source or combination of light sources on the intraoral scanner that emit at a diffuse wavelength, or using one or both of different image sensors on the intraoral scanner that acquire the image.

[0041] In some variations, obtaining a plurality of diffuse images may include obtaining three or more diffuse images.

[0042] Obtaining a plurality of penetration images of the tooth surface in various orientations may include, for example, obtaining penetration images by small-angle illumination / observation, and in each penetration image, the angle between the emitted light and the light received by the image sensor is 0 to 15 degrees. For example, one method of imaging the interior of a tooth to detect cracks and caries is the step of scanning the tooth from a plurality of positions, including the step of using an intraoral scanner to obtain a plurality of penetration images of the tooth in various orientations, where the intraoral scanner emits light at a penetration wavelength, and in each penetration image, the angle between the emitted light and the light received by the image sensor is 0 to 15 degrees, the step of obtaining a plurality of penetration images, and the step of using the intraoral scanner to identify surface position information, which is repeated at each position, the step of scanning the tooth, and the step of generating a three-dimensional (3D) model of the tooth using the penetration images and the surface position information may be included.

[0043] As described above, in addition to the scanning and modeling apparatuses (e.g., scanning apparatuses, tooth modeling apparatuses, etc.) and methods, and the method of operating the scanning and / or modeling apparatuses, this specification further describes a method of reconstructing a volumetric structure using images generated from one or more penetration wavelengths.

[0044] For example, this specification describes a method of reconstructing a volumetric structure from an object that includes a region (such as a tooth) that is translucent and has strong scattering with respect to a radiation wavelength range. This method may include illuminating the object with a light source that emits an infiltration wavelength (for example, exclusively or mainly radiates), acquiring a plurality of images of the object with a camera that is sensitive to the infiltration wavelength (for example, records in the radiation wavelength range), receiving position data representing the position of the camera relative to the object in each of the plurality of images, generating an upper limit of the scattering coefficient from the plurality of images and the position data for each point in the volume, and generating an image of the object from the upper limit of the scattering coefficient at each point. The infiltration wavelength light applied to the object may be emitted from substantially the same direction as the camera. One or more of the generated images may be capable of showing the shape within the volume of the object, and this image may further include (or be modified to include) not only the internal structure but also the outer boundary of the object.

[0045] In this specification, a tooth may be described as an object that includes one or more regions that are translucent and have strong scattering. Generally, however, a tooth may also include a region with strong scattering (such as dentin) and a region with weak scattering and high transparency (such as enamel) at near-infrared wavelengths. A tooth may also include a region with intermediate or mixed scattering characteristics, such as caries. The methods and apparatuses for performing volumetric scans described in this specification are suitable for mapping these various regions of one or more teeth.

[0046] One method of reconstructing a volumetric structure from an object that includes a region that is translucent and strongly scattering for a radiation wavelength range includes, in the radiation wavelength range, acquiring a plurality of images of the object with a camera, wherein illumination for the plurality of images is projected substantially from the direction of the camera; receiving position data representing the position of the camera relative to the object in each of the plurality of images; generating, for each point in the volume, an upper bound on a scattering coefficient from the plurality of images and the position data; and generating an image of the object from the upper bound on the scattering coefficient at each point.

[0047] The radiation wavelength range may be an infrared wavelength or a near-infrared wavelength.

[0048] Any of these methods may also include receiving surface data representing an external surface of the object, and the generating step is performed for each point in the volume within the external surface of the object.

[0049] The object may include a tooth having an external enamel surface and an internal dentin surface. The tooth is just one type of object that includes a region that is translucent and strongly scattering, but in other examples, may include other both tissues (e.g., a backbone, etc.) (including soft tissue and / or hard tissue). These objects that include a region that is translucent and strongly scattering may typically include a region that is translucent and strongly scattering for a penetration wavelength (e.g., an infrared wavelength or a near-infrared wavelength) as described herein.

[0050] The position data generally includes data on the position and orientation of the camera at the time each of the plurality of images is captured. For example, the position data may include three numerical coordinates of the camera in three-dimensional space and pitch, yaw, and roll.

[0051] The step of generating the upper limit of the scattering coefficient at each point of the volume may include projecting each point of the 3D point cloud grid corresponding to the volume of the object onto each of its plurality of images using a first calibration, generating a list of luminance values of the projected points, converting each luminance value of the list of luminance values into a scattering coefficient according to the volume response, and storing, for each grid point, the minimum scattering coefficient value in the list of scattering coefficient values.

[0052] For example, the first calibration may include a fixed pattern noise calibration for calibrating the sensor problems of the camera and image ghosts. The first calibration may include a camera calibration for determining the transformation of the camera that projects known points in space onto points in the image.

[0053] Furthermore, this specification also describes a method for reconstructing a volumetric structure from teeth that are translucent in the radiation wavelength range. This method includes receiving, in a processor, a representation of the surface of the teeth in a first coordinate system, receiving, in the processor, a plurality of images of the teeth in the radiation wavelength range, where the plurality of images are acquired by illumination projected substantially from the direction of the camera, receiving, in the processor, position data representing the position of the camera in each of the plurality of images, projecting each point of the point cloud grid corresponding to the volume within the surface of the teeth onto each of the plurality of images using a first calibration, generating a list of luminance values of the projected points, converting each luminance value on the luminance value list into a scattering coefficient according to the volume response, and storing the minimum scattering coefficient at each point in a minimum scattering coefficient list.

[0054] Any of these methods may further include the step of generating an image from the minimum scattering coefficient list.

[0055] The position data may include data on the position and orientation of one or more cameras at the time each of the plurality of images is captured.

[0056] The first calibration may include a fixed pattern noise calibration to calibrate the camera's sensor problems and image ghosts. In some embodiments, the first calibration may include a camera calibration that determines the transformation of the camera that projects known points in space onto points in the image.

[0057] This method may further include receiving surface data representing the outer surface of the object, and the projecting step is performed within the outer surface of the object for each point within the volume.

[0058] The point cloud grid may include a cubic grid.

[0059] Any of the methods described herein may be implemented as software, firmware, and / or hardware. For example, any of these methods may be configured as a non-transitory computer-readable medium storing instructions for performing this method.

[0060] For example, a non-transitory computer-readable medium storing instructions for reconstructing a volumetric structure from teeth that are translucent in the radio wavelength range is described. These instructions include receiving a representation of the surface of the teeth in a first coordinate system, receiving a plurality of images of the teeth in the radio wavelength range, the plurality of images being acquired by illumination projected substantially from the direction of the camera, receiving position data representing the position of the camera in each of the plurality of images, projecting each point of a point cloud grid corresponding to the volume of the teeth onto each of the plurality of images using the first calibration, generating a list of luminance values for each projected point, converting each luminance value on the luminance value list into a scattering coefficient according to the volume response, storing the minimum scattering coefficient at each point in a minimum scattering coefficient list, and generating an image from the minimum scattering coefficient list, and may be executable by a processor to cause the computing device to perform.

[0061] The position data may include data on the position and orientation of the camera at the time when each of a plurality of near-infrared images is captured. The position data may include three numerical coordinates in three-dimensional space and pitch, yaw, and roll of the camera.

[0062] The first calibration may include a fixed pattern noise calibration for calibrating the sensor problems and image ghosts of the camera. The first calibration may include a camera calibration for determining the transformation of the camera that projects known points in space onto points in the image.

[0063] The point cloud grid may be inside the tooth and, as described above, the point cloud grid may include a cubic grid.

[0064] As an alternative or addition to the use of the scattering coefficient, any suitable method of forming the internal structure of a patient's tooth using a penetration wavelength image. For example, any of the devices (e.g., systems, devices, software, etc.) and methods described herein may use a two-dimensional penetration image together with information on the position and / or orientation of the scanner relative to the object being imaged (e.g., a tooth) to segment the 2D penetration image, thereby forming a three-dimensional model of the tooth including the internal structure within the tooth. As described above, the penetration image may mean an image of the interior of an object acquired at a near-infrared wavelength and / or an infrared wavelength. The position and / or orientation of the scanner may be in place of the position and / or orientation of a camera on the scanner (e.g., on a handheld wand) that acquires an image.

[0065] For example, this specification describes a method of modeling a subject's teeth, the method including capturing, by an intraoral scanner, a plurality of images inside the subject's teeth and the position and orientation of the intraoral scanner unique to each of the plurality of images, segmenting the plurality of images to form an internal structure corresponding to the structure within the subject's teeth, projecting the internal structure onto a three-dimensional model of the subject's teeth using the position and orientation of the plurality of images, and displaying a three-dimensional model of the subject's teeth including the internal structure.

[0066] In any of these methods and apparatuses, it is possible to capture a 3D surface model using a non-penetrating wavelength (e.g., surface scan) while simultaneously capturing a penetration image. For example, capturing may include capturing a surface image of a subject's teeth while capturing a plurality of images inside the subject's teeth. This method may also include forming a three-dimensional model of the subject's teeth from the captured surface image. For example, forming a three-dimensional model of the subject's teeth may include identifying a three-dimensional surface topology using confocal focusing. Capturing a surface image of the subject's teeth may include using confocal scanning, stereoscopy, or structured light triangulation.

[0067] Generally, the same apparatus (e.g., scanner) may model and / or display a 3D representation of a tooth including its internal structure, or alternatively or additionally, a separate (e.g., remote from the scanner) processor may be used. Any of these methods may also include storing and / or transmitting a plurality of penetration images and the position and orientation of an intraoral scanner while capturing a plurality of two-dimensional images, and this transmission includes transmitting to a remote processor that performs steps subsequent to the segmentation.

[0068] In any of the methods and apparatuses described herein, a 3D model including the internal structure may be displayed during operation of the scanner. This may advantageously enable a user to view the internal structure of a subject's teeth in real time or near real time. Accordingly, any of these methods may include displaying a three-dimensional model when an image is captured.

[0069] Segmenting a plurality of images may include applying edge detection to the plurality of images to identify closed boundaries within the plurality of images. Segmenting a plurality of images may include forming a volumetric density map from the plurality of images to identify internal structures. Segmenting the volumetric density map may include performing segmentation by identifying one or more isosurfaces within the volumetric density map to identify internal shapes. Any of these methods may include segmenting the volumetric density map to identify internal shapes (e.g., cracks, caries, dental fillings, dentin, etc.).

[0070] For example, an intraoral scanning device configured to generate a model of a subject's teeth may include an intraoral scanner having a plurality of light sources and a sensor for position and orientation, the light sources being configured to emit light in a first spectral range and a second spectral range, and further, the second spectral range being penetrative, and an intraoral scanner, and a processor operatively connected to the intraoral scanner, the processor being configured to cause the scanner to capture a plurality of images and the position and orientation of the intraoral scanner corresponding to each of the plurality of images when the intraoral scanner is emitting light in the second spectral range, and one or more processors configured to further segment the plurality of images to form an internal structure corresponding to a structure within the subject's teeth, and display or transmit a three-dimensional model of the subject's teeth including the internal structure.

[0071] The processor may be configured to segment the plurality of images by applying edge detection to the plurality of images to identify closed boundaries within the plurality of images. The processor may be configured to identify internal structures by forming a pixel density map from the plurality of images to segment the plurality of images. The processor may be configured to identify internal structures by identifying closed segments within the pixel density map.

[0072] Furthermore, this specification also describes a non-transitory computer-readable medium storing instructions executable by a processor, the instructions causing an intraoral scanning device to perform steps of: capturing a plurality of images using penetrating wavelength light and the position and orientation of an intraoral scanner specific to each of the plurality of images; segmenting the plurality of images to form an internal structure corresponding to a structure within a subject's teeth; projecting the internal structure onto a three-dimensional model of the subject's teeth using the position and orientation of the intraoral scanner specific to each image; and displaying a three-dimensional model of the subject's teeth including the internal structure.

[0073] The non-transitory computer-readable medium having instructions may be further configured to cause the intraoral scanning device to perform steps of segmenting the plurality of images by applying edge detection to the plurality of images to identify closed boundaries therein. The non-transitory computer-readable medium having instructions may be further configured to cause the intraoral scanning device to perform steps of forming an internal structure by forming a pixel density map from the plurality of images and segmenting the plurality of images. The non-transitory computer-readable medium having instructions may be further configured to cause the intraoral scanning device to perform steps of forming an internal structure by identifying closed segments within the pixel density map and segmenting the plurality of images.

[0074] Furthermore, this specification also describes a non-transitory computer-readable medium storing instructions executable by a processor, the instructions causing a computing device to perform steps of: receiving from a scanner three-dimensional surface model data of a subject's teeth; receiving from the scanner a plurality of images inside the subject's teeth and the position and orientation of an intraoral scanner specific to each of the plurality of images; segmenting the plurality of images to form an internal structure of the subject's teeth; projecting the internal structure of the subject's teeth onto the three-dimensional surface model; and displaying a three-dimensional surface model showing the internal structure.

[0075] For example, this specification describes a method of generating a three-dimensional (3D) volumetric model of a subject's teeth using an intraoral scanner. This method includes moving an intraoral scanner over the subject's teeth and capturing 3D surface model data of at least a portion of the teeth using the intraoral scanner; moving the intraoral scanner over the teeth and acquiring a plurality of images inside the teeth using a near-infrared wavelength with the intraoral scanner, such that a plurality of images of the same internal region of the teeth are imaged; for each of the plurality of images inside the teeth, identifying the position of the intraoral scanner relative to the subject's teeth using the 3D surface model data; and forming the 3D volumetric model of the subject's teeth, including the internal shape, using the plurality of images and the position of the intraoral scanner relative to the subject's teeth.

[0076] One method of generating a three-dimensional (3D) volumetric model of a subject's teeth using an intraoral scanner includes moving an intraoral scanner over the subject's teeth and capturing 3D surface model data of at least a portion of the teeth using the intraoral scanner; moving the intraoral scanner over the teeth and acquiring a plurality of images inside the teeth using a near-infrared wavelength, by emitting near-infrared light from the intraoral scanner with a first polarization and detecting the near-infrared light returning to the intraoral scanner with an image sensor within the intraoral scanner, wherein the near-infrared light returning to the intraoral scanner is filtered to remove specular reflection, and this removal is performed by filtering the near-infrared light of the first polarization from the near-infrared light returning to the intraoral scanner before reaching the image sensor, the step of acquiring a plurality of images inside the teeth; for each of the plurality of images inside the teeth, identifying the position of the intraoral scanner relative to the subject's teeth using the 3D surface model data at the time when each of the plurality of images is captured; and forming the 3D volumetric model of the subject's teeth, including the internal shape, using the plurality of images and the position of the intraoral scanner relative to the subject's teeth.

[0077] In any of these methods and apparatuses, the near-infrared light returning to the intraoral scanner may be filtered to remove specular reflections, and this removal is performed by filtering all or substantially all of the near-infrared light of the first polarization from the near-infrared light returning to the intraoral scanner before it reaches the image sensor.

[0078] Furthermore, this specification also describes an intraoral scanner that scans both surface structures and internal structures. For example, an intraoral scanning system that generates a three-dimensional (3D) volumetric model of a subject's teeth may include a handheld wand having at least one image sensor and a plurality of light sources, the light sources being configured to emit light in a first spectral range and a second spectral range, the second spectral range being within the near-infrared wavelength range, a handheld wand, and one or more processors operatively connected to the handheld wand, the one or more processors being configured to capture 3D surface model data of at least a portion of the teeth while moving the intraoral scanner over the subject's teeth, to acquire a plurality of images of the interior of the teeth using light in the second spectral range while moving the intraoral scanner over the teeth such that a plurality of images of the same internal region of the teeth are imaged, to determine the position of the handheld wand relative to the subject's teeth for each of the plurality of images of the interior of the teeth using the 3D surface model data, and to form the 3D volumetric model of the subject's teeth including the internal shape using the plurality of images and the position of the intraoral scanner relative to the subject's teeth.

[0079] An intraoral scanning system that generates a three-dimensional (3D) volumetric model of a subject's teeth includes a handheld wand having at least one image sensor and a plurality of light sources, the light sources being configured to emit light in a first spectral range and a second spectral range, the second spectral range being within the near-infrared wavelength range; a filter located in front of the image sensor and configured to filter light in the second spectral range and light of a first polarization; and one or more processors operatively connected to the handheld wand, the one or more processors configured to capture 3D surface model data of at least a portion of the teeth while moving the intraoral scanner over the subject's teeth; and obtain a plurality of images inside the teeth using light in the second spectral range by emitting near-infrared light from the intraoral scanner in the first polarization and detecting, with an image sensor within the intraoral scanner, the near-infrared light returning to the intraoral scanner, the near-infrared light returning to the intraoral scanner being filtered to remove specular reflections, the removal being performed by filtering out near-infrared light of the first polarization from the near-infrared light returning to the intraoral scanner before it reaches the image sensor; the step of obtaining a plurality of images inside the teeth; for each of the plurality of images inside the teeth, identifying the position of the handheld wand relative to the subject's teeth using the 3D surface model data; and forming a 3D volumetric model of the subject's teeth, including the internal shape, using the plurality of images and the position of the intraoral scanner relative to the subject's teeth.

[0080] Furthermore, this specification also describes a method for imaging tooth cracks and caries. For example, one method described herein for imaging the interior of a subject's teeth using an intraoral scanner to detect cracks and caries includes scanning the subject's teeth with an intraoral scanner, acquiring a plurality of near-infrared images of the interior of the subject's teeth in various orientations using an intraoral scanner that emits both near-infrared wavelengths and non-penetrating wavelengths, identifying the position of the intraoral scanner relative to the subject's teeth using the non-penetrating wavelength for each location in the images of the plurality of near-infrared images, and generating a three-dimensional (3D) volumetric model of the subject's teeth using the plurality of near-infrared images and the position of the intraoral scanner relative to the subject's teeth in each of the plurality of near-infrared images.

[0081] Any of these methods may include analyzing the volumetric model to identify cracks or caries (or other internal tooth regions).

[0082] For example, one method for performing imaging through a subject's teeth to detect cracks and caries includes scanning the subject's teeth from a plurality of positions, wherein at each position, an intraoral scanner is used to acquire a plurality of near-infrared images of the interior of the teeth in various orientations, the intraoral scanner emits light of a near-infrared wavelength in a first polarization, and in each near-infrared image, the angle between the emitted light and the light received by the image sensor is between 0 degrees and 15 degrees, and further, the received near-infrared light is filtered to block the near-infrared light of the first polarization, acquiring a plurality of near-infrared images, repeating the steps of identifying the position of the intraoral scanner relative to the subject's teeth for each location in the images of the plurality of near-infrared images, and generating a three-dimensional (3D) volumetric model of the teeth using the penetration images and the surface position information.

[0083] Furthermore, this specification also describes a method of using a scattering coefficient to generate an internal image of a tooth based on a penetration image and a camera sensor position. For example, one method of forming a three-dimensional (3D) volumetric model of a subject's tooth includes obtaining a plurality of near-infrared images of the subject's tooth with a camera sensor, wherein the near-infrared illumination for the plurality of near-infrared images is projected from substantially the direction of the camera sensor; obtaining the plurality of near-infrared images; receiving position data representing the position of the camera relative to the subject's tooth for each of the plurality of near-infrared images; generating an upper limit of the scattering coefficient from the plurality of near-infrared images and the position data for each point in the volume; combining the upper limits of the scattering coefficient for each point in the volume to form a 3D volumetric model of the subject's tooth; and outputting the 3D volumetric model of the subject's tooth.

[0084] Any of these methods may include forming an isosurface from a 3D volumetric model of a subject's tooth. The formation of the isosurface may be performed by selecting a threshold value or range of values of the scattering coefficient. Sub-ranges may correspond to various internal regions (e.g., structures). For example, the outputting step may include forming an isosurface corresponding to the internal dentin surface from the 3D volumetric model of the subject's tooth.

[0085] One method for reconstructing a volumetric structure from a tooth that is translucent in the emission wavelength range includes receiving, in a processor, a representation of the surface of the tooth in a first coordinate system; receiving, in the processor, a plurality of images of the tooth in the emission wavelength range acquired by a camera, the plurality of images being acquired by illumination projected substantially from the direction of the camera; receiving, in the processor, position data representing the position of the camera in each of the plurality of images; projecting, using a first calibration, each point of a point cloud grid corresponding to a volume within the surface of the tooth onto each of the plurality of images; generating a list of luminance values for each projected point; converting each luminance value on the luminance value list into a scattering coefficient according to a volume response; and storing a minimum scattering coefficient at each point in a minimum scattering coefficient list.

[0086] Any of these methods may be implemented in an apparatus including software, hardware, and / or firmware for implementing the method. For example, described herein is a non-transitory computer-readable medium storing instructions for reconstructing a volumetric structure from a tooth that is translucent in the emission wavelength range, the instructions being executable by a processor to cause the computing device to: receive a representation of the surface of the tooth in a first coordinate system; receive a plurality of images of the tooth in the emission wavelength range acquired by a camera, the plurality of images being acquired by illumination projected substantially from the direction of the camera; receive position data representing the position of the camera in each of the plurality of images; project each point of a point cloud grid corresponding to the volume of the tooth onto each of the plurality of images using a first calibration; generate a list of luminance values for each projected point; convert each luminance value on the luminance value list into a scattering coefficient according to a volume response; store a minimum scattering coefficient at each point among the scattering coefficients; and output an image generated from the minimum scattering coefficient list.

[0087] Furthermore, this specification also describes a method of forming an internal structure using segments. For example, one method of modeling a subject's teeth includes capturing, by an intraoral scanner, a plurality of images inside the subject's teeth and the position and orientation of the intraoral scanner that are unique to each of the plurality of images; segmenting the plurality of images to form an internal structure corresponding to the structure within the subject's teeth; projecting the internal structure onto a three-dimensional model of the subject's teeth using the positions and orientations of the plurality of images; and displaying a three-dimensional model of the subject's teeth that includes the internal structure.

[0088] Furthermore, this specification also describes an intraoral scanning device configured to generate a model of a subject's teeth, the device including an intraoral scanner having a plurality of light sources and a sensor for position and orientation, the light sources being configured to emit light in a first spectral range and a second spectral range, and the second spectral range being penetrative; and one or more processors operatively connected to the intraoral scanner and configured to cause the scanner to capture a plurality of images and the position and orientation of the intraoral scanner corresponding to each of the plurality of images when the intraoral scanner is emitting light in the second spectral range, the processors being further configured to segment the plurality of images to form an internal structure corresponding to the structure within the subject's teeth and to display or transmit a three-dimensional model of the subject's teeth that includes the internal structure.

[0089] Furthermore, this specification also describes a non-transitory computer-readable medium storing instructions executable by a processor, the instructions causing an intraoral scanning device to perform steps of: capturing a plurality of images using penetrating wavelength light and the position and orientation of an intraoral scanner that is unique to each of the plurality of images; segmenting the plurality of images to form an internal structure corresponding to a structure within a subject's teeth; projecting the internal structure onto a three-dimensional model of the subject's teeth using the position and orientation of the intraoral scanner that is unique to each image; and displaying a three-dimensional model of the subject's teeth including the internal structure.

[0090] Furthermore, this specification also describes a method of forming a 3D volume of teeth (including a volumetric volume). For example, one method described herein includes receiving data associated with an intraoral scan of a subject; identifying at least a portion of a volume of a first internal shape of the subject's teeth from the received data; identifying at least a portion of a volume of a second internal shape of the subject's teeth that is different from the first internal shape from the received data; mapping together a portion of the volume of the first internal shape and a portion of the volume of the second internal shape; and outputting together a portion of the volume of the first internal shape and a portion of the volume of the second internal shape as a 3D volume.

[0091] The received data may include data from an intraoral scan of the subject's tooth surface with penetration. The received data may further include data from an intraoral scan of the subject's tooth surface.

[0092] This method may further include identifying the surface of the subject's teeth from the received data; mapping the tooth surface together with a portion of the volume of the first internal shape and a portion of the volume of the second internal shape; and outputting the tooth surface together with a portion of the volume of the first internal shape and a portion of the volume of the second internal shape as a 3D volume.

[0093] The received data may further include data from an intraoral scan of the tooth surface color of the subject.

[0094] This method may further include identifying the color of the tooth surface of the subject from the received data, mapping the color of the tooth surface to the tooth surface, and outputting a 3D volume having the tooth surface and the color of the tooth surface.

[0095] The first internal shape of the tooth may include dentin of the tooth, and the second internal shape of the tooth includes enamel of the tooth. The intraoral scan may include a second intraoral scan of the subject, and this method may further include receiving data associated with a previous intraoral scan of the subject, identifying at least a portion of the enamel or dentin volume from the received data associated with the previous intraoral scan of the subject, and identifying a portion of the enamel or dentin volume identified from the received data associated with the second intraoral scan and a portion of the enamel or dentin volume identified from the received data associated with the previous intraoral scan. By comparing, identifying a change in the enamel or dentin volume, and outputting the identified volume change.

[0096] This method may further include detecting tooth caries of the tooth by comparing the second internal shape and the first internal shape, and outputting a signal to the user associated with the detected tooth caries. Comparing the second internal shape and the second internal shape may include analyzing whether the volume of the second internal shape extends from the surface of the volume of the first internal shape. The analyzing step may include determining whether the volume of the second internal shape extends from the surface of the volume of the first internal shape to a portion associated with dentin of the second internal shape.

[0097] This method may further include calculating a volume of a second internal shape extending from a surface of a volume of a first internal shape, and outputting a signal associated with the calculated volume.

[0098] Furthermore, a method is described that includes receiving data associated with an intraoral scan of a subject, identifying from the received data a volume of dental caries of the subject's teeth, quantifying the volume of dental caries of the subject's teeth, and outputting a signal associated with the quantified volume of dental caries of the subject's teeth.

[0099] This method may further include identifying from the received data a volume of enamel of the subject's teeth, mapping the volume of enamel to the volume of dental caries, and outputting to the user a 3D volume of the mapped volumes of enamel and dental caries. For example, this method may further include identifying from the received data a volume of dentin of the subject's teeth, mapping the volume of dentin to the volumes of enamel and dental caries, and outputting as a 3D volume the mapped volumes of enamel and dental caries together with the volume of dentin.

[0100] The intraoral scan of the subject may include a second intraoral scan of the subject. This method may further include receiving data associated with a previous intraoral scan of the subject, identifying from the received data associated with the previous intraoral scan a previous volume of dental caries of the subject's teeth, and outputting a signal associated with a volume difference between the volume of dental caries and the previous volume of dental caries. This method may further include outputting a 3D model of the volume of dental caries of the subject's teeth.

[0101] Furthermore, this specification also describes an illumination adapter sleeve device for an intraoral scanner, which device comprises a sleeve body configured to fit snugly over a wand of an intraoral scanner, the sleeve body including at a distal end a light passage region configured to allow near-infrared light to pass through the sleeve, a first wing region extending from the distal end of the sleeve body adjacent to the light passage region, and a near-infrared light source configured to emit near-infrared light from the first wing region. This near-infrared light source may be configured to emit near-infrared light across the light passage region.

[0102] The device may further include a second wing region extending from the distal end of the sleeve body adjacent to the light passage region, the second wing region having a second near-infrared light source configured to emit near-infrared light from the second wing region. The device may further include electrical contacts at a proximal end of the sleeve body configured to apply electrical energy to the near-infrared light source. The device may further include a flexible circuit coupling the electrical contacts to the near-infrared light source. Any of these devices may include a camera sensor operatively connected to a second wing extending from the distal end of the sleeve body adjacent to the light passage region.

[0103] In the claims that follow, the novel features of the invention are specifically set forth. The features and advantages of the invention will be better understood by reference to the following detailed description, which describes exemplary embodiments in which the principles of the invention are utilized, and the following accompanying drawings.

Brief Description of the Drawings

[0104]

Figure 1A

Figure 1B

Figure 2A

Figure 2B

Figure 2C

Figure 2D

Figure 2E

Figure 3A

Figure 3C

Figure 3D

Figure 4A

Figure 4B

Figure 4F

Figure 5A

Figure 5I

Figure 6

Figure 7

Figure 8

Figure 9A

Figure 9B

Figure 10A

Figure 10B

Figure 11A

Figure 11C

Figure 11D

Figure 11F

Figure 11G

Figure 11I

Figure 12

Figure 13

Figure 14

Figure 15A

Figure 15E

Figure 16

Figure 17A

Figure 17B

Figure 18A

Figure 18C

Figure 19A

Figure 19C

Figure 20A

Figure 20C

Figure 21A

Figure 21C

Figure 22A

Figure 22C

Figure 23A

Figure 23C

Figure 24A

Figure 24B

Figure 25A

Figure 25B

Figure 26A

Figure 26C

Figure 27A

Figure 27G

Figure 28A

Figure 28B

Figure 29A

Figure 29B

Figure 30A

Figure 30C

Figure 31A

Figure 31B

Figure 31C

Figure 32A

Figure 32B

[0105] In this specification, an intraoral scanner that generates a three-dimensional (3D) model of the intraoral region of a subject (e.g., one or more teeth, gums, jaws, etc.), which may include the internal shape of the teeth and may further include a surface model, and a method of using such a scanner will be described. For example, FIG. 1A shows an example 101 of an intraoral scanner that may be configured or adapted as described herein to generate a 3D model having both a surface shape and an internal shape. As schematically shown in FIG. 1B, an exemplary intraoral scanner may include a wand 103 that can be manually held by an operator (e.g., a dentist, dental hygienist, technician, etc.) and can move over one or more teeth of the subject to scan both the surface structure and the internal structure. The wand may include one or more sensors 105 (e.g., a camera such as a CMOS, a CCD, a detector, etc.) and one or more light sources 109, 110, 111. In FIG. 1B, three light sources are shown, namely, a first light source 109, a second light source (color light source), and a third light source 111. The first light source 109 is configured to emit light in a first spectral range for detecting the surface shape (e.g., visible light, monochromatic visible light, etc. This light may not be visible light), the second light source (color light source) is, for example, white light in the range of 400 to 700 nm (e.g., about 400 to 600 nm), and the third light source 111 is configured to emit light in a second spectral range for detecting the internal shape of the teeth (e.g., transillumination, small-angle transillumination imaging, laser fluorescence, etc., which may be collectively referred to as (e.g., in near-infrared) penetration imaging). Although a plurality of illumination light sources are separately shown in FIG. 1B, in some alternative embodiments, a switchable light source may be used. The light source may be any suitable light source, including LEDs, optical fibers, etc. The wand 103 may include one or more control means (buttons, switches, dials, touchscreens, etc.) to assist with control (e.g., turning the wand on and off, etc.). Alternatively or additionally, one or more control means (not shown) may be present in another part of the intraoral scanner, for example, a foot pedal, a keyboard, a console, a touchscreen, etc. may be present.

[0106] Generally, any suitable light source may be used, and in particular, a light source suitable for the detected mode may be used. For example, any of these devices may include a visible light source or other (including non-visible light sources) light source for surface detection (e.g., at or around 680 nm or other suitable wavelengths). A color light source for color imaging, typically a visible light source (e.g., a "white light" source), may also be included. Further, a penetrative light source for penetrative imaging (e.g., an infrared light source, specifically, a near-infrared light source) may also be included.

[0107] The intraoral scanner 101 may include one or more processors, which may include linked processors or remote processors, control the operation of the wand 103 including the adjustment of the scan, and control the scan and generation of a 3D model including the surface shape and internal shape during review and processing. As shown in FIG. 1B, one or more processors 113 may include or be coupled to a memory 115 that stores scan data (surface data, internal shape data, etc.). A communication circuit 117 including a wireless or wired communication circuit for communicating with components of the system (including the wand) or external components (including external processors) may also be included. For example, the system may be configured to transmit and receive scans or 3D models. One or more additional outputs 119 may be included to output or present information, for example, a display screen, a printer, etc. may be included. As described above, an input 121 (buttons, touch screen, etc.) may be included, and this device may enable user input for controlling the scan and other operations, or may require such user input.

[0108] The devices and methods described herein may all be used to perform scans for internal structures such as enamel and / or dentin cracks, caries (tooth decay), damage, etc., and / or to identify such internal structures. Thus, any of the devices described herein may be configured to perform scans that can be used to detect internal structures using an infiltration wavelength or a spectral range of infiltration wavelengths. Also, herein, methods for detecting cracks, caries, and / or damage, or other internal shapes (e.g., dental fillings, etc.) will also be described. Various infiltration scan techniques (infiltration imaging) may be used or may be incorporated into the device, such techniques including transillumination and small-angle infiltration imaging, but not limited thereto, both of which detect the path of light at an infiltration wavelength from or through tissue (e.g., from or through one or more teeth).

[0109] Transillumination is a technique that may be used to know the internal shape of teeth. Conventionally, there are two basic configurations for tooth transillumination. FIGS. 2A and 2B show those configurations, namely, the 180° configuration and the 90° configuration. Both configurations can be used for visualizing the interior of teeth, mainly for visualization through enamel. As shown in FIG. 2A, in the 180° configuration, an incident wavelength (including the spectral range of one or more incident wavelengths) is emitted from a light source 203, transmitted from one side of a tooth 201, and a sensor 205 (e.g., a camera) on the opposite side detects the light transmitted through the tooth. This light is neither scattered nor absorbed. Similarly, in FIG. 2B, the tooth 201 is illuminated with light from light sources (203, 203') on both sides, and a camera 205 oriented at 90° with respect to both light sources detects light at an angle perpendicular to the light sources. Transillumination has typically been limited to single-projection type use for capturing an image of the interior of a tooth (similar to the case of using X-rays). In this specification, an incident wavelength (e.g., 700 - 1300 nm, 700 - 1090 nm, etc., e.g., 850 nm) is used to obtain multiple projections or orientations from a single position of a scanner relative to one or more teeth and / or with respect to multiple angles of a sensor relative to a tooth (specifically, three or more orientations or projections may be employed for each internal region to be imaged), and a method and apparatus for visualizing the enamel-dentin region will be described. By employing multiple (e.g., three or more) projections, better imaging can be achieved, because multiple (e.g., three or more) images of the tooth transmitted through can be generated from a specific position of a wand relative to one or more teeth. Performing one or more 180° projections may be useful when the light propagation distance is short and light scattering is low, but combining multiple different projections (orientations) from the same location (e.g., within a few milliseconds of each other, at approximately the same scan time) can enable the system to construct a volumetric model of the enamel-dentin region.

[0110] In a deformation mode that performs a 90° configuration and / or 180° configuration projection, the intraoral scanner may be adapted to perform transillumination imaging in this configuration. For example, FIGS. 2C and 2D show an example of the distal end of a wand of an intraoral scanner adapted to collect transillumination images at 90° and 180°, and the wand 213 includes a pair of protrusions or wings 215 each housing a combination 217 of a light source (LED) and a camera. In FIGS. 2C and 2D, both wings and the base of the wand may include a light source and a sensor (camera), whereby at least three transillumination images can be acquired from a single position of the wand relative to the tooth, as shown in FIGS. 3A - 3C. In FIG. 3A, a first orientation is shown, where the right LED 303 is turned on and illumination is passed through the tooth for detection / capture (180°) by the camera 305 on the left. FIG. 3D, similar to FIG. 3A, shows how light is applied from the right and transmitted into the tooth (arrow), passes through the tooth, and reaches the camera sensor 305 (also referred to herein as an image sensor, camera, or simply "sensor") or scatters from the internal region. The orientation of the camera sensor and the illumination source may be exchanged. In FIG. 3B, the left LED 303' is turned on and illumination is passed through the tooth for detection / capture (180°) by the camera 305' on the right. In FIG. 3C, both LEDs 303, 303' are turned on, illumination is performed from both the right and the left, and the camera 305'' located 90° off the axis of the LEDs captures a transillumination image.

[0111] Generally, the transillumination imaging data as described above may be combined with and collected simultaneously with the 3D surface data of the tooth (e.g., 3D surface model data), thereby enabling the addition of a data layer on internal structures such as caries and cracks. Further, by using multiple projections as described above (obtained from multiple orientations), it is possible to reconstruct a volumetric model of the internal structure of the tooth enamel, thereby enabling the visualization of shapes that would not be visible by other methods.

[0112] The 90° and 180° configurations of tooth transillumination can be useful, but particularly useful is the case where an infiltration imaging configuration is provided in which the angle between the emitted light beam (vector) and the received light beam (vector) is very small (e.g., 0 to 30°, 0 to 25°, 0 to 20°, 0 to 15°, 0 to 10°, etc.). In particular, an angle of 0 to 15° (or 0 to 15° excluding 0°) can be useful.

[0113] In transillumination with 180° and 90° configurations, the movable range of the intraoral scanner wand may be limited to the periphery of the tooth, which is due to the angle limitation of the camera with respect to the light source in those configurations (as shown in FIGS. 2C and 2D). Therefore, in this specification, a method and apparatus for infiltration imaging / visualization using a small angle including 0 to 15° (e.g., in the enamel-dentin region) will also be described. In one example, a light source (LED) that emits an infiltration spectral range (e.g., 850 nm) with a line-of-sight vector at a small angle of 0 to 15° with respect to the field of view angle of the camera is used. As described above, this infiltration imaging may be combined with the simultaneous 3D surface modeling of the tooth. The relative positions of the light source and the camera are typically known, and it is possible to acquire one or more infiltration images at each position of the wand. Since the line-of-sight vectors available for the wand are at small angles, the intraoral scanner wand may be configured to have only a slight curve, which allows the intraoral scanner wand to fit into the oral cavity and move easily within the oral cavity, which is different from a wand configured to measure 90° and 180° transillumination (such a wand can hold the LED and the sensor using a device shape including side wings so that the wand can wrap around the tooth for imaging (see, for example, FIG. 2C)). By using small-angle reflection imaging, scans in the buccal and lingual directions can be enabled, while the 90° (transillumination) scans described in this specification will be limited to scans in the occlusal direction.

[0114] Performing penetration imaging using a small angle may well include imaging the interior of a tooth using a wand that can move unrestrictedly around the tooth, and without the need for a dedicated structure and / or operating mode, it is possible to capture internal structure data while also performing a scan for 3D (surface) model data. However, reducing the angle between the emitted light and the detector can also become complicated due to direct reflections. For example, direct reflections can occur in regions where the illumination angle and the imaging angle are approximately equal on the surface of the tooth (e.g., in the light cone and imaging NA). Such direct reflections can be a problem if they saturate the sensor or if they obscure the deep structure information while showing surface information. To overcome such problems, the devices and methods described herein are capable of capturing and using multiple illumination orientations obtained from the same position. In the context of a handheld wand herein, obtaining multiple images from the same position may well substantially mean obtaining the multiple images almost simultaneously, such that no significant amount of movement occurs. For example, it is possible for multiple images to be obtained within a few milliseconds of each other (within less than 500 milliseconds, within less than 400 milliseconds, within less than 300 milliseconds, within less than 200 milliseconds, within less than 100 milliseconds, within less than 50 milliseconds, etc.) and / or it is possible for small movements to be corrected.

[0115] Alternatively or in addition, the device and / or method can reduce or eliminate problems arising from saturation due to direct reflections by using only non-saturated pixels. In some variations, as part of the processing, surface information may be subtracted from the penetration image. For example, a visible light image ("viewfinder image") or surface imaging may be used to remove direct surface reflections.

[0116] Generally, the devices (e.g., systems) described herein are capable of always recognizing the position of the wand based on a surface scan, which is possible even when acquiring images at a plurality of different (and even smaller) angles. Thus, when surface scans and penetration scans are performed simultaneously or almost simultaneously (e.g., within 600 milliseconds, 500 milliseconds, 400 milliseconds, etc. relative to each other), including interleaving these scans with other scan types, the position of the wand relative to the object being scanned may be recognized. Based on this information, the device can estimate which portions of the plurality of images or signals are arriving from the surface and what is arriving from the deep structure.

[0117] FIG. 4A shows an example of a configuration including a penetrative light source 403, 403' (e.g., a light source in a penetrative spectral range) and one or more cameras 405 that can be used as part of an intraoral scanner wand shown at different positions around a target object (tooth 401). In FIG. 4A, three camera positions are shown, and at each position, a pair of LEDs (e.g., 403 and 403') are arranged on both sides of each camera, and those LEDs emit light in a penetrative spectral range (penetration wavelength). Alternatively, a single light source (e.g., an LED) may be used instead of the pair. At various wand positions relative to the tooth, different images by the penetrative modality may be acquired. Alternatively, the wand may be configured to have a plurality of imaging sensors (cameras) and a plurality of light sources such that a plurality of penetrative images can be acquired almost simultaneously (e.g., by turning on a plurality of sensors when illuminating from the direction of one or more LEDs (e.g., FIGS. 5G and 5E, etc.)). In FIGS. 5A - 5I, as shown, penetrative images in at least nine different directions may be acquired. Alternatively or additionally, a plurality of orientations may be sequentially used, such as within a very short period (e.g., within less than 500 milliseconds, within less than 400 milliseconds, within less than 300 milliseconds, etc.).

[0118] Figures 4B - 4F show other emitters and detectors used at any penetration wavelength that can be used to acquire internal images of an object (e.g., a tooth) having a translucent and highly scattering region. These images typically collect in reflection mode (e.g., collecting light at the penetration wavelength that can travel into the tooth, scatter / reflected from the internal structure, and be collected by the detector). In Figure 4B, a classical (e.g., 90°, 180°) transillumination angle is combined with a small angle illumination angle. In Figures 4C - 4F, the light rays emitted and collected have a very small angle (e.g., about 0°), and the assemblies of emitters 403, 403' and detector 405 (e.g., CMOS, CCD, etc.) are arranged adjacent to each other as shown in Figure 4C, arranged in combination with each other as shown in Figure 4D, or simply share a common or nearly common beam path as shown in Figures 4E and 4F (in this case, reflection or waveguides may be used, including the use of beam splitters (dichroic beam splitters) and / or filters to direct the emitted light and / or received light).

[0119] As described above, any suitable sensor may be used, including a CMOS or CCD camera, or any other sensor capable of detecting the appropriate wavelength (e.g., a near - infrared wavelength detector).

[0120] Applying transillumination light from close to the sensor (camera) may intensify the illumination in the area closest to the camera, and thus may make the illumination distribution non-uniform. However, unexpectedly, this does not pose as much of a problem as expected. In the situation where transillumination imaging is performed, the light that generates the captured image propagates through the object. The longer the path, the longer the scattering that occurs, and as a result, compared to direct illumination, the illumination becomes smoother. In the case of front illumination, as a result of small-angle illumination, the light intensity is strongest in the area closest to the illumination source (e.g., LED), which causes backscattering. This nearby area (e.g., the first 1 - 2 mm) is an important area for detecting caries. However, there may be cases where it is desirable to compensate for the resulting non-uniform illumination profile distribution as described above.

[0121] By using transillumination imaging, and particularly small-angle illumination / imaging (also sometimes referred to as reflection imaging), it is possible to obtain information on the internal regions of teeth (e.g., cracks, caries, damage, etc.) that would not be obtainable by other methods. The internal shape (or internal region) information may be incorporated into a 3D model, which can be particularly powerful when combined with surface information (e.g., a 3D surface model or depth information). This enables the user to seamlessly capture diagnostic data during a 3D scanning procedure while allowing unrestricted movement around the tooth to capture data from various angles to provide a 3D model of the inside of the tooth.

[0122] Combining surface data and internal shape data As described above, it can be particularly useful to combine and / or correlate 3D surface data with any internal shape data (including, but not limited to, transillumination imaging data). For example, internal shape data such as transillumination imaging data and surface data (surface imaging data) collected from the same or approximately the same position of an intraoral scanner may be combined so that the same coordinate system can be applied to both types of data.

[0123] As described above, an illumination device that emits light in two or more different spectral ranges may be equipped in a color 3D intraoral scanner as shown in FIG. 1A to capture various surface shapes and internal shapes. By correlating and combining the collected data (e.g., surface data and internal shape data), it is possible to form a 3D model that includes information not only about the internal structure of the teeth but also damage, aging, enamel incomplete fractures, etc. The internal shape data may be collected by any suitable penetration imaging technique, such as the reflection (e.g., small angle) illumination and imaging techniques and transillumination imaging techniques described above, or may be collected by other techniques known in the art, such as UV / blue fluorescence and red light fluorescence, but is not limited thereto.

[0124] The internal shape data may be collected together with (and may include images of damage and internal tooth structure in) surface data including the color 3D surface model data of the teeth and may be combined with those surface data. The combination of the surface data and the internal data may be represented as a 3D model or 3D rendering, which may include full-color 3D data (including models and renderings) of the damage and internal tooth structure, as well as the surface of the scanned portions of the teeth, gums, and any other intraoral regions. However, depending on the deformation form, the spread of the internal data and the surface data may be the same, and depending on the deformation form, the spread of the surface data may be larger than the spread of the internal data. For example, the 3D model may include internal data only for a part of the 3D model, and other regions may not include the internal shape (or may include only an incomplete internal shape).

[0125] In use, a 3D model of one or more teeth, including both surface and internal elements, may be analyzed automatically or manually, and the internal shape may be identified and / or marked. For example, labeling, including color-coding, may be performed for damage, caries, and / or cracks, which may be represented, for example, in one or more images available and / or as part of a data file generated to display those images, depending on the type and severity of the damage, caries, and / or cracks. Alternatively or additionally, the meaning / explanation of these findings may be written in.

[0126] An intraoral scanner that generates a 3D model including both surface and internal structures as described herein may include one or more image sensors. For example, the image sensor may be configured to capture color 3D (surface) images or data, and may also capture images of damage and internal tooth structures. Optionally or additionally, the system may have multiple sensors. The surface data may be acquired using the intraoral scanner in any suitable manner. The intraoral scanner is generally configured to perform scans (by a wand), for example, simultaneously, in both a surface imaging mode and an internal imaging mode. For example, the capture of surface data may be performed using a color intraoral 3D scanner by confocal stereoscopy, or structured light triangulation, or any other 3D surface scan technique capable of intraoral scanning.

[0127] As shown in FIGS. 10A and 10B, an illumination light source (including a light source for a first modality (e.g., surface scan), a light source for a second modality (e.g., penetrative imaging such as penetrative imaging), and / or a light source for a third modality (e.g., color scan)) may be disposed at the front tip of an intraoral scanner wand (e.g., near the object to be scanned or inside the scanner head). The illumination configuration of the front tip may be configured by changing the front tip according to the needs of the application, regardless of the presence or absence of any specific light source suitable for the desired diagnostic function. The light source and the sensor (e.g., a camera) may be arranged in any suitable manner, including those shown in FIGS. 10A-10B and FIG. 4. For example, the light source and the camera may be adjacent to each other. In some variations, in this system and method, small sensors 1005, 1007 (e.g., wrapped around the front tip) are used to capture three-dimensional 3D internal shape data (e.g., images) and / or to facilitate penetrative imaging in a more efficient manner.

[0128] As described above, in some variations, the damage / internal tooth structure capture method may be any composite penetrative imaging that includes one or more of transillumination, red light laser fluorescence, blue / UV laser fluorescence, etc., that penetrates the tooth. Generally, the internal shape data may be used in combination with the surface data including the coordinate system of the surface data to reconstruct a 3D representation of the tooth structure. For example, the 3D reconstruction of the tooth data may be reconstructed by an algorithm that combines several (e.g., multiple) 2D images typically acquired at several different angles or orientations using any of the internal shape imaging techniques described herein.

[0129] In particular, data captured by an intraoral scanner, including 3D models of one or more teeth having both surface and internal shapes, may be stored by the device and / or sent to a doctor, medical record, dentist, etc. For example, all data captured by the intraoral scanner, i.e., a color 3D model combining the topography of tooth damage and the internal tooth structure, may be retained in a patient database designated for long-term monitoring and preservation of the patient's oral health status. These data may or may not be annotated (including markings referring to the date and / or internal shape).

[0130] For example, a comparison over a long period of time may be performed at one or more levels using the 3D models described herein. For example, it may be performed by comparing surface changes, visual color changes, internal / volumetric changes, or any combination thereof over time. For example, each may be shown as before and after the change (e.g., by manual evaluation or automated differential comparison). In some embodiments, two or more 3D models may be overlaid on a display such that the differences between the 3D models are highlighted. The overlaid models may, for example, be useful for highlighting changes in enamel thickness, dentin volume, color, opacity, and / or the size of caries. Optionally, a 3D model of a patient's tooth condition at an earlier time may be morphed to a 3D model of the patient's tooth condition at a later time, which is useful for highlighting how the patient's tooth condition has changed over time. In some embodiments, a time series of 3D models may be provided as a video or animation of the change in the patient's tooth condition by progressively morphing from one model to the next. The automated comparison may be performed by applying or transforming to a common coordinate system, which may be performed in particular using surface information (e.g., based on 3D surface model data included as part of the generated 3D volumetric model). Typically, all three types of data (surface, color, volumetric, etc.) may be interconnected by the same coordinate system as described above. The methods and apparatuses described herein, including 3D models, may generally be used to predict the future tooth or orthodontic condition of a patient (e.g., as described in U.S. Patent Application Publication No. 2016 / 0135925, the entire contents of which are incorporated herein by reference).

[0131] When comparing scans that include 3D volumetric scans, those scans may be adjusted or normalized relative to each other in preparation for automatic, semi-automatic, or manual comparison. For example, scans of one or more teeth (e.g., a scan of the entire jaw, a partial scan, etc.) may not be 100% reproducible, and in particular, may not be reproducible to an accuracy beyond the voxel resolution. When performing voxel-by-voxel comparison, a matching function and / or a morphing function may be applied to one or both scans to enable a more direct comparison. For example, a matching function and / or a morphing function may be used. The morphing function can enable voxel-by-voxel comparison by aligning the external surfaces with each other. This can also enable comparison between the overall scan and a partial scan.

[0132] As described above, generally, the captured data may be stored and saved in the same coordinate system. Thus, surface data (including 3D surface model data) may use a certain coordinate system (e.g., x, y, z. Thus, the 3D surface model is S(x,y,z)), and the internal shape data may use or refer to the same coordinate system (thus, for example, the internal shape data is I(x,y,z)). Thus, a common shape or structure will have the same address (coordinates) between the two data sets.

[0133] FIG. 6 is a diagram showing an example of a method for generating a 3D model or 3D rendering of one or more teeth using surface data and internal shape data. In this example, first, a handheld intraoral scanner wand (scanner) may be placed adjacent to a target intraoral region to be scanned (601). When the scan is initiated, the apparatus may collect surface data (e.g., 3D model surface data) including depth information in a first coordinate system (603). The surface data may typically be collected while the sample is illuminated using a first illumination spectrum (e.g., visible light (e.g., monochromatic light or broadband light)). Internal shape data may also be collected, which may be performed, for example, using a second illumination spectrum that penetrates inside one or more teeth (which may include only a single wavelength or a small wavelength range) (605). This data may use the same coordinate system as the surface data, which may be implemented as detailed later. After collection, analysis and / or filtering (including differences, smoothing, etc.) of the data, and combination are performed to form a 3D model rendering of the intraoral region (e.g., one or more teeth, gums, jaws, etc.) using both the surface data and the internal shape data (607). For example, when constructing the 3D geometry of the internal shape data (which is typically essentially two-dimensional), the algorithm can enhance the accuracy of the internal shape data using a reference to a known 3D surface scan.

[0134] In general, in any of the devices and methods described herein, it is possible to reconstruct a volumetric model of one or more teeth, including the internal shape, using the internal shape data collected at 605. In particular, tomographic reconstruction (e.g., optical tomography) may be used. Full volumetric modeling may be performed. Typically, depending on the material properties and the light used, refraction, reflection, scattering, and / or absorption (including combinations thereof) of all transmitted light rays may occur. In some variant forms, the method and / or the device divide the volume of the tooth into small voxels and, for each voxel, estimate these four parameters (refractive index, reflection, scattering, absorption) using a coordinate system corresponding to the coordinate system of the surface data, based on the imaging data collected. More complex models (e.g., based on anisotropic scattering or complex surface scattering) may be used. After the parameter set for each voxel has been estimated, the method or the device can compare how well the captured image fits this model. Thus, in some variant forms, the device and / or the method may attempt to minimize the difference between the captured image and the modeled and predicted image. An initial estimate, including an estimate of the enamel parameters and width, may be constructed from the capture results of the 3D surface.

[0135] As an alternative or in addition, multi-faceted modeling may be used. In multi-faceted modeling, a set of (possibly homogeneous) optical properties of a substance is assumed, for example, a set of properties for air, dentin, and enamel (although more than these three may be included). This technique is capable of attempting to discover boundaries between substances. There are multiple ways to achieve this, for example, a technique quite similar to the one described above for full volumetric modeling, but there is a technique that does not use a voxel representation. As an alternative or in addition, a contour line method may be used. In this method, a first (e.g., air and enamel) boundary is given from the capture result of a 3D surface, and then it is possible to approximate a smooth 3D surface that best fits this silhouette by discovering the edges of regions in a 2D penetration image. For example, refer to "3D Shape from Silhouette Points in Registered 2D Images Using Conjugate Gradient Method, (Andrzej Szymczaka, William Hoffb, Mohamed Mahfouzc, etc.)", the entire content of which is incorporated herein by reference. Outside of the contours, as is known in the art, other shapes, similarity points, corners may be used. These shapes may be detected from different viewpoints, may be 3D located by triangulation, and they are part of the boundary.

[0136] In practice, recording surface data and internal shape data in the same coordinate system can be achieved by scanning both the surface shape and the internal shape at the same location and / or at the same time. As described above, in the case of a handheld intraoral scanning device (e.g., a wand) controlled by a user, it may be difficult to scan the same area at different wavelengths at different times. Therefore, all of the devices and methods described herein are capable of adjusting scans in a plurality of different modalities or modes (e.g., surface data scan, and / or internal shape / permeation data scan).

[0137] For example, FIG. 7 shows one method in which an intraoral scanner alternately performs a surface scan and one or more other scan modalities (e.g., an internal shape scan (such as an optical coherence tomography scan)). In FIG. 7, the scanner is placed adjacent to the target intraoral structure to be modeled (701), and then the wand may be moved above the target, during which the device automatically scans the target with respect to both surface data and internal data (703). As part of this method, the system may alternately (switchably) collect surface data (e.g., 3D surface model data) by scanning a portion of a tooth using a first modality (e.g., a surface scan by emission of an appropriate wavelength or wavelength range) (705) and perform a scan using a second modality (e.g., an optical coherence tomography wavelength) (707). The method and device may switch to the second modality (e.g., an optical coherence tomography wavelength or optical coherence tomography wavelength range) for a short time after an appropriate duration in the first modality, and collect internal shape data over a short time (a second duration) over substantially the same area of the object that was scanned in surface mode (707). As long as the second duration is appropriately short (e.g., less than 500 milliseconds, less than 400 milliseconds, less than 300 milliseconds, etc., less than 200 milliseconds, less than 100 milliseconds, less than 50 milliseconds, etc.), the coordinate system between the two modalities is substantially the same and the wand is in substantially the same position during the switch. Alternatively or additionally, the method and device may extrapolate the relative position of the wand with respect to the surface based on surface data information collected immediately before and after the collection of the internal data. Thus, in any of the methods described herein, including what is shown in step 703 of FIG. 7, the device may interpolate the position between each scan (e.g., a first modality scan (e.g., a surface scan), a second modality scan (e.g., an optical coherence tomography, e.g., one or more near-infrared scans), and a third modality scan (e.g., a color scan), etc.). This interpolation makes it possible to correct for small but potentially non-negligible movements of the wand during the scan.In particular, when adjusting between the surface structure and the internal structure, the scan is performed manually, and for each scan image, interpolation (and / or extrapolation) is performed to more accurately approximate the 3D position of the tooth (or the 3D relative position of the tooth with respect to the scanner wand). Therefore, the portion of the tooth scanned using the penetration wavelength may be linearly interpolated between the surface scans performed before and after the penetration scan. For example, refer to FIG. 8 described later. FIG. 8 shows an exemplary relative timing of the scans in each mode. Alternatively or additionally, the position of the tooth and / or wand / scanner during the scan may be extrapolated from the previous surface scan position based on the movement speed of the scanner wand (e.g., the rate of change across the surface from the previous surface scan and / or estimated from the movement sensor within the wand). By thus correcting the coordinate system (e.g., the x, y, z positions, and azimuth angles) of each scan, it may be possible to closely align images of different modalities with respect to each other regardless of how the scanner is operated by the user. In the penetration scan, multiple scans may be acquired from the same relative position and used for reconstructing the internal shape, and the accuracy of this coordinate system may make it possible to increase the resolution of the internal shape modeling.

[0138] Generally, when collecting penetration wavelength images, the emitted light and the received light may have different polarizations. In the reflection light mode, for example, when using small-angle penetration imaging, a portion of the energy is penetrative, but a portion is also reflected from the surface. It may be preferable to block this direct surface reflection, which may be done in any suitable way, including using polarization. For example, the sample (e.g., a tooth) may be illuminated with a penetration wavelength of a specific polarization to block the surface reflection, and this polarization may be blocked in the imaging path. This polarization may also help to block the direct light from the illumination source in transillumination (e.g., when there is a direct line of sight to the illuminator as in the case of 180° transillumination).

[0139] Many of the methods and apparatuses described herein include a mode switch for distinguishing between a surface structure and an internal structure, although in some variations they may be detected entirely simultaneously, which may be done, for example, using a dichroic beam splitter and / or a filter. Thus, by separating wavelengths and / or polarizations that are penetrative and include internal reflection and / or scattering from wavelengths and / or polarizations that include only (or primarily) the surface shape, surface data may be collected and processed separately from the internal shape, and these two data sets may be recombined later. This technique may originally use the same coordinate system.

[0140] For example, FIG. 2E shows a schematic of an intraoral scanner configured to perform both a surface scan (e.g., visible light, non-penetrative) and a penetrative scan using a near-infrared (NIR) wavelength (850 nm in this example). In FIG. 2E, the scanner blocks near-infrared light (P-polarized light) reflected from the surface of tooth 290, while collecting near-infrared light (S-polarized light) scattered from the internal tooth structure / region, and includes a near-infrared illumination source 289 and a first polarizer 281, as well as a second polarizer 283 in front of image sensor 285. The near-infrared light source illuminates the tooth with P-polarized light, and specular light reflected from the surface of the tooth (e.g., enamel) is reflected by specular reflection since its P-polarized state is preserved. Near-infrared light that penetrates into the internal tooth shape such as dentin scatters and becomes randomly polarized (S and P). The wavelength-selective quarter-wave plate 293 does not change the polarization of the near-infrared light (e.g., keeps the polarization state of the delivered near-infrared light unchanged), but changes the polarization of the return scan light from P to S so that only surface reflections are captured at the scan wavelength. The return near-infrared light has a mixture of S-polarized and P-polarized light and is first filtered through a polarization beam splitter (PBS) 294 and a polarization filter 283 so that only S-polarized light is sent to the image sensor. Thus, only the near-infrared S-polarized light coming from the internal tooth structure is captured by the image sensor, and the specular light with the original P-polarization is blocked. Other intraoral scanner configurations may be used as part of the probe, with or without a polarization filter as shown in FIG. 2E.

[0141] In FIG. 2E, the surface scan may be performed by illuminating the surface (using scanner illumination unit 297), which illumination is performed with P-polarization, and the polarization is reversed by a wavelength selective quarter-wave plate 293 that transmits S-polarized light to the image sensor.

[0142] As shown in FIG. 7, the scan mode may be adjusted manually or automatically (709), including the duration of the scan modality (e.g., the second scan modality) for calculating the internal shape data. For example, the scan procedure (time sharing and sequence) may be changed for each case, and the system can automatically optimize the scan resources so that a high-quality scan and / or a more complete reconstruction is performed. The method or apparatus can determine the quality of the scan data (e.g., the quality of the surface scan data) (709) and accordingly adjust the scan duration (e.g., the second duration). The quality estimation may be performed automatically, e.g., based on blur, saturation, or unsaturation, etc. For example, the duration of the scan mode may be dynamically adjusted (e.g., increased or decreased) based on the quality of the scan in this modality, and if the previous x scans in this modality fall below a first (e.g., minimum) quality threshold that quantifies one or more of blur, saturation, unsaturation, etc., then the scan duration d for that modality iIt may be lengthened. If the scan duration is longer than the minimum duration and the quality exceeds a second quality threshold (which may be the same as the first quality threshold or higher than the first quality threshold), the scan time may be shortened. By shortening the scan duration, it may be possible to lengthen the duration of other scan modalities and / or increase the switching speed of scan modalities. Alternatively or in addition, the scan duration of a modality may be adjusted based on the completeness of the 3D model to be reconstructed. For example, when scanning an area with a more complete surface model in the 3D model (e.g., an area where the surface model has already been created), by shortening the duration of the surface scan and lengthening the duration of the penetration scan (e.g., a reflection scan using a near-infrared wavelength or a transillumination scan using a near-infrared wavelength), it is possible to increase the resolution and / or expand the range of the internal structure. Similarly, the frequency of the scan in each mode may be dynamically adjusted by the apparatus. Both the methods and apparatuses described herein may be configured to provide feedback to the user to slow down the scan and / or add scans from specific angles by displaying these missing areas or angles on a 3D graphical display.

[0143] As shown in FIG. 7 (e.g., optional step 708) and FIG. 8, three or more scan modalities may be used. FIG. 8 shows an exemplary method of operating an intraoral scanner to switch between a plurality of different scan modalities, such modalities including surface scan 801, laser fluorescence 803, color visible light scan (viewfinder) 805, penetration scan 807, UV scan, etc. The system may initially perform the scan modality switching in accordance with a default scan method. Thereafter, the system may (in real time) analyze the defaults coming from each scan modality as described above, and may prioritize scan modalities with low data integrity, which may be done, for example, by increasing the scan frequency and / or increasing the duration (d). In some embodiments, the system may compare the collected data from one or more of those scan modalities with a predetermined data resolution threshold to determine which scan modality should be prioritized. For example, when the system confirms that sufficient surface data has been collected by the surface imaging modality and that the resolution of the internal shape data is still insufficient, it may increase the frequency of surface penetration imaging or increase the duration. Alternatively or additionally, in some variations, the scans may be performed simultaneously for various modalities. Once the scans of the sufficient scan area are completed, a 3D model of the entire intraoral area may be assembled using those scan data (711). Alternatively, the 3D model may be assembled continuously during the scan. The scan frequency 809 is shown as the frequency of the scan amplitude in FIG. 8. As the depth of the confocal scan increases or decreases, the surface scan is performed at the maximum of the scan amplitude, and the penetration scan is performed at the minimum of the scan amplitude. The frequency of the depth scan 809 may be dynamically increased or decreased during the scan. This is, for example, to enable a longer scan duration or to accommodate the user moving the wand / scanner faster.Depending on the deformation mode, the wand may include a motion sensor (e.g., an accelerometer, etc.) to detect the movement speed, and the scan speed and duration may be adjusted based on the detected movement of the scanner.

[0144] As shown in FIG. 6, the resulting 3D model including the surface structure and internal structure may be utilized in various ways for the health management of a subject (e.g., a patient). For example, the 3D model may be used to (automatically or manually) identify and analyze tooth damage, caries, and / or cracks. The 3D model may be used, for example, to measure the size, shape, and location of damage including caries, to evaluate the type of caries based on translucency, color, shape, and / or to evaluate the type of surface problems (e.g., cracks, caries, etc.) based on surface illumination (609).

[0145] This 3D data (or data derived therefrom) may be monitored over time for a particular patient (611). For example, the 3D data may be checked over time for changes in shape, size, and type, either visually or by an algorithm.

[0146] Generally, it is possible to annotate the 3D data. For example, a clinician may mark the area of interest after the first scan, and that area may be evaluated manually or automatically in subsequent scans. Further, the 3D data may be used to assist in treatment, or to provide treatment guidance and monitoring (613). For example, when a clinician determines a tooth restoration, 3D data showing the surface area and internal area generated as described herein may be used to provide tooth reduction guidelines for reliably removing the carious part. During the treatment, additional (e.g., intermediate) scans may be performed to give the physician further instructions regarding the reduction and immediate feedback.

[0147] Figures 9A and 9B show an example of a rendering of a 3D model 900 of an intraoral region of a subject, which includes both the surface (the entire surface is shown in the projection of FIG. 9A) and the internal structure (an enlarged region is shown in FIG. 9B). In FIG. 9B, a dark region 903 revealed by a combination of diffuse optical imaging using 850 nm light and 3D surface data indicates the region of interest. The region of interest may be a carious region, a dental filling, or the like. By manipulating the image in this way, it is possible to significantly enhance the treatment and understanding of the dental needs of the subject by observing the 3D model or each region of the 3D model in rotation, zoom, cross-section, or other forms.

[0148] Depth scan Figures 11A - 11I show an example of volumetric modeling of internal tooth structure using a diffuse wavelength such as transillumination (TI). In this example, there is damage within the tooth, and it is possible to detect the damage when the light is below the damage or at the height of the damage. When the light is below the damage, the damage absorbs the light, so the damage is displayed as a dark spot in the image. In FIG. 11D, a damaged tooth is shown together with a scanner sensor 1101 that is above the tooth (located above the occlusal surface of the tooth). The scanner includes one or (as shown in FIGS. 11D - 11F) two light sources (emitters) 1105, 1105' that emit near-infrared light as indicated by the arrows. The light penetrates the tooth, and the sensor 1101 detects the occlusion of light due to the damage as shown in FIG. 11A.

[0149] Moving the scanner upward together with the light source (i.e., lifting the wand of the scanner high in the longitudinal direction of the tooth) causes the damaged image to change as shown in FIG. 11B. The corresponding relative position of the light source with respect to the tooth is schematically shown in FIG. 11E and illustrated in the illustration of FIG. 11H. As the scanner is moved further above the tooth, the dark spot 1113 representing the damage begins to shrink and eventually disappears completely and the light saturates. Finally, when the light sources 1105, 1105' are higher than the damage, there is no dark spot (e.g., FIG. 11C), and only the central occluded region (dentin) is shown. As already described above, the outer surfaces of the tooth and the gum may be scanned simultaneously using separate light sources, thereby obtaining a 3D outer surface of the tooth and thus the distance from the tooth to the scanner. This information may be used, as described above, to map the depth and / or shape of the damage.

[0150] Such depth scans may be performed manually or automatically and may be useful in providing backup and / or alternative to the volumetric modeling of one or more teeth (e.g., 0-degree volumetric modeling). In fact, this longitudinal scan of the tooth, which may be performed in either direction (e.g., from the bottom to the top, from the top to the bottom of the tooth), may be used as one type or subtype of volumetric scan that can provide information about the shape and position of the dentin and / or damage.

[0151] For example, a method of performing a longitudinal (z-axis) scan of one or more teeth with an intraoral scanner, particularly an intraoral scanner having both an infiltration (e.g., near-infrared) scan wavelength and a surface scan wavelength, may be an alternative to volumetric scanning. Generally, data can be obtained by scanning one or more teeth upward or downward (in the z-axis direction).

[0152] As described above, one configuration of the described scanning device is capable of optically imaging the internal region of one or more teeth, which is possible, for example, by using transillumination (transmitting through the side surface) at a certain angle (e.g., 90°) between a light source and a camera. When dental caries are present in the tooth, it is possible to show the caries as an occluded region by observing the tooth from above at the penetration wavelength (e.g., in transillumination). Depending on the relative z (depth) position of the light source with respect to the caries, the occluded region corresponding to the caries appears in the x,y image. Therefore, in order to identify one or both of the z position and the shape of the caries, a scan through the z-axis (depth) may be performed as described above. In some variants, at the beginning of the method of scanning using the penetration wavelength (or penetration scan and surface scan), it may be done to illuminate from both sides, image from above, and place the light source as close as possible to the gingival margin. This method may then proceed to move upward along the z-axis of the tooth and move away from the occluded surface of the tooth. This may make it possible to apply light to the damage from various depths (along the z-axis). As shown in FIGS. 11A - 11C, although the caries will initially be present, as the scanner is pulled upward, it shrinks within the imaging plane (x,y) and eventually stops blocking the light. All of these methods are also capable of calculating or identifying the longitudinal z position of the tooth when the scanner moves upward, from which the relative depth with respect to the tooth is known, and thus the depth of the damage from the enamel layer is known. From this information, it is also possible to identify the dimensions of the damage (e.g., an estimate of how far the damage extends in the z position direction), as well as the width and spread (e.g., how far it extends in the x,y directions). This information may be used together with a surface 3D model showing the outer shape of the tooth in order to provide a model of the entire tooth and the damage.

[0153] Therefore, it is possible to determine models of both the external and internal structures of teeth using both the penetrating wavelength (e.g., near-infrared wavelength) and the non-penetrating (surface scan) wavelength. To identify the depth and / or dimensions of the internal structure of one or more teeth, a depth scan (and more particularly a discontinuous scan) in the z-axis direction of the tooth would be particularly useful. In any of the methods described herein, as noted above, the 3D scan of the tooth may be performed simultaneously with the penetrating scan (including depth).

[0154] Therefore, in any of the tooth scan methods described herein, the method may include the step of identifying a depth (z) dimension for each scan to indicate the relative depth of a light source (e.g., near-infrared light source) with respect to the tooth. This information may be provided by a 3D surface scan that corresponds / correlates with the penetrating scan. Depth information (e.g., indicating how far the scanner has moved in the z-axis direction) can provide substantial volumetric information.

[0155] As described above, the depth (z) scan described in this specification may be performed manually or automatically. For example, this scan may be performed by manually scanning a wand upward along the teeth. During the scan, both 3D surface modeling and internal modeling / imaging may be performed simultaneously and continuously. The scan may be performed at any suitable rate (e.g., 20 scans per second). Thus, the user may perform the scan at a reasonable speed, and the output may be performed in real time, including displaying damage, and / or the damage (and any other internal structures) may be displayed later after software analysis is performed. In one example, a surface scan (by laser) may be performed over a period of about 35 milliseconds, followed by a 15-millisecond window for other types of imaging, including color, near infrared, etc., such that simultaneous scans may be performed, and this may be repeated during the scan period. In some examples, a near-infrared scan may be performed over 5 milliseconds within that 15-millisecond window. Sampling may advantageously be shorter (e.g., less than 20 milliseconds, less than 15 milliseconds, less than 12 milliseconds, less than 10 milliseconds, less than 7 milliseconds, less than 5 milliseconds, etc.) as this can reduce image artifacts thereby. However, shortening the scan time may require more energy, e.g., more power / current may be required for the illumination source. Imaging data may be collected much longer. Alternatively, the scan (e.g., surface scan, near-infrared scan, color scan, etc.) may be performed over a longer period or a shorter period and / or simultaneously (e.g., a laser surface scan and a near-infrared scan may be performed simultaneously using separate emitters / detectors). In this way, for example, by performing a surface scan and a penetration scan, or any other different types of scans simultaneously or switching between them in a short time (within 200 milliseconds, within 150 milliseconds, within 100 milliseconds, within 50 milliseconds, etc.), it is possible to make adjustments between the surface (e.g., 3D) type structure and the internal structure as described above.

[0156] Imaging the internal structure using the scattering coefficient Further, this specification describes a method and apparatus for generating an image of the internal structure of a tooth or other semi-transparent and highly scattering object based on a plurality of penetration images (also referred to herein as "penetration images") that have passed through the object and for which the position of the camera (relative to the object) is given. Accordingly, these methods and apparatuses are capable of generating an image of the internal structure (including a three-dimensional model) without requiring a model of the external surface.

[0157] For example, this specification describes a method and apparatus (including a computer-readable medium) for reconstructing the volumetric structure of an object that includes semi-transparent and highly scattering regions, such as a tooth. More specifically, these apparatuses (e.g., systems) and methods are capable of providing a technique for reconstructing the internal structure of an object, such as dentin within a tooth.

[0158] In general, imaging of an object that is semi-transparent and highly scattering with respect to a particular wavelength can be performed by the methods (and use of any apparatus) described herein. If the position and orientation of the camera relative to the object are known, the internal structure of the object can be reconstructed with a low computational complexity that is proportional to the volume and number of images to be reconstructed.

[0159] An intraoral scanner for acquiring an image that penetrates an intraoral region of a subject (e.g., one or more teeth, gums, jaws, etc.) described in this specification can be used as long as it provides information regarding the relative position of the scanner (e.g., a camera of the scanner that acquires the image). For example, referring back to FIGS. 1A and 1B, FIG. 1A shows an example of an intraoral scanner 101 that may be configured or adapted as described in this specification to generate a 3D model having both a surface shape and an internal shape. As schematically shown in FIG. 1B, an exemplary intraoral scanner may include a wand 103 that can be manually held by an operator (e.g., a dentist, dental hygienist, technician, etc.) and can move over one or more teeth of the subject to scan both the surface structure and the internal structure. The wand may include one or more sensors 105 (e.g., a camera such as a CMOS, CCD, detector, etc.) and one or more light sources 109, 110, 111.

[0160] In FIG. 1B, two separate light sources are shown, namely, a first light source 109 and a second light source 111. The first light source 109 is configured to emit light in a first spectral range for detecting the surface shape (e.g., visible light, monochromatic visible light, etc.), and the second light source 111 is configured to emit light in a second spectral range for detecting the internal shape of the tooth (e.g., by transillumination, small-angle transillumination imaging, laser fluorescence, etc., which may be collectively referred to as transillumination imaging). Although a plurality of illumination light sources are separately shown in FIG. 1B, in some variations, a switchable light source may be used. The light source may be any suitable light source, including LEDs, optical fibers, etc. The wand 103 may include one or more control means (buttons, switches, dials, touch screens, etc.) to assist with control (e.g., turning the wand on and off, etc.). Alternatively or in addition, one or more control means (not shown) may be present in another part of the intraoral scanner, such as a foot pedal, keyboard, console, touch screen, etc.

[0161] Furthermore, the wand 103 may include one or more position and / or orientation sensors 123, which may be, for example, accelerometers, magnetic field sensors, gyroscope sensors, GPS, etc. Alternatively or additionally, the wand may include an optical sensor, a magnetic sensor, or some other, or combinations thereof, for detecting the relative position of the wand (particularly the camera) with respect to the object to be imaged (e.g., one or more teeth). Alternatively or additionally, the apparatus may be capable of detecting the relative position of the wand based on the surface image (e.g., surface scan) and / or discovery scan obtained as described above.

[0162] Generally, any suitable light source may be used, particularly a light source adapted to the mode to be detected. For example, any of these devices may include a visible light source or other light source for surface detection (e.g., at or around 680 nm or other suitable wavelengths), a visible light source for conventional imaging including color imaging (e.g., a white light source), and / or a penetrative light source for penetrative imaging (e.g., an infrared light source and / or a near-infrared light source).

[0163] The relative positions of the light source and the camera are typically known, and one or more penetrative images may be acquired at each position of the wand. The positions of the light source and the camera may include three numerical coordinates in three-dimensional space (e.g., x, y, z) and the pitch, yaw, and roll of the camera.

[0164] The intraoral scanner 101 may include one or more processors, which may include linked processors or remote processors, and control the operation of the wand 103, including scan adjustment, and control the scanning and generation of 3D models, including surface and internal shapes, during review and processing. As shown in FIG. 1B, one or more processors 113 may include, or be coupled to, a memory 115 that stores scan data (such as surface data, internal shape data, etc.). A communication circuit 117 may also be included, including wireless or wired communication circuits for communicating with components of the system (including the wand) or external components (including external processors). For example, the system may be configured to transmit and receive scans or 3D models. One or more additional outputs 119 may be included for outputting or presenting information, such as a display screen, printer, etc. As described above, an input 121 (such as a button, touch screen, etc.) may be included, and the device may enable or require user input for controlling scans and other operations.

[0165] Any of the devices and methods described herein may be used to perform scans for detecting internal structures such as enamel and / or dentin cracks, caries (cavities), damage, etc., and / or identifying such internal structures. Accordingly, any of the devices described herein may be configured to perform scans for detecting internal structures using an incident wavelength or a spectral range of incident wavelengths. Various incident scan techniques (incident imaging) may be used or incorporated into the device, but particularly interesting may be transillumination and small-angle incident imaging. These both detect the path of incident wavelength light that passes through tissue (e.g., through one or more teeth).

[0166] A method and apparatus for visualizing the enamel-dentin region using an infiltration wavelength (e.g., 850 nm, etc.) described in this specification may acquire a plurality of projections or orientations from a single position of a scanner relative to one or more teeth. In particular, three or more orientations or projections may be acquired at each position. By acquiring a plurality of (e.g., three or more) projections, better imaging is possible because a plurality of (e.g., three or more) images that penetrate the teeth can be generated from a specific position of a wand relative to one or more teeth.

[0167] FIG. 12 shows an example of a part of a scanner configured to include infiltration light sources 1202, 1202' (e.g., light sources in an infiltrative spectral range) and a camera that can be used as part of an intraoral scanner wand. In FIG. 12, a camera 1200 is shown, and a pair of LEDs 1202, 1202' are arranged on both sides thereof, and these LEDs emit light in an infiltrative spectral range in substantially the same direction as the camera is facing the target T (e.g., tooth 1201). A single light source 1202 (e.g., an LED) instead of a pair may be used. Generally, according to the present disclosure, the light source of the wand projects in substantially the same direction as the camera, but in some embodiments, the light source may be offset by up to ±15 degrees with respect to the direction of the camera as described above.

[0168] FIG. 13 shows a flowchart 1300 illustrating a method for reconstructing a volumetric structure in a radiation wavelength range from an object having a translucent and highly scattering region. The object having a translucent and highly scattering region may be, for example, a tooth that includes an enamel surface on the outside and a dentin surface on the inside.

[0169] In step 302 of flowchart 1300, the method includes the step of acquiring, with a camera, a plurality of images of an object in a radiation wavelength range, and illumination for the plurality of images is projected substantially from the direction of the camera. In some embodiments, the radiation wavelength range is an infrared wavelength or a near-infrared wavelength. The infrared wavelength or the near-infrared wavelength can be used, for example, for penetration into a translucent object. In one embodiment, the illumination for the plurality of images may be offset up to ±15 degrees with respect to the direction of the camera. The plurality of images may be stored in a computer memory coupled to the camera.

[0170] Any of these methods may also include the step of receiving position data representing the relative position of the camera with respect to the object in each of the plurality of images. Generally, the position data includes the position and orientation of the camera with respect to the object. This position data can be calculated from the plurality of images, or alternatively or additionally, the position and orientation can be measured with sensors 123 on the wand (such as a gyroscope sensor, an accelerometer, a GPS, etc.). Alternatively or additionally, the position and orientation can be calculated by aligning the scanned surface data. In some embodiments, the position data includes three numerical coordinates in three-dimensional space (for example, x, y, and z in a Cartesian coordinate system) and the pitch, yaw, and roll of the camera. The position data may be quantified as a vector metric (for example, a rotation metric and a vector position).

[0171] In step 306 of flowchart 1300, the method further includes the step of generating, for each point in the volume, an upper limit of the scattering coefficient from the plurality of images and the position data. Each of the plurality of images may be a projection from the real world (3D environment) to a 2D plane (image), and depth is lost in this process. Each 3D point corresponding to a specific image point may be constrained to be on the line of sight of the camera. The position of each 3D point in the real world can be detected as the intersection of two or more projection lines through a triangulation process.

[0172] In step 306, the upper limit of the scattering coefficient at each point of the volume representing the object to be scanned is calculated. The upper limit is selected from the plurality of images by triangulating the position of each point using the position data from the camera for each point. The plurality of images generate the luminance of each point, which is the result of the amount of light reflected by the object. Using the luminance of each point, the scattering coefficient of each point is generated. The upper limit of the scattering coefficient of each point may be stored in the memory coupled to the camera.

[0173] The step of generating the upper limit of the scattering coefficient at each point of the volume may include projecting each point of the 3D point cloud grid corresponding to the volume of the object onto each of the plurality of images using a first calibration, generating a list of scattering coefficient values of the projected points, correcting each scattering coefficient value in the list of scattering coefficient values according to the volume response, and storing the minimum scattering coefficient value in the list of scattering coefficient values for each grid point.

[0174] To facilitate projecting each point of the 3D point cloud grid onto each of the plurality of images, several calibrations may be performed. For example, in one embodiment, the first calibration may include a fixed pattern noise calibration for calibrating the sensor problems and image ghosts of the camera. In another embodiment, the first calibration includes a camera calibration that determines the transformation of the camera that projects known points in space onto points in the image. In some embodiments, all of the above calibrations may be performed before projecting the points onto the image.

[0175] When generating the upper limit of the scattering coefficient from the penetration image and the position data, the upper limit of the scattering coefficient may be determined only for points within the outer surface of the object to be imaged. For example, the method described herein may further include receiving surface data representing the outer surface of the object (e.g., scan data representing the outer surface of a tooth or the enamel surface). With the outer surface data, it is possible to generate the scattering coefficient using only the points within this outer surface (e.g., internal points). This may make it possible, for example, to focus on and image only the dentin surface inside the enamel surface of a tooth.

[0176] Finally, each of these methods may include step 308 of generating an image of the object from the upper limit of the scattering coefficient at each point. Examples of generating these images are shown herein, and this example may include steps of forming lines and / or surfaces based on a threshold value of the scattering coefficient or a value based on the scattering coefficient.

[0177] FIG. 14 is a flowchart 400 showing a method for reconstructing a volumetric structure from a tooth. The tooth may be translucent in the radiation wavelength range. In an optional step 402, the method includes receiving, in a processor, a representation of the surface of the tooth in a first coordinate system. The representation of the surface of the tooth may be, for example, a 3D model of the tooth generated by scanning the tooth or by taking a mold of the tooth.

[0178] The method may further include step 404 of receiving, in a processor, a plurality of images of the tooth in the radiation wavelength range, the plurality of images being acquired by illumination projected substantially from the direction of the camera. In some embodiments, the wavelength is a penetration wavelength in the infrared or near-infrared region, or a range of penetration wavelengths in the infrared or near-infrared region. Infrared (IR) or near-infrared wavelengths can be used, for example, for penetration into the tooth. The illumination for the plurality of images may be offset up to ±15 degrees with respect to the direction of the camera. The plurality of images may be stored in a computer memory coupled to the camera.

[0179] In step 406, the method further includes receiving, at the processor, position data representing the position of the camera in each of the plurality of images. Generally, the position data includes the position and orientation of the camera relative to the object. This position data can be calculated from the plurality of images, or alternatively, the position and orientation can be measured by sensors on the camera (e.g., gyroscope sensors, accelerometers, GPS, etc.). Alternatively or additionally, the position and orientation can be calculated by aligning the scanned surface data. In some embodiments, the position data includes three numerical coordinates in three-dimensional space (e.g., x, y, and z in a Cartesian coordinate system) and the pitch, yaw, and roll of the camera. The position data may be quantified as a vector metric (e.g., a rotation metric and a vector position).

[0180] The method may further include step 408 of projecting each point of a point cloud grid corresponding to a volume within the surface of the tooth onto each of the plurality of images using a first calibration. The generated point cloud grid may be inside the outer surface of the tooth. The grid may be located, for example, on a cubic grid. Each grid point may be projected onto each of the plurality of images using the calibration. Several calibrations may be performed to facilitate projecting each point of the grid onto each of the plurality of images. For example, the calibration may include a fixed pattern noise calibration for calibrating sensor problems of the camera and image ghosts. In another embodiment, the calibration may include a camera calibration that determines the transformation of the camera that projects known points in space onto points in the image. In some embodiments, all of the above calibrations may be performed before projecting the points onto the image.

[0181] The method may further include step 410 of generating a list of luminance values for each of the projected points. The plurality of images generate the luminance of each point, which is the result of the amount of light reflected by the object. The luminance value of each point may be stored.

[0182] In step 412, the method may further include the step of converting each luminance value on the list of luminance values into a scattering coefficient according to the volume response. This step may be performed to calibrate the luminance value of each pixel. In this process, for each point based on the position of the camera, the scattering coefficient that would result in such a luminance value is calculated. The result is the scattering coefficient that normalizes the luminance according to the volume response.

[0183] Finally, in FIG. 14, the method may further include step 414 of storing the minimum scattering coefficient at each point in a minimum scattering coefficient list. The method may further include the step of generating an image from the list of minimum scattering coefficients for each point.

[0184] As described above, the method and technique may include multiple calibrations for projecting points from the real world onto its multiple images. One such calibration is the pixel fixed pattern noise calibration (PRNU), which addresses sensor problems and system ghosts that are independent of the object being scanned. FIGS. 15A - E show an example of pixel fixed pattern noise calibration, which gives a constant response in the case of a homogeneous and flat target. FIG. 15A shows the original image of the flat and homogeneous target, which contains two particles 1501, 1502 in the middle of the image. FIG. 15B shows the intermediate image after moving the target parallel to its plane. As a result, the two particles "disappear" from the image. FIG. 15C shows the image after applying a bias coefficient value to each pixel, and strong electronic noise is generated in the image due to this application. In FIG. 15D, a slope is applied to each pixel, and as a result, a smooth pattern is given by the optical system. Finally, FIG. 15E shows the final image after response equalization.

[0185] Another applicable calibration is what is called camera calibration, which enables projecting points in the real world (3D) onto 2D image pixels. Camera calibration determines the transformation of the camera that projects known points in space onto points in the image.

[0186] Volumetric response calibration may also be applied, which gives the scattering coefficient for all points in the world that are given luminance in the images within the camera's field of view. This calibration gives a standard scattering coefficient for a given response anywhere within the field of view.

[0187] Finally, calibration from the scan to the world camera may be applied, which is a rigid body transformation that converts from the scan coordinate system (of the 3D scan of the object) to the camera calibration coordinate system (of the 2D image of the object).

[0188] Other techniques may be used to calculate the volumetric scattering coefficient from the penetration image and the camera position. For example, depending on the deformation mode, the error backpropagation method may be used. The error backpropagation method may include estimating (e.g., tracing) the light rays that pass through the tooth volume and enter the camera. The actual luminance reaching the sensor for each light ray may be obtained from the penetration image and the position and orientation of the camera. For each light ray, the attenuation of the luminance due to scattering within the volume through which the light ray passes may be estimated. For example, the transmission of light through a substance with strong scattering and weak absorption may be modeled by a hybrid calculation method of scattering by the Monte Carlo method in order to obtain the temporal change of the transmittance of the light passing through the substance. A series of projection data may be estimated by temporally extrapolating the difference in optical density between the absorptive object and the non-absorptive reference to the shortest flight time. Therefore, this technique can give the difference in absorption coefficient. See, for example, Yamada et al., "Simulation of fan-beam-type optical computed-tomography imaging of strongly scattering and weakly absorbing media," Appl. Opt. 32, pp. 4808-4814 (1993). Then, the volumetric scattering may be estimated by obtaining the actual luminance reaching the sensor.

[0189] Any of the methods described herein may be implemented by an apparatus that includes a data processing system (or subsystem), which may include hardware, software, and / or firmware for performing many of the above-described steps, including, for example, as part of the processor of an intraoral scanner (see, e.g., FIG. 1B). For example, FIG. 16 is a simplified block diagram of a data processing subsystem 500. The data processing system 500 typically includes at least one processor 502 that communicates with several peripheral devices via a bus subsystem 504. These peripheral devices may typically include a storage subsystem 506 (memory subsystem 508 and file storage subsystem 514), a series of user interface input / output devices 518, an interface 516 to an external network such as the public switched telephone network, etc. This interface is schematically shown as a "modem and network interface" block 516 and is coupled to a corresponding interface device in another data processing system via a communication network interface 524. The data processing system 500 may be a terminal or a low-end personal computer, or a high-end personal computer, a workstation, or a mainframe, etc.

[0190] The user interface input device may be a keyboard or the like, or further may be a pointing device, a scanner, or the like. The pointing device may be an indirect pointing device such as a mouse, a trackball, a touchpad, a graphics tablet, etc., or may be a direct pointing device such as a touch screen incorporated in the display. Other types of user interface input devices (e.g., a voice recognition system) may also be used.

[0191] The user interface output device may be a printer, a display subsystem, etc. The display subsystem includes a display controller and a display device coupled to this controller. The display device may be a cathode ray tube (CRT), a flat panel device (e.g., a liquid crystal display (LCD)), or a projection device. The display subsystem may provide a non-visual display such as an audio output.

[0192] The memory subsystem 506 can hold the basic programming constructs and data constructs that provide the functionality of the present invention. The methods described herein may be configured as software, firmware, and / or hardware, and those (configured as software / firmware) may be stored in the memory subsystem 506. The memory subsystem 506 typically includes a memory subsystem 508 and a file storage subsystem 514.

[0193] The memory subsystem 508 typically includes several memories, including a main random access memory (RAM) 510 for storing instructions and data during program execution, and a read-only memory (ROM) 512 in which fixed instructions are stored. In the case of a Macintosh-compatible personal computer, the ROM includes part of the operating system, and in the case of an IBM-compatible personal computer, the ROM includes the BIOS (basic input / output system).

[0194] The file storage subsystem 514 is capable of performing persistent (non-volatile) storage of program files and data files, and may include at least one hard disk drive and at least one floppy disk drive (and associated removable media). There may also be other devices such as CD-ROM drives or optical drives (and all associated removable media). Further, the system may include a drive of the type having a removable media cartridge. One or more of these drives may be located remotely, for example, on a server on a local area network or on a World Wide Web site on the Internet.

[0195] In this context, the term "bus subsystem" may be used as a general term encompassing any mechanism that enables various components and subsystems to communicate with each other as intended. With the exception of input devices and displays, other components need not be in the same physical location. Thus, for example, the various parts of the file storage system may be connected via various local area network media or wide area network media including telephone lines. Similarly, the input device and the display need not be in the same location as the processor, although the present invention is expected to be implemented in the context of PCs and workstations in most cases.

[0196] The bus subsystem 504 is schematically shown as a single bus, but a typical system has several buses, for example, a local bus and one or more expansion buses (e.g., ADB, SCSI, ISA, EISA, MCA, NuBus, or PCI), as well as serial ports and parallel ports. Network connections are typically established through devices such as network adapters on one of these expansion buses or modems on serial ports. The client computer may be a desktop system or a portable system.

[0197] Scanner 520 may correspond to a wand and other components that acquire from a patient or an orthodontist a scan of the patient's tooth form and pass the scanned digital dataset information to data processing system 500 for further processing. In a distributed environment, scanner 520 may be located remotely and transmit the scanned digital dataset information to data processing system 500 via network interface 524.

[0198] Various alternative, modified, and equivalent forms may be used in place of the components described above. Further, the techniques described herein may be implemented in hardware or software, or a combination of both. These techniques may be implemented as a computer program executed by a programmable computer, such a computer including a processor, a processor-readable storage medium (such as volatile and non-volatile memory and / or storage elements), and appropriate input / output devices. Program code is applied to data input by the input device to perform the described functions and generate output information. The output information is applied to one or more output devices. Each program may be implemented in a procedural or object-oriented high-level programming language to operate in combination with a computer system. However, the programs may be implemented in assembly language or machine language, if desired. In any case, the language may be a compiler language or an interpreter language. Each such computer program may be stored on a general-purpose or special-purpose programmable computer-readable storage medium or storage device (such as a CD-ROM, hard disk, or magnetic diskette) for configuring and operating a computer when the computer reads the storage medium or storage device to perform the described procedures. The system may also be implemented as a computer-readable storage medium configured by a computer program, and such a configured storage medium causes a computer to operate in a specific and predetermined manner.

[0199] Figures 26A-26C and 27A-27G illustrate steps that may form part of a method of forming a 3D volumetric model of a patient's teeth and that are usable in one or more treatments using the methods and apparatus described above. In any of these methods, the teeth of a patient are scanned (e.g., an image and scan of the jaw including the teeth) using an intraoral scanner 2801 capable of measuring both surface and internal structures (including color, e.g., R-G-B color, in some variations). The apparatus is capable of scanning in a variety of modalities including surface wavelengths (non-penetrating or substantially non-penetrating, e.g., visible light, white light) and penetrating wavelengths (e.g., near infrared / infrared). Scanning typically involves scanning from a plurality of positions around the oral cavity and stitching together the resulting images to generate a three-dimensional model of the teeth, which is done, for example, by resolving the relative positions of the scans with respect to the jaw (Figure 26C). The surface scan may be used, as shown in Figure 26C, to construct a model (e.g., a 3D digital model and / or rendering) of the outer surface of the jaw / tooth 2803.

[0200] In any of these methods and apparatuses described herein, the internal structure of the tooth may be formed or modeled to form a volumetric model of the tooth that includes the internal structure extracted by an infiltration scan (e.g., near-infrared scan and / or infrared scan) as shown in FIGS. 27A-27G. FIGS. 26A-27G show one method of reconstructing the internal structure using the scattering coefficient (alternatively or additionally, another method may be used). In FIG. 27A, a grid of points representing the internal volume of the jaw / tooth is constructed. All grid points are projected onto the acquired infiltration (e.g., near-infrared) image, and for each grid point, all pixel positions can be saved as shown in FIG. 27B. For each pixel position and each grid position, the apparatus can calculate the scattering coefficient that results in the observed gray level of the pixel, as graphically shown in FIG. 27C. In these drawings (e.g., FIG. 27C), the eye may be considered to represent the field of view angle of the sensor (e.g., camera). The apparatus can acquire the minimum scattering coefficient calculated for each grid point (FIG. 27D). Then, the point cloud grid, along with the corresponding minimum scattering coefficient, can provide a sampleable volume 2909 at the grid points based on a threshold or correlation (e.g., an isosurface) of the minimum scattering coefficient, as shown in FIG. 27E. FIG. 27G shows an isosurface 2911 generated by identifying a constant value of the sampled density function. FIG. 27F is an enlarged view of the same region of the tooth and shows both the isosurface of FIG. 27G and a (partially transparent) ghost image 2915 of the enamel around the isosurface. This isosurface can represent the dentin and the tooth caries extending from the external surface of the tooth (as described below) towards the dentin.

[0201] In the example shown in FIG. 27F, the isosurface indicates the boundary 2911 between dentin and enamel that is visible under the enamel 2915. The example of FIG. 27F further shows dental caries in the region 2913 surrounded by a circle. In this example, the dental caries appears (similarly to dentin) as an isosurface within or surrounded by enamel. The dental caries can be identified by extending from the internal dentin region to the outer surface of the tooth. Since the methods and apparatuses described herein can accurately reconstruct both the outer surface and the internal structure, it is possible to identify dental caries from this characteristic form (showing an arm or extension extending from the outer surface through the infrared / near-infrared transmissive enamel). In FIG. 27F, a similar region of dental caries is surrounded by a circle 2913, which indicates an extension or bridge between two teeth, and in this region, the surface scan shows that those teeth are actually separate. Thus, by combining the surface scan with an internal scan (e.g., from an infrared / near-infrared image), it may be possible to correct internal data for errors that may occur, such as due to limited field of view. Both the apparatuses and methods described herein may be configured to automatically or semi-automatically identify these regions or irregularities corresponding to dental caries and the like. They may be highlighted in a dental model, image, or representation, and / or flags, alerts, or other notifications may be presented, transmitted, and / or stored along with the estimated location. Alternatively or additionally, the threshold used to determine the isosurface may be selected to distinguish one or more internal shapes such as dentin, caries, fillings, cracks, etc.

[0202] Alternatively or additionally, the apparatus can automatically (or semi-automatically) identify and distinguish the internal structure of teeth based on the shape of the isosurfaces and / or their relative positions within those teeth. As described above, caries may have a density (e.g., scattering coefficient) equivalent to that of dentin. However, caries can be distinguished from dentin by its morphology. The apparatus can detect an "arm" or appendage that extends from the outer surface of enamel while being a substance with a density (e.g., scattering coefficient) equivalent to that of dentin. Since the outer surface of the tooth can be well characterized together with the internal structure, it is possible to identify the spread of caries by mapping the outer surface of the isodensity map of the region extending from the outer surface to a larger, well-defined internal dentin pattern. The boundary between the internal spread of dentin and caries can be identified by approximating a continuous surface of dentin that includes the region around the "projected" region and / or by examining the rate of change of the direction of the dentin surface. Other internal structures (e.g., fillings, cracks, etc.) can be distinguished based on their scattering coefficient value ranges and / or based on their positions or morphologies. The apparatus can display them in color-coding, annotation, etc.

[0203] Accordingly, in any of these methods and apparatuses, the scanner can examine the interior of the enamel and reconstruct the margin line. Additionally, it may be possible to use a further wavelength (e.g., green light), or even another, tissue-penetrating radiation modality (e.g., ultrasound) imaging, which enables the construction of the margin line and even the root of the tooth and / or helps to distinguish structures such as differentiating dental caries from dentin or other internal structures.

[0204] The resulting volumetric 3D model of the tooth may be used to reconstruct the tooth based on histological teeth. As described above, the volumetric model may be used to create dental prostheses (such as implants) that look more realistic and / or feel more comfortable.

[0205] Furthermore, the methods and apparatuses described herein enable a user (e.g., a dentist, physician, dental technician, etc.) to track the time course of a tooth, including tracking dentin, caries, etc. by comparing models acquired over time, as well as tracking the overall health of the tooth. For example, it is possible to construct a time-lapse video (image). FIG. 28A shows an example of an initially acquired volumetric reconstruction, showing dentin 3001 (filled) and enamel 3003 (slightly made transparent). FIG. 28B shows another example of a volumetric model of a tooth showing dentin 3001 and enamel 3003.

[0206] The volumetric model may include width information and may also show an estimate of wear over time. For example, it is readily possible to track changes in the width of enamel over time and across various regions of the tooth. Knowing the width of the enamel makes it possible to estimate tooth wear and provide a snapshot of the severity of the wear.

[0207] Classification and categorization Any suitable method and apparatus (e.g., system, device, software, etc.) may be used to generate an image of the internal structure of a tooth (or other translucent and highly scattering object). For example, as an alternative or addition to the use of the scattering coefficient described above, any of the apparatuses and methods described herein use a two-dimensional penetration image together with information on the relative position and / or orientation of an intraoral scanner with respect to the object to be imaged (e.g., a tooth) to classify the two-dimensional penetration image and form a three-dimensional model of the tooth including one or more internal structures of the object. The penetration image may mean an image that reveals the internal structure of an object (e.g., a tooth) acquired by near-infrared and / or infrared wavelengths. The position and / or orientation of the scanner may be instead of the position and / or orientation of a camera located on the scanner (e.g., on a handheld wand) that acquires an image.

[0208] The apparatuses and methods described herein are capable of constructing a three-dimensional (3D) volumetric model of teeth from segmented two-dimensional (2D) images. These methods and apparatuses are also capable of segmenting the 3D model of the teeth.

[0209] Generally, the methods and apparatuses described herein enable directly segmenting the penetration image. This may enable identification and localization of dentin within the tooth, including the position and morphology of the dentin, as well as identification and localization of cracks, damage, and / or caries within the tooth, including within the dentin. By performing the segmentation, it may be possible to reconstruct a volumetric model based on the penetration image and information on the camera position corresponding to the penetration image. The volumetric model of the tooth may be segmented, and these segments (relating to various internal structures of the tooth) may be projected onto a surface image and / or better segmented within the internal structure of the tooth by back-projecting these segments into the image and / or combining them with a surface model of the tooth (e.g., the outer surface of the tooth).

[0210] Accordingly, the penetration image of the tooth obtained by the penetration wavelength (e.g., near-infrared wavelength and / or infrared wavelength) may include the internal tooth structure and / or 3D data. These images can be obtained using any of the dental scanners described herein, and the tooth volume may be divided into different regions according to the opacity, color, and other characteristics of the images and 3D data. These regions may be, for example, healthy enamel, dentin, damage, dental fillings, etc. The division may be performed on a 2D image or a volumetric model. The division may be performed to classify the image and / or 3D model according to the presence of various divisions. The user may be able to manually or automatically (or semi-automatically) detect and classify various internal structures, such as dental caries, enamel erosion, and other dental problems, by this division. Further, it is possible to measure the internal regions of one or more tooth divisions using these images or models to perform better dental treatment, including tooth positioning or other treatment planning. For example, the user may be able to plan an accurate filling that minimizes enamel extraction by accurately identifying the location of tooth damage. Accordingly, by performing the division described herein, it may be possible to capture the internal tooth structure without performing the ionizing radiation currently used by X-rays. Dental problems may be presented on a 3D volumetric model. Further, as will be detailed later, the division and classification of the internal structure can be automated. Finally, it is possible to perform accurate measurements of the internal structure for better treatment planning.

[0211] Figures 17A and 17B show an example of the data flow when teeth are scanned with an intraoral scanner to identify the internal structure. The method shown in Figures 17A and 17B includes three parts. First, the teeth may be scanned with an intraoral scanner 1701 (or any other scanner), which is configured to perform an infiltration scan using optical (e.g., infrared, near-infrared, etc.) wavelengths or an optical wavelength range on the interior of the teeth. Any of these scanners can also simultaneously perform scans to identify surface shape (e.g., by one or more non-infiltration wavelengths), color, etc., as described above. During the scan, a plurality of infiltration scans 1703, 1703' are performed, and for each infiltration image, camera positions 1705, 1705' (e.g., x, y, z positions and / or pitch, roll, yaw angles) may be identified and / or recorded. In some variations, as described above, imaging of the tooth surface may also be performed simultaneously, and a 3D surface model 1707 of the tooth may also be determined simultaneously. In this example, the scan of the patient's teeth may be performed by an intraoral 3D scanner 1702 capable of imaging the internal tooth structure (e.g., by near-infrared imaging). The position and orientation of the camera can be determined in part from the 3D scan data and / or the 3D tooth surface model 1707.

[0212] Thereafter, the penetration image may be segmented (1711). In this example, the segmentation may be performed in either of two ways. On the internal tooth structure image, the image may be segmented by contour finding 1713, 1713’. To further automate this process, a machine learning method may be applied. Alternatively or additionally, to accurately locate segments such as enamel, a proximity image (where the camera position is close) may be used to determine the proximity shape and further back-project the shape from the 3D model onto the image. The method may further include back-projecting the pixels of the internal tooth image onto the tooth to calculate a density map of the internal tooth reflectance coefficient. Finding or estimating the bounding surfaces of the various segments may be possible by using the isosurfaces or thresholds of the density map and / or by a machine learning method. Further, by segmenting the images and back-projecting those segments onto a model (e.g., a 3D surface model (e.g., back-projected into the world)), it is possible to find the segments from the intersections of the projections of the segments with the tooth surface.

[0213] The results may be displayed (1717), transmitted, and / or stored. For example, the results may be displayed by the scan system during an intraoral scan procedure. The results may be shown in an image by the bounding contours of the various segments, the 3D density map, etc. In the example shown in FIG. 17, a density map 1715 is shown, which represents dentin under the enamel of the outer surface. This image may be color-coded to show the different segments. In this example, the internal segments (structures) are shown within the 3D surface model (shown transparently), but only a part is shown as not all teeth are scanned with the penetration image. Alternative views, cross-sections, slices, projections, etc. may be provided. The image of the example of FIG. 17 includes artifacts 1716 present outside the tooth, and these may be removed or trimmed based on the surface model 1718.

[0214] Each pixel of the image may be marked by category. Internal structures such as dentin, enamel, cracks, damage, etc. can be automatically identified by category and can be identified manually or automatically (e.g., based on machine learning such as 3D structure, etc.). Each category may be displayed separately or together (e.g., in different colors, shades, etc.), regardless of the presence or absence of a surface model (e.g., 3D surface model).

[0215] Therefore, in FIG. 17, the patient is first scanned with a 3D scanner capable of both surface scanning and penetration scanning (e.g., near-infrared imaging), and the orientation and / or position of the camera are recognized (based on the position and / or orientation of the wand and / or surface scan). This position and orientation may be relative to the tooth surface. Thus, the method and apparatus may have an estimation of the camera position (where the camera is located. For example, the x, y, z positions of the camera, and its rotational position).

[0216] Generally, the penetration image (e.g., near-infrared image or infrared image) may be automatically segmented. FIGS. 18A-18C show a first example of automatic segmentation of a near-infrared image. FIG. 18A shows a first automatic segmentation of the outer surface of a tooth (identified, e.g., by edge detection). In FIG. 18A, the outer peripheral edge 1803 is shown. In this example, only the first level of edge detection for exploring the outer periphery is performed. In FIGS. 18B and 18C, a continuous edge region 1805 is shown, which is derived by edge detection and mapped to the near-infrared image (original image). FIGS. 19A-19C show the identification and mapping of other edges in the same image. FIG. 19A shows only the edges detected using a threshold setting value from the near-infrared image (e.g., FIG. 19C). In FIG. 19B, five (overlapping (1905)) segments 0-4 are traced from the detected edges by forming a continuous line. The separate segments are shown color-coded, and a color-coding legend for identifying the segments is shown on the right side. The present apparatus is capable of automatically segmenting an image from a near-infrared image. In FIGS. 18A-18C and FIGS. 19A-19C, markings are made in different segments, respectively, which may correspond to different regions (or different internal structures) on the image. When multiple images are analyzed, these estimated segments may be back-projected onto a 3D model and / or shown within the image. FIGS. 20A-20C and FIGS. 21A-21C show another example of a near-infrared image from the same patient as shown in FIGS. 18A-19C, showing segmentation based on edge detection and identification of an estimated continuous line region from the detected edges. In FIGS. 21A-21C, another region of the same patient's tooth is shown, and in this image, eight segments (0-7) are identified as shown in FIG. 21B. FIG. 21A shows the edge detection from the original image shown in FIG. 21C. FIGS. 22A-22C show the segmentation of another region of the patient's tooth. FIG. 22A shows the edges detected from the original near-infrared image. FIGS. 22B and 22C show the eight segments (0-7) identified on the near-infrared image. Similarly, FIGS. 23A-23C show the segmentation of another region of the patient's tooth, FIG. 23A shows the edge detection, FIG. 23B shows the segments identified from these edges, and FIG. 23C shows the original near-infrared image.

[0217] The sectional images as shown in FIGS. 18A to 23C may be used to form a model of the internal structure of a scanned object (e.g., a tooth). A surface 3D model may also be used. For example, FIGS. 24A to 24B show a three-dimensional model of the tooth region of a patient formed by sectional images including those shown in FIGS. 18A to 23C. In FIG. 24A, the 3D reconstruction includes the outer surface of the tooth (shown partially transparent), and various internal sections may be shown with different colors and / or transparencies. For example, in FIG. 24A, dentin (the inner part of the tooth) 2404 is shown within the boundary of tooth 2405. In FIG. 24A, the section showing dentin is a plane (a volume in FIG. 24B), but may be shown as a density map as shown later in FIGS. 25A and 25B. The resulting 3D volume including the sectional images may be repeatedly used to acquire an image that penetrates the obtained volume, which may be a "projection" that allows for a direct comparison with the original near-infrared image, and this comparison may be made to modify the model. This process may be repeated (iterated) for model improvement, thereby making it possible to better segment the image.

[0218] As described above, the segmentation may include edge detection. Any suitable edge detection method, including machine learning, may be used. The segmentation of the plurality of near-infrared images may be used in combination with the position information of the camera to reconstruct the volume. Since a plurality of different cross-sections (a plurality of different cones) are known and segmented, the segmentation inside all the projections of those cones, obtained as a result from various positions, is recognized, and thus, the intersection of these segmentations can be specified. This process can be performed more easily by using the outer surface boundary of the tooth, which can be provided by surface imaging and / or a 3D model. As described above, this process may be iterative, and in this method, a simulated penetration (e.g., near-infrared) image may be projected using 3D data, and by comparing this with the original image, the segmentation can be improved to derive the next evolved internal structure model. Similarly, the segmentation or segmentation area 2407 outside the outer surface of the tooth may be removed.

[0219] As described above, the model of the tooth including the internal structure may be displayed in various ways. FIG. 24B shows a cross-section of a tooth and shows the internal structure including dentin 2404 and the thickness of enamel between the outer surface 2405 and dentin 2404.

[0220] FIGS. 25A and 25B show the reconstruction of a tooth including the internal structure (already shown in FIGS. 17A and 17B as well). In this example, the internal structure is shown by density mapping (e.g., segmentation). For example, dentin 2505 is shown in detail within a portion 2503 of the surface model of FIG. 25B. The outer surface of the tooth can also be identified as a segmentation (as shown in FIGS. 25A and 25B), and in this example, the segmented outer surface and the outer surface identified by surface imaging are almost completely coincident.

[0221] Sleeve for an intraoral scanner with transillumination Any of the devices described herein may include one or more sleeves configured to protect an intraoral scanner wand, but may be configured to extend its functionality and / or adapt the scanner for use at an infiltration wavelength, such as transillumination. The sleeves shown in FIGS. 29A - 31B may be used as a barrier (e.g., a hygiene barrier) to prevent contamination of the wand portion of the intraoral scanner, since the scanner may be used on various patients and may also be used as an adapter to provide transillumination by infrared / near - infrared wavelength imaging. The sleeves in these drawings are configured as transillumination sleeves having electrical couplings. For example, the sleeves described herein may include both illumination at an infiltration wavelength (e.g., near - infrared and / or infrared LEDs) and one or more sensors (e.g., CCDs), or may use the same camera already on the wand.

[0222] In FIG. 29A, the wand of the intraoral scanner is shown together with a sleeve 3101 disposed around the end of the wand 3105. The sleeve is shown semi - transparent so that the internal structure (connector) is visible. In FIG. 29B, only the sleeve 3105 for the intraoral scanner (wand) is shown in solid. Generally, the sleeve 3105 is slid onto and covers the end of the wand, such that the light source and camera (sensor) already on the wand are visible through the sleeve, and electrical contacts 3123 that can provide control, power, and / or data transmission to the LED and / or sensor 3125 are integrated within or on the sleeve. The sleeve includes a pair of wing regions 3103 on opposing sides, and when the sleeve is placed over the wand, the wing regions 3103 face each other and extend from the distal end of the wand.

[0223] The sleeve 3101 may be held at the end of the wand by a friction or connection mechanism (not shown). Thus, the sleeve can be easily removed from the wand, and a new sleeve can be put on the wand each time the scanner is used on another patient. In this example, the sleeve may be configured to transmit infrared light (e.g., near-infrared), and thus may include one or more protrusions 3103 (such as for transillumination, etc.) as shown in FIG. 29B. Electrical contacts and connectors integrated with the sleeve can make the scanner compatible with infrared / near-infrared transillumination.

[0224] Accordingly, the sleeve may include a circuit (e.g., a flexible circuit) connected to an LED illumination (infrared / near-infrared) source and / or one or more sensors, particularly for transillumination. For example, FIGS. 30A - 30C. FIG. 30A shows an example 3201 of the frame of the sleeve, which may be rigid or semi-rigid. The frame can support a flexible circuit 3203 and / or a connector 3205 (shown in FIG. 30B) and can also perform shielding (e.g., light shielding). A flexible outer sleeve 3207 can be put on these frames and circuits as shown in FIG. 30C.

[0225] The sleeve may be assembled by injection molding each component part including the overall sleeve, a window for illumination and image capture, a connector for the circuit, and one or more LED holding areas (e.g., by injecting a material that forms a window passing through the sleeve and transmits infrared and visible light, and then injecting a rigid sleeve material). Next, a flexible circuit positioning device may be used to position the flexible circuit, and an LED encapsulation mechanism may be arranged. Then, a flexible outer sleeve may be injected.

[0226] Figures 31A - 31C are diagrams showing the flexible circuit 3301, the connector 3303, and the LED holder / shield 3305 in more detail. Figures 32A - 32B show examples of the LED positioner and light shield portions at the distal end of the sleeve. The example shown in Figure 32A includes a support frame or arm 3404, which extends downward and includes a light shroud or light shield area 3406 that seals a portion of the LED. Exemplary dimensions are illustrated.

[0227] As used herein, when a feature or element is referred to as being "on" another feature or element, that feature or element may be in direct contact with the other feature or element, or intervening features and / or elements may be present. In contrast, when a feature or element is referred to as being "directly on" another feature or element, no intervening features and / or elements are present. Also, of course, when a feature or element is referred to as being "connected", "attached", or "coupled" to another feature or element, that feature or element may be directly connected, attached, or coupled to the other feature or element, or intervening features and / or elements may be present. In contrast, when a feature or element is referred to as being "directly connected", "directly attached", or "directly coupled" to another feature or element, no intervening features and / or elements are present. Such described or illustrated features and elements are described or illustrated with respect to one embodiment, but may also apply to other embodiments. Also, as will be understood by those skilled in the art, when a structure or feature is referred to as being "adjacent" to another feature, that reference may include the structure or feature partially overlapping the adjacent feature or having a portion underlying the adjacent feature.

[0228] The terms used in this specification are for the sole purpose of describing particular embodiments and are not intended to limit the disclosure. For example, as used in this specification, the singular forms "a", "an", and "the" include the plural forms as well, unless the context clearly dictates otherwise. Further, as a matter of course, the terms "comprises" and / or "comprising", when used in this specification, specify the presence of the stated features, steps, operations, elements, and / or components, and do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. In this specification, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0229] Spatially relative terms such as "under", "below", "lower", "over", "upper", etc. may be used in this specification for ease of explanation to describe the relationship of one element or feature to another as shown in the drawings. Of course, such spatially relative terms are intended to encompass orientations in addition to the orientation depicted in the drawings for the device during use or operation. For example, if the device in the drawing is inverted, an element described as "under" or "beneath" another element or feature will be oriented "over" that other element or feature. Thus, for example, the term "under" may encompass both the "over" and "under" orientations. The device may be oriented in other directions (rotated 90 degrees or otherwise), and accordingly, the spatially relative descriptors used in this specification may be interpreted accordingly. Similarly, terms such as "upwardly", "downwardly", "vertical", "horizontal", etc. are used in this specification for purposes of explanation only, unless otherwise specified.

[0230] The terms "first" and "second" may be used herein to describe various features / elements (including steps), but these features / elements should not be limited by these terms, except where the context is inconsistent. These terms may be used to distinguish one feature / element from another. Thus, unless departing from the teachings of the present invention, a first feature / element may be referred to as a second feature / element hereinafter, and similarly, a second feature / element may be referred to as a first feature / element hereinafter.

[0231] Throughout this specification and the following claims, unless otherwise specified, after the word "comprise" and its variations such as "comprises" and "comprising", it is meant that various components may be used in association with each other in a method and an article (for example, a configuration and an apparatus including a device and a method). For example, it should be understood that the term "comprising" means the inclusion of all the recited elements or steps, and does not mean the exclusion of any other element or step.

[0232] As used in this specification and the claims, including when used in the examples, and unless otherwise specified, all numerical values may be read as being preceded by the term "about" or "approximately", even if the term does not explicitly appear. The term "about" or "approximately" may be used to indicate that the stated value and / or position, when indicating size and / or position, falls within a reasonable expected range of values and / or positions. For example, a numerical value may be a value within ±0.1% of the stated value (or range of values), within ±1% of the stated value (or range of values), within ±2% of the stated value (or range of values), within ±5% of the stated value (or range of values), within ±10% of the stated value (or range of values), or other such values. Any numerical value given in this specification should be understood to include approximate values before and after that value, unless contextually inconsistent. For example, if the value "10" is disclosed, "about 10" is also disclosed. Any numerical range described in this specification shall be considered to include all sub-ranges subsumed therein. Also, of course, as would be properly understood by those skilled in the art, if a value is disclosed, values "less than" that value, values "greater than" that value, and possible ranges between those values are also disclosed. For example, if the value "X" is disclosed, values "less than X" and values "greater than X" (e.g., if X is a numerical value) are also disclosed. Also, throughout this application, it should be understood that data is provided in several different formats, and that this data represents endpoints and starting points and has ranges over any combination of these data points. For example, if a particular data point "10" and a particular data point "15" are disclosed, not only values between 10 and 15, but also values greater than 10 and 15, values greater than or equal to 10 and 15, values less than 10 and 15, values less than or equal to 10 and 15, and values equal to 10 and 15 are considered to be disclosed. It should also be understood that each individual between two particular singulars is also disclosed.For example, if 10 and 15 are disclosed, then 11, 12, 13, and 14 are also disclosed.

[0233] Although various exemplary embodiments have been described thus far, various changes may be made to the various embodiments without departing from the scope of the invention as set forth by the claims. For example, the order in which the various method steps described are performed may often be changed in alternative embodiments, and in some alternative embodiments, one or more method steps may be skipped altogether. Optional features of the various embodiments of the apparatus and system may or may not be included depending on the embodiment. Accordingly, the above description is primarily for illustrative purposes and should not be construed as limiting the scope of the invention as recited in the claims.

[0234] The examples and specific examples included in this specification show specific embodiments in which the subject matter of the invention can be implemented, as illustrations rather than limitations. As mentioned, other embodiments may be utilized or derived, and structural or logical substitutions or changes may be made without departing from the scope of the present disclosure. Such embodiments of the subject matter of the invention may be referred to individually herein, or collectively by the term "the invention," which is used for convenience only and is not intended to spontaneously limit the scope of this application to any one invention or inventive concept, even if more than one is disclosed. Accordingly, although specific embodiments have been illustrated and described herein, the specific embodiments shown may be replaced by any configuration made to achieve the same purpose. The present disclosure encompasses any adaptation or variation of the various embodiments. Those skilled in the art will, by examining the above description, become aware of combinations of the above-described embodiments and other embodiments not specifically described herein.

Claims

1. 1. A method for generating a three-dimensional (3D) volumetric model of a subject's teeth using an intraoral scanner, comprising: capturing 3D surface model data of at least a portion of the subject's teeth using an intraoral scanner while moving the intraoral scanner over the subject's teeth; acquiring multiple images of the interior of the tooth using near infrared wavelengths with the intraoral scanner while moving the intraoral scanner over the tooth such that multiple images of the same interior area of ​​the tooth are imaged; for each of the plurality of images of the interior of the teeth, determining a position of the intraoral scanner relative to the subject's teeth using the 3D surface model data; forming the 3D volumetric model of the subject's teeth, including internal geometry, using the plurality of images and the position of the intraoral scanner relative to the subject's teeth; The method includes:

2. 1. A method for generating a three-dimensional (3D) volumetric model of a subject's teeth using an intraoral scanner, comprising: capturing 3D surface model data of at least a portion of the subject's teeth using an intraoral scanner while moving the intraoral scanner over the subject's teeth; acquiring a plurality of images of the interior of the tooth using near infrared wavelengths while moving the intraoral scanner over the tooth by emitting near infrared light from the intraoral scanner at a first polarization and detecting the near infrared light returning to the intraoral scanner with an image sensor within the intraoral scanner, wherein the near infrared light returning to the intraoral scanner is filtered to remove specular reflections by filtering the near infrared light of the first polarization from the near infrared light returning to the intraoral scanner before it reaches the image sensor; for each of the plurality of images of the interior of the teeth, identifying a position of the intraoral scanner relative to the subject's teeth at the time each of the plurality of images was captured using the 3D surface model data; forming the 3D volumetric model of the subject's teeth, including internal geometry, using the plurality of images and the position of the intraoral scanner relative to the subject's teeth; The method includes:

3. 2. The method of claim 1, wherein the step of acquiring the images of the interior of the tooth comprises emitting near-infrared light from the intraoral scanner at a first polarization and detecting the near-infrared light returning to the intraoral scanner with an image sensor within the intraoral scanner, wherein the near-infrared light returning to the intraoral scanner is filtered to remove specular reflections, the removal being achieved by filtering near-infrared light of the first polarization from the near-infrared light returning to the intraoral scanner before it reaches the image sensor.

4. 3. The method of claim 2, wherein the near infrared light returning to the intraoral scanner is filtered to remove specular reflections, the removal being achieved by filtering all or substantially all of the near infrared light of the first polarization from the near infrared light returning to the intraoral scanner before it reaches the image sensor.

5. The method of claim 1 or 2, wherein the step of capturing the 3D surface model data comprises identifying the 3D surface topology using confocal focusing.

6. The method of claim 1 or 2, wherein the step of capturing the 3D surface model data includes using confocal scanning, stereoscopic vision.

7. The method of claim 1 or 2, wherein the step of capturing the 3D surface model data includes using structured light triangulation.

8. The method of claim 1 or 2, wherein the step of capturing 3D surface model data comprises capturing a model of the teeth and gums.

9. The method of claim 1 or 2, further comprising capturing a 3D color model of the teeth while moving the intraoral scanner over the teeth.

10. 3. The method of claim 1, further comprising alternating between capturing 3D surface model data and obtaining the images of the interior of the tooth while moving the intraoral scanner over the tooth.

11. 3. The method of claim 1 or 2, wherein the step of acquiring the plurality of images comprises capturing the 3D surface model data and the plurality of images of the interior of the tooth using a same sensor on the intraoral scanner.

12. The method of claim 1 or 2, wherein the step of acquiring a plurality of images of the interior of the tooth includes low angle penetration images.

13. 3. The method of claim 1 or 2, wherein obtaining the plurality of images of the interior of the tooth comprises using a separate sensor on the intraoral scanner to capture 3D surface model data and the plurality of images of the interior of the tooth.

14. The method of claim 1 or 2, wherein the step of forming the 3D volumetric model of the subject's teeth comprises combining the 3D surface model data with a 3D model of the internal structure.

15. The method of claim 1 or 2, further comprising marking the internal structure.

16. 1. An intraoral scanning system for generating a three-dimensional (3D) volumetric model of a subject's teeth, comprising: a handheld wand having at least one image sensor and a plurality of light sources configured to emit light in a first spectral range and a second spectral range, the second spectral range being within a near infrared wavelength range; one or more processors operatively connected to the handheld wand, capturing 3D surface model data of at least a portion of the subject's teeth while moving an intraoral scanner over the teeth; acquiring multiple images of the interior of the tooth using light in the second spectral range while moving the intraoral scanner over the tooth such that multiple images of the same interior area of ​​the tooth are imaged; for each of the plurality of images of the interior of the tooth, determining a position of the handheld wand relative to the subject's tooth using the 3D surface model data; forming the 3D volumetric model of the subject's teeth, including internal geometry, using the plurality of images and the position of the intraoral scanner relative to the subject's teeth; the one or more processors configured to implement A system including:

17. 1. An intraoral scanning system for generating a three-dimensional (3D) volumetric model of a subject's teeth, comprising: a handheld wand having at least one image sensor and a plurality of light sources configured to emit light in a first spectral range and a second spectral range, the second spectral range being within a near infrared wavelength range; a filter in front of the image sensor configured to filter light in the second spectral range and in the first polarization; one or more processors operatively connected to the handheld wand, capturing 3D surface model data of at least a portion of the subject's teeth while moving an intraoral scanner over the teeth; acquiring a plurality of images of the interior of the tooth using light of the second spectrum while moving the intraoral scanner over the tooth by emitting near infrared light from the intraoral scanner at a first polarization and detecting the near infrared light returning to the intraoral scanner with an image sensor within the intraoral scanner, wherein the near infrared light returning to the intraoral scanner is filtered to remove specular reflections by filtering near infrared light of the first polarization from the near infrared light returning to the intraoral scanner before it reaches the image sensor; for each of the plurality of images of the interior of the tooth, determining a position of the handheld wand relative to the subject's tooth using the 3D surface model data; forming the 3D volumetric model of the subject's teeth, including internal geometry, using the plurality of images and the position of the intraoral scanner relative to the subject's teeth; the one or more processors configured to implement A system including:

18. 18. A system according to claim 16 or 17, wherein the handheld wand further comprises at least one secondary image sensor in addition to the image sensor.

19. 18. The system of claim 16 or 17, wherein the one or more processors are configured to identify surface information by using confocal focusing.

20. 18. The system of claim 16 or 17, wherein the one or more processors are configured to identify surface information by using confocal scanning, stereoscopic vision, or structured light triangulation.

21. 18. The system of claim 16 or 17, wherein the handheld wand is configured to cycle between emitting light in the first spectral range to perform the surface scan and emitting light in the second spectral range to detect internal structures by switching between the first and second spectral ranges such that images acquired with the second spectral range share a coordinate system with the surface scan.

22. 18. The system of claim 16 or 17, wherein the image sensor is configured to capture both the 3D surface model data and the internal data of the teeth.

23. 18. The system of claim 16 or 17, wherein the plurality of light sources includes a white light source.

24. 18. The system of claim 16 or 17, further comprising: the plurality of light sources and the image sensor positioned such that light emitted in the second spectral range is reflected off the teeth and received by the one or more sensors at an angle between 0° and 15°.

25. 1. A method of generating a model of a subject's teeth, comprising the steps of: scanning a portion of a subject's teeth with an intraoral scanner using a first modality to capture three-dimensional (3D) surface model data of the teeth; scanning the portion of the subject's teeth with the intraoral scanner using a second modality to image the interior of the teeth using a penetrating wavelength to capture internal data of the teeth; cycling between the first modality and the second modality, the cycling rapidly switching between the first modality and the second modality such that images using the penetrating wavelength share a coordinate system with the 3D surface model data captured with the first modality; The method includes:

26. 1. A method of generating a model of a subject's teeth, comprising the steps of: scanning a portion of a subject's teeth with a handheld intraoral scanner using a first modality to capture three-dimensional (3D) surface model data of the teeth; scanning the portion of the subject's teeth with the handheld intraoral scanner with a second modality that images the interior of the teeth using a penetrating wavelength to capture internal data of the teeth; rotating between the first modality and the second modality using a scanning scheme in which the first modality and the second modality are rapidly switched in a cycling manner such that the internal data uses the same coordinate system as the 3D surface model data acquired with the first modality; adjusting the scanning strategy based on the captured 3D surface model data, the internal data, or both the 3D surface model data and the internal data; The method includes:

27. 27. The method of claim 25 or 26, further comprising combining the 3D surface model data with the internal data of the tooth to form a 3D model of the tooth.

28. 27. The method of claim 25 or 26, further comprising combining the 3D surface model data with the internal data of the tooth to form a 3D model of the tooth and marking internal structures.

29. 27. The method of claim 25 or 26, wherein the step of capturing the 3D surface model data comprises identifying the 3D surface topology using confocal focusing.

30. 27. The method of claim 25 or 26, wherein the step of capturing the 3D surface model data comprises using confocal scanning, stereoscopic vision, or structured light triangulation.

31. 27. The method of claim 25 or 26, further comprising scanning the subject's teeth and gums.

32. 27. The method of claim 25 or 26, further comprising cycling between the first modality, the second modality, and a third modality, the cycling switching being performed rapidly between the first modality, the second modality, and the third modality such that images using the penetrating wavelength share a coordinate system with the 3D surface model captured with the first modality.

33. 27. The method of claim 25 or 26, further comprising scanning the portion of the subject's teeth with the intraoral scanner with a third modality that captures a color image of the subject's teeth.

34. 27. The method of claim 25 or 26, further comprising using a sensor on the intraoral scanner to capture both the 3D surface model data and the internal data of the teeth.

35. 27. The method of claim 25 or 26, wherein the step of scanning the portion of the subject's teeth with the intraoral scanner using the second modality includes illuminating the teeth at an angle of 0° to 15° relative to a sensor receiving the illumination.

36. 27. The method of claim 25 or 26, wherein the step of scanning the portion of the subject's teeth with the intraoral scanner using the second modality comprises acquiring a plurality of trans-illumination or low angle penetration images at a plurality of different angles between an illumination source and a sensor.

37. 27. The method of claim 25 or 26, further comprising capturing the 3D surface model data and the internal data of the teeth using another sensor on the intraoral scanner.

38. 27. The method of claim 25 or 26, further comprising capturing internal data of the teeth using a plurality of sensors on the intraoral scanner.

39. 27. The method of claim 25 or 26, wherein the step of scanning the portion of the subject's teeth with the intraoral scanner using the second modality includes illuminating with one or more of white light trans-illumination, UV / blue fluorescence, and red light fluorescence.

40. 32. The method of claim 30 or 31, wherein the step of scanning the portion of the subject's teeth with the intraoral scanner using the second modality comprises illuminating with infrared light.

41. 26. The method of claim 25, further comprising automatically adjusting the duration spent scanning in the first modality, the duration spent in the second modality, or the duration spent in the first modality and the second modality when cycling between the first modality and the second modality.

42. 26. The method of claim 25, further comprising automatically adjusting the duration spent scanning in the first modality, the duration spent in the second modality, or, when cycling between the first and second modalities, the duration spent in the first and second modalities based on the captured 3D surface model data, the internal data, or both the 3D surface model data and the internal data.

43. 27. The method of claim 26, wherein the step of adjusting the scanning strategy comprises adjusting based on a quality determination of the captured 3D surface model data.

44. 27. The method of claim 26, wherein the step of adjusting the scan strategy comprises automatically adjusting the scan strategy.

45. 27. The method of claim 26, wherein the step of adjusting the scan modality comprises adjusting a duration of a scan in the first modality.

46. 27. The method of claim 26, wherein the step of adjusting the scan modality comprises adjusting a duration of a scan with the second modality.

47. 1. An intraoral scanning system for generating a model of a subject's teeth, comprising: a handheld intraoral wand having at least one sensor and a plurality of light sources configured to emit light in a first spectral range and a second spectral range, and further wherein the second spectral range is penetrative; one or more processors operatively connected to the handheld intraoral wand, the one or more processors configured to cycle the wand between a first mode, where in the first mode the wand emits light in the first spectral range for a first duration and the one or more processors receive three-dimensional (3D) surface data in response, and in the second mode the wand emits light in the second spectral range for a second duration and the one or more processors receive image data in response; A system including:

48. 1. An intraoral scanning system for generating a model of a subject's teeth, comprising: a handheld intraoral wand having at least one sensor and a plurality of light sources configured to emit light in a first spectral range and a second spectral range, and further wherein the second spectral range is penetrative; one or more processors operatively connected to the wand, the one or more processors configured to cycle the wand between a first mode, where in the first mode the wand emits light in the first spectral range for a first duration and the one or more processors receive three-dimensional (3D) surface data in response, and in the second mode, where the wand emits light in the second spectral range for a second duration and the one or more processors receive image data in response; the one or more processors are configured to adjust the first duration and the second duration based on the received 3D surface data, the received image data, or both the 3D surface data and the image data. system.

49. 49. The system of claim 47 or 48, wherein the wand includes a plurality of sensors.

50. 49. The system of claim 47 or 48, wherein the one or more processors are configured to identify surface information from the three-dimensional (3D) surface data using confocal focusing.

51. 49. The system of claim 47 or 48, wherein the one or more processors are configured to responsively identify surface information from the three-dimensional (3D) surface data by using confocal scanning, stereoscopic vision, or structured light triangulation.

52. 49. The system of claim 47 or 48, wherein the first duration and the second duration are set such that the image data acquired in the second spectral range shares a coordinate system with the 3D surface data acquired in the first spectral range.

53. 49. The system of claim 47 or 48, wherein the first duration and the second duration are less than 100 milliseconds, whereby the image data acquired in the second spectral range shares a coordinate system with the 3D surface data acquired in the first spectral range.

54. 49. The system of claim 47 or 48, wherein the at least one sensor is configured to capture both the 3D surface model data and the internal data of the teeth.

55. 49. The system of claim 47 or 48, wherein the at least one sensor comprises a first sensor configured to capture the 3D surface model data from light in the first spectral range, and a second sensor configured to capture the internal data of the tooth from light in the sensor spectral range.

56. 49. The system of claim 47 or 48, further comprising a plurality of sensors adjacent to the plurality of light sources.

57. 49. The system of claim 47 or 48, wherein the light source configured to emit light in the second spectral range is configured to emit one or more of white light trans-illumination, UV / blue fluorescence, and red light fluorescence.

58. 49. The system of claim 47 or 48, wherein the light source configured to emit light in the second spectral range is configured to emit infrared light.

59. 49. The system of claim 47 or 48, further comprising: the plurality of light sources and the at least one sensor arranged on the handheld wand such that light emitted in the second spectral range is reflected off the teeth and received by the one or more sensors at an angle between 0° and 15°.

60. 49. The system of claim 47 or 48, wherein the processor is configured to generate a three-dimensional (3D) model of the teeth from the 3D surface data and the image data.

61. 49. The system of claim 47 or 48, wherein the one or more processors are configured to automatically adjust the first duration and the second duration based on the received 3D surface data, the received image data, or both the 3D surface data and the image data.

62. 1. A method for imaging the interior of a subject's teeth to detect cracks and caries using an intraoral scanner, comprising: scanning the subject's teeth with the intraoral scanner; acquiring a plurality of near-infrared images of the interior of the subject's teeth at various orientations using the intraoral scanner emitting both near-infrared and non-penetrating wavelengths; determining, for each image location of the plurality of near infrared images, a position of the intraoral scanner relative to the subject's teeth using the non-penetrating wavelength; generating a three-dimensional (3D) volumetric model of the subject's teeth using the plurality of near-infrared images and the position of the intraoral scanner relative to the subject's teeth in each near-infrared image of the plurality of near-infrared images; The method includes:

63. 63. The method of claim 62, further comprising analyzing the volumetric model to identify cracks or caries.

64. 63. The method of claim 62, wherein the step of acquiring a plurality of near-infrared images of the interior of the subject's teeth at various orientations comprises acquiring low-angle images.

65. 63. The method of claim 62, wherein the step of acquiring the multiple near-infrared images of the inside of the subject's teeth at various orientations includes acquiring low-angle images, wherein the angle between near-infrared light emitted and received by the intraoral scanner is between 0 and 15 degrees.

66. 63. The method of claim 62, wherein the intraoral scanner emits light at a wavelength of 850 nm.

67. 63. The method of claim 62, further comprising emitting near infrared light from the intraoral scanner to acquire the plurality of near infrared images at a first polarization; and filtering the infrared light returning from the patient's teeth to remove specular reflections by filtering the infrared light returning from the patient's teeth to remove light having the first polarization.

68. 1. A method for imaging through a subject's teeth to detect cracks and caries, comprising: Scanning the subject's teeth from a plurality of positions, wherein at each position: acquiring a plurality of near infrared images of the interior of the tooth at various orientations using an intraoral scanner, the intraoral scanner emitting light at a near infrared wavelength at a first polarization, the angle between the emitted light and the light received by the image sensor in each near infrared image being between 0 and 15 degrees, and further wherein the received near infrared light is filtered to block near infrared light of the first polarization; determining a position of the intraoral scanner relative to the subject's teeth for each image location of the plurality of near infrared images; the scanning step including repeating generating a three-dimensional (3D) volumetric model of the tooth using the penetration image and the surface position information; The method includes:

69. 70. The method of claim 68, further comprising the step of capturing 3D surface model data of the subject's teeth.

70. 70. The method of claim 69, wherein the step of capturing the 3D surface model data includes using confocal scanning, stereoscopic vision, or structured light triangulation.

71. 69. The method of claim 68, wherein the scanning step includes switching between the near-infrared wavelength and a non-penetrating wavelength.

72. 1. A method for forming a three-dimensional (3D) volumetric model of a subject's teeth, comprising: acquiring a plurality of near-infrared images of the subject's teeth with a camera sensor, where near-infrared illumination for the plurality of near-infrared images is projected from approximately the direction of the camera sensor; receiving, for each of the plurality of near infrared images, position data representative of a position of the camera relative to the subject's teeth; generating an upper bound on a scattering coefficient from the plurality of near infrared images and the position data for each point within the volume; combining the upper bounds of scattering coefficients for each point within a volume to form a 3D volumetric model of the subject's teeth; outputting the 3D volumetric model of the subject's teeth; The method includes:

73. 73. The method of claim 72, wherein the outputting step includes forming an iso-surface from the 3D volumetric model of the subject's teeth.

74. 73. The method of claim 72, further comprising the step of receiving surface data representing an exterior surface of the subject's teeth, and wherein the generating step is performed within the exterior surface of the subject's teeth for each point within the volume.

75. 73. The method of claim 72, wherein the outputting step includes generating an iso-surface corresponding to an interior dentin surface from the 3D volumetric model of the subject's teeth.

76. 73. The method of claim 72, wherein the location data includes position and orientation data of the camera at the time each of the plurality of near infrared images is captured.

77. 73. The method of claim 72, wherein the position data includes three numerical coordinates in three-dimensional space, the pitch, yaw, and roll of the camera.

78. The step of generating the upper limit of the scattering coefficient for each point in the volume comprises: projecting each point of a 3D point cloud grid corresponding to the volume of teeth of the subject onto each of the plurality of near infrared images using a first calibration; generating a list of luminance values ​​for each projection point; converting each intensity value on the list of intensity values ​​into a scattering coefficient according to a volume response; storing, for each grid point, a minimum scattering coefficient value of said scattering coefficients; Including, 73. The method of claim 72.

79. 80. The method of claim 78, wherein the first calibration includes a fixed pattern noise calibration to calibrate for sensor issues and image ghosting of the camera.

80. 80. The method of claim 78, wherein the first calibration comprises a camera calibration that determines a transformation of the camera that projects known points in space to points on an image.

81. 1. A method for reconstructing volumetric structures from a tooth that is translucent in a radiation wavelength range, comprising: receiving in a processor a representation of the tooth surface in a first coordinate system; receiving in the processor a plurality of images of the teeth in the radiation wavelength range acquired by a camera, the plurality of images being acquired with illumination projected generally from the direction of the camera; receiving, at the processor, position data representative of a position of the camera in each of the plurality of images; projecting each point of a point cloud grid corresponding to a volume within the surface of the tooth onto each of the plurality of images using a first calibration; generating a list of intensity values ​​for each projected point; converting each intensity value on the list of intensity values ​​into a scattering coefficient according to a volume response; storing the minimum scattering coefficient at each point in a minimum scattering coefficient list; The method includes:

82. 82. The method of claim 81, further comprising the step of outputting an image generated from the minimum scattering coefficient list.

83. 82. The method of claim 81, wherein the position data includes position and orientation data of the camera at the time each of the plurality of images is captured.

84. 82. The method of claim 81, wherein the first calibration includes a fixed pattern noise calibration to calibrate for sensor issues and image ghosting of the camera.

85. 82. The method of claim 81, wherein the first calibration comprises a camera calibration that determines a transformation of the camera that projects known points in space to points on an image.

86. 82. The method of claim 81, further comprising the step of receiving surface data representing an external surface of the subject's teeth, and wherein the projecting step is performed within the external surface of the subject's teeth for each point within the volume.

87. 82. The method of claim 81 , wherein the point cloud grid comprises a cubic grid.

88. 82. The method of claim 81, wherein the camera comprises a camera sensor.

89. 1. A non-transitory computing device readable medium having stored thereon instructions for reconstructing volumetric structures from teeth that are translucent in a radiation wavelength range, the instructions comprising: receiving a representation of the tooth surface in a first coordinate system; receiving a plurality of images of the tooth in the radiation wavelength range acquired by a camera, the plurality of images being acquired with illumination projected from approximately the direction of the camera; receiving position data representative of a position of the camera in each of the plurality of images; projecting each point of a point cloud grid corresponding to the tooth volume onto each of the plurality of images using a first calibration; generating a list of intensity values ​​for each projected point; converting each intensity value on the list of intensity values ​​into a scattering coefficient according to a volume response; storing a minimum scattering coefficient at each point among the scattering coefficients; outputting an image generated from the minimum scattering coefficient list; Executable by a processor to cause a computing device to perform A computing device readable medium.

90. 90. The device of claim 89, wherein the location data includes position and orientation data of the camera at the time each of the plurality of near-infrared images is captured.

91. 90. The device of claim 89, wherein the position data includes three numerical coordinates in three-dimensional space: pitch, yaw, and roll of the camera.

92. 90. The device of claim 89, wherein the first calibration includes a fixed pattern noise calibration to calibrate for sensor issues and image ghosting of the camera.

93. 90. The device of claim 89, wherein the first calibration includes a camera calibration that determines a transformation of the camera that projects known points in space to points on an image.

94. 90. The device of claim 89, wherein the point cloud grid is inside the tooth.

95. 90. The device of claim 89, wherein the point cloud grid comprises a cubic grid.

96. 1. A method of modeling a subject's teeth, comprising: capturing with an intraoral scanner a plurality of images of the interior of the subject's teeth and a position and orientation of the intraoral scanner specific to each image of the plurality of images; segmenting the plurality of images to generate internal structures corresponding to internal structures of the subject's teeth; using the positions and orientations of the plurality of images to project the internal structures onto a three-dimensional model of the subject's teeth; displaying the three-dimensional model of the subject's teeth, including the internal structure; The method includes:

97. 97. The method of claim 96, wherein the capturing step includes capturing surface images of the subject's teeth while capturing the multiple images of the interior of the subject's teeth.

98. 98. The method of claim 97, further comprising generating the three-dimensional model of the subject's teeth from the captured surface images.

99. 97. The method of claim 96, further comprising storing the position and orientation of the intraoral scanner while capturing the plurality of two-dimensional images.

100. 97. The method of claim 96, wherein the step of segmenting the plurality of images comprises applying edge detection to the plurality of images to identify closed boundaries within the plurality of images.

101. 97. The method of claim 96, wherein segmenting the plurality of images comprises forming a volumetric density map from the plurality of images to identify the internal structures.

102. 102. The method of claim 101, further comprising partitioning the volumetric density map to identify the internal structure.

103. 103. The method of claim 102, further comprising partitioning the volumetric density map to identify the internal structure by identifying isosurfaces in the volumetric density map.

104. 104. The method of claim 103, wherein the step of forming the three-dimensional model of the subject's teeth comprises identifying three-dimensional surface topology using confocal focusing.

105. 98. The method of claim 97, wherein the step of capturing an image of the subject's tooth surface comprises using confocal scanning, stereoscopic vision, or structured light triangulation.

106. 97. The method of claim 96, wherein the step of capturing the plurality of images includes using a penetrating wavelength by the intraoral scanner.

107. 97. The method of claim 96, wherein the step of capturing a plurality of images includes capturing an infrared image.

108. 97. The method of claim 96, further comprising displaying the three-dimensional model as the images are captured.

109. 1. An intraoral scanning device configured to generate a model of a subject's teeth, comprising: an intra-oral scanner having a plurality of light sources and a position and orientation sensor, the light sources configured to emit light in a first spectral range and a second spectral range, and further wherein the second spectral range is penetrative; a processor operatively connected to the intraoral scanner, the one or more processors configured to cause the scanner to capture a plurality of images and a position and orientation of the intraoral scanner corresponding to each of the plurality of images when the intraoral scanner emits light in the second spectral range; Including, The processor is further configured to segment the plurality of images to form internal structures corresponding to structures within the subject's teeth, and display or transmit a three-dimensional model of the subject's teeth including the internal structures. Device.

110. 110. The apparatus of claim 109, wherein the processor is configured to segment the plurality of images by applying edge detection to the plurality of images to identify closed boundaries within the plurality of images.

111. 110. The system of claim 109, wherein the processor is configured to identify the internal structure by forming a pixel density map from the plurality of images to segment the plurality of images.

112. 110. The system of claim 109, wherein the processor is configured to identify the internal structure by identifying closed sections within the pixel density map.

113. 1. A non-transitory computing device readable medium having stored thereon instructions executable by a processor, the instructions causing an intraoral scanning device to: capturing a plurality of images using penetrating wavelength light and a position and orientation of the intraoral scanner specific to each image of the plurality of images; segmenting the plurality of images to generate internal structures corresponding to structures within the subject's teeth; projecting the internal structures onto a three-dimensional model of the subject's teeth using the position and orientation of the intraoral scanner specific to each image; displaying the three-dimensional model of the subject's teeth, including the internal structure; To carry out the following: A computing device readable medium.

114. 114. The device of claim 113, wherein the non-transitory computing device readable medium having instructions is further configured to cause the intraoral scanning device to perform the step of segmenting the plurality of images by applying edge detection to the plurality of images to identify closed boundaries within the plurality of images.

115. 114. The device of claim 113, wherein the non-transitory computing device readable medium having instructions is further configured to cause the intraoral scanning device to perform the step of forming a pixel density map from the plurality of images to segment the plurality of images to form the internal structure.

116. 114. The device of claim 113, wherein the non-transitory computing device readable medium having instructions is further configured to cause the intraoral scanning device to perform the step of identifying closed sections in the pixel density map to segment the plurality of images to form the internal structure.

117. A non-transitory computing device readable medium having stored thereon instructions executable by a processor, the instructions being configured to cause a computing device to: receiving three-dimensional surface model data of the subject's teeth from a scanner; receiving from a scanner a plurality of images of the interior of the subject's teeth and a position and orientation of the intraoral scanner specific to each image of the plurality of images; segmenting the plurality of images to form an internal structure of the subject's teeth; projecting the internal structure of the subject's tooth onto the three-dimensional surface model; displaying the three-dimensional surface model showing the internal structure; To carry out the following: A non-transitory computing device readable medium.

118. receiving data associated with an intraoral scan of a subject; identifying from the received data at least a portion of a volume of a first internal shape of the subject's teeth; identifying from the received data at least a portion of a volume of a second internal shape of the tooth of the subject, the second internal shape being different from the first internal shape; mapping together said portion of said volume of said first internal shape and said portion of said volume of said second internal shape; outputting the portion of the volume of the first internal shape and the portion of the volume of the second internal shape together as a 3D volume; The method includes:

119. 119. The method of claim 118, wherein the received data includes data from a tooth surface penetration intraoral scan of the subject.

120. 120. The method of claim 119, wherein the received data further comprises data from an intraoral scan of the subject's dental surfaces.

121. 121. The method of claim 120, further comprising the steps of: identifying surfaces of the teeth of the subject from the received data; mapping the surfaces of the teeth together with the portion of the volume of the first internal shape and the portion of the volume of the second internal shape; and outputting the surfaces of the teeth together with the portion of the volume of the first internal shape and the portion of the volume of the second internal shape as the 3D volume.

122. 122. The method of claim 121, wherein the received data further comprises data from a tooth surface color intraoral scan of the subject.

123. 123. The method of claim 122, further comprising the steps of: identifying a color of the surface of the tooth of the subject from the received data; mapping the color of the surface of the tooth to the surface of the tooth; and outputting the 3D volume having the surface of the tooth and the color of the surface of the tooth.

124. 119. The method of claim 118, wherein the first internal shape of the tooth comprises the dentin of the tooth and the second internal shape of the tooth comprises the enamel of the tooth.

125. 125. The method of claim 124, wherein the intraoral scan includes a second intraoral scan of the subject, the method further comprising the steps of: receiving data associated with a previous intraoral scan of the subject; identifying at least a portion of the enamel or dentin volume from the received data associated with the previous intraoral scan of the subject; identifying a volume change in the enamel or dentin by comparing the portion of the enamel or dentin volume identified from the received data associated with the second intraoral scan with the portion of the enamel or dentin volume identified from the received data associated with the previous intraoral scan; and outputting the identified volume change.

126. 125. The method of claim 124, further comprising: detecting dental caries of the tooth by comparing the second internal shape to the first internal shape; and outputting a signal to a user associated with the detected dental caries.

127. 127. The method of claim 126, wherein comparing the second internal shape to the first internal shape comprises analyzing whether the volume of the second internal shape extends from a surface of the volume of the first internal shape.

128. 128. The method of claim 127, wherein the analyzing step includes determining whether the volume of the second internal shape extends from the surface of the volume of the first internal shape to a portion of the second internal shape associated with the dentin.

129. 128. The method of claim 127, further comprising the steps of: calculating a volume of the second internal shape extending from the surface of the volume of the first internal shape; and outputting a signal associated with the calculated volume.

130. receiving data associated with an intraoral scan of a subject; determining a dental caries volume of the subject's tooth from the received data; quantification of the volume of dental caries in the teeth of the subject; outputting a signal associated with the quantified volume of dental caries in the tooth of the subject; The method includes:

131. 131. The method of claim 130, further comprising the steps of: identifying a volume of the subject's dental enamel from the received data; mapping the volume of the enamel to the volume of the dental caries; and outputting a 3D volume of the mapped volume of the enamel and the dental caries to a user.

132. 132. The method of claim 131, further comprising the steps of: identifying a volume of dentin of the subject's teeth from the received data; mapping the volume of dentin to the volume of enamel and the volume of dental caries; and outputting the mapped volumes of enamel and dental caries together with the volume of dentin as the 3D volume.

133. 131. The method of claim 130, wherein the intraoral scan of the subject includes a second intraoral scan of the subject, the method further comprising the steps of receiving data associated with a previous intraoral scan of the subject, determining a previous volume of dental caries of the teeth of the subject from the received data associated with the previous intraoral scan of the subject, and outputting a signal associated with a volumetric difference between the volume of dental caries and the previous volume of dental caries.

134. 131. The method of claim 130, further comprising the step of outputting a 3D model of the volume of the dental caries of the tooth of the subject.

135. 1. A transillumination adapter sleeve device for an intraoral scanner, comprising: a sleeve body configured to fit over a wand of an intraoral scanner, the sleeve body including a light passing region at a distal end configured to allow near infrared light to pass through the sleeve; a first wing region extending from the distal end of the sleeve body adjacent the light passing region; a near-infrared light source configured to emit near-infrared light from the first wing region; An apparatus comprising:

136. 136. The apparatus of claim 135, wherein the near-infrared light source is configured to emit near-infrared light that traverses the light passing region.

137. 136. The device of claim 135, further comprising a second wing region extending from the distal end of the sleeve body adjacent the light passing region, the second wing region having a second near-infrared light source configured to emit near-infrared light from the second wing region.

138. 136. The device of claim 135, further comprising electrical contacts at a proximal end of the sleeve body configured to apply electrical energy to the near-infrared light source.

139. 139. The apparatus of claim 138, further comprising a flexible circuit coupling the electrical contacts to the near infrared light source.

140. 139. The apparatus of claim 138, further comprising a camera sensor operatively connected to a second wing extending from the distal end of the sleeve body adjacent the light passing region.

Images