Intraoral 3D scanning with automatic charting

Abstract

To provide a method and a device for generating a dental chart from an image of a tooth.SOLUTION: One or more output imaging signals are generated from the intraoral probe, multi-modal image content is acquired from intraoral surface locations according to the tissue response from the one or more imaging signals, and spatial coordinates are associated with the acquired multi-modal image content. The surface contour of the patient's dentition is generated by stitching the acquired multi-modal image content and maintaining a spatial coordinate association with the stitched multi-modal image content. An outer shape of the one or more teeth is generated from the generated surface contour, and the generated outer shape is arranged as a dental chart representing a spatial order of the one or more teeth and supporting gingival tissue adjacent to the teeth. The dental chart is input and displayed by analyzing the acquired multi-modal image content and associating the analysis with a position on the dental chart according to the stored association of spatial coordinates.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates generally to intraoral imaging, and more particularly to methods and apparatus for automatically generating dental charts and related content from intraoral scans. [Background technology]

[0002] Dental charts are widely used tools by dental practitioners to plan and track patient treatment. Traditionally, dental charts are manually annotated by practitioners and treatment staff and stored in patient folders to maintain an up-to-date record of the patient's condition, treatment progress, and diagnostic concerns.

[0003] In recent years, there has been interest in adapting various electronic tools to support imaging and diagnosis, and in storing and maintaining dental chart information online for rapid reference and updating. One solution developed to address this need is SOFTDENT™ software from Carestream Dental LLC. Electronic solutions of this type have helped improve the efficiency of record-keeping functions and provide a mechanism for better integration of patient information systems with imaging. Using automated dental charting with segmentation and automatic tooth identification utilities, dental practitioners can create, reference, and maintain dental charts that are more closely related to a specific patient's set of teeth rather than a standard model. The displayed dental chart can serve as a convenient index and link to information acquired separately from several different types of imaging and measurement systems. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] US Patent Application Publication No. 2012 / 189182 [Patent Document 2] US Patent Application Publication No. 2018 / 357766 [Patent Document 3] US Patent Application Publication No. 2016 / 125601 [Patent Document 4] US Patent Application Publication No. 2011 / 109616 [Patent Document 5] US Patent Application Publication No. 2018 / 028064 Summary of the Invention [Problem to be solved by the invention]

[0005] For some practitioners, the need to manually enter notes and measurement information associated with each tooth using a computer keyboard makes such systems unattractive in practice, despite the recognized advantages of digital data storage. Thus, there is a need for an automated dental charting method and apparatus that can both generate an appropriate dental chart for a particular patient from dental images and populate the generated chart with information obtained by applying automated diagnostics to dental image data.

[0006] While several automated tools have been developed to generate displayable dental charts and integrate them with image content, there appears to be considerable room for improvement. For example, electronic dental charting allows for indexing image content from various radiological, optical, and ultrasound sources, but in practice, integrating this image content is typically not straightforward. The task of spatially registering data from various sources can be challenging, potentially resulting in unsatisfactory results and less than the accuracy required for diagnosis and follow-up. A related complication is the need to temporally synchronize data from various sources with other image data to provide up-to-date information on the patient's condition. Existing solutions mimic the "snapshot" data recording functionality of manually maintained charts, thereby ignoring the benefits of stored data and automatic updates, and can make it difficult to track treatment progress. Yet another drawback relates to limitations in data representation.

[0007] Therefore, it can be seen that there is a need for improved systems, devices, and methods that automate the dental charting process and, in particular, provide solutions that integrate dental charting with multi-function and / or multi-modal intraoral imaging systems. [Means for solving the problem]

[0008] Broadly described, the present invention comprises an apparatus, method, and system for automating the dental charting process by integrating dental charts with multi-function and / or multi-modal intraoral imaging systems. According to one example embodiment, a method for intraoral imaging is provided, including: (a) generating one or more output imaging signals from an intraoral probe; (b) acquiring multimodal image content from each of a plurality of intraoral surface locations of the patient's dentition according to tissue responses from the one or more imaging signals and associating spatial coordinates with the acquired multimodal image content; (c) generating a surface contour of the patient's dentition by reconstructing and stitching from a data subset of the acquired multimodal image content and maintaining an association between the spatial coordinates of the multimodal image content and the stitched surface contour; (d) generating outlines of one or more teeth from the generated surface contours and arranging the generated outlines as a dental chart representing the spatial order of the one or more teeth and the spatial order of supporting gum tissue adjacent the teeth; (e) inputting data into the dental chart by analyzing the acquired multimodal image content and presenting the analysis results at one or more locations on the dental chart according to the stored spatial coordinate associations; and (f) displaying, communicating, or saving the input dental chart.

[0009] Advantageously, the present invention acquires multimodal image content, automatically associates and stores spatial coordinates associated with the image content, and automatically analyzes the multimodal image content to input or update a patient's dental chart with minimal input from a dental technician. By automating the dental charting process, the present invention significantly reduces the time it takes a dental technician to create and / or update a patient's dental chart. As an additional advantage, the accuracy of the patient's dental chart may be improved over manually generating or updating a patient's dental chart. As yet another advantage, substantially more information is integrated into the patient's dental chart, thereby providing a dental practitioner with more information about the patient's dentition.

[0010] Other desirable advantages and benefits inherently achieved by the disclosed systems, devices, and methods may occur or become apparent to those skilled in the art. The present invention is defined by the appended claims. Specifically, the present invention has the following features (1) to (32). (1) A method for intraoral imaging, the method comprising: (a) generating one or more output imaging signals from an intraoral probe; (b) acquiring multimodal image content from each of a plurality of intraoral surface locations of the patient's dentition according to tissue response from the one or more imaging signals, and associating spatial coordinates with the acquired multimodal image content; (c) generating a surface contour of the patient's dentition by reconstructing and stitching from a data subset of the acquired multimodal image content, and maintaining the association between spatial coordinates of the multimodal image content and the stitched surface contour; (d) generating one or more tooth contours from the generated surface contours and arranging the generated contours as a dental chart representing the spatial order of the one or more teeth and the spatial order of supporting gum tissue adjacent to the teeth; (e) inputting data into the dental chart by analyzing the acquired multimodal image content and displaying analysis results at one or more locations in the dental chart according to the stored spatial coordinate associations; (f) displaying the input dental chart; and The method includes the steps of: (2) The method of (1), wherein the step of acquiring multimodal image content includes directing a first output imaging signal and a second output imaging signal along a common imaging path. (3) The method comprises: (i) obtaining updated multimodal image content from one or more intraoral surface locations; (ii) reconstructing an updated surface view from a data subset of the updated multimodal image content; and (iii) stitching the reconstructed updated surface view to the surface contour of the patient's dentition; (iv) mapping the updated multimodal image content to spatial coordinates according to the input dental chart; Step 2: Update the input dental chart by The method according to (1), further comprising: (4) The method of (3), further comprising highlighting the relative position of the updated multimodal image content on the input dental chart. (5) The method of (4), wherein the highlighting is color-coded according to the mode of the updated multimodal image content. (6) The method of (1), further comprising the step of indicating a relative position of the intraoral scanner with respect to the patient's dentition on the input dental chart. (7) The method of (1), wherein the displaying indicates the status of one or more modes of acquisition or processing of the image content. (8) The method described in (1), wherein the step of displaying the input dental chart further includes presenting either a planar 2D view or an oblique 3D view in response to an operator's instruction. (9) The method described in (1), wherein the step of displaying the input dental chart further includes displaying partial analysis results of one or more imaging modes in response to an operator's instruction. (10) The method of (1), further comprising the steps of storing multimodal image content from successive imaging sessions and rendering successive versions of the stored image content on a display. (11) The method comprises: (i) accepting operator instructional entry specifying one or more intraoral surface locations requiring updated multimodal image content; (ii) acquiring the updated multimodal image content from the one or more intraoral surface locations; and (iii) mapping the acquired updated image content to spatial coordinates according to the input dental chart; (iv) highlighting the relative position of the updated image content on the input dental chart; and Step 2: Update the input dental chart by The method according to (1), further comprising: (12) The method of (11), further comprising providing visual or audible operator feedback regarding acquisition status. (13) The method of (1), wherein the one or more output imaging signals differ from each other in at least one of wavelength range, wavelength sequence, bandwidth, or coherence. (14) The method of (1), wherein the first output imaging signal includes color light from three or more primary colors. (15) The method of (1), wherein the first output imaging signal includes a swept-source laser. (16) The method of (1), wherein the tissue response to the first output imaging signal includes fluorescence. (17) A method for intraoral imaging, the method comprising: (a) generating a plurality of output imaging signals from an intra-oral probe, the output imaging signals being directed along a common axis, and the generated signals differing from one another in at least one of wavelength range, wavelength sequence, bandwidth, or coherence; (b) acquiring multimodal image content from each of a plurality of intraoral surface locations of the patient's dentition according to tissue responses to the plurality of output imaging signals, and associating spatial coordinates with the acquired multimodal image content; (c) generating a surface contour of the patient's dentition by reconstructing and stitching from a data subset of the acquired multimodal image content, and maintaining the association between spatial coordinates of the multimodal image content and the stitched surface contour; (d) generating one or more tooth outlines from the generated surface contours and arranging the generated outlines as a dental chart representing the spatial order of the one or more teeth according to the retained spatial coordinate associations; (e) inputting data into the dental chart by analyzing the acquired multimodal image content and associating the analysis with a location on the dental chart according to the maintained spatial coordinate association; (f) displaying the input dental chart; and The method includes the steps of: (18) The method of (17), wherein the step of displaying the input dental chart includes displaying spatial coordinates of intraoral features. (19) The method of (17), wherein the plurality of output imaging signals comprises a coherent laser beam. (20) The method according to (19), wherein the coherent laser beam repeatedly changes wavelength. (21) The method of (17), wherein the plurality of output imaging signals include ultrasound signals. (22) The method of (17), wherein the step of inputting data into the dental chart includes associating image content from two or more different imaging modes with teeth on the dental chart. (23) A method for intraoral imaging, the method comprising: (a) acquiring multimodal image content from an intraoral probe at each of a plurality of intraoral surface locations, the intraoral probe comprising: (i) directing polychromatic visible illumination at the surface location and obtaining surface image content associated with the surface location from the reflected polychromatic light; (ii) directing a surface contour imaging signal to said surface location; the obtaining is configured to successively perform (b) reconstructing a surface contour of the patient's dentition from the acquired multi-modal image content, wherein the reconstruction preserves the spatial association between the acquired surface image content and the reconstructed surface contour; (c) generating contours of one or more teeth from the reconstruction and arranging the generated contours as a dental chart representing the spatial order of the one or more teeth and the spatial order of supporting gum tissue adjacent to the teeth; (d) analyzing the captured image content; (e) inputting data into the dental chart by associating the analysis results with a location on the dental chart; (f) displaying the input dental chart; and The method includes the steps of: (24) The method of (23), wherein the step of inputting data into the dental chart includes highlighting one or more portions of the analysis results. (25) The method of (24), wherein the acquired surface image content includes a color image. (26) The method of (23), wherein the step of inputting data into the dental chart includes highlighting the tooth contours using color or shading depending on the analysis results. (27) The method of (23), further comprising the step of storing, transmitting, or communicating the input dental chart. (28) The method of (23), further comprising the steps of directing excitation illumination in an excitation wavelength range to the surface location and acquiring fluorescence image content associated with the surface location, wherein the fluorescence wavelength of the acquired fluorescence image content is outside the excitation wavelength range, and the reconstructing step further preserves the spatial association between the acquired surface image content and the fluorescence image content and the reconstructed surface contour. (29) The method according to (28), wherein the excitation illumination is provided from a light source that is also used for the polychromatic illumination. (30) The method according to (23), wherein the surface contour imaging signal obtains depth-resolved image content. (31) A method for intraoral imaging, the method comprising: (a) acquiring multimodal image content from an intraoral probe at each of a plurality of surface locations within a patient's mouth, the intraoral probe comprising: (i) directing polychromatic visible illumination at the surface location and obtaining 2D image content associated with the surface location from the reflected polychromatic light; (ii) directing a point-by-point scanning sequence of a coherent light beam transmitted through points along the surface location to modulate the wavelength of the scanning coherent light beam at each transmitted point over a range of wavelengths to obtain an interference signal having depth-resolved image content; acquiring the interference signal, wherein the polychromatic visible illumination and the sequence of coherent light beams are directed along a common optical axis of the intra-oral probe; the obtaining is configured to successively perform (b) reconstructing a surface contour of the patient's dentition from the acquired multi-modal image content, wherein the reconstruction preserves the association of the acquired 2D image content and depth-resolved image content with each surface location; (c) generating contours of one or more teeth from the reconstruction and arranging the generated contours as a dental chart representing the spatial order of the one or more teeth and the spatial order of supporting gum tissue adjacent to the teeth; (d) inputting data into the dental chart by analyzing the acquired image content and associating the analysis results with locations on the dental chart; (e) displaying the input dental chart; and The method includes the steps of: (32) An intraoral probe, (a) a signal generation circuit energizable to generate one or more output imaging signals; (b) one or more imaging sensors capable of generating multimodal image content from each of a plurality of intraoral surface locations of the patient's dentition according to detected tissue responses to the one or more generated imaging signals; and (c) in signal communication with the signal generating circuit and the one or more sensors; (i) associating spatial coordinates corresponding to the intraoral surface locations with the acquired multimodal image content; (ii) generating a surface contour of the patient's dentition by reconstructing and stitching from a data subset of the acquired multimodal image content, while preserving the association between spatial coordinates of the multimodal image content and the stitched surface contour; (iii) generating one or more tooth profiles from the generated surface contours; and (iv) arranging the generated contours as a dental chart representing the spatial order of the one or more teeth and the spatial order of supporting gum tissue adjacent to the teeth; (v) inputting data into the dental chart by analyzing the acquired multimodal image content and associating the analysis with a location on the dental chart according to the maintained spatial coordinate association; said control logic processor configured with programmed instructions to: (d) a display, in signal communication with the control logic processor, for displaying the input dental chart; The intraoral probe comprising:

[0011] These and other objects, features, and advantages of the present invention will become apparent from the following more particular description of illustrative embodiments thereof, as illustrated in the accompanying drawings, in which elements are not necessarily to scale relative to each other. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram showing components of an imaging device for multi-modal image acquisition and automated dental chart generation. [Figure 2] 1 is a schematic diagram illustrating an alternative exemplary embodiment of an imaging device that combines OCT ("optical coherence tomography") scanning with color image acquisition. [Figure 3] FIG. 1 is a schematic diagram illustrating another alternative exemplary embodiment of an imaging device for multimodal image acquisition. [Figure 4] FIG. 10 is a schematic diagram illustrating yet another alternative exemplary embodiment of an imaging device for multimodal image acquisition. [Figure 5A] 1 illustrates functional and geometric aspects of OCT imaging. [Figure 5B] 1 illustrates functional and geometric aspects of OCT imaging. [Figure 6] 1 shows in schematic form a combined OCT and color scan sequence. [Figure 7A] 1 illustrates various embodiments of color light emitters / color light detectors. [Figure 7B] 1 illustrates various embodiments of color light emitters / color light detectors. [Figure 7C] 1 illustrates various embodiments of color light emitters / color light detectors. [Figure 7D] 1 illustrates various embodiments of color light emitters / color light detectors. [Figure 7E] 1 illustrates various embodiments of color light emitters / color light detectors. [Figure 7F] 1 illustrates various embodiments of color light emitters / color light detectors. [Figure 7G] 1 illustrates various embodiments of color light emitters / color light detectors. [Figure 7H] 1 illustrates an exemplary embodiment in which RGB (“red-green-blue”) light detection is performed in an OCT spectrometer. [Figure 8] 1 is a flowchart illustrating an exemplary method for composite color calibration. [Figure 9]1 is a graph showing the spectral range of OCT imaging and reflected light imaging. [Figure 10A] FIG. 1 is a schematic diagram showing the relationship between the acquired image content and each scanned x, y, z position within the scanner field of view. [Figure 10B] FIG. 10 is a schematic diagram illustrating the remapping of captured image content to a set of positions after a stitching transformation. [Figure 11] 1 is a flowchart illustrating an OCT processing method for obtaining OCT imaging content along with a surface point cloud extracted from the OCT content, according to an exemplary embodiment of the present disclosure. [Figure 12] A-E illustrate the different types of imaging content acquired and generated as part of the OCT processing method using an example image of a tooth with severe caries. [Figure 13] 1 is a flowchart illustrating a method for generating a dental chart using multi-modal scan image data, according to an embodiment of the present invention. [Figure 14A] 1 illustrates various aspects of a generated and displayed dental chart according to an exemplary embodiment of the present invention. [Figure 14B] 1 illustrates various aspects of a generated and displayed dental chart according to an exemplary embodiment of the present invention. [Figure 14C] 1 illustrates various aspects of a generated and displayed dental chart according to an exemplary embodiment of the present invention. [Figure 14D] 1 shows an example of a generated dental chart with a top view of a patient's dentition. [Figure 14E] 1 shows an example of a generated dental chart with a perspective view of the patient's dentition. [Figure 15] 1 is a flowchart illustrating a method of steps for automating the process of dental charting according to an exemplary embodiment of the present disclosure. [Figure 16] FIG. 1 is a schematic diagram illustrating an example of a highlight placed on a perspective view of a dental chart to indicate the current position of a scanner or probe relative to the patient's entire dentition for acquiring image content. [Figure 17]FIG. 10 is a schematic diagram illustrating an example of entering touchscreen commands to identify a particular tooth or other feature. DETAILED DESCRIPTION OF THE INVENTION

[0013] Exemplary embodiments will now be described in detail with reference to the drawings, in which like reference numerals identify like apparatus elements or method steps in each of the several views.

[0014] When used in the context of this disclosure, the terms "first," "second," etc. do not necessarily denote any ordinal, sequence, or priority relationship, but are merely used to more clearly distinguish one step, element, or set of steps or elements from another, unless otherwise specified.

[0015] As used herein, the term "energizable" relates to a device or set of components that performs a specified function upon receiving electrical power and, optionally, upon receiving an enabling signal.

[0016] In the context of this disclosure, the terms "viewer," "operator," and "user" are considered equivalent and refer to a viewing practitioner, technician, or other person who views and manipulates images, such as dental images, on a display monitor. "Operator instructions" or "viewer instructions" are obtained from explicit commands entered by the viewer, such as by clicking a button on a camera, using a computer mouse, or by touchscreen or keyboard input.

[0017] In the context of this disclosure, the phrase "in signal communication" indicates that two or more devices and / or components can communicate with each other via signals traveling along some type of signal path. The signal communication may be wired or wireless. The signals may be communication signals, power signals, data signals, or energy signals. The signal path may include a physical connection, an electrical connection, a magnetic connection, an electromagnetic connection, an optical connection, a wired connection, and / or a wireless connection between a first device and / or component and a second device and / or component. The signal path may also include additional devices and / or components between the first device and / or component and the second device and / or component.

[0018] In the context of this disclosure, the term "optics" is used generally to refer to lenses and other refractive, diffractive, and reflective components or apertures used to shape and direct beams of light. Individual components of this type are called optical components.

[0019] In the context of this disclosure, the term "scattered light" is used generally to include light that is reflected and backscattered from an object.

[0020] The general term "scanner" refers to an optical system used to acquire various types of intraoral images of a patient's dentition, including support structures. In OCT (optical coherence tomography) imaging, the scanner optics projects a scanning light beam of broadband near-infrared (BNIR) light, which is directed through a sample arm toward the tooth surface and captured as scattered light returned to the sample arm to detect interference with light from a reference arm used for OCT imaging of the surface. The general term "raster scanner" refers to a combination of hardware components that scan light toward a sample, as described in more detail below.

[0021] In the context of the present disclosure, the general term "camera" more specifically refers to a device capable of acquiring reflected light, 2D ("two-dimensional") digital images from reflected visible or NIR light, such as structured light, from the surfaces of the teeth and supporting structures. According to exemplary embodiments of the present disclosure, a camera operating at video or near-video rates is used to acquire images that are used to generate 3D ("three-dimensional") contour images of the teeth and supporting intraoral surfaces.

[0022] The term "subject" refers to the patient's teeth or other part being imaged, and in optical terms can be considered equivalent to the "target" of a corresponding imaging system.

[0023] In the context of this disclosure, the phrase "broadband light emitter" refers to a light source that emits a continuous spectral output over a range of wavelengths at any given time. Short-coherence or low-coherence broadband light sources can include, for example, superluminescent diodes, short-pulse lasers, many types of white light sources, and supercontinuum light sources. Most such short-coherence length light sources have coherence lengths on the order of tens of microns or less.

[0024] In the context of this disclosure, two wavelengths may be considered "close" to each other if they are within + / - 10 nm of each other.

[0025] In the context of this disclosure, the terms "color light," "polychromatic light," and "RGB light" describe visible light illumination provided for reflected-light imaging. A color image of an intraoral surface location can be considered a reflected-light image or a color texture image. As is well known in the art of color imaging, a color combiner, such as a dichroic surface that transmits one spectral band and reflects another, can be used to combine the colors of light traveling in one direction along an optical axis and separate the colors of light traveling in the opposite direction along the axis. Thus, the general term "combiner" is typically used for a "combiner / separator" device that both mixes and separates light traveling along an optical path according to wavelength and direction.

[0026] The term "highlighting" for a displayed feature has its conventional meaning as understood by those skilled in the art of information and image display. Generally, highlighting is used to locally enhance a display in some way to draw the viewer's visual attention. For example, highlighting a portion of an image, such as an individual organ, bone, or structure, or a connecting path from one chamber to the next, can be accomplished in any of several ways, including, but not limited to, annotating, displaying a symbol nearby or overlaid, outlining or tracing, displaying in a different color or at a significantly different intensity or grayscale value from other image or information content, flashing or animating a portion of the display, or displaying with greater sharpness or contrast.

[0027] As used herein, the term "set" refers to a non-empty set, as the concept of a collection of elements or an element of a set is commonly understood in elementary mathematics. Unless otherwise specified, the term "subset" is used herein to refer to a non-empty proper subset, i.e., a subset of a larger set having one or more elements. For a set "S", a subset may constitute the complete set "S", where a "proper subset" of set "S" is one that is strictly contained in set "S" and excludes at least one element of set "S".

[0028] Exemplary embodiments of the present disclosure acquire multimodal image content from a single intraoral scan. According to exemplary embodiments of the present disclosure, the multimodal image content that can be obtained from each scanned location of an intraoral surface can include image content from two or more imaging modes, where an imaging mode can acquire, for example, color / polychromatic or monochrome reflected-light image content, fluorescence image content, and depth-resolved image content. Still other image modes can use signal content from different types of sources, including x-ray or ultrasound image content. Image modes can be distinguished from one another in terms of the signals provided to acquire the image and the signals acquired to acquire the image.

[0029] Image stitching algorithms, which combine content from several 2D images captured from adjacent views in proper alignment to form a more complete composite image, provide a data subset of the full set of acquired image content (elements of this data subset share adjacent features), and are well known to those skilled in the art of image processing.

[0030] FIG. 1 is a schematic diagram illustrating the components of an intraoral imaging device 300 for multimodal image acquisition and automated dental chart generation. An intraoral scanner 314 is energized to acquire multimodal image content and can generate different types of optical and other signals required to acquire image content from intraoral surfaces. Optical signals may include, for example, polychromatic visible illumination, excitation illumination for fluorescence, or a scanning sequence of a coherent light beam. Other exemplary signal types may include ultrasound and photoacoustic signals. By operating under multiple imaging modes, the single scanner 314 in the device of FIG. 1 can provide multiple functions, such as 3D surface contour imaging and depth-resolved imaging. The scanner 314 is in signal communication with a processor 320, such as a computer. Signal communication can be wired, wireless, or may use some combination of wired and wireless transmission. The processor 320 is in signal communication with a memory 324 or data storage device that provides long-term data retention and archiving. A display 326, also in signal communication with the processor 320, displays the generated dental chart and associated operator interface.

[0031] The images obtained from surface contour imaging form a data subset of the acquired multi-modal image content obtained from the scanner 314. This data subset is processed by surface contour reconstruction and stitching to provide a spatial reference for mapping the data content acquired in any imaging mode.

[0032] Depth-resolved imaging mode OCT, ultrasound, and photoacoustic imaging techniques are each capable of providing depth-resolved imaging from a properly configured intraoral scanner. Each of these depth-resolved imaging modes uses generated signal energy capable of penetrating below the tissue surface to direct a series of depth-probing output imaging signals to the scanned surface location, providing depth-resolved image content in addition to surface contour information. Depth-resolved imaging modes such as OCT, ultrasound, and photoacoustic imaging not only provide imaging data for reconstructing surface contours, but can also provide useful information related to features below the surface of the imaged tissue, accurate to some depth depending on the limitations of the signal energy used. In addition to depth-related information, various depth-resolved imaging devices and methods can also provide useful data regarding feature density, distribution, and dimensions.

[0033] Exemplary embodiments of the present disclosure can address the need for enhanced characterization of intra-oral features, such as teeth, using depth-resolved imaging. The method of such exemplary embodiments adds weight and utility to dental charts maintained for patients using the additional information gained from depth-resolved images, complementing standard tooth-by-tooth assessments traditionally obtained by visual inspection with data traditionally hidden below the surface.

[0034] The following discussion focuses on image acquisition using OCT as an exemplary type of depth-resolved imaging system that can be utilized for enhanced automated dental chart characterization and generation, but it should be recognized and understood that similar processes and methods can be used to assemble surface content using alternative types of depth-resolved imaging, such as when using ultrasound and photoacoustic imaging.

[0035] Advantageously, depth-resolved image content is acquired using the same scanner that acquires reflected-light and fluorescence images. By combining image acquisition capabilities in a single intraoral scanner, the disclosed method addresses spatial and temporal synchronization issues that have limited the usefulness, accuracy, and practicality of previous imaging solutions.

[0036] Exemplary embodiments of the present disclosure utilize results obtained from devices that perform depth-resolved imaging, such as optical coherence tomography (OCT), to generate dental charts, and can input results from image analysis to aid in the recognition of prevalent tooth and gum conditions into the generated dental chart. The OCT results can be combined with 3D surface contour imaging results and analysis obtained using the same imaging device, which are acquired simultaneously and co-registered with the OCT data.

[0037] The following description provides more detailed information regarding OCT and multimodal imaging subsystems that can be used to provide dental chart content, according to exemplary embodiments of the present disclosure. It should be recognized and understood that the multimodal imaging device, scanner 314 in FIG. 1, can acquire any of several types of image data, all registered to a shared set of spatial coordinates.

[0038] OCT Imaging Subsystem OCT, described as a form of "photo-ultrasound," acquires cross-sectional data by imaging reflected energy from within biological tissue. In an OCT imaging system, light from a broadband light source, such as a superluminescent diode (SLD) or other light source, is directed along two distinct optical paths as a depth-probing output signal: a reference arm of known length and a sample arm that illuminates the tissue or other object being investigated. The reflected and backscattered light from the reference and sample arms is then recombined within the OCT device, where interference effects are used to characterize the surface and underlying structures of the sample. Interference data is acquired by rapidly scanning the sample illumination across the entire sample. At each of several thousand points, the OCT device acquires an interference profile that can be used to reconstruct an A-scan with axial depth into the material, a factor that accounts for source coherence. For most tissue imaging applications, OCT uses a broadband illumination source and can provide image content at depths of up to several millimeters (mm).

[0039] Early OCT devices used a time-domain (TD-OCT) architecture, which achieved depth scanning by rapidly varying the length of a reference arm using some type of mechanical mechanism, such as a piezoelectric actuator. TD-OCT uses point-by-point scanning, requiring the illumination probe to be moved, or scanned, from one position to the next during an imaging session. More recent OCT devices can use a Fourier-domain (FD-OCT) architecture, which distinguishes reflections from different depths according to the optical frequency of the signal they produce. FD-OCT simplifies or eliminates the axial scanning requirement by collecting information from multiple depths simultaneously, improving acquisition rates and signal-to-noise ratios (SNRs).

[0040] Due to their potential to achieve high performance at low cost, FD-OCT systems based on swept-frequency laser sources have attracted considerable attention for medical applications requiring subsurface imaging of highly scattering tissues. Two implementations of Fourier-domain OCT exist: spectral-domain OCT (SD-OCT) and swept-source OCT (SS-OCT).

[0041] SD-OCT imaging can be accomplished by illuminating the sample with a broadband illumination source and dispersing the reflected and scattered light with a spectrometer onto an array detector, such as a CCD (charge-coupled device) detector. SS-OCT imaging illuminates the sample with a fast wavelength-tuning laser and collects the reflected light during a wavelength sweep using only a single photodetector or a balanced photodetector. In both SD-OCT and SS-OCT, a profile of the scattered light reflected from different depths is obtained by performing an operation on the recorded interference signal using a Fourier transform, such as a fast Fourier transform (FFT), which is well known to those skilled in the art of signal analysis.

[0042] A recent advancement in swept-source OCT is the use of a Fourier-domain mode-locked (FDML) laser source. FDML-OCT scanning significantly improves the acquisition speed of OCT sampling and improves depth resolution compared to other OCT methods.

[0043] Exemplary embodiments of the present disclosure can utilize any of various types of OCT scanning methods, including time-domain, spectral, or frequency-domain OCT. Because the speed advantage is particularly important, the following description is primarily directed to exemplary embodiments using swept-source OCT, a type of frequency-domain OCT that generally favors faster speed and overall scan throughput. However, it should be noted that compressive sampling or other available OCT methods can be used to improve the response of SS-OCT, as well as time-domain OCT and other types of OCT. The methods of the present disclosure can also be used when a spectrometer is used for sensing in the OCT system.

[0044] According to an exemplary embodiment of the present disclosure, a hybrid imaging device is provided that acquires OCT scan data with accompanying color texture content for intra-oral features along with fluorescence image content, the generated image content being provided using the same optical probe.

[0045] 2, an imaging device 100 for multimodal image acquisition combining OCT scanning, reflected light imaging, and fluorescence image acquisition modes is shown, with image data from each mode sharing the same spatial registration. Having the same spatial registration refers to acquiring image content from each mode with reference to the same sensor position and orientation, eliminating the need to map one type of image onto another.

[0046] The imaging device 100 comprises an intra-oral probe 30 that combines optical paths for directed and acquired light of different / multiple imaging modes along a common or shared optical path, shown in FIG. 2 as optical axis OA, that extends outside the probe 30. Separate directed output and acquired input signals can then be separated and channeled to / from corresponding optical subsystems within the probe 30.

[0047] For example, in the OCT optical path 40, the OCT light source 10 provides illumination for the OCT image scan. The light source 10 can be a superluminescent diode (SLD) or other light source that emits continuous wavelength broadband light. Alternatively, the light source 10 can be another type of suitable light source, such as a swept source that emits light with continuously varying spectral content. One advantageous type of swept source is a Fourier domain mode-locked (FDML) laser, which has a high sweep speed and a wavelength range suitable for depth-resolved imaging.

[0048] In the configuration of FIG. 2 , coherent laser light is directed through a first fiber coupler FC1 or a wavelength division multiplexer (WDM) to a second fiber coupler FC2. The fiber coupler FC2 splits the optical path into a reference arm 42 and a sample arm 44. The light in the reference arm 42 reflects back from a reference mirror 48. This light is coupled back through the fiber coupler FC2 and directed to an OCT signal detector 46. The light directed to the sample arm 44 is directed by the scanner 24 toward a subject or sample S. For intraoral imaging, the sample "S" is a surface location in the patient's mouth, which may include one or more teeth along with supporting features such as the gums. For OCT depth-resolved image acquisition, reflected and scattered light from the sample "S" is coupled back through the sample arm 44 to the fiber coupler FC2 and transmitted to the OCT signal detector 46. The light from the reference arm 42 interferes with the light from the reference arm 44 to provide OCT scan data for processing and reconstruction.

[0049] In the color reflected light imaging path 50, polychromatic or colored light is emitted from a color light emitter / color light detector (CLED) 52 and directed via a fiber coupler FC1 or WDM to a second fiber coupler FC2. Coupler FC2 functions as a combiner / separator. Polychromatic visible light is mixed with the OCT sample light and simultaneously directed to the sample "S" via an output scanner 24, which is part of the intraoral probe 30. Reflected colored light returning from surface locations, such as tooth surfaces or other intraoral features, is transmitted back to the CLED 52 via fiber coupler FC2. The CLED 52 senses the color content from the reflected light and forms a reflected light image of the sample "S." A control logic processor 60, in signal communication with the OCT signal detector 46, the CLED 52, and the light source 10, records and processes OCT output data from the interference and combines this data with color data from the intraoral surface. The resulting combined image content can then be presented on the display 72 and, in turn, communicated, transmitted, and / or stored.

[0050] The reflected light image can be used for a variety of purposes, including providing a high-resolution 2D intraoral image for documentation and patient communication, and its color content (e.g., R, G, and B image values) can also be used to determine tooth shade.

[0051] Fluorescence imaging uses components of the reflected light imaging path 50, and optionally a separate light source to direct excitation illumination in the appropriate wavelength range to stimulate the surface location, to generate fluorescence image content. In this case, a long-pass spectral filter can be used on the detector of the CLED 52 to sense the fluorescence signal and form a fluorescence image of the sample "S."

[0052] The schematic diagrams of Figures 3 and 4 show similar imagers 120 and 140, respectively, with slightly different optical path configurations for combining OCT and reflected light imaging functions. In the configuration of imager 120 shown in Figure 3, the OCT path can be the same as that previously described for imager 100 shown in Figure 2. Light from color light emitter / detector (CLED) 52 is directed through fiber coupler FC3 and shares the sample arm with the OCT light. This combined light is directed toward the subject or sample "S" by scanner 24. Backscattered color light from the intraoral surface is transmitted through fiber coupler FC3 to color light emitter / detector (CLED) 52 to measure color content, which is then recorded and processed by processor 60. The resulting combined image content can then be presented on display 72 and, in turn, communicated, transmitted, and / or stored.

[0053] In the configuration of imager 140 of Figure 4, the OCT pathway is the same as that previously described for imager 100 of Figure 2. Light from color light emitter / detector (CLED) 52 is directed to the sample pathway via a dichroic combiner 54, which in the illustrated configuration has reflective and dichroic surfaces. Backscattered color light from the intraoral surface is transmitted back to CLED 52 via combiner 54 for detection and measurement, and is recorded and processed by processor 60. The resulting combined image content can likewise be presented on display 72 and, in turn, communicated, transmitted, and / or stored.

[0054] According to an exemplary embodiment of the present disclosure, the different signals directed from the scanner to the intraoral surface locations share a common axis.

[0055] Scanning methods for OCT imaging The schematic diagrams in Figures 5A and 5B show scan sequences that can be used to generate tomographic images using the OCT device of the present disclosure in Fourier domain acquisition. The sequence shown in Figure 5A illustrates how a single B-scan image is generated. The raster scanner 24 (Figure 2) scans a selected light sequence point-by-point across the object, sample "S." As shown in Figure 5A, a periodic drive signal 92 is used to drive the galvo mirrors of the raster scanner 24 to control a horizontal scan or B-scan extending across each row of the sample, shown as horizontally extending discrete points 82 in Figures 5A and 5B. At each of the multiple points 82 along the line or row of the B-scan, an A-scan or depth scan is generated using successive portions of a selected wavelength band, acquiring data in the z-axis direction. Figure 5A illustrates the drive signal 92 for generating a simple ascending sequence using the raster scanner 24 across a wavelength band, with corresponding pixel-by-pixel actuation of a micromirror or other spatial light modulator. The retroscan signal 93, which is part of the drive signal 92, simply returns the scan mirror to its starting position in preparation for the next line. During the retroscan signal 93, no OCT data is acquired.

[0056] Note that the B-scan drive signal 92 drives the galvo mirror of the scanner 24 for the raster scanner 90, as shown in FIGS. 2-4. At each incremental position, or point 82 along the B-scan row, an A-scan is acquired. To acquire the A-scan data, a tuned laser or other OCT light source sweeps through a spectral sequence controlled by the programmable filter of the OCT light source 10. Thus, in an exemplary embodiment in which the light source sweeps a 30 nm wavelength range, this sequence is performed at each point 82 along the B-scan path. As shown in FIG. 5A, a set of A-scan acquisitions is performed at each point 82, i.e., each position of the scanner 24. By way of example, there may be 2,048 measurements to generate an A-scan at each position 82.

[0057] FIG. 5A shows a schematic representation of the information acquired during each A-scan. An interference signal 88, shown with the DC signal content removed, is acquired over a time interval for each point 82, where the signal is a function of the time interval required for the spectral sweep, and the acquired signal represents the spectral interference fringes produced by mixing light from the reference and feedback arms of the OCT interferometer components. A Fourier transform generates a transform T for each A-scan. One transform signal corresponding to an A-scan is shown in FIG. 5A as an example.

[0058] From the above discussion, it can be seen that a significant amount of data is acquired over a single B-scan sequence. To efficiently process this data, a Fast Fourier Transform (FFT) is used to convert the time-based signal data into corresponding frequency-based data from which image content can more easily be generated.

[0059] In Fourier domain OCT, an A-scan corresponds to one line of spectral acquisition that generates a line of depth (z-axis) resolved OCT signal. B-scan data generates a 2D OCT image along the corresponding scan line.

[0060] Raster scanning is used to acquire multiple B-scan data by incrementing the acquisition of the raster scanner 24 in the C-scan direction. This is represented diagrammatically in Figure 5B, which shows how the A-scan, B-scan, and C-scan data are used to generate three-dimensional volume information.

[0061] As previously mentioned, the wavelength or frequency sweep sequence used at each A-scan point 82 can be varied from the commonly used ascending or descending wavelength sequence. Arbitrary wavelength sequencing can be used alternatively. Arbitrary wavelength selection, which may be useful in some particular implementations of OCT, results in each sweep providing only a subset of the available wavelengths. In arbitrary wavelength sequencing, wavelengths to be used by the OCT system during a single sweep can be randomly selected in any order.

[0062] Multimodal Imaging Sequences FIG. 6 shows in schematic form a combined OCT and color scanning scheme. At each point 82, the scanner 24 directs both the color light beam and the OCT light beam onto the sample "S" in a two-dimensional (x,y) raster scan, where x∈[0,L-1] is indexed along the x-scan axis and the orthogonal component y∈[0,M-1] is indexed along the y-scan axis. A color signal having three reflectance values ​​(R(x,y), G(x,y), B(x,y)) and an OCT signal I of "N" size in the depth direction (for "N" points of data) are generated. OCT (x,y) is obtained corresponding to each scanned position (x,y).

[0063] As the 2D scanner 24 scans continuously, the 2D color image is filled with L×M color pixels, and the 3D CT volume is correspondingly reconstructed with L×M×N values. The values ​​of (R(x,y), G(x,y), B(x,y)) are expressed as I along the lateral direction. OCT Part (b) of FIG. 6 shows the intrinsic mapping provided to the color content at each point 82 corresponding to the OCT scan, where the values ​​(R(x,y), G(x,y), B(x,y)) are intrinsically registered to the corresponding I OCT (x,y) measurement. Part (c) of Figure 6 shows the (x,y,z i ) with surface point intensity I OCT (x,y,z i ) and z iis the surface depth from the zero delay line along the A-line OCT signal. Therefore, the color texture (R(x,y), G(x,y), B(x,y)) is directly mapped to the OCT signal at (x,y,z0).

[0064] Exemplary embodiments of the present disclosure preserve spatial mapping of data from different imaging modes, including OCT and reflected-light imaging, in the generated surface reconstruction. Dental charts generated from the surface reconstruction can then incorporate data on multiple characteristics of features, such as teeth and gums, without the need for separate alignment processes from different types of scans. This configuration not only enables dental charts to be generated without separate alignment steps, but also allows for easy updates to the dental chart content using subsequent scans. Thus, for example, the initial generation of a dental chart can use a complete scan incorporating reflected-light image content, color shade, 3D surface topography, OCT depth-resolved data, and fluorescence data, preserving the spatial relationship of each data type. Following this initial scan, partial scans can be performed, such as scanning implant sites, individual teeth requiring treatment, or other localized portions of the dentition. The reconstructed surfaces of the new scans can be stitched to the initially generated 3D surface of the dental arch. The updated information for the scanned portions, including updated data from multiple modes, can then be easily aligned with the previously scanned content and used to modify the existing dental chart.

[0065] Structural and functional components of CLED 7A, 7B, and 7C show different exemplary embodiments of the color light emitter / detector CLED 52. Laser diodes LD1, LD2, and LD3 are red, green, and blue laser diodes, respectively. Lenses L1, L2, and L3 are corresponding collimation lenses used with LD1, LD2, and LD3, respectively, to generate collimated light beams.

[0066] In the arrangement of FIG. 7A, the collimated beams are combined onto the same optical path by dichroic mirrors DM1 and DM2. Mirrors DM1 and DM2 have cutoff wavelengths appropriate for corresponding routing of light between the optical paths, such as the center wavelengths of red and green laser diodes LD1 and LD2. Lens L4 couples light from the shared path into a single-mode optical fiber 74 to provide full-color or polychromatic light. Full-color light backscattered from sample "S" is coupled back to CLED 52. Each color light is combined back into its original channel via beam splitters BS1, BS3, and BS4, and a portion of the optical power is directed to the corresponding photodiodes PD1, PD2, and PD3 for measurement.

[0067] In the configuration of Figure 7B, a trichroic beam splitter TBS with Philips prism-like dichroic filters F1 and F2 combines light from red, green, and blue laser diodes LD1, LD2, and LD3 into optical fiber 74, which is coupled to optical fiber 74 via lens L4. The full-color light backscattered from the object sample "S" is coupled back to CLED 52. Each color light is then coupled back to its original channel via trichroic beam splitter TBS.

[0068] In the configuration of FIG. 7C, a wavelength division multiplexer WDM is used to mix the red, green, and blue light and to separate these lights from their respective color channels.

[0069] In the configuration of FIG. 7D, a fiber combiner 76 is used to combine the red, green, and blue light and to separate these lights from their respective color channels.

[0070] In the configuration of FIG. 7E, two fiber combiners 76 are used: one to combine the outgoing red, green, and blue light into one channel, and the other to separate the returning red, green, and blue light into their respective color channels.

[0071] In the configuration of Figure 7F, a broadband visible light source, such as a supercontinuum laser SCL, serves as the light source for color imaging. The SCL has a continuous visible spectrum output. A wavelength division multiplexer WDM in the return light path separates the backscattered light and redirects the light to the respective photodiodes PD1, PD2, and PD3. A fiber coupler FC is used to couple the light to and from fiber 74.

[0072] In the configuration of Figure 7G, a pair of wavelength division multiplexers (WDM) and a variable attenuator (VA) are used to modulate the SCL light in the emission path. The WDM in the return path separates the backscattered light and redirects it to the respective photodiodes PD1, PD2, and PD3. A fiber coupler (FC) is used to couple the light to and from fiber 74.

[0073] The configuration of Figure 7H shows an exemplary embodiment in which RGB light detection is performed in an OCT spectrometer. Input polychromatic light is coupled into the OCT scanning system, as shown above in Figures 7A-7G. Polychromatic light detection utilizes a spectral separator 124, such as a grating or prism, to spectrally separate the light and direct the red, green, and blue light toward corresponding detectors 126r, 126g, and 126b at appropriate angles determined by the characteristics of the grating, prism, or other separator.

[0074] Light Source Options The visible light (Vis) used in the scanner optics can be multiple wavelengths in the visible light range. Vis light sources can be used, for example, to color-code projected structured light patterns. Vis light sources can alternately be used for white-light image preview, tooth shade measurement, or color or texture characterization.

[0075] Vis light can be provided by a conventional bulb light source or may be generated by a solid-state light-emitting device such as a laser or one or more light-emitting diodes (LEDs). Individual red, green, and blue LEDs are used to provide the primary wavelengths for reflected light imaging.

[0076] Vis light sources can provide structured light patterns as well as alternately provide specific wavelengths or broadband light that are scanned over the object for conventional reflected light imaging, e.g., detecting tooth shading, or for obtaining surface contour data in a manner that does not use a light pattern, e.g., structure-from-motion imaging.

[0077] Violet light in the near-ultraviolet region can be used as excitation light for dental fluorescence imaging. Backscattered fluorescence can be collected by the OCT optical path. Fluorescence images can be detected in the same detector path as Fourier-domain OCT but at a different transverse spectral position.

[0078] Color image processing and calibration For system calibration and imaging, the reflected light imager must be calibrated to a reference standard. The R, G, and B laser emissions are adjusted to provide balanced light intensities. A background signal is captured with sample "S" removed from the sample arm. R, G, and B photodiodes PD1, PD2, and PD3, respectively, detect the background signal reflected from components in the optical path. The background signal is subtracted from the R, G, and B signals, respectively. The calibration method for color images is similar to the method used for color photography, which is also adapted from the calculation flowchart in Figure 8.

[0079] FIG. 8 shows a color calibration sequence that can be used for a color scanner that performs both OCT and RGB imaging. RGB signals are acquired from a calibration target, such as a gray or white light reference patch, in a reference imaging step 700. Values ​​from a standard color model, such as sRGB, of a reference white patch or other calibration target are acquired in a standard color model step 710. A least-squares calculation or other suitable method to obtain a calibration transformation between the RGB signals and sRGB is performed in a calibration step 720, generating a calibration matrix in a conversion step 730. The calibration matrix is ​​applied to the RGB signals obtained from a reflected light imaging step 732 to generate a calibrated RGB signal 734, which is combined with the OCT surface detection obtained in an OCT surface imaging step 740. An attachment step 750 then combines the OCT surface detection data in register with the calibrated RGB data to provide a combined output.

[0080] The difference in the spectral range of the two imaging modes allows for the combination of OCT light with RGB color light using either spectral or amplitude splitting.

[0081] Figure 9 shows the spectral distribution of visible R, G, and B light and infrared OCT light waves. As can be easily seen from this mapping, the spectral wavelength ranges do not overlap. Visible light spans the wavelength range from approximately 380 nm to approximately 740 nm. Infrared light spans the wavelength range from greater than 740 nm to approximately 1600 nm. Figure 9 also shows the cutoff and bandpass wavelengths of the associated dichroic mirrors for WDM operation. As shown in the spectral diagram in Figure 9, additional color content can be added to provide more accurate shade matching, such as the violet V wavelength (below approximately 450 nm).

[0082] Illumination for fluorescence imaging can be at wavelengths approaching the visible blue region, or in the violet or ultraviolet range.

[0083] An alternative approach to meeting the need to combine OCT and color texture image data uses an OCT scanner coupled with a color preview camera to acquire the required image content. When using this alternative approach, processing is required to register the color texture data to the OCT scan content.

[0084] Maintaining association between image data and corresponding surface locations In each of the configurations of FIGS. 2-4, CLED signal detection and OCT signal detection can be synchronized, and the same optical path in the sample arm can be shared between the sample probe 30 and its scanner 24.

[0085] Rapid image acquisition in different modes allows a significant amount of image content to be acquired in a single scan using imaging device 100. Instead of requiring some kind of manual "mode switching" or requiring changes in scan execution, imaging device 100 can be configured for multi-mode operation so that each scan of a patient's mouth can acquire image content in multiple modes. This configuration allows the various types of image content acquired in each scan to be integrated into a surface reconstruction, stitching, and display of the complete dental arch. Data identifying each imaged surface location, typically identified using Cartesian x, y, and z coordinates, can be spatially associated with each type of image data acquired at that surface location.

[0086] Referring to the schematic diagram of FIG. 10A, an association of acquired image content to each scanned x, y, z spatial location within the field of view of the scanner 314 is shown. Repeated image acquisition sequences during scanning, 3D surface reconstruction, and stitching create an association between the acquired image content and the calculated x, y, z location coordinates. This spatial association can be formed, stored, and represented in a variety of ways. The example of FIG. 10A illustrates the creation of a set of association vectors 510 corresponding to each scanned location of the patient's dentition. The data structure of the vectors 510 is merely one example mechanism for representing the resulting association between a spatial location (x, y, z) and the data acquired at that location.

[0087] According to exemplary embodiments of the present disclosure, maintaining the association between spatial coordinates and tissue response results for a signal follows a similar sequence for each imaging mode. Working backward from reconstruction and stitching, the control logic can associate one image frame with each surface point, such as by using an image frame orthogonal to a vector orthogonal to the surface point. The data value corresponding to a particular point in the frame can be the most accurate measure of the tissue response at the corresponding point on the surface. Data associated with stitched surface points can be alternately averaged, such as by taking a weighted average of all points used for stitching at that location.

[0088] Maintaining an association between spatial coordinates and the resulting tissue response to light or other stimuli at each intraoral location simplifies the problem of matching and mapping various types of acquired data to an automatically generated dental chart. Instead of requiring computationally intensive methods for feature detection and registration of individual data content, the maintained associations allow 3D surface reconstruction and stitching operations to serve as a reference for organizing and displaying information acquired from multiple imaging modalities.

[0089] To preserve the association between the multimodal image content and its spatial coordinates, the stitching algorithm used can transform the image coordinates and link the corresponding content to the new coordinates of the transformation. The schematic diagram in FIG. 10B shows the remapping of the acquired image content to a set of locations after the stitching transformation. The acquired association vectors 510 provide structure to the measurement data of the tissue response at the locations indicated by the corresponding spatial coordinates (x1, y1, z1), (x2, y2, z2), and (x3, y3, z3). Following the stitching process, these initial coordinates are remapped to the spatial coordinates (x 1b ,y 1b ,z 1b ), (x 2b ,y 2b ,z 2b ), and (x 3b ,y 3b ,z 3b ) into coordinates of the stitched reconstructed 3D surface, expressed as . The stitching transformation thus works in conjunction with this to bring the corresponding data to bear and maintain the original correspondence between the multimodal measurements taken at each specific intraoral location and the coordinates assigned to the reconstruction from the acquired images. In this way, the 3D surface reconstruction and stitching maintains the registration of the various elements of the multimodal data relative to each other, simplifying the analysis and display of the multimodal image content.

[0090] OCT imaging processing The flowchart in FIG. 11 illustrates an OCT processing method for acquiring OCT imaging content along with a surface point cloud extracted from the OCT content, according to an exemplary embodiment of the present disclosure. Raw 2D spectral data 150, comprising multiple A-scans per B-scan, is provided as "N" lines with "M" pixels per line across a range of optical signal wavelengths λ. Mapping 152 then provides a wavenumber value "k" for each corresponding wavelength λ. Background subtraction 154 is performed, calculated along the B direction for each k value, to obtain a line of background signal. The background subtraction 154 performed on each A-line serves to remove fixed pattern noise. Zero-padding operation 156 and phase correction process 160 correct spectral sampling and account for dispersion-induced broadening of the OCT signal. An FFT processing step 162 processes and scales the phase-corrected data to provide input for 3D volume rendering and 2D cross-sectional frame display rendering 166, useful for visualization and diagnostic support. At the end of step 162, the OCT image content is available.

[0091] The data is then processed using the method of FIG. 11 to extract a point cloud for surface characterization and subsequent matching / stitching. A segmentation step 170 is performed to extract surface contour data from the OCT volume data. An object surface point cloud generation step 172 provides a surface point cloud of the measured object. The point cloud can then be used for further processing in a mesh rendering step 174. To help correct shape distortion, calibration of the OCT image's geometric distortion can be performed. Unless properly corrected, distortions can be caused by the scan pattern or the optical arrangement used. Distortion processing can use spatial calibration data obtained by using a calibration target of a given geometry. Scanning the target and acquiring the scanned data establishes a basis for adjusting the registration of the scanned data in 3D space and compensating for errors in the scanning system. The calibration target can be a 2D target imaged at one or more positions, or a 3D target.

[0092] 11, surface contour data is obtained from the OCT volume measurements. Importantly, the result of these steps is a reconstructed surface of the object measured by OCT. If reflected light image data and fluorescence image data are also captured during the scan, the OCT volume, reconstructed 3D surface, 2D reflected light image, and 2D / 3D fluorescence image are all linked together and registered to each other to identify the same view of the sample S.

[0093] The generated 3D surface is then stitched with a new 3D surface reconstructed from the scan data acquired in the new view using a matching method commonly known in the art, such as iterative nearest neighbor (ICP) merging. The spatial calibration data described above is also used in the stitching process. If stitching is successful, the spatial coordinates of each point on the imaged surface are determined, thus obtaining the correct spatial relationship between the different 3D surfaces. In this way, the association between the spatial positions of the various types of acquired data is maintained. Therefore, the OCT image data content and the multimodal image data content acquired by the scanner can be automatically registered without the need for additional steps.

[0094] The 3D surface data, OCT depth-resolved volume, 2D reflected light image, and 2D / 3D fluorescence image can be displayed, saved, or transmitted to another computer or storage device. According to exemplary embodiments of the present disclosure, these multimodal image data can be used to create and populate a dental chart, as described in more detail below. The results of the processing of the method of FIG. 11 can be directed to subsequent control logic processing for image and diagnostic analysis to generate the information necessary to populate a dental chart.

[0095] Depending on the application and imaging conditions, various image segmentation algorithms can be used to extract the object surface in the segmentation step 170 of the method of Figure 11. Image segmentation algorithms such as simple direct thresholding, active contour level set, watershed methods, supervised and unsupervised image segmentation, neural network-based image segmentation, spectral embedding, and max-flow / min-cut graph-based image segmentation are well known in the image processing field and can be utilized. These may be applied to the entire reconstructed 3D volume or to each 2D frame of OCT data individually.

[0096] 12A-12E illustrate various types of imaging content acquired and generated as part of the OCT processing method, using the example of an image of a tooth with severe caries. FIG. 12A shows a 2D slice corresponding to a B-scan of the OCT imaging. FIG. 12B shows a depth-encoded color projection of the tooth relative to an optional color bar 180. FIG. 12C shows a corresponding slice of a volume rendering obtained from the OCT image content. FIG. 12D shows the result of the segmentation process of FIG. 12A, in which points along the tooth surface have been extracted. FIG. 12E shows a tooth surface point cloud 64 generated from the OCT volume data. The surface point cloud 64 can be obtained from the OCT volume data following segmentation, as previously described with respect to the method of FIG. 11.

[0097] Mapping and analysis of OCT data As shown in the example images of FIGS. 12A-12E, the acquired depth-resolved OCT data also supports the reconstruction of the surface contours of features such as teeth and gums of the patient's dentition and supporting structures. Furthermore, the depth-resolved data itself can be correlated or mapped to the reconstructed surface contours. By correlating the depth-resolved data with spatial locations on the surface, the resulting dental chart can also be spatially correlated with the analysis or evaluation of lesions such as cavities, the integrity of temporary and permanent fillings, and the evaluation of various prosthetic devices, hardware, and implants. This correlation can also help provide an indication of the health of specific areas of the gums and other supporting and nearby tissues.

[0098] Surface contour imaging using reflected light Conventional surface contour imaging, unlike the aforementioned OCT imaging, uses reflected light imaging to provide data for characterizing a surface, such as surface structure, curvature, and contour features, but does not provide information about the material beneath the surface. Contour imaging data or surface contour image data can be obtained from a structured light imaging device or from an imaging device that obtains structural information about a surface from a sequence of 2D reflected light images acquired using visible light illumination, typically in the wavelength range above about 380 nm and below the 740 nm threshold, near-infrared light near or above 740 nm, or ultraviolet light below 380 nm. Alternative techniques for contour imaging include structured light imaging as well as known techniques for characterizing surface structure, such as triangulation-based feature tracking, structural-from-motion photogrammetry, time-of-flight imaging, interferometer-based imaging, and depth-from-focus imaging. Contour image content can alternatively be extracted from volumetric image content, such as OCT volume content (described above with respect to FIG. 11), by identifying and collecting only voxels representing surface texture.

[0099] The phrase "patterned light" is used to refer to light having a predetermined spatial pattern, such as light having one or more distinguishable parallel lines, curves, grid or checkerboard patterns, or other features having areas of light separated by areas of no illumination. In the context of this disclosure, the phrases "patterned light" and "structured light" are considered equivalent and are used together to identify light projected onto a subject to derive contour image data.

[0100] In structured light imaging, a pattern of lines or other structured patterns is projected from an imaging device onto the surface of an object at a given angle. The projected pattern is then viewed from the surface using triangulation to generate a contour image from a different angle to analyze the surface information based on the contour line overview. Phase shifting, which spatially shifts the projected pattern in incremental steps to take additional measurements at new locations, is commonly applied as part of structured light imaging to complete the contour mapping of the surface and increase the overall resolution of the contour image.

[0101] The multimodal imaging device of the present invention may include depth-resolved imaging or conventional surface contour imaging. For example, the TRIOS® dental intraoral scanner from 3Shape, Copenhagen, Denmark, performs surface contour imaging using a depth-from-focus technique and captures color reflected light images and tooth shades during scanning. The CEREC™ Omnicam™ system from Dentsply Sirona, Salzburg, Austria, performs surface contour imaging using a structured light triangulation technique and captures color reflected light images and tooth shades during scanning. International Patent Application No. US2014 / 070719, entitled "Intra-Oral 3-D Fluorescence Imaging," by Inglese et al., captures 3D surface contours using structured light triangulation, color reflected light images, and 2D / 3D fluorescence images.

[0102] 3D Arch Surface Reconstruction Reconstructing a 3D mesh corresponding to the complete arch is typically done by acquiring a series of slightly overlapping intraoral 3D surface views and stitching them together. The process of identifying which portions of the mesh being constructed the newly acquired views will overlap is called "matching" or "stitching." By employing matching methods well known in the art of surface contour imaging, intraoral 3D scanners can generate a 3D mesh of the patient's entire arch as the individual views are acquired.

[0103] As previously described with reference to Figure 11, multiple OCT measurements can be used alternately to perform 3D surface reconstruction following the same stitching process described above. The OCT data from successive scans is processed to identify surface content. As with the description in conjunction with Figures 5A and 5B, surface features are readily provided by identifying the outermost A-scan data for each B-scan and C-scan location.

[0104] Dental chart generation and input According to exemplary embodiments of the present disclosure, image content from any of the available imaging modes can be spatially associated with the surface reconstruction generated after a scan. The spatial location of any of the image data acquired in a scan is automatically determined as a result of successful stitching. The generated surface reconstruction maintains spatial coordinate associations with the image data showing the tissue responses in the various modes. Thus, for example, a given coordinate in the surface reconstruction can be associated with a specific reflected light imaging value, such as R, G, or B value, shade, fluorescence value, and OCT depth imaging content. Thus, the combined information from multiple imaging modes can be associated in coordinate space without requiring separate calibration, registration, or matching logic.

[0105] For example, reflected light image content and OCT scan content for a particular intraoral surface location can be associated with a location on a 3D arch surface reconstruction, which can then be used to create and populate a dental chart, maintaining spatial coordinate information associated with each corresponding region of the dental chart.

[0106] The flowchart in Figure 13 illustrates a method for generating a dental chart according to scan data from a scanner device capable of capturing multimodal image content, such as reflected light imaging, OCT scans, and fluorescence images. In a scanning step S1920, 3D contour data and other multimodal image data content are obtained. Next, in a reconstruction and stitching step S1930, a 3D surface contour is reconstructed using a subset of the scanned data. As part of step S1930, the reconstruction and stitching operation obtains spatial coordinate information associated with the image content from the multimodal imaging.

[0107] Continuing with the method of FIG. 13 , a segmentation step S1940 then performs segmentation and labeling of the teeth based on the spatial relationships of the surface points obtained in step S1930, using utilities and approaches well known to those skilled in the art of image reconstruction. A contour generation step S1950 generates contour data suitable for constructing a dental chart, such as providing line drawings of side and top views. An analysis step S1960 then analyzes the scan data of the teeth and associated gum structures and associates the analysis with elements of the dental chart. A display step S1970 presents the generated dental chart for display on a display or stores the chart for future reference or for transmission or communication to another computing device or data storage unit.

[0108] The generated chart can be updated with subsequent scans from the same multimodal scanner device. Because the scanner's internal spatial reference remains unchanged, 3D surfaces reconstructed from newly scanned data can be stitched to the existing dentition, thereby establishing the spatial location of the new multimodal image data. For any part of the teeth, existing and current analyses can be easily compared to note, for example, the presence or absence of fillings (e.g., amalgam, composite) or artificial teeth (e.g., crowns, bridges, veneers, dentures), the presence or absence of teeth, and changes in condition such as caries development, crack development, gum line recession, and enamel erosion.

[0109] Because it captures a certain amount of depth data, depth-resolved imaging data (e.g., OCT scan data or ultrasound or photoacoustic data) contains information that can be used to analyze the condition of teeth and gums. Ultrasound images are useful for detecting periodontal pocket depth. Reflected light image content can provide precise information about the contours and overall shape of teeth, as well as detailed surface features of teeth, gums, and other structures. Fluorescence image content is useful for detecting caries, mineralization, plaque, and dental prostheses. Each type of information can be used to input data into the generated dental chart, providing instant access to detailed data about specific features of the patient's dentition.

[0110] The presentation of detailed information can take a variety of forms. According to exemplary embodiments of the present disclosure, various types of information can be selectively enabled or hidden from view, as desired by the practitioner. This reduces clutter while still allowing for the association and preservation of important information accessible from the dental chart. Layered models can be adapted so that certain types of more specialized information are displayed as "overlays." Thus, for example, information about gum condition can be enabled separately from information about teeth, such as cavities, fillings, implants, or other features.

[0111] In addition to generating dental charts in standard view formats, exemplary embodiments of the present disclosure can provide some degree of manipulation of the standard views, for example, allowing for oblique views. Cross-sectional views (e.g., corresponding to the OCT B-scans mentioned above) can be displayed at points of interest. A progressive sequence of views can be displayed alternately, with the view angle adjusted based on cursor position. The coordinate locations of various features can also be displayed, allowing traditional "flat" 2D dental charts to include annotations describing the actual positions of teeth or other features.

[0112] 14A, 14B, and 14C illustrate various aspects of a generated and displayed dental chart according to an exemplary embodiment of the present invention. In FIG. 14A, a displayed dental chart 270 is shown. Individual entries in dental chart 270 may be linked to a surface contour image 276, shown in low-resolution or thumbnail format in FIG. 14A. Coordinate references 272 obtained as part of the scan may be provided on the display. Features such as hovering with a mouse or other pointer may provide both coordinate reference data and any associated image content related to the specified location.

[0113] By way of example, Figure 14B shows an expanded view of region E1 of dental chart 270, schematically illustrating several teeth 120. The relevant information for one of the teeth 120 includes a suspected caries area 122. Figure 14C includes additional information highlighting nearby portions of gum tissue 124 that may be bleeding or exhibiting other conditions. The display of various detected conditions of the teeth and gums can be individually enabled or disabled, as described above, allowing the practitioner to focus on conditions of particular interest or concern.

[0114] FIG. 14D shows an example of a generated dental chart 270 with a top view of the patient's dentition.

[0115] Figure 14E shows an example of a generated dental chart 270 with a perspective view of the patient's dentition. As shown in Figures 14D and 14E, a reference origin "R" can be displayed to provide reference data for the association of spatial coordinates stored in the reconstruction. Several thumbnail images 280 can be provided to show the various types of information available about the selected tooth or gum region.

[0116] According to exemplary embodiments of the present disclosure, the automatically generated graphical arrangement of dental chart data can be displayed in a standard dental chart format familiar to practitioners, or in a 3D or other format using tooth outlines generated according to the procedures described above. A toggle or other command can be provided for the practitioner to select the display format preferred by the practitioner in accordance with input operator instructions. Display format options can include a flat 2D alignment of a traditional dental chart, as shown in the example of FIG. 14A; a flat 2D plan or top view from a standard chart or from a patient image, as shown in the example of FIG. 14D; or a perspective 3D view of the upper or lower jaw, as shown in FIG. 14E. The various dental chart representations shown in the examples of FIGS. 14A-14E can be used to display pan, zoom, and 3D rotation of individual teeth, sets of teeth, or jaw structures. The location of the origin "R" can be adjusted with corresponding changes to other coordinates of the input dental chart.

[0117] Various dental chart configurations can be displayed. For example, the traditional dental chart configuration shown in FIGS. 14A-14C may be most preferred by experienced practitioners; this configuration is most easily usable and can utilize additional features that allow viewing different types of information available from the input dental chart. The alternative views of FIGS. 14D and 14E are highly useful for viewing specific information regarding the spatial arrangement of teeth and may be more useful in assessing how teeth are positioned relative to one another in the jaw. Different dental chart embodiments can be annotated, for example, allowing practitioners and staff to record various data during an examination or cleaning. FIG. 14D illustrates an annotation 282 configuration that may be displayed, for example, indicating the practitioner's observations or details detected in an automated analysis of the tooth's condition. Labels, such as tooth numbers, can be provided to track tooth position along with the displayed content, such as when rotating the 3D view of the dentition or when zooming or panning.

[0118] The flowchart in FIG. 15 illustrates a method including steps that may be performed to automate the process of dental chart creation using acquired multimodal image data, according to an exemplary embodiment of the present disclosure. Multimodal image content is acquired in an acquisition step S1510. The multimodal image content can be acquired using a single imaging signal, such as a visible light source with a spectral bandwidth that provides both reflected light imaging content and fluorescent light. Spatial coordinates are maintained during image acquisition, providing a mechanism for automatically associating image content resulting from multiple imaging modes. Alternatively, the acquisition step S1510 can use multiple output imaging signals, such as using optical signals of different bandwidths oriented along a common axis and acquired simultaneously or closely together. In a surface contour generation step S1520, the intraoral probe's control logic generates a surface contour using the acquired multimodal image content. Stitching logic is used to combine adjacent images while maintaining the association of spatial coordinates of the multimodal image content. Next, a tooth outline generation step S1530 is performed, in which the probe logic can generate tooth outlines for use in the dental chart. A dental chart setup step S1540 may then assemble a dental chart using the sequence of tooth outlines obtained in step S1530 to show the spatial order of the teeth and the supporting gingival tissue adjacent to the teeth. Each assembled element of the dental chart may then be associated with corresponding multimodal image content in a dental chart input step S1550. Step S1550 analyzes the multimodal image content and associates the results of this analysis with locations and features in the assembled dental chart. A display step S1560 then generates any data files necessary to render the dental chart on a display as well as store, transmit, or communicate the input dental chart to other systems.

[0119] In addition to image content acquired using intraoral imaging devices, exemplary embodiments of the present disclosure can also use image data obtained from extraoral imaging devices, such as bitewing and periapical x-rays or cone-beam computed tomography (CBCT) systems, as multimodal image content. Radiography-based imaging modes can be associated with the spatial coordinates of intraoral image content using various utilities for feature recognition. For example, volumetric images acquired using CBCT processing can be registered to the surface contours or surface meshes acquired by the intraoral imaging device using volumetric image registration methods commonly known in the art. Thus, x-ray and CBCT image content can be associated with dental charts for ready reference and display.

[0120] Advantageously, using exemplary embodiments of the present disclosure, spatial correlation of image content from multiple imaging modes can be easily achieved. No processing delays are required for separate registration processing of image content acquired from scanner 314 (FIG. 1). Surface reconstruction coordinates can be associated with specific reflected light imaging values, such as R, G, B values, shade, fluorescence values, and OCT depth imaging content, as well as X-ray and CBCT image content. Thus, the combined information of multiple imaging modes can be inventively associated with intraoral coordinate space without requiring separate calibration, alignment, or matching logic and the processing overhead required to achieve registration of different types of image data relative to a patient's dentition.

[0121] Utility and workflow updates The ability to directly spatially correlate image data from different modalities provides many options and features that help improve the update process. As mentioned above, it may be useful to image a single tooth or a small portion of the jaw during treatment using the same or another calibrated multimodal intraoral scanner periodically at various stages of implant preparation or to track the progress of a suspected caries condition. Exemplary embodiments of the present disclosure support updates in several ways, including:

[0122] (i) Dental Chart Generation and Update Options. As one option, the practitioner may choose to generate a new dental chart, such as by performing a full scan of the intraoral surfaces and repeating the chart generation process described above. By default, the previously acquired chart and associated image content may be saved or archived to maintain historical patient data when a new chart is generated. Alternatively, the practitioner may choose to update an existing chart with new information specific to one or more teeth or intervening tissue regions, or new image content associated with a particular imaging mode. This type of partial update requires stitching a new surface reconstruction to the surface of the existing dentition, thereby mapping the new content associated with the new surface reconstruction to coordinates in the existing dental chart. Techniques that support stitching a newly reconstructed surface to an existing reconstructed surface are well known to those skilled in the imaging arts. The dental chart update option may be fully automated or may benefit from some means of operator input, such as a touchscreen, mouse, or keyboard input, where the operator roughly identifies the location of the probe and the teeth to be scanned.

[0123] (ii) Localized highlighting related to update location and / or content. As previously mentioned, the practitioner may have the ability to select the display format of the dental chart, such as showing a traditional 2D array view (FIGS. 14A-14C), a plan or top-view schematic view (FIG. 14D), or a perspective 3D view (FIG. 14E). To support update scanning operations with the intraoral probe or scanner 314 (FIG. 1), any of several highlighting schemes can be selected. FIG. 16 shows highlighting 330 placed on a perspective view of the dental chart, indicating the current position of the scanner 314 relative to the patient's entire dentition for newly acquired image content. This type of visual operator feedback can help the practitioner or technician ensure that the appropriate areas for the update are being imaged. Other types of highlighting can be useful to indicate the various imaging modes in which data is acquired. For example, color coding can be used to indicate which types of image content are available or currently being updated, such as orange for OCT image content and green for fluorescent content. Color or other highlighting methods may also be used to indicate processing status. Boundary coordinates to relate the scanned area to the overall dental chart and origin "R" may also be displayed if useful to the technician or practitioner.

[0124] (iii) Chart-Guided Updates. According to an exemplary embodiment, a markup utility is provided to identify areas requiring image content updates through entries made in the dental chart. Using this scheme, electronically generated and populated dental charts can become a convenient tool for improving communication and advancing the workflow of dental treatment teams. For example, a practitioner may request updated image content for the gingival tissue between teeth 8-11 and teeth 2-5. The practitioner can electronically mark the dental chart by entering operator instructions, such as with a mouse or touchscreen. The schematic diagram in FIG. 17 shows an example of touchscreen instruction entry identifying specific teeth or other features, such as with a simple outline as shown. A pop-up menu 290 can provide selections that allow the practitioner to specify the imaging mode of the scan information to be acquired. The same or another calibrated multimodal intraoral scanner can be used to perform the update process without any other detailed positioning sequence. The technician can operate the probe or scanner 314 to scan the tooth of interest and receive on-screen or audible operator feedback indicating that some or all of the information requested from the practitioner's entry has been acquired and updated. This sequence simplifies the acquisition and processing of information without the need for written or verbal instructions and the resulting risk of confusion.

[0125] (iv) Time-lapse display using "history" content saved in one or more modes. As part of the update process, previously saved image content from one or more imaging modes can be retained for future analysis and display. Thus, for example, the progression of caries status in one or more teeth over time can be easily observed and displayed in fluorescent 2D images or OCT cross-sectional (B-scan) images, with successive saved image content registered and displayed in chronological order, following the familiar model of time-lapse video. Temporal information associated with each saved image can also be displayed as an aid in tracking the status and rate. A selection button or other prompt input tool on the input dental chart allows for the selection and "playback" of previously saved views, including the ability to specify the specific mode of image content to be displayed.

[0126] Although the invention has been described in detail with particular reference to exemplary embodiments thereof as currently understood, it is to be recognized and understood that changes and modifications may be effected within the spirit and scope of the invention.

[0127] For example, control logic processor 340 may be any of a number of types of logic processing devices, including, but not limited to, a computer or computer workstation, a dedicated host processor, a microprocessor, a digital signal processor, a logic array, or other device that executes stored program logic instructions.

[0128] The presently disclosed exemplary embodiments are therefore considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the appended claims, and all changes that come within the meaning and range of equivalency are intended to be embraced therein.

[0129] Consistent with at least one exemplary embodiment, an exemplary method / apparatus may employ a computer program having stored instructions that operate on image data accessed from an electronic memory. As will be appreciated by those skilled in the image processing arts, the computer program of the exemplary embodiments herein may be utilized by a suitable general-purpose computer system, such as a personal computer or workstation. However, many other types of computer systems may be used to execute the computer program of the described exemplary embodiments, including, for example, single or networked processor arrangements.

[0130] A computer program for executing the methods of certain exemplary embodiments described herein may be stored on a computer-readable storage medium. This medium may be composed of, for example, but not limited to, a magnetic storage medium such as a magnetic disk, a hard disk, a removable device, or a magnetic tape; an optical storage medium such as an optical disk, an optical tape, or a machine-readable optical encoding; a solid-state electronic storage device such as a random access memory (RAM) or a read-only memory (ROM); or other physical device or medium adapted to store a computer program. A computer program for executing the exemplary methods of the described embodiments may also be stored on a computer-readable storage medium connected to an image processing device via the Internet or other network or communication medium. Those skilled in the art will further readily recognize that the equivalent of such a computer program product may also be constructed in hardware.

[0131] It should be noted that the term “memory” is equivalent to “computer-accessible memory” in the context of this disclosure and may refer to any type of temporary or more permanent data storage workspace accessible to a computer system, including, for example, a database, used to store and manipulate image data. Memory can be non-volatile, using long-term storage media such as magnetic or optical storage. Alternatively, memory can be more volatile in nature, using electronic circuitry such as random access memory (RAM) used by a microprocessor or other control logic processor device as a temporary buffer or workspace. For example, display data is typically stored in a temporary storage buffer that may be directly associated with a display device and is periodically refreshed as needed to provide the display data. This temporary storage buffer can also be considered a memory, as the term is used in this disclosure. Memory is also used as a data workspace for performing or storing intermediate and final results of calculations and other processing. Computer-accessible memory can be volatile, non-volatile, or a hybrid combination of volatile and non-volatile.

[0132] It is understood and appreciated that computer program products according to example embodiments herein can utilize various well-known image manipulation algorithms and / or processes. It is further appreciated and understood that example computer program product embodiments herein can embody algorithms and / or processes useful for implementation that are not specifically shown or described herein. Such algorithms and processes may include conventional utilities that are within the ordinary skill in the art of image processing. Such algorithms and systems, as well as additional aspects of hardware and / or software for generating and otherwise processing images or for cooperating with the computer program products of the present application, are not specifically shown and can be selected from such algorithms, systems, hardware, components, and elements known in the art.

[0133] Exemplary embodiments according to the present disclosure may include various features described herein individually or in any combination.

[0134] While this application has been described with respect to one or more implementations, changes and / or modifications can be made to the illustrated examples without departing from the scope and spirit of the appended claims. In addition, while a particular feature of the present invention may be disclosed with respect to only one of several implementations / exemplary embodiments, such feature can be combined with one or more other features of other implementations / exemplary embodiments, as may be desirable and advantageous for any given or specific function. The terms "a" or "at least one" are used to mean that one or more of the listed items can be selected. The term "about" indicates that a stated value can be varied somewhat unless such variation results in a process or structure being incompatible with the illustrated exemplary embodiment. Other embodiments of the present invention will be apparent to those skilled in the art from consideration of the specification and embodiments of the invention disclosed herein. It is intended that the specification and examples be considered exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

Claims

[Claim 1] 1. A method for intraoral imaging, the method comprising: (a) generating one or more output imaging signals from an intra-oral probe; (b) acquiring multimodal image content from each of a plurality of intraoral surface locations of the patient's dentition according to tissue response from the one or more imaging signals, and associating spatial coordinates with the acquired multimodal image content; (c) generating a surface contour of the patient's dentition by reconstructing and stitching from a data subset of the acquired multimodal image content, and maintaining the association between spatial coordinates of the multimodal image content and the stitched surface contour; (d) generating one or more tooth contours from the generated surface contours and arranging the generated contours as a dental chart representing the spatial order of the one or more teeth and the spatial order of supporting gum tissue adjacent to the teeth; (e) inputting data into the dental chart by analyzing the acquired multimodal image content and presenting analysis results at one or more locations in the dental chart according to the stored spatial coordinate associations; (f) displaying the input dental chart; and The method includes the steps of:

Images