Dental Shade Matching for Multiple Anatomical Regions

Abstract

A shade matching method for teeth obtains a plurality of digitized tooth shade standards, obtains a three-dimensional (3D) surface representation of the teeth and associated color information defined on the three-dimensional (3D) surface, selects regions on the three-dimensional (3D) surface representation of the teeth, determines a plurality of correspondences of all the tooth shade standards with a common tooth shape, calculates a color difference for each of the tooth shade standards from the plurality of tooth shade standards, and records a label corresponding to the shade standard having the smallest difference.

Description

[Technical field]

[0001] The present disclosure relates generally to methods and apparatus for three-dimensional (3D) dental intraoral scanning, and more particularly to methods and apparatus for shade matching for multiple anatomical regions in three-dimensional (3D) dental intraoral scanning. [Background technology]

[0002] Accurate modeling of tooth color shades is especially valued for patient appearance and satisfaction in restorative dentistry. In conventional practice, dental technicians or practitioners manually estimate tooth color and overall appearance using a set of shade tabs. Tooth color shade tabs are often provided by restorative material suppliers and represent the available shades or color ranges available for a filling, crown, implant, bridge, or another restorative feature. However, even with an extensive set of dental shade tabs, color shade determination can be difficult to perform and subject to human error. The overall accuracy of dental shade determination often depends on the practitioner's relative experience and aesthetic sense.

[0003] In recent years, digital shade tabs have been created for use in shade value matching. Digital shade tabs have been used in conjunction with digital models of the patient's teeth in various ways to automate the process of determining tooth shade. Spectrophotometers, including the SpectroShade Micro II and VITA Easyshade spectrophotometers, have also become available for obtaining dental shade value measurements. With some devices, such as the SpectroShade Micro II spectrophotometer, shade information is taken and analyzed based on a two-dimensional (2D) image. With other devices, such as the VITA Easyshade spectrophotometer, shade information is taken within an area defined by the position of the device on the tooth.

[0004] Other devices, such as three-dimensional (3D) dental scanners, have become available with the ability to measure dental shades. When these devices are used, a comparison step between the area on the patient's tooth surface that the device is targeting and the digitized shade information is performed by computer software. The digitized shade information may be taken from a shade tab, such as a VITA shade guide. A shade tab typically has a color profile that varies across the tooth surface, thereby representing a natural color gradation between the dentin and enamel areas. Generally, one shade value may be represented by a particular area of ​​the shade tab, such as the central area representing the dentin color. As an example, for a Vita Classical shade tab, if the desired location is typically near the "X" mark 115 as shown in the example of FIG. 9, it represents a shade tab with areas of different shade values. Near the "X" mark 115 in FIG. 9, the shade value is correctly identified as A2 for this A2 shade tab. However, further away from the "X" mark, the enamel contains less colored pigment and therefore appears more similar to the "X" mark in shade tab D2, so the shade may be detected as D2. In this example, one shade value is represented only by a specific area of ​​the shade tab, and since the shade tab is not uniform, the shade values ​​for the different areas will not be the same. This is a source of confusion for the technician who has to see the shade appearance near the "X" mark for the D2 shade tab and apply it to the enamel, which is a different location. Thus, a detection device and method is required such that for the A2 shade tab, all the different areas are detected as A2. This would give the technician a clear instruction that "I just need to reproduce the color gradation of the A2 shade tab."

[0005] 14 is a schematic diagram showing the shade distribution in one shade tab with shade variation from dentin to enamel. Since the mesial / distal / central parts of the mid / cervical / gingival parts usually have different appearances, an experienced dentist may measure the corresponding anatomical regions between the patient's teeth and the shade tab.

[0006] Figures 10A and 10B are schematic diagrams depicting various anatomical mappings between regions of a patient's teeth and shade tabs. As shown in Figure 10A, the mesial incisal edge of the patient's teeth is compared to the central region of the shade information digitized from the A2 shade tab, as described above. Figure 10B depicts another correspondence between the teeth and the shade tabs that compares to equivalent anatomical regions, which is a more natural way of defining the correspondence.

[0007] Therefore, a need exists in the industry for a tooth shade matching method and apparatus that takes into account the correspondence between a digital three-dimensional representation of a tooth and a shade tab for an anatomical region of the tooth, and that addresses this and other problems, difficulties, deficiencies, and shortcomings of the present method and apparatus. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] U.S. Patent No. 9,295,532 [Patent Document 2] U.S. Patent No. 9,349,182 [Non-patent literature]

[0009] [Non-Patent Document 1] Rusu, RB, "Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments", Artificial Intelligence, German Society for Computer Science, August 17, 2009, pp. 57-61. [Non-Patent Document 2] R. Montes and C. Urena, "An Overview of BRDF Models", Technical Report LSI-2012-001, University of Granada, 2012. [Non-Patent Document 3] M. Langovoy, "Novel metric for analysis, interpretation and visualization of BRDF data," Transaction on Spatial Algorithms and Systems, 2014. Summary of the Invention [Means for solving the problem]

[0010] The present invention comprises a tooth shade matching method and apparatus that considers the correspondence between the digital three-dimensional representation of the teeth and the shade tabs for the anatomical regions of the teeth. According to one aspect of an exemplary embodiment of the present invention, the tooth shade matching method comprises the steps of: acquiring a plurality of digitized tooth shade references, each tooth shade reference corresponding to a common tooth shape, having a label, and including color information defined over the tooth shape; acquiring a three-dimensional surface representation of the teeth and associated color information defined on the three-dimensional surface; selecting an area on the three-dimensional surface representation of the teeth; determining a plurality of correspondences of all the tooth shade references with the common tooth shape, the plurality of correspondences connecting points from the same anatomical region between the selected area on the three-dimensional surface representation of the teeth and the common tooth shape; calculating a color difference for each of the tooth shade references from the plurality of tooth shade references, each color difference being calculated by comparing the color information from the tooth shade reference with the associated color information from the selected area on the three-dimensional surface representation of the teeth, the comparison of the color information using the plurality of correspondences; and recording the label corresponding to the shade reference with the smallest difference.

[0011] According to another aspect of an exemplary embodiment of the present invention, an intra-oral imaging device includes an image capture device for acquiring a plurality of digitized tooth shade references, each tooth shade reference corresponding to a common tooth shape and having a label and including color information in the tooth shape; another image capture device for acquiring a three-dimensional surface representation of the teeth and associated color information defined on the three-dimensional surface; and a processor programmed with processing logic instructions for: selecting an area on the three-dimensional surface representation of the teeth; determining a plurality of correspondences of all the tooth shade references with the common tooth shape, the plurality of correspondences connecting points from the same anatomical area between the selected area on the three-dimensional surface representation of the teeth and the common tooth shape; calculating a color difference for each of the tooth shade references from the plurality of tooth shade references, each color difference being calculated by comparing color information from the tooth shade reference with associated color information from the selected area on the three-dimensional surface representation of the teeth, the comparison of the color information using the plurality of correspondences; and recording a label corresponding to the shade reference having the smallest difference.

[0012] According to yet another aspect of an exemplary embodiment of the present invention, the intra-oral imaging device includes an image capture device for acquiring a plurality of digitized tooth shade references, each tooth shade reference corresponding to a common tooth shape and having a label and including color information in the tooth shape, the image capture device acquiring a three-dimensional surface representation of the teeth and associated color information defined on the three-dimensional surface, and a processor programmed with processing logic instructions for: selecting an area on the three-dimensional surface representation of the teeth; determining a plurality of correspondences of all the tooth shade references with the common tooth shape, the plurality of correspondences connecting points from a same anatomical region between the selected area on the three-dimensional surface representation of the teeth and the common tooth shape; calculating a color difference for each of the tooth shade references from the plurality of tooth shade references, each color difference being calculated by comparing color information from the tooth shade reference with associated color information from the selected area on the three-dimensional surface representation of the teeth, the comparison of the color information using the plurality of correspondences; and recording a label corresponding to the shade reference having the smallest difference. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a schematic diagram illustrating an imaging device for shade matching according to an exemplary embodiment of the present invention. [Diagram 2] FIG. 1 illustrates a Lambertian sphere with uniform optical brightness. [Diagram 3] 1A-1D are schematic diagrams depicting an intra-oral imaging probe in different positions relative to the teeth. [Figure 4A] FIG. 1 is a schematic diagram showing the basic geometric structure for Bidirectional Reflectance Distribution Function (BRDF) variables. [Figure 4B] FIG. 2 is a schematic diagram illustrating the definition of polar angles and how measured reflectance data for a surface portion is generally represented. [Diagram 5] 1 is a flowchart of a method for obtaining and using BRDF data for improved color shading identification and display, according to an exemplary embodiment of the present invention. [Figure 6]6 is a flow chart of a method for creating a BRDF library for use in the method of FIG. 5. [Figure 7] A diagram showing a collection of shade tabs with their corresponding BRDF snapshots. [Figure 8] FIG. 1 is a schematic diagram showing the BRDF grid, the BRDF snapshot, and the growing 3D surface contour in the same coordinate system. [Figure 9] FIG. 2 is a schematic diagram illustrating a shade tab having regions of different shade values ​​defined by a BRDF. [Figure 10A] 1A-1C are schematic diagrams illustrating corresponding anatomical regions between a patient's teeth and shade tabs in various approaches. [Figure 10B] 1A-1C are schematic diagrams illustrating corresponding anatomical regions between a patient's teeth and shade tabs in various approaches. [Figure 11] 1 is a flowchart of a shade matching method according to an exemplary embodiment of the present invention. [Figure 12] 4 is a flow chart of a method for calculating shade distance for a plurality of reference shade distributions according to an exemplary embodiment of the present invention. [Figure 13] 1 is a flowchart of a method for calculating the distance between a scanned shade distribution and a reference shade distribution using anatomical mapping according to an exemplary embodiment of the present invention. [Figure 14] FIG. 2 is a schematic diagram showing the shade distribution in one shade tab having a shade variation from dentin to enamel. [Figure 15] FIG. 1 is a schematic diagram representing all shade tabs from a shade guide, all of which have a common tooth shade, albeit with different shade distributions, and may be represented within the same coordinate system. [Figure 16] 1A-1D are schematic diagrams depicting various examples of three-dimensional regions on a tooth. [Figure 17] FIG. 1 is a schematic diagram depicting anatomical mapping illustrated for coarse spatial resolution of shade distribution according to an exemplary embodiment of the present invention. [Figure 18]FIG. 1 is a schematic diagram depicting anatomical mapping illustrated for fine spatial resolution of shade distribution according to an exemplary embodiment of the present invention. [Figure 19] FIG. 2 is a schematic diagram illustrating an anatomical coordinate system defined for dental reference surfaces according to an exemplary embodiment of the present invention. [Figure 20] FIG. 20 is a schematic diagram illustrating an anatomical coordinate system mapped onto the teeth from the tooth reference surfaces of FIG. 19 according to an exemplary embodiment of the present invention. [Figure 21] 1 is a flowchart of a shade matching method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] In the following detailed description of exemplary embodiments of the present invention, reference is made to the drawings in which like reference numerals identify like elements or steps throughout the several views. When used in the context of this disclosure, the terms "first," "second," and the like do not necessarily indicate any sequential, sequential, or preferential relationship, but are merely used to more clearly distinguish one step, element, or collection of elements from another, unless otherwise specified.

[0015] In the context of this disclosure, the term "energizable" refers to a device or collection of components that performs a stated function upon receiving electrical power, and optionally, upon receiving an enabling signal.

[0016] For purposes of this disclosure, the terms "observer," "operator," and "user" are considered equivalent and refer to a viewer, technician, or another human being who views and processes an image, such as a dental image, on a display. "Operator commands" or "observer commands" may be obtained from explicit commands entered by the observer, for example, by clicking a button on a camera, using a computer mouse, or by touch screen or keyboard entry.

[0017] In the context of this disclosure, the phrase "in signal communication" indicates that two or more devices and / or components can communicate with each other via signals traveling over some type of signal path. The signal communication may be wired or wireless. The signals may be communication, power, data, control, or energy signals. The signal path may include physical, electrical, magnetic, electromagnetic, optical, wired, and / or wireless connections between a first device and / or component and a second device and / or component. The signal path may also include additional devices and / or components between the first device and / or component and the second device and / or component.

[0018] For the purposes of this disclosure, the term "set" refers to a non-empty set, as the concept of a collection of components or a set of elements is widely understood in basic mathematics. The terms "subset" or "some subset", unless otherwise specified, are used herein to refer to a non-empty proper subset, i.e., a subset of a larger set having one or more elements. For a set S, the subset may be composed of the complete set S. However, a "proper subset" of set S is strictly contained within set S and excludes at least one element of set S.

[0019] In the context of this disclosure, "surface data" may include surface contour characterization in the form of a point cloud, polygonal mesh, or another surface representation or image. The surface data may include a diagram of the surface contour, such as a dynamically growing surface contour actively assembled by an imaging system during data collection, and may include a low-resolution image representing the surface contour. The surface data may alternatively be generated as part of a refinement step, as a result of additional data acquisition or calculations. High-resolution surface data is typically required by engineers for precision design of repair devices. According to one exemplary embodiment of the present invention, the surface data may include two-dimensional (2D) surface information extracted from acquired three-dimensional (3D) scan data.

[0020] Intraoral Scanner 1 is a schematic diagram depicting an imaging device 90 for three-dimensional (3D) scanning according to an exemplary embodiment of the present invention. A camera 16, typically a handheld digital camera, a color depth camera, a handheld 3D scanner, or an intraoral 3D scanner, is scanned through the oral cavity of a patient 14 to obtain a set of image data including reflected images and associated depth information. A control logic processor 80 in signal communication with the camera 16 obtains the image data from the camera 16, processes the reflected images to assign color to the associated depth information, and calculates the placement of the color and associated depth information on the growing surface contour. If the placement can be successfully calculated, the control logic processor 80 generates an updated growing surface contour and shade label 92. The control logic processor 80 combines the growing surfaces, typically provided as point clouds, and renders the surfaces and the resulting shade labels on a display 84 in signal communication with the processor 80.

[0021] Surface Data and Color Assignment 1, the camera 16 acquires a set of image data including reflected images and associated depth information. A control logic processor 80 processes the image data to provide a local shade indication for the surface contour.

[0022] In one exemplary embodiment, the reflected images may be combined to form a color image, where the number of color channels is three or more. A color vector of dimensions matching the number of color channels may be used to store color values ​​in the combined color image. In one exemplary embodiment, the sensor is monochrome and requires multiple exposures with illumination of different central wavelengths. In another exemplary embodiment, the color values ​​are stored in an RGB color vector. Another equivalent three-dimensional (3D) color space may alternatively be used. For example, for color distance, it may be advantageous to use the CIELAB three-dimensional (3D) color space, which has been designed to provide perceptually uniform color distance according to the sensitivity of the eye. In another exemplary embodiment, if a spectrum of color channels with four or more central wavelengths is acquired, the corresponding color values ​​are stored in a color vector.

[0023] In one exemplary embodiment, the associated depth information is also obtained by processing the reflected image using structured illumination, such as that described in commonly assigned U.S. Patent No. 9,295,532 entitled "3D Intraoral Measurements Using Optical Multiline Method" and U.S. Patent No. 9,349,182 entitled "3D Intraoral Measurements Using Optical Multiline Method," both of which are issued to Milch and are incorporated herein by reference in their entireties. In another exemplary embodiment, the associated depth information may be obtained using any of the following techniques, including but not limited to triangulation using a projected pattern, depth from focus using confocal geometry, or active or passive stereogrammetry.

[0024] In practice, the reflected image and associated depth information are acquired as fast as possible to minimize scanner displacement during the sequence of camera exposures as the image data is generated. Following this processing, the associated depth information may be related to pixel locations from the reflected image, also called a distance image, and colors may be assigned to the distance image. Pixel data from the distance image may be converted to 3D point coordinates.

[0025] Shade Matching Method (Prior Art) Recently, shade information is generally obtained and analyzed based on two-dimensional (2D) texture images. Figure 21 is a flow chart of the shade matching method.

[0026] In step S2110, a texture image is identified.

[0027] In step S2120, a mapping from the three-dimensional (3D) surface onto the texture image is determined.

[0028] In step S2130, for each triangle on the three-dimensional (3D) surface, find the associated color information obtained during the scan and draw the corresponding triangle on the texture image accordingly.

[0029] In step S2140, the mapping from the three-dimensional (3D) surface to texture coordinates is stored in the mesh characteristic (u,v) texture coordinates for each vertex.

[0030] Since only the correspondence between three-dimensional (3D) surface vertices and texture coordinates is stored, the texture image must be preserved and accessible to visualize the three-dimensional (3D) surface or shade information measurements.

[0031] Shade matching for multiple anatomical regions FIG. 11 is a flow chart illustrating a shade matching method according to an embodiment of the present invention.

[0032] Step S1110 In step S1110, a three-dimensional surface representation of the tooth defined on a three-dimensional surface and associated color information is obtained. This provides a three-dimensional (3D) surface and associated shade distribution across the surface. The shade distribution is spatially resolved across the surface. The resolution depends on the type of shade data used. According to one exemplary embodiment of the invention, the shade distribution is texture data, i.e. an image with a resolution of about 50 μm. The texture is a spatial distribution providing color values ​​at any location on the tooth surface. According to another exemplary embodiment of the invention, the shade distribution is a BRDF cloud with a resolution between 0.5 mm and 1 mm.

[0033] Many commercially available intraoral 3D scanners, such as the CS3600 model from Carestream Dental LLC, may be used just to obtain the individual three-dimensional (3D) surface representations of the teeth. In this step, besides the three-dimensional (3D) surface representation, the associated color information defined on the three-dimensional (3D) surface needs to be obtained as well. The associated color information may be measured by the same intraoral scanner during the same scanning operation to capture the three-dimensional (3D) geometric structure data of the surface. Indeed, the color information may be acquired or input separately and then linked to the reconstructed three-dimensional (3D) tooth surface positions. For example, some spectrophotometers, such as the SpectroShade Micro II and VITA Easyshade spectrophotometers, are used to acquire the color information of the tooth surface, and then their shade values ​​are manually input or linked to the individual positions of the three-dimensional (3D) surface. According to an exemplary embodiment of the present invention, the linking process between dental shade values ​​and three-dimensional (3D) surfaces may also be performed automatically by mapping two-dimensional (2D) images of tooth surfaces linked with dental shade values ​​onto the corresponding three-dimensional (3D) surfaces.

[0034] Color information, meaning the basic colors acquired by the scanner, may be represented or recorded in different means or formats. One typical means is using color values ​​acquired from a two-dimensional (2D) image corresponding to three-dimensional (3D) surface locations in the form of RGB, CIELAB XYZ (defined by the International Commission on Illumination), or the visible spectrum.

[0035] According to an exemplary embodiment of the present invention, obtaining a 3D surface representation of the teeth and associated color information defined on the three-dimensional (3D) surface is performed using an intraoral scanner. According to an exemplary embodiment of the present invention, the associated color information defined on the three-dimensional (3D) surface comprises red, green and blue color data values. According to an embodiment of the present invention, the associated color information defined on the three-dimensional (3D) surface comprises CIELAB color data values. According to an exemplary embodiment of the present invention, the associated color information defined on the three-dimensional (3D) surface comprises a spectrum having wavelengths covering visible wavelengths.

[0036] Step S1120 In step S1120, a region on the three-dimensional (3D) surface representation of the tooth is selected. This is a selection or segmentation task that may be manual, semi-automatic, or automatic. Figure 16 is a schematic diagram showing various examples of three-dimensional (3D) regions on a tooth.

[0037] For manual processing, the selection manipulated by the user may be an area defined by a polygon or a circle.

[0038] For semi-automatic selection, the user first selects multiple points on the three-dimensional (3D) tooth surface, and then a computational algorithm will help fit an initial polygon onto the tooth contour. This is achieved first by attracting the contour towards areas of small radius of curvature. This optimization of the contour follows a strategy similar to limit line detection used in dental applications. According to one exemplary embodiment of the present invention, the algorithm is path following, where user clicks are control points, which follow the path of minimum radius of curvature. Regularization may be applied to prevent and maintain the contour from being overly deformed. Control points may be added or removed, and the contour is updated accordingly. Surfaces within the closed contour become the selected region.

[0039] For automatic selection, only minimal user action is required, such as a single click to define the target tooth. Control points may then be automatically placed radially when the radius of curvature of the surface reaches a predefined threshold or when the surface normal forms an angle with the screen normal that exceeds a predefined threshold. A path following algorithm may then determine a closed contour to define the selected region. According to one exemplary embodiment of the present invention, the closed contour may be automatically cut into a plurality of three-dimensional (3D) regions, which generates a section of the surface contained within the closed contour. The plurality of three-dimensional (3D) regions may be a cervical third, a middle third, and an enamel third. According to another exemplary embodiment of the present invention, the plurality of three-dimensional (3D) regions may be defined by a 3×3 grid on the tooth surface. Each three-dimensional (3D) region from the plurality of three-dimensional (3D) regions is then processed.

[0040] According to an embodiment of the present invention, selecting a region on a three-dimensional (3D) surface representation of a tooth involves a segmentation process, which may be manual, semi-automatic, or automatic.

[0041] Step S1130 In step S1130, a plurality of correspondences of all the tooth shade references with the common tooth shape are determined. The plurality of correspondences connect points from the same anatomical region between the selected region on the three-dimensional (3D) surface representation of the tooth and the common tooth shape. That is, each of the correspondences is a mapping from the selected region on the three-dimensional (3D) surface representation of the tooth to the same anatomical region on the common tooth shape of the shade references. According to an exemplary embodiment of the present invention, each of the shade references is a shade tab from a shade guide, so all the tooth shade references must have a common tooth shape. FIG. 15 is a schematic diagram showing that all the shade tabs from a shade guide have a common tooth shape but have different shade distributions and may be represented in the same coordinate system.

[0042] Typically, the digitized tooth shade references may be acquired in advance, i.e., before the scanning operation on the patient's teeth. Of course, the digitized tooth shade references may also be performed after the scanning operation on the patient's teeth. According to an exemplary embodiment of the present invention, acquiring the plurality of digitized tooth shade references is performed before or after step S1110 of acquiring a three-dimensional (3D) surface representation of the teeth and associated color information defined on the 3D surface.

[0043] Each tooth shade standard, i.e., each shade tab from the digitized shade guide, corresponds to a common tooth shape, has a label, and includes color information defined across the tooth shape. Typically, all shade tabs have the same shape resembling human incisors, for which the correct shade value is most important, since they are revealed when smiling. This common tooth shape aids in the method for determining multiple corresponding anatomical regions between the three-dimensional (3D) dental teeth and the common tooth shape of the shade standard. The same anatomical regions may be described using common dental terms such as gingival, cervical, central, incisal, mesial, distal, interdental, coronal, enamel, dentin, pulp, translucent, lingual, buccal, occlusal, etc.

[0044] 17 is a schematic diagram illustrating an anatomical mapping shown for a coarse spatial resolution of the shade distribution according to an exemplary embodiment of the present invention. The density of the correspondences is approximately equal to the number of regions of the coarse shade distribution covered by the three-dimensional (3D) region. The user may easily draw those correspondences. The coarse number of correspondences will typically range between 1 and 20.

[0045] FIG. 18 is a schematic diagram showing an anatomical mapping shown for fine spatial resolution of the shade distribution according to an embodiment of the present invention. Correspondences between pixels may be extrapolated from a coordinate system defined by only a few control points. In this case, the user of the algorithm would place a few control points (as indicated by crosses) to define a similar coordinate system on the respective surface. Correspondences would then be sampled from those coordinate systems using interpolation between the control points. FIG. 18 merely represents a series of correspondences for illustration purposes. Other correspondences are generated by moving this series sideways to cover the entire three-dimensional area defined by the control points. Dense correspondences require automatic mapping and would typically correspond to a set with more than 20 correspondences. The number of correspondences may be as high as 10,000 or even higher.

[0046] A coordinate system may also be defined for the shade reference surface and mapped onto the tooth surface. FIG. 19 is a schematic diagram of an exemplary anatomical coordinate system defined for the shade reference surface according to an exemplary embodiment of the present invention. FIG. 20 is a schematic diagram of an exemplary anatomical coordinate system mapped onto the tooth from the tooth reference surface shown in FIG. 19 according to an exemplary embodiment of the present invention. The projective transformation used in this example transforms the original rectangle from FIG. 19 into the polygon of FIG. 20. A projective transformation is a mapping between two coordinate systems. More generally, the mapping between coordinate systems is an injective function that takes three-dimensional (3D) coordinates and provides corresponding three-dimensional (3D) coordinates everywhere inside the three-dimensional (3D) region. Sometimes, three-dimensional (3D) points on the surface of the shade tab are mapped onto empty space on the tooth surface because the geometry is not exactly the same. In this case, their correspondence does not hold.

[0047] According to an exemplary embodiment of the present invention, a BRDF library represents a collection of reference BRDFs and their associated shade labels (details of BRDF libraries will be presented in another section). A BRDF library may be recorded directly in production and reused during scanning of the patient's teeth. A BRDF library may also be generated by a practitioner using a preferred shade tab. A BRDF library is a collection of shade labels, each of which has its associated shade distribution, which is represented by a spatially resolved BRDF distribution. The distance between two BRDF distributions, or the distance between the two BRDF distributions, is described in more detail in another section of this document.

[0048] According to an exemplary embodiment of the present invention, determining the correspondences is a manual, automatic, or semi-automatic process. For manual processing, a user determines and selects the same anatomical region on the common tooth shape using a computer mouse or touch screen. For automatic processing, there are several optional steps, including (a) a three-dimensional (3D) surface representation of the teeth is segmented from the gingiva using color and morphological operations, and / or (b) an injective transformation is used to map from the selected region on the three-dimensional (3D) surface representation of the teeth to a region on the common tooth shape. The transformation is determined by the surface of the three-dimensional (3D) surface representation of the teeth and the shade reference, and typically a bijective space transformation will be suitable, although interpolation may be used if necessary.

[0049] For semi-automatic processing, the user operates further adjustments of the region provided by the automatic process, and the adjustments include further modifications to the position, scale, or orientation of the region provided by the automatic process. For example, scrolling a computer mouse wheel may provide the scale adjustment, and a combination of computer mouse clicks and movements may provide changes in orientation, position, or rotation. According to one exemplary embodiment of the present invention, the user will use the mouse to extend the surface common to all shade tabs into the selected three-dimensional (3D) region, either with some transparency or with an overlay on the screen. The transparency or overlay is present to help the user estimate when the extension is satisfactory. Or, conversely, extend the selected three-dimensional (3D) region from the three-dimensional (3D) tooth surface into the surface common to all shade tabs. The correspondence may then be generated using an injective transformation by sampling it. Some samples may not be on the tooth surface and so are not recorded as correspondences. According to an exemplary embodiment of the present invention, the correspondence determined in this step is defined / stored within the data of the selected three-dimensional (3D) region.

[0050] Step S1140 In step S1140 of calculating a color difference for each of the tooth shade standards from the plurality of tooth shade standards, each color difference is calculated by comparing color information from the tooth shade standard with associated color information from selected regions on the three-dimensional (3D) surface representation of the teeth, and the plurality of correspondences obtained from step S1130 are used in this comparison of the color information.

[0051] More details regarding step S1140 are further described with reference to Figure 12. Figure 12 is a flow chart illustrating a process for calculating shade distance for multiple reference shade distributions according to an exemplary embodiment of the present invention.

[0052] In step S1210, a tooth shade reference with an associated label is selected. A shade distribution is defined over the reference tooth surface. The tooth shade reference is a shade tab digitized from a shade guide, typically all shade tabs in a shade guide have a common tooth shape, so the same reference tooth surface may be used without loss of generality. The label is typically "A1", "B3", etc. for Vita Classic set, or "2L1.5", "3M1", etc. for Vita 3DMaster.

[0053] In step S1220, the distance between the scanned shade distribution and the corresponding reference shade distribution is calculated according to the anatomical mapping. Further details of step S1220 are further described with reference to FIG.

[0054] FIG. 13 is a flow chart illustrating a process for calculating the distance between a scanned shade distribution and a reference shade distribution using anatomical mapping according to an exemplary embodiment of the present invention.

[0055] In step S1310, the anatomical mapping is described with a set of correspondences and associated weights. The density of the correspondences in the three-dimensional (3D) domain must be comparable to the resolution of the shade distribution. For textures, it will typically be the size of one pixel of the texture image. For BRDFs, it is approximately the spatial resolution of the structures that hold the various BRDFs defined across the surface. The weights may be uniform or may vary based on the region represented in the anatomical mapping. The sum of the weights is ideally normalized to 1 to simplify the averaging in what follows.

[0056] In step S1320, for each correspondence, a shade-color distance is calculated. The color difference is preferably expressed in the CIELAB color space, which provides a distance that matches the human perception of color variation. The correspondence defines two points at which the shade distribution is evaluated. The distance is defined as the norm of the difference between the shade distribution sampled from the three-dimensional (3D) tooth surface and the tooth shade reference.

[0057] In step S1330, a weighted average of the shade color distances is calculated. If the weights are normalized, it is simply a weighted sum.

[0058] The weighted average is returned in step S1340. If CIELAB space is used, the returned distance is in DeltaE (ΔE) units, ie, the visual perception of color difference.

[0059] Referring back to FIG. 12, following step S1220, in step S1230, if the distance is the smallest obtained, the associated label is recorded.

[0060] In step S1240, a determination is made whether there are any more labels remaining in the set: if yes, the method returns to step S1210, if no, the method proceeds to step 1250, i.e., the end, which means that the color difference calculation in step 1140 has been performed.

[0061] Step S1150 In step S1150, the label corresponding to the shade criterion with the smallest difference is recorded. Up to this step, the method mainly focuses on one selected region. It should be understood that this is only for the purpose of a brief and clear description. According to another exemplary embodiment of the present invention, multiple regions may be selected, and shade labels for different regions are determined accordingly. Furthermore, a grid may be defined on the buccal surface of the tooth. For example, a 3×3 grid or a 1×3 grid may be considered by the dentist. A denser grid may be used as needed. Thus, according to another exemplary embodiment of the present invention, a shade matching method for teeth includes: (a) selecting a tooth and defining a grid on the buccal surface of the tooth; (b) defining a mapping of the grid from the buccal surface to the upper surface of the common tooth shape; (c) calculating a shade label for each cell of the grid according to the process previously introduced in the present invention, where each cell defines a selected three-dimensional region; and (d) storing the shade label for the cell of the grid.

[0062] Stimuli for using the angular distribution of color vectors. Texture characterization falls short of accurately modeling the true color shading of teeth, which can be easily seen in side-by-side comparisons of different views of the same tooth or shade tab with slightly different lighting.

[0063] One of its drawbacks is that texture characterization inherently fails to take into account illumination and detection factors. Accurate color shade characterization of a tooth or another intraoral feature includes consideration of the surface geometry and the angle of illumination incident on the tooth. To illustrate shape-related effects, FIG. 2 shows a Lambertian sphere with uniform albedo, i.e., uniform optical luminance response, where the ratio of incident to reflected energy is constant. However, in appearance, the intensity profile of the luminance decreases as the cosine of the surface normal to detection. When the luminance decreases, the captured color texture changes accordingly. However, color shade is a property of the material of the object being scanned, and its measurement should be independent of the illumination / observation geometry.

[0064] An exemplary embodiment of the present invention utilizes the capabilities of an intra-oral imaging device to acquire multiple reflectance images of the same feature, such as the same surface of a tooth, from different angles from an intra-oral imaging camera 16, as shown in Figure 3. The combined angular illumination and reflectance information extracted from the different images may then be used to more closely characterize the color shade of each imaged tooth or other surface feature.

[0065] The angular distribution of color vectors represents a characteristic of a mapping that relates angular coordinates to corresponding color vectors, where the angular coordinates depend on the illumination direction, the observation direction and the surface normal, and the color vectors represent values ​​in a standard color space such as RGB, CIELAB XYZ (defined by the International Commission on Illumination), or the visible spectrum. According to an exemplary embodiment of the present invention, the angular distribution of color vectors may be represented by a BRDF (Bidirectional Reflectance Distribution Function) structure or a BRDF snapshot.

[0066] In the context of this disclosure, the phrase "spatially resolved angular distribution of color vectors" describes an angular distribution of color vectors with spatial variation. The spatial variation is used to parameterize the variation of the tooth optical response across the tooth surface. The spatially resolved angular distribution of color vectors is a function that relates spatial coordinates to the corresponding angular distribution of color vectors. This function is used to relate the spatial coordinates of one or more points from the surface data to the corresponding angular distribution of color vectors. The spatially resolved angular distribution of color vectors may provide a variation of spatial coordinates for a continuous or discrete variation of the corresponding angular distribution of color vectors.

[0067] Bidirectional Reflectance Distribution Function (BRDF) The Bidirectional Reflectance Distribution Function (BRDF) provides a way to more accurately characterize the effects on color shade of illumination and observation at various angles. Use of this function may characterize the color shade of the material itself while reducing the confusion resulting from the contributions of illumination and detection angles. For a single color, the BRDF for surface characterization of a material by a scanner is expressed as a function of the relevant variables as follows:

number

[0068] The BRDF is the inverse steradian sr -1 The basic geometry of the measurements to obtain the BRDF parameters for a point on a surface is shown diagrammatically in FIG. 4A. The line h is expressed in units of ω i and ω rand are described in more detail subsequently.

[0069] The global BRDF for monochromatic surface data has four dimensions: two for each of the incident and reflected light angles (azimuth and zenith). For Applicant's system for intraoral imaging, two constraints apply: (i) fixed illumination and detection angles are dictated by the design of the intraoral camera 16, and (ii) the BRDF data for a surface point is assumed to be invariant to rotations about the normal.

[0070] The BRDF data for the monochromatic surface data may be extended to BRDF color data for the color surface data by considering each color channel from the color surface data separately. In one exemplary embodiment, the angular distribution of color vectors is represented as BRDF color data, where the BRDF color data records the correspondence between the illumination direction, the reflectance direction, and the surface normal on the one hand, and the obtained color vectors on the other hand. In one exemplary embodiment of the invention, the BRDF data is represented by a two-dimensional RGB image.

[0071] 4B depicts the angular geometry on the left and the BRDF two-dimensional "section" content in graphical form, referred to in this disclosure as the BRDF structure, on the right. The two-dimensional (2D) structure is a known representation for a section of a general BRDF having four dimensions. For the present disclosure, due to the fixed illumination / sensor symmetry of Applicant's intraoral imaging system, the BRDF structure represents BRDF information corresponding to color shades.

[0072] The definition of the angle on the left side of FIG. 4B is obtained from the half-direction h and the surface normal n. The zenith angle θ and the azimuth angle φ provide the polar coordinates of the surface normal n with respect to the half-direction h. The direction of the origin and the angle φ are related to the input direction (ω i ,ω r ), may be chosen arbitrarily but must be aligned with the scanner coordinate system throughout the scan.

[0073] The two-dimensional (2D) BRDF structure is shown in Figure 4A with respect to the incident light direction ωi The illumination optical axis and the reflected light direction ω r θ, which represents the mid-vector between the detection optical axis for the surface normal n and the half-direction h. In the graphical representation of FIG. 4B, the horizontal axis represents the zenith angle θ between the surface normal n and the half-direction h, ranging from 0 to 90 degrees, with the leftmost axis being 0 degrees. The vertical axis represents the azimuthal angle between the surface normal n and the half-direction h, ranging from 0 to 360 degrees. This type of mapping array includes the location of standard contributions in the plot, such as specular reflection peaks, retroreflection, and scattered light.

[0074] Specular reflection is typically a strong, bright reflection observed on a metal surface when the observer is on the opposite side of the illumination with respect to the surface normal. The specular response depends primarily on the surface polish, but does not affect the choice of block shade used in milling. The specular response often saturates the detector, corrupting the shade measurement in that region of the two-dimensional (2D) cross section. This specular peak occurs when the normal direction n and half direction h are nearly aligned, as shown along the left side of the 2D BRDF mapping in Figure 4B.

[0075] The diffuse response is measured over a wide range of zenith angles θ and azimuth angles φ over most of the cross section. Small angle retroreflection is measured over a narrow range of angles, where the incident ω i , reflection ω r , and half-direction h are near alignment when designed by handpiece geometry constraints, but the surface normal n is tilted by 90° to them. When modeling physical properties, small angle retroreflection usually determines the type of microfacet distribution, i.e., microscopic level surface roughness. This indicates the angular separation of various physical effects that characterize the appearance. Small angle retroreflection represents properties under small angulation incidence relative to the normal direction, which is not the native way of observing a patient's teeth. Furthermore, depth information, and therefore surface normals, are more difficult to identify and measure in such geometries.

[0076] For intraoral shade matching, the diffuse region of the mapping in FIG. 4B is of most interest. The specular and small angle retroreflected light are of much smaller value for the reasons just explained. A single point P1 in the BRDF structure shown in FIG. 4B represents the BRDF value f for each color. r (ω i ,ω r , n), where the BRDF values ​​represent the surface normals at corresponding points along the surface contour, for a given relationship to the normal n, and for a given detector direction ω r With a given lighting direction ω i This results from lighting in

[0077] The assigned colors and associated depth information are measured by the camera in the scanner coordinate system, which is independent of the placement of the scanner in a common coordinate system. In the scanner coordinate system, the incident and reflected light directions are constant, so that the direction ω i , ω r , h are constant, and only the local surface normal n depends on the associated depth information. In the scanner coordinate system, the associated depth information provides the local surface normal n at a given point. The local surface normal n and the assigned color are used to compute a BRDF value for that point. The following sections explain how the BRDF values ​​acquired by the cameras can be organized into a common coordinate system in which the growing surface contour is defined.

[0078] Scanner orientation for updating growing surface contour Color and associated depth information extracted from the image data is used to create a continuously enhanced or growing surface contour. Although a point cloud with normals and colors provides good flexibility for point cloud evolution during acquisition (fast point insertion, deletion), a colored / textured surface mesh is typically presented to the user on the display 84 (FIG. 1).

[0079] A point cloud or mesh data structure may be equivalently used to represent a three-dimensional (3D) surface. A point cloud with normals and colors may be extracted from a colored / textured mesh by using vertices from the mesh and their average triangle normals and vertex colors, discarding triangle data, etc. A mesh with color may be generated from a point cloud with normals and colors using an algorithm such as that described in Kazhdan, M., & Hoppe, "Screened Poisson Surface Reconstruction," ACM Transactions on Graphics, 32(3), 1-13, (2013), or a variant of Vrip as described in Brian Curless and Marc Levoy, "A Volumetric Method for Building Complex Models from Range Images," Proceedings of SIGGRAPH 96, adapted to take a point cloud with normals and color data as input instead of the range images from the original algorithm. Without loss of generality, the term "surface contour" may be equivalently expressed using "surface mesh" or "surface point cloud." The conversion between point clouds and meshes is well known to those skilled in the art of surface contour imaging.

[0080] According to one embodiment, both mesh and point cloud structures may be used to generate the surface contour shown on the display. The point cloud with normals and color is a dynamic structure that combines information from all input stitched image data. The color mesh may be generated from the point cloud at low resolution as a temporary structure for display. The color mesh may be regenerated with respective updates and synchronized with the accurate point cloud with colors and normals.

[0081] The color and associated depth information may be converted to a colored surface contour. Normal information for each point of the surface contour may be estimated using well-known techniques in computer graphics, such as those described on the pointclouds.org website under documentation / tutorials / normal_estimation.php. At the query point, a local neighborhood of points is defined. A covariance matrix is ​​calculated using the local neighborhood of points. The covariance matrix C has the following form:

number

[0082] Stitching Procedure The stitching procedure of surface contours with normals and colors from the acquired image data onto the moving surface contour includes the following general steps: (a) estimating one set of feature descriptors for the surface contours with normals and colors from the image data and for the growing surface contour, (b) performing feature matching using both sets of feature descriptors to generate a relative configuration and a score, (c) accepting or rejecting the relative configuration based on the score, (d) improving the relative configuration using an iterative closest point algorithm to generate a distance measure, and (e) accepting or rejecting the relative configuration based on the generated distance measure. Details of these steps are as follows: (a) The feature descriptor for the surface contour represents a local surface description that is invariant to rigid transformations (rotation / translation). For example, a Fast Point Feature Histogram (FPFH) may be computed for each query point of the surface contour with its normals. This is described, for example, in Rusu, RB, "Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments", pp. 57-61, (2009, August 17). Another descriptor that also includes color information from the point cloud may be used. (b) Feature matching between one set of feature descriptors from the moving surface contour with normals and one set of feature descriptors from the target surface contour with normals includes generating a correspondence between the feature descriptors. The FPFH feature descriptors may be histograms, and the distance may be defined as the norm of the histogram difference between the two feature descriptors. The correspondence is defined as the minimum distance from one feature descriptor to another set of feature descriptors. Feature matching then includes selecting a set of correspondences to generate a relative configuration. This step is usually performed using a Random Sample Consensus (RANSAC) algorithm that constitutes a random selection of three correspondences to compute a candidate relative transformation, and then counting the number of correspondences that are consistent with this candidate relative transformation. The correspondence is formed from two feature descriptors, one from the moving point cloud and one from the target point cloud with normals. Each feature descriptor corresponds to a query point. A correspondence is consistent with a candidate relative transformation if the moving query point that is moved with the candidate relative transformation is within a predefined distance from the target query point. The candidate relative transformation with the greatest number of matching correspondences becomes the final relative alignment. The score indicates the quality of the final relative alignment and may be the corresponding number of matching correspondences. (c) A relative configuration may be rejected if its score is below a predetermined threshold. In this case, stitching is not possible and the image data is discarded. (d) Refining the relative configuration using an iterative closest point (ICP) algorithm defines point correspondences between the moving surface contour and the target surface contour by finding the closest fit when the relative configuration is applied on the moving surface contour. Minimizing the least squared distance between the fits provides the update of the relative configuration. The selection of point correspondences and updating of the relative configuration are then repeated until convergence or a predefined number of iterations is reached. The distance measure may be the value of the cost function used in the least squared distance minimization that corresponds to the final updated relative configuration. (e) The updated relative placement may be rejected if the distance measure is below a predetermined threshold. In this case, stitching is not possible and the image data is discarded. Otherwise, stitching passes. If stitching passes, the final relative placement of the surface contours with normals and colors from the image data onto the growing surface contour is known. This results in a common coordinate system for all calculated surface contours with normals and colors, to which points are added, updated, combined or deleted to generate the growing surface contour. There is also an original coordinate system for the scanner, which corresponds to the axes of the image data as originally acquired before stitching is performed. The common coordinate system is usually defined for any orientation of the scanner during acquisition of the first image data.

[0083] Sequence for Acquiring and Using BRDF for Intraoral Imaging The flow chart of Figure 5 depicts a sequence for acquiring and using a BRDF structure for improved color shade discrimination and is displayed in accordance with an embodiment of the present invention. In an acquisition and processing step S510, a surface contour image is acquired as a point cloud or mesh with normal and color information from the acquired image data. A color correction step S520 adjusts the f r (ω i ,ω r , n) and the illumination direction ω in the scanner coordinate system i and detector direction ω r The colors from the point cloud are converted to BRDF color data using: Although step S520 may be postponed until a later stage, direct color correction to generate BRDF values ​​allows for better color normalization early in the pipeline processing such that the corrected colors are a more accurate representation of the tooth material and less dependent on the illumination / detection geometry. Using this method, all subsequent data structures generated benefit from the corrected color values.

[0084] Each identified point from the surface contour includes at least a spatial 3D position, a normal vector, and a color vector (typically RGB color). The stitching step S530 defines the relative position of the surface contour from the acquired image data to the coordinate system of the "growing" surface contour, i.e., the surface contour is actively generated. Once the stitching is successful, the spatial point position in the coordinate system of the growing surface contour may be obtained.

[0085] A spatially varying BRDF (SVBRDF) uses a spatially resolved BRDF that adds point location information to the distribution function. The SVBRDF may be represented by using a regular BRDF grid structure defined in the same coordinate system as the growing surface contour. The grid cell size (voxel size) is typically the desired spatial shade resolution, which typically varies between 1 mm and 20 mm when using intraoral scanners. Dental technicians or practitioners generally separate teeth into one to three shades, so a typical grid cell size of 5 mm is used to reach this spatial resolution. The point locations after stitching provide coordinates in the BRDF grid structure. The corresponding BRDF grid cells contain the BRDF structure to be updated.

[0086] The general BRDF function f introduced above r (ω i ,ω r,n) are sampled during the scan under various observation / measurement geometries around a given point with normal direction n. Each sample is one of the BRDF values ​​introduced above, which may be stored in a BRDF structure of the same dimensions as the BRDF function. For Applicant's system, the BRDF structure is a two-dimensional image as previously noted. Updates to the BRDF resulting from a point are made as follows: the point position in the scanner coordinate system and the point normal are used to calculate the angular coordinate in the BRDF structure. The corresponding pixel stores the weighted sum of the corrected colors along with the total weight for that pixel. The average BRDF color for that pixel may be obtained at any time by dividing the weighted sum of the corrected colors by the total weight. A pixel represents the angular range of the BRDF function.

[0087] The weight corresponding to a point represents the relative confidence of the measurement. The default weight may be 1 and may be reduced according to, for example, the relative error of the point position, the uncertainty of the point normal direction, or the uncertainty from the matching criterion. For example, the uncertainty of the normal direction may be calculated from the ratio of the smallest eigenvalue to the second smallest eigenvalue. If the ratio is close to 1, both eigenvalues ​​are similar and the choice of normal is misleading. Beyond a predefined threshold, for example to 0.5, the corresponding weight for that point may be reduced. Similarly, the stitching is completed with an estimation of the distance measure, which may be compared to a predefined threshold to finally reduce the weights for all points in the cross section.

[0088] The pixel size is chosen so that the angular range is similar to the angular noise in the point cloud normal estimates. Typically, each pixel represents an angular variation of 5-20 degrees along each axis. In this two-dimensional (2D) image, the horizontal and vertical axes represent the zenith and azimuth angles, respectively, between the surface normal and the half direction.

[0089] More complex models exist, such as the Bidirectional Scattering Surface Reflectance Distribution Function (BSSRDF), or the Bidirectional Texture Function (BTF), or the Bidirectional Scattering Distribution Function (BSDF). Each of these models represents different characteristics of the scanned object. The physical properties modeled by these functions may represent reflection, scattering and / or shadowing, or masking. These distribution functions share the common principle of the angular resolution of the color vector as a function of the incident and reflected light direction. The term "BRDF" as used herein may refer to these similar angle-resolved bidirectional distribution functions that characterize the light interactions at different points along a surface.

[0090] The two-dimensional (2D) BRDF data structure, represented diagrammatically on the right side of Figure 4B, stores color data for the BRDF. Alternatively, a model of the corresponding surface properties may be stored, along with value distributions for reflection, as well as transmission, scattering, and shadowing. These alternative data structures may represent parameters that may be adjusted to fit the observed color for light at given incidence and output angles.

[0091] Alternative expressions for expressing a particular BRDF function may be used where the BRDF distance between data points may be specified as a metric. A default distance may be obtained by using the following BRDF distance for a BRDF structure that samples the BRDF function at pixel locations specified herein. For example, alternative expressions for BRDFs may be analytical expressions such as the Torrance-Sparrow BRDF, Cook-Torrance BRDF, Phong BRDF, etc., which are prevalent in computer drawing applications. A more detailed list of BRDF models may be found in "An Overview of BRDF Models" by R. Montes and C. Urena in technical report LSI-2012-001 from the University of Granada, which is incorporated herein by reference. This report describes an analytical model with free parameters that is adjusted to fit the observed BRDF values ​​described in this disclosure. A typical adjustment method involves a least-squares minimization of the residual distance between the BRDF values ​​measured from the scanner and the BRDF values ​​calculated from the analytical model.

[0092] Continuing with the FIG. 5 sequence, a contour generation step S540 generates a representation of the growing surface contour including an indication of the associated color shade. For each BRDF structure, the associated color shade is calculated using a separate BRDF library. A BRDF library is a collection that includes color shade values ​​and associated BRDF structures. The creation of a BRDF library is described in more detail below.

[0093] BRDF distance The evaluation of two BRDF functions at one point P1, as shown in FIG. 4B, gives two BRDF values. In the context of this disclosure, the BRDF values ​​are also called color vectors. The concept of distance between two BRDF values ​​is defined below. The BRDF distance between two BRDF functions may be defined as the integral of the color distance over all allowed positions of the point P1. In other words, the integration domain is a function of the input variable ω i and ω rCovers a range of angles.

[0094] If the BRDF data structure is a two-dimensional (2D) image (similar to the BRDF structure shown in FIG. 4B) where each pixel stores a weighted sum of colors and a total weight, then the BRDF snapshot may be defined as a two-dimensional (2D) image with an average color vector (obtained by dividing each pixel value by the total weight). If the total weight is zero or less than a predefined threshold, then the weighted sum of colors may be considered unreliable. In this case, the average color vector may be replaced with an invalid color vector, such as RGB=(-1.0,-1.0,-1.0).

[0095] The BRDF structure accumulates information over the entire scan period. The stored information represents the sum of the contributions to each pixel. The BRDF snapshot performs the averaging (keeping the original BRDF structure unchanged) and has a short lifespan. The data is only valid at the time it is generated during the BRDF distance calculation. The averages for the BRDF snapshot may be refreshed as soon as a new three-dimensional (3D) point from the subsequent image data acts on the BRDF structure.

[0096] In the general case, the color difference in either dimension may be represented as a vector difference, such as by a computed norm of the color difference. If one of the two color vectors is invalid, the color distance may be set to a predetermined distance value, as described above for values ​​considered unreliable. The BRDF distance between two BRDF snapshots is the weighted sum of the color distances for all corresponding pixels.

[0097] The weights from this weighted sum are derived from a discretization of the integral of the color distance over the integration domain. For applicant's system, due to the BRDF structure and the polar coordinates used for the BRDF snapshots, the integration elements (for the weights) are of the form sinθdθdφ evaluated with respect to the angular coordinate at the center of each pixel.

[0098] The BRDF snapshots are also used to convert colors from the original BRDF structure (such as RGB values) to a color space more suitable for evaluating color distances (such as the standard CIELAB color space familiar to those skilled in the art of imaging). For a given BRDF structure, the associated color shade from the BRDF library is the shade value that corresponds to the minimum distance between the BRDF snapshot from the given BRDF structure and the BRDF snapshot from the associated BRDF structure in the BRDF library. The associated color shade is the shade value from the BRDF library for each of the BRDF structures from the BRDF grid.

[0099] An indication of an associated color shade may be a label that corresponds to the associated color shade. For example, the label may be corresponding text or any color that represents the associated color shade from a BRDF library. In general, a "shade value indication" is a labeling that identifies a shade value. This type of labeling may be provided for multiple shade values, for example, by combining several text labels or color labels.

[0100] As an example, a color shade prescription for a tooth may include two or more shades, such as "D2-A2-A3," which may represent the upper, middle, and lower thirds of the tooth according to a particular implementation. Alternatively, the shade values ​​for the tooth may be indicated using text labels displayed on the surface data, where each shade value provides a text label that is represented at a point from the surface data that corresponds to the angular distribution of the color vector used to determine the shade value. According to an alternative exemplary embodiment, the indication of the one or more shade values ​​may include one or more color labels displayed on the surface data, where each shade value is associated with a color label that is represented at a point from the surface data that corresponds to the angular distribution of the color vector used to determine the shade value.

[0101] The growing surface contour and the BRDF grid are in the same coordinate system. Thus, an associated color shade may be displayed according to the corresponding surface location. If color is used for labeling, each point from the growing surface contour has a corresponding coordinate in the BRDF grid and may be assigned the color of the color shade label from the BRDF library associated with the BRDF structure at that coordinate. In this way, all of the displayed surfaces that fall within one BRDF cell may be rendered with the same color label. In other words, two adjacent points of the growing surface contour are likely to correspond to the same cell in the BRDF grid (described below). The two points may therefore be assigned the same color label. Going further, surface regions clipped by the BRDF structure may be rendered with the same color label. If text is used for labeling, the location of the text label relative to the growing surface contour may be the center of the BRDF cell in the same coordinate space. It may also be the center of the region selected by the operator.

[0102] A next position step S550 repeats the process of moving to the next set of image data from the camera for processing in step S510. According to an alternative exemplary embodiment, surface contour generation may be completed first using the tooth color as originally acquired. BRDF data is acquired but not applied during the initial surface contour generation. Surface contour stitching may be followed by a shade matching process using the acquired BRDF data. For this alternative method, step S540 is effectively performed after the next position step S550 and its corresponding processing loop.

[0103] According to an alternative exemplary embodiment, shade matching is performed after a conventional three-dimensional (3D) scanning process, whereby shade data is added to the existing three-dimensional (3D) surface data. Shade matching using angle information may then begin following a conventional three-dimensional (3D) scan or may use the loaded three-dimensional (3D) scan data for the patient. Stitching for shade matching using angle information is then performed on the existing surface contour. This sequence may allow the operator to focus on a particular tooth or teeth to be treated. This procedure allows, for example, BRDF data acquisition and shade labeling to be provided only to the region of interest for the restoration. In practice, the same shade acquisition sequence follows. Although the initial BRDF data structure is empty, the surface contour is not empty.

[0104] According to another alternative exemplary embodiment, the operator may select a limited area of ​​the patient's dentition of interest for which the shade label is intended, for example based on a treatment plan. In such a case, the drawing of the shade label on the displayed surface is limited to the selected area or limited to within a predefined distance range from the selected area. The selection may be performed using a pointer such as a computer mouse, using input from a tablet, or using some other digital input interface. The selection may be made before or after the three-dimensional (3D) scanning. For the latter, shade data is generated during the three-dimensional (3D) scanning process but is not displayed on the stitched surface contour. The shade-matched results may be displayed with a shade indicator either in the selected area or in a separate area on the monitor screen after the operator selects the area of ​​interest. According to an alternative exemplary embodiment, the shade label is some text corresponding to a shade tab from a shade library.

[0105] The flow of image data from the scanner may be stopped automatically, such as when valid information is no longer detected after a timeout period. Image data may also be stopped by an operator, such as by repositioning the scanner back in the holder. These or similar events may be detected to stop the scan. The user then interacts with the displayed surface, causing the application to display, store, or transmit a final shade-matched image. The final shade-matched image, which may be a color-shaded image, includes an indication of the displayed surface and associated color shades, for example, in a display step S560.

[0106] Building a BRDF library The logic flow of Figure 6 represents steps for forming a BRDF library for use in the method of Figure 5. A collection of conventional shade tabs commercially available from dental material suppliers may be used to generate a BRDF library to be used for matching values ​​in step S540 of the method of Figure 5. Common shade values ​​may be obtained, for example, from Vita Classical® shade tabs (such as "A1", "A3.5", "D2") or Vita Linearguide 3D-MASTER® (such as "2L1.5" or "3M2").

[0107] In a selection step S610, one of the shade tabs is selected for subsequent processing. A color shade association step S620 combines steps S510, S520, S530, and S550 of FIG. 5, which are repeated during the surface scan of the shade tabs. Upon completion of the scan, a calculate reference BRDF step S630 automatically combines several BRDF structures from the scan into a unique reference BRDF. Step S630 aligns the shade tabs with a reference coordinate system that is common to all of the reference shade tabs. Step S630 also automatically selects a group of cells and combines the BRDF structures into a single BRDF structure designated as the reference BRDF for that shade tab.

[0108] A BRDF library represents a collection of reference BRDFs and their associated shade labels. The BRDF library may be directly recoded in production and reused during scanning of the patient's teeth. The BRDF library may also be generated by the practitioner using the preferred shade tab. The practitioner simply needs to follow the model outlined here using a three-dimensional (3D) scanning device. This also allows the dentist to create a collection of shades by simply scanning them. The optical characteristics of the measurement system remain the same. The reference BRDFs stored in the BRDF library may be a collection of original BRDF data, BRDF constructions, or fitted models. The reference BRDF may be a combination of a portion of cells containing BRDF functions.

[0109] The surface contours are preferentially aligned in a similar coordinate system common to all shade tabs. The alignment is performed by stitching the growing surface contour onto a fixed reference surface of one of the shade tabs. All shade tabs from a collection usually have similar surface conditions, but their appearance represents a gradient. The selected shades are based on industrial notation. Therefore, it is important to align all growing surface contours to a similar coordinate system. In this way, corresponding BRDF cells in a given domain may be selected across the BRDF library to have consistent meanings.

[0110] FIG. 9 shows a shade tab with regions of different shade values ​​defined by the BRDF. The central region is identified as A2, while the incisal edge is D2 or another shade value. The present invention aims for more accurate shade measurement by comparing equivalent anatomical regions. If the approach from FIG. 10B is used, it may be possible to predict a three-dimensional (3D) surface that is globally detected as A2 for that A2 shade tab.

[0111] The reference BRDF is defined as the sum of all BRDF structures from selected BRDF cells within a given region. The summation is performed by a weighted sum of all colors individually and a pixel-wise accumulation of all weights.

[0112] The law of large numbers in statistics implies that collecting more observations leads to a more accurate average, i.e. a smaller standard deviation. The purpose of the pixel-by-pixel accumulation of BRDF structures within a given area is to obtain a more accurate average estimate when computing the BRDF snapshot of the reference BRDF during BRDF distance measurement.

[0113] The predefined region may be a sphere of radius about 2 mm centered about a predefined location on the reference surface. The predefined location may be, for example, the intersection of the reference surface with the (Oz) axis. A storing step S640 stores each BRDF reference and associated label in a shade library. A deciding step S650 controls a looping operation for systematically processing the entire array of shade tabs from the collection. At the conclusion of the measurement, a BRDF library is generated and stored. It may then be loaded prior to the start of the intraoral scan.

[0114] The sphere radius serves to define the boundary for the accumulation of BRDF information. The sphere radius for a given region is determined based on the local variability of shade appearance across the shade tab. At a minimum, a single BRDF structure is selected from the cell that contains the given location. Conventional methods for shade determination usually prescribe at most three shades for the canine represented within the band, and the radius is generally less than 5 mm to ensure uniformity of shade.

[0115] 7 depicts a collection of shade tabs 94 with corresponding BRDF reference structures contained within a BRDF library 100. Each BRDF reference structure is associated with a text label, also referred to as a shade label 104 (D2, D3, D4, C1, C2, C3, etc.), and / or a color label 112. The shade labels may also be stamped or otherwise recorded onto the metallic support of the shade tabs 94.

[0116] For conversion to CIELAB values, a signal from a white standard, defined as a surface with 90% reflectance, is acquired. Also, to avoid saturation of the detector, a signal from an 18% reflectance standard (N5 gray card) may also be used. The signal from the 18% reflectance standard is then multiplied by 5 to provide the equivalent of the white standard. Acquisition of the reflectance standard may be performed before acquisition of the reference surfaces and shade tabs for the shade library (before step S610, in the sequence of steps of FIG. 6). The signal from the reflectance standard may be stored in the shade library. Data from the reflectance standard is then available for shade acquisition and also during scanning of the actual teeth, since the shade library is loaded into the computer memory.

[0117] In practice, data for the reflectance standard is also acquired using a three-dimensional (3D) scanner by scanning the surface of the reflectance standard. After steps S510 to S560 are completed, a BRDF grid is created. All BRDF structures from the BRDF grid may be combined into an accumulated BRDF structure, which is then converted into a BRDF snapshot. The average color vector from this BRDF snapshot provides an average reflectance standard color vector, which may be used to convert to the CIELAB color space. The color vector may include a spectrum having wavelengths that encompass the visible wavelengths. For the present disclosure, the visible wavelengths are in the range between 360 nm and 830 nm.

[0118] Conversion of a color vector to CIELAB values ​​using a reflectance standard color vector is a well-known procedure, which may be summarized as (1) separating each color component from the color vector corrected by the color components of the reflectance standard color vector, (2) converting this normalized color vector to tristimulus XYZ values, and (3) converting the tristimulus XYZ values ​​to CIELAB colors.

[0119] Scanning the reflectance standard may provide more complex reflectance standard data. In one exemplary embodiment, the average BRDF structure of the reflectance standard is recorded directly instead of a single average reflectance standard color vector. When this is performed, the corresponding pixels from the BRDF snapshot of the reflectance standard may be used for color vector normalization when converting the color vector from the BRDF snapshot to CIELAB color. This technique may help to indicate the cause of angular variation in the reflectance of the reflectance standard. In practice, the scanner's elongated light source, a non-ideal illumination spectrum (because the illumination is not D65), may introduce slight angular variations.

[0120] In another exemplary embodiment, the scan of the reflectance standard provides spatially resolved color data. In particular, the spatial resolution may be expressed in the coordinate system of the scanner. This allows for the measurement of the variation of illumination and detection across the field of view. It may be represented as a three-dimensional grid aligned with the coordinate system of the scanner, where the X / Y axes represent the imaging plane and the Z axis represents the main optical axis of the scanner. Using a spatially resolved profile of the reflectance standard color vector allows for a more correct normalization of the color vector across the field of view. This helps to better generate CIELAB colors that will be independent of the position of the surface within the field of view.

[0121] BRDF Grid Structure 8 is a diagram representing multiple elements within the same coordinate system. A three-dimensional (3D) BRDF grid 30 made up of cells 72 defines a spatial partition encompassing a three-dimensional (3D) growing surface contour represented by shaded tabs 94. Each cell 72 contains a BRDF structure 70 represented by an associated BRDF snapshot at that point in the scan. Each cell 72 is a single 1 mm 3 BRDF data structure 70. During stitching step S530 (FIG. 5), the point cloud with normals and color is stitched onto the three-dimensional (3D) growing surface contour, so that those points with color and normals from the point cloud that correspond to a BRDF cell 72 are merged with the BRDF data structure 70 in that cell. If necessary, the BRDF grid 30 is expanded so that all stitched points are within a valid BRDF cell 72. All points from the point cloud are then merged into the growing surface contour, which is updated on the display.

[0122] The updates to the BRDF structure within each cell 72 include: (i) the parameters of the normal to the tooth region corresponding to the segment; (ii) the incident illumination direction ω i , (iii) the reflection direction ω of the detection light r , and (iv) a color vector in a well-known color space, such as RGB, CIELAB, or another suitable color space. The number of color channels may be four or more.

[0123] Illumination incident direction ω i , the reflected direction of the detection light ω r , and local surface normal n, the BRDF structure stores an angular distribution, where each element (pixel) of the angular distribution stores a weighted sum of all color vectors and the corresponding total weight. Thus, for a given geometric structure, the BRDF structure may be used to calculate an average color vector, which is stored in the BRDF snapshot.

[0124] The weights used in the weighted sum of color vectors and the corresponding total weights may be related to the confidence (or quality) level of the obtained three-dimensional (3D) point as previously described, which represents a variable level of confidence in the accuracy of the three-dimensional (3D) point location, surface normal, or color. In addition, the weights may also vary due to factors including: (1) decreasing saturation regions (specular cones) as the exact point location becomes imprecise, (2) decreasing as the local normal tilts away from the incident light or measurement angle, and (3) increasing if the normals are consistent within a local neighborhood.

[0125] Lighting direction ω i , reflection direction ω r , and the geometric structure for normal n may be encoded in two-dimensional (2D) angular coordinates in the BRDF structure in cell 72. A two-dimensional (2D) image structure is used to represent the angular extent of the BRDF structure. Each pixel contains a weighted sum of color vectors and associated total weights.

[0126] The resulting data structure for the tooth region may be incomplete since the scanner position in the measurement sequence may not explore all the various angle combinations. At the beginning of the acquisition sequence, the BRDF distance is initially driven by a default color distance. As additional data is accumulated in the BRDF structure during the scan, the BRDF snapshot will contain more plausible color values ​​and the identification of shades from the shade library will become more and more accurate. The calculation may use a minimum threshold distance below which the associated color shade from step S540 may be invalid. In such a case, no shade label is displayed for that region of the surface. This prevents very fast changes in associated color shades at the beginning of the scan. A shade appears when enough orientations have been accumulated within the pixels of the BRDF structure to be meaningful. According to one embodiment, the BRDF distance may be displayed in units of DeltaE, a standard metric for color distance in the CIELAB color space. It is assumed that 50% of dentists can visually detect a difference of 3DeltaE (3ΔE). A threshold for the maximum distance below which an associated color shade is invalid may then be set to 3DeltaE or a similar value (typically between about 1DeltaE and 8DeltaE).

[0127] The BRDF data may be stored in memory or generated from an intraoral camera and scanning device as needed. Figure 8 provides one visualization of a data structure that may be used to accumulate and organize the BRDF data obtained for the teeth resulting from step S540. It may be appreciated that such a data structure need not itself be visible to an operator.

[0128] Often, specular reflections saturate values ​​from the image data, meaning that the associated depth information does not have valid data for the region corresponding to the specular reflection. After stitching the rest of the surface contour, there is no point to calculate coordinates in the corresponding BRDF structure, and therefore no normal. If associated depth information is still available near the outer limit of the region of specular reflection, the depth information is often incorrectly estimated in that region. This means that the normals may be very noisy. It may also happen that the reflection image from the image data is saturated even if the depth information is valid. Saturation occurs because the operating range of the sensor is not sufficient. It is better to reject those erroneous measurements than to store them in the BRDF structure. Rejection may be implemented by setting a rejection threshold on the cosine of the point normal and the half direction (e.g., rejected if greater than 0.98). Another implementation would be to reject storage in the BRDF structure if the color of a point from a point cloud with color and normal is close to the camera's saturation value (e.g., if all color channels are above value 250 for a camera range that goes up to 255). The reference BRDF obtained from the shade tab 90 may be stored, for example, as an element of a BRDF library, as previously described with reference to the flow chart of FIG.

[0129] An exemplary embodiment of the present invention adjusts the standard BRDF relationship in step S520 to the following:

number

[0130] Improved measurements of the distance between two BRDF structures may use an underlying model specified by a fixed number of parameters. The paper by M. Langovoy et al., "Novel metric for analysis, interpretation and visualization of BRDF data", ACM Transactions on Spatial Algorithms and Systems (2014), gives a possible implementation (see equations (8) or (15) as examples of metrics). This means fitting the recorded two-dimensional (2D) BRDF structures to the models by optimizing the parameters and then calculating the distance between them.

[0131] The presently disclosed embodiments are therefore considered in all respects to be illustrative and not restrictive The scope of the invention is indicated by the appended claims, and all changes that come within the meaning and range of equivalents thereof are intended to be embraced therein.

[0132] FIG. 9 represents a surface contour with shade indicators, where the surface contour corresponds to the shade tab. Exemplary positions of reference positions 115 for creating a BRDF library are represented with example shade indicators using text labels 116 and color labels 112 for detected shades. A portion of a growing surface contour 114 is represented. FIG. 9 represents a possible representation 110 of a three-dimensional (3D) surface contour with labels for the identified color shades from step S540. For each cell of the BRDF grid, each BRDF structure may provide one shade value. Once a shade value is successfully identified, the surface in the corresponding region of space is rendered with the label 112 associated with that shade value. For example, the A2 shade from the Vita Classical® shade tab may be rendered using a light green color, which is its arbitrary default color label.

[0133] The BRDF grid leads to spatial variation of the labels on the representation of the labeled three-dimensional (3D) surface. If the shade value determination is not successful because the minimum BRDF distance is greater than a predetermined distance threshold, the surface is not rendered with the labels since color or texture is independent of the BRDF shade library, thus preserving the original appearance of the colored or textured mesh 114. In this example, the original appearance of the colored mesh 114 results from the fact that this metallic surface does not correspond to a similar material present in the BRDF library.

[0134] According to one exemplary embodiment of the present invention, the intra-oral imaging device 90 of Fig. 1 acquires a three-dimensional (3D) tooth surface representation and acquires a spatially resolved angular distribution of color vectors, which associates one or more point locations from the surface data with a corresponding angular distribution of color vectors. The processor 80 is programmed with processing logic instructions to acquire the tooth surface representation and identify one or more shade values, each shade value being associated with one angular distribution of color vectors from the spatially resolved angular distribution of color vectors by comparing the angular distribution of color vectors to a set of reference angular distributions of color vectors, each reference angular distribution in the set being associated with a corresponding shade value. The display 84 then displays the three-dimensional tooth surface representation and displays an indication of the identified shade values.

[0135] Consistent with one exemplary embodiment of the present invention, a computer program having stored instructions that control system functions for image acquisition and image data processing for image data is stored in and accessed from electronic memory. As can be appreciated by those skilled in the image processing art, the computer program of one embodiment of the present invention may be utilized by a suitable general-purpose computer system, such as a personal computer or workstation, acting as an image processor, when the processor is equipped with suitable software programs operative to acquire, process, transmit, store and display data as described herein. Many other types of computer system configurations may be used to execute the computer program of the present invention, including, for example, an array of networked processors.

[0136] The computer program for carrying out the method of the present invention may be stored in a computer readable storage medium. The medium may comprise, for example, a magnetic recording medium such as a magnetic disk or magnetic tape, such as a hard disk or a removable device, an optical storage medium such as an optical disk, an optical tape, or a machine-readable optical encoding, a solid-state electronic storage device such as a random access memory (RAM) or a read-only memory (ROM), or another physical device or medium used to store a computer program. The computer program for carrying out the method of the present invention may also be stored in a computer readable storage medium connected to the image processor by the Internet or another network or communication medium. Those skilled in the art of image data processing will further readily recognize that the equivalent of such a computer program product may also be constructed in hardware.

[0137] The term "memory", which is equivalent to "computer-accessible memory" in the context of this disclosure, may refer to any type of temporary or more permanent data storage working area that is used to store and operate on image data, including databases, and that is accessible to a computing system. For example, the memory may be non-volatile, using long-term storage media such as magnetic or optical storage devices. Alternatively, the memory may be of a more volatile nature, using electronic circuitry such as random access memory (RAM) used as a temporary buffer or working area by a microprocessor or another control logic processor device. Display data, for example, is typically stored in a temporary storage buffer that is directly associated with a display device and is periodically refreshed as needed to provide the display data. This temporary storage buffer may also be considered to be a memory as that term is used in this disclosure. Memory is also used as a data working area for performing and storing intermediate and final results of calculations and other processing. Computer-accessible memory may be volatile, non-volatile, or a hybrid combination of volatile and non-volatile types.

[0138] It is understood that the computer program products of the present invention may utilize various image processing algorithms and processes that are well known. It will be further understood that computer program product embodiments of the present invention may embody algorithms and processes not specifically depicted or described herein that are useful for implementation. Such algorithms and processes may include conventional utilities within the common knowledge of image processing technology. Additional aspects of such algorithms and systems, as well as hardware and / or software for generating and otherwise processing images or cooperating with the computer program products of the present invention, are not specifically depicted or described herein and may be selected from such algorithms, systems, hardware, components, and elements known in the art.

[0139] Although the present invention has been described in detail and with particular reference to preferred or presently preferred exemplary embodiments, variations and modifications may be effected within the spirit and scope of the invention. The presently disclosed embodiments are therefore considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the appended claims, and all changes that come within the meaning and range of equivalents thereof are intended to be embraced therein.

Claims

1. 1. An intraoral imaging system comprising: an image capture device for capturing a plurality of digitized tooth shade references, each tooth shade reference corresponding to a common tooth shape, each tooth shade reference having a label and including color information on the tooth shape; a separate image capture device that captures a 3D surface representation of the teeth and associated color information defined on said 3D surface; a processor programmed with processing logic instructions; wherein the processing logic instructions include: (a) selecting an area on the 3D surface representation of the tooth; (b) determining, among all said tooth shade references, a plurality of tooth shade references which belong to the same anatomical region as the selected region on the 3D surface representation of the tooth, said determining being performed by mapping from said selected region on the 3D surface representation of the tooth to the same anatomical region on a common tooth shape of all said tooth shade references, (c) calculating a color difference for each of the determined tooth shade standards, each color difference being calculated by comparing color information from each of the determined tooth shade standards with the associated color information from the selected region on the 3D surface representation of the tooth; (d) recording the label corresponding to the tooth shade standard having the smallest difference; An intraoral imaging system.

2. 2. The system of claim 1, wherein a plurality of digitized tooth shade references are acquired and stored in the system prior to acquiring the 3D surface representation of the teeth and associated color information defined on the 3D surfaces.

3. The system of claim 1 , wherein the step of obtaining the 3D surface representation of the teeth and associated color information defined on the 3D surfaces is performed using an intraoral scanner.

4. The system of claim 1 , wherein the associated color information defined for the 3D surface includes red, green, and blue color data values.

5. The system of claim 1 , wherein the associated color information defined for the 3D surface comprises CIELAB color data values.

6. The system of claim 1 , wherein the associated color information defined on the 3D surface includes a spectrum having wavelengths covering visible wavelengths.

7. The system of claim 1 , wherein selecting a region on the 3D surface representation of the tooth comprises a segmentation process, the segmentation process being manual, semi-automatic, or automatic.

8. determining the plurality of responses is a manual process, an automatic process, or a semi-automatic process; The manual process includes a step of a user using a computer mouse or a touch screen to determine and select the same anatomical region on the common tooth form; The automated processing includes: (a) the 3D surface representation of the tooth is segmented from the gingiva using color and morphological operations; and / or (b) an injective transformation is used to map from the selected region on the 3D surface representation of the tooth to the region on the common tooth shape; wherein the transformation is determined by the 3D surface representation of the tooth and a surface of the tooth shade reference, 2. The system of claim 1, wherein the semi-automatic process comprises a step in which a user manipulates further adjustments of the region provided by the automatic process, the adjustments including making further modifications to a position, scale, or orientation of the region provided by the automatic process.

9. The system of claim 1 , wherein each of the tooth shade references is a shade tab from a shade guide.

10. The system of claim 1 , wherein the color information comprises a spatially resolved angular distribution of color vectors of the teeth or tooth shade references.

11. The system of claim 10 , wherein the spatially resolved angular distribution of color vectors is a bidirectional reflectance distribution function value.

12. The system of claim 1 , wherein the color information comprises surface texture information of the tooth or tooth shade reference.

13. The system of claim 1 , wherein the label is a text label, or a color-coded label, or a spatial profile of color variation across the 3D surface representation.

14. The system of claim 1 , wherein the tooth shade information corresponds to at least one of an upper, middle, or lower third of the tooth surface.

15. The system of claim 1 , wherein the selected region is defined by a grid on the 3D surface representation of a tooth.

16. 1. An intraoral imaging device comprising: an image capture device for acquiring a plurality of digitized tooth shade references, each tooth shade reference corresponding to a common tooth shape and having a label and including color information for the tooth shape, the image capture device acquiring a 3D surface representation of the teeth and associated color information defined on the 3D surface; a processor programmed with processing logic instructions; wherein the processing logic instructions include: (a) selecting an area on the 3D surface representation of the tooth; (b) determining, among all said tooth shade references, a plurality of tooth shade references which belong to the same anatomical region as the selected region on the 3D surface representation of the tooth, said determining being performed by mapping from said selected region on the 3D surface representation of the tooth to the same anatomical region on a common tooth shape of all said tooth shade references, (c) calculating a color difference for each of the determined tooth shade standards, each color difference being calculated by comparing color information from each of the determined tooth shade standards with the associated color information from the selected region on the 3D surface representation of the tooth; (d) recording the label corresponding to the tooth shade standard having the smallest difference; An intraoral imaging device.

Images