Device and method for acquiring multi-point chromaticity coordinate values on surface of shot object

Abstract

The invention provides a device and a method for acquiring multi-point chromaticity coordinate values on the surface of a shot object, and mainly relates to the fields of color vision, colorimetry, color communication, image processing, computer control, deep learning, applied mathematics and the like. The problems that the multi-point chromaticity coordinate value of the surface of an existing shot object is difficult to measure, inaccurate in measurement and high in measurement cost are solved, and the multi-point chromaticity coordinate value of the surface of the shot object is calculated by obtaining a group photo image of the shot object and a color block group under the irradiation of different spectral distribution light sources.

Description

Device and method for obtaining multi-point chromaticity coordinate values ​​on the surface of a photographed object Technical Field

[0001] The present invention relates to the fields of color vision, colorimetry, color communication, image processing, computer control, deep learning, and applied mathematics, and in particular to a device and method for obtaining multi-point chromaticity coordinate values ​​on the surface of a photographed object. Background Art

[0002] Accurately, quickly, and cost-effectively acquiring multi-point chromaticity coordinate values ​​on the surface of an object is a crucial foundation for color communication. Currently, colorimeters or spectrophotometers are primarily used to obtain these values, but these methods only capture the chromaticity coordinates of a single point at a time. Furthermore, these measurements are easily affected by the three-dimensional shape of the object, leading to inaccurate results. While hyperspectral cameras can capture the spectral distribution of multiple points on the surface and calculate the chromaticity coordinates for each point, their high cost limits their application. Conventional color cameras only produce a three-channel color image of the object, but cannot accurately capture the chromaticity coordinates of multiple points on the surface.

[0003] Summary of the Invention

[0004] The present invention provides a device and method for using an image sensor to capture a combined image of a subject and a group of color blocks under illumination by a multi-spectral distribution light source, and then using the combined image to calculate the chromaticity coordinate values ​​of multiple points on the surface of the subject, thereby solving the problems of difficulty in measuring the chromaticity coordinate values ​​of multiple points on the surface of the subject, inaccurate measurement, and high measurement cost.

[0005] The present invention relates to various aspects of devices and methods. Implementations of the present invention may include any combination of one or more of the various aspects described herein.

[0006] In a first aspect, a device for obtaining chromaticity coordinate values ​​of multiple points on the surface of a photographed object is provided, comprising: a light source group, a color block group, a control unit, an imaging unit, an image segmentation and alignment unit, and a chromaticity coordinate value calculation unit, wherein the light source group includes multiple light sources with different spectral distributions, and the color block group includes multiple color blocks with different reflection spectral distributions. The control unit controls the light source group to use light sources with different spectral distributions to illuminate the subject and the color block group in different time periods, and controls the imaging unit to synchronously obtain multiple group images of the subject and the color block group under the illumination of light sources with different spectral distributions in corresponding time periods. The image segmentation and alignment unit segments an image containing a single subject and an image containing a single color block from the group image, and aligns the images of the same subject and the same color block obtained under different light source illumination conditions. The chromaticity coordinate value calculation unit uses a regression model to calculate the chromaticity coordinate values ​​of multiple points on the surface of the subject.

[0007] In a second aspect, a method for obtaining multi-point chromaticity coordinate values ​​on the surface of a subject is provided, wherein a group image of the subject and a color block group containing multiple color blocks with different reflective spectral distributions under illumination by light source groups with different spectral distributions is obtained, an image containing a single subject and an image containing a single color block are segmented from the group image, and images of the same subject and the same color block obtained under different light source illumination conditions are aligned respectively. The chromaticity coordinate value calculation unit uses a regression model to calculate the multi-point chromaticity coordinate values ​​on the surface of the subject.

[0008] The half-maximum width range near the peak wavelength of the light source spectrum distribution of at least part of the light source groups intersects with the half-maximum width range near the sensitive peak wavelength of the three types of human cone cells, S type, M type, and L type.

[0009] The color block group includes multiple color blocks, and these color blocks have different reflective spectral distributions in the half-width range near the sensitive peak wavelength of three types of human cone cells: S-type, M-type, and L-type.

[0010] The calculation steps of the chromaticity coordinate value calculation unit include:

[0011] First, a regression model is established, using color channel component values ​​of color block images obtained under illumination of light sources with different spectral distributions as model input data, and chromaticity coordinate values ​​of the corresponding color blocks as model output annotation data, and using color channel component values ​​of subject images obtained under illumination of light sources with different spectral distributions as model input data, and chromaticity coordinate values ​​of corresponding positions of the corresponding subjects obtained by other means as model output annotation data, training the regression model, and determining the regression model parameters;

[0012] Then, the trained regression model is used to input the color component values ​​of the image of the subject obtained under the illumination of light sources with different spectral distributions, and the chromaticity coordinate values ​​corresponding to the subject are output. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present invention may be better understood through the following detailed description of preferred but non-limiting embodiments with reference to the accompanying drawings, in which:

[0014] FIG1 is a schematic diagram showing a composite image of a subject and a group of color blocks taken under illumination by a multi-spectral light source group;

[0015] FIG2 is a schematic diagram showing the image segmentation and alignment unit segmenting and aligning a composite image of a subject and a color block group;

[0016] FIG3 is a schematic diagram of a chromaticity coordinate calculation unit;

[0017] FIG4 shows the spectral distribution curve of the multi-spectral light source group and the spectral sensitivity curves of three types of human cone cells: S-type, M-type, and L-type;

[0018] Figure 5 shows the reflectance spectrum distribution curve of the color block group and the spectral sensitivity curves of three types of human cone cells: S-type, M-type, and L-type;

[0019] FIG6 shows a schematic diagram of the neural network regression model structure. DETAILED DESCRIPTION

[0020] In order to make the technical problems and technical solutions solved by the present invention clearer, the technical solutions in the embodiments of the present invention are further clearly and completely described below in conjunction with the accompanying drawings and embodiments. The described embodiments are part of the embodiments of the present invention, rather than all the embodiments. The description herein is only for explaining the present invention and is not intended to limit the present invention. For the sake of clarity, some features that are not particularly important for understanding the present invention or are obvious to those skilled in the relevant art may not be shown.

[0021] This embodiment uses multiple teeth as the photographing objects to illustrate an embodiment of the invention.

[0022] The light source groups in this embodiment are all implemented using light-emitting diodes. Nine LEDs are used to emit different colors: red, cool white, medium white, purple, warm white, green, yellow, white, and blue. This embodiment uses the example of photographing multiple teeth to obtain multi-point chromaticity coordinate values ​​on the tooth surfaces. The color patch group in this embodiment includes 24 color patches. The image captured by the image sensor in this embodiment is a three-channel color image.

[0023] 1 , a light source group 101 includes a plurality of light sources with different spectral distributions. A control unit 102 controls the light source group 101 to illuminate a subject 105 and a color block group 104 using light 107 emitted by light sources with different spectral distributions in different time periods. The color block group 104 includes a plurality of color blocks 106 with different wavelength spectral absorption rates. The control unit 102 controls the imaging unit 103 to synchronously acquire, during corresponding time periods, a plurality of group images of the subject and the color block group illuminated by light sources with different spectral distributions.

[0024] The control unit can be implemented by a general microcontroller, the imaging unit can be implemented by a general color CMOS image sensor and an optical lens, and the control unit can use a general relay to control the on and off of the LED light source.

[0025] 2 , a group image 201 captured by an imaging unit under illumination with different spectral distributions is processed by an image segmentation and alignment unit 202. The image segmentation and alignment unit 202 first segments the group image 203 into a color block group image and an object image 205. The image 206 containing a single object and the image 204 containing a single color block are then further segmented into images of the same object and the same color block obtained under different illumination conditions. The images are then aligned.

[0026] A currently known image instance segmentation method based on deep neural networks, such as Mask RCNN, can be used to segment the group image into a color block group image and a subject image, and further segment the group image into an image containing a single subject and an image containing a single color block.

[0027] First, frontal, full-mouth, tooth-and-color-block group photos of 50 subjects were captured. Using a known image annotation tool, the outlines and numbers of individual teeth, as well as the outlines and numbers of individual color blocks, were annotated in the group photos. Based on the pre-trained Mask RCNN network, which is well-known in the industry, the acquired single tooth images, single color block images, and corresponding annotated data were used to train the Mask RCNN deep neural network for image instance segmentation using known methods to obtain the trained neural network. The trained neural network was then used to perform image instance segmentation on the combined tooth and color block images, obtaining single tooth images and single color block images.

[0028] After segmentation, images of the same instance under different lighting conditions can be aligned using their geometric centers as reference points under different lighting conditions.

[0029] 3 , the chromaticity coordinate value calculation unit 301 first uses the color channel component values ​​of the color block image 303 obtained under illumination of light sources with different spectral distributions as model input data, and the chromaticity coordinate value 302 of the corresponding color block as model output annotation data, and uses the color channel component values ​​of the subject image 304 obtained under illumination of light sources with different spectral distributions as model input data, and the chromaticity coordinate value 305 of the corresponding position of the corresponding subject obtained by other means as model output annotation data, to train the regression model 306 and determine the parameters of the regression model 306; then, the color components of the subject image 304 obtained under illumination of light sources with different spectral distributions are input into the regression model 306, and after calculation by the regression model 306, the chromaticity coordinate value 307 corresponding to the subject is output.

[0030] The chromaticity coordinates of the subject used as training data for the regression model can be obtained using other devices that measure chromaticity coordinates, such as a hyperspectral camera. In this case, tooth shade charts can be used instead of actual teeth to facilitate the acquisition of chromaticity coordinates of the tooth color spectrum.

[0031] Referring to FIG4 , the half-width at half maximum (FWHM) range 405 near the peak wavelength of the light source spectral distribution 404 of at least some light source groups intersects with the half-width at half maximum (FWHM) range 406 near the peak wavelength sensitivity of three types of human cone cells: S-cone, M-cone, and L-cone. These include the spectral sensitivity curve 401 for S-cone cells, the spectral sensitivity curve 402 for M-cone cells, and the spectral sensitivity curve 403 for L-cone cells.

[0032] Figure 4 shows the spectral distribution curves for red, cool white, neutral white, purple, warm white, green, yellow, white, and blue LED light sources, as shown by the thin solid lines. The thick dashed lines in the figure show the spectral sensitivity curves for S-type, M-type, and L-type human cone cells. The horizontal axis of the curves in the figure represents wavelength in nanometers, while the vertical axis represents the normalized, dimensionless value. The light source represents intensity, while the sensitivity of S-type, M-type, and L-type cone cells is represented by the vertical axis.

[0033] Referring to FIG5 , the color patch group includes multiple color patches, each of which has a different reflectance spectral distribution within a half-width range 505 near the peak sensitivity wavelength of S-type, M-type, and L-type human cone cells. The color patches include a spectral sensitivity curve 501 for S-type human cone cells, a spectral sensitivity curve 502 for M-type human cone cells, a spectral sensitivity curve 503 for L-type human cone cells, and a reflectance spectral distribution curve 504 for the color patches.

[0034] Figure 5 shows the reflectance spectral distribution curves corresponding to 24 color blocks, as shown by the thin solid lines. The thick dashed lines show the spectral sensitivity curves for S-type, M-type, and L-type human cones. The horizontal axis of the curves is wavelength in nanometers, and the vertical axis is the dimensionless, normalized value. The color blocks represent intensity, while the S-type, M-type, and L-type cones represent sensitivity.

[0035] Referring to Figure 6 , the regression model is implemented using a 9-layer fully connected neural network. The input layer 601 contains 27 nodes, corresponding to the three color channel component values ​​of the color image acquired under nine light sources. The output layer contains 3 nodes, corresponding to the chromaticity coordinate values. The intermediate layers 603, including layers 2 through 8, contain 81, 243, 729, 729, 243, 81, and 81 nodes, respectively. Each layer uses the sigmoid activation function. Nodes 605 in each layer are fully connected 602.

[0036] It should be noted that, in order to more clearly understand the present invention, the drawings and descriptions in the present invention have been simplified, and elements well-known in the art have been eliminated for the sake of clarity. Moreover, the eliminated elements themselves do not promote a better understanding of the present invention, so they will not be described in detail. The omitted details and modified or alternative implementations are within the knowledge of ordinary technicians in this field. The present invention is not limited to the above embodiments, and those skilled in the art can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein; however, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A device for obtaining chromaticity coordinate values ​​of multiple points on the surface of a photographed object, characterized in that: include: A light source group, a color block group, a control unit, an imaging unit, an image segmentation and alignment unit, and a chromaticity coordinate value calculation unit, wherein the light source group includes multiple light sources with different spectral distributions, and the color block group includes multiple color blocks with different reflection spectral distributions. The control unit controls the light source group to use light sources with different spectral distributions to illuminate the subject and the color block group in different time periods, and controls the imaging unit to synchronously obtain multiple group images of the subject and the color block group under the illumination of light sources with different spectral distributions in corresponding time periods. The image segmentation and alignment unit segments an image containing a single subject and an image containing a single color block from the group image, and aligns the images of the same subject and the same color block obtained under different light source illumination conditions. The chromaticity coordinate value calculation unit uses a regression model to calculate the chromaticity coordinate values ​​of multiple points on the surface of the subject.

2. The device according to claim 1, characterized in that The half-width at half maximum range near the peak wavelength of the light source spectrum distribution of at least part of the light source groups intersects with the half-width at half maximum range near the sensitive peak wavelength of the three types of human cone cells, namely, S-type, M-type, and L-type.

3. The device according to claim 1, wherein: The color block group includes multiple color blocks, and these color blocks have different reflective spectral distributions within the half-width range near the sensitive peak wavelength of three types of human cone cells: S-type, M-type, and L-type.

4. The device according to claim 1, wherein The calculation steps of the chromaticity coordinate value calculation unit include: First, a regression model is established, using color channel component values ​​of color block images obtained under illumination of light sources with different spectral distributions as model input data, and chromaticity coordinate values ​​of the corresponding color blocks as model output annotation data, and using color channel component values ​​of subject images obtained under illumination of light sources with different spectral distributions as model input data, and chromaticity coordinate values ​​of corresponding positions of the corresponding subjects obtained by other means as model output annotation data, training the regression model, and determining the regression model parameters; Then, the trained regression model is used to input the color component values ​​of the subject image obtained under the illumination of light sources with different spectral distributions, and the chromaticity coordinate values ​​corresponding to the subject are output.

5. A method for obtaining chromaticity coordinate values ​​of multiple points on the surface of a photographed object, characterized in that: A composite image of a subject and a color block group containing multiple color blocks with different reflective spectral distributions, obtained under different light source groups, is obtained. An image containing a single subject and an image containing a single color block are segmented from the composite image. Images of the same subject and the same color block obtained under different light source conditions are aligned respectively, and the chromaticity coordinate values ​​of multiple points on the surface of the subject are calculated using a regression model.

6. The method according to claim 5, wherein: The half-width at half maximum range near the peak wavelength of the light source spectrum distribution of at least part of the light source groups intersects with the half-width at half maximum range near the sensitive peak wavelength of the three types of human cone cells, namely, S-type, M-type, and L-type.

7. The method according to claim 5, wherein: The color block group includes multiple color blocks, and these color blocks have different reflective spectral distributions within the half-width range near the sensitive peak wavelength of three types of human cone cells: S-type, M-type, and L-type.

8. The method according to claim 5, wherein: The step of calculating the chromaticity coordinate value comprises: First, a regression model is established, using color channel component values ​​of color block images obtained under illumination of light sources with different spectral distributions as model input data, and chromaticity coordinate values ​​of the corresponding color blocks as model output annotation data, and using color channel component values ​​of subject images obtained under illumination of light sources with different spectral distributions as model input data, and chromaticity coordinate values ​​of corresponding positions of the corresponding subjects obtained by other means as model output annotation data, training the regression model, and determining the regression model parameters; Then, the trained regression model is used to input the color component values ​​of the subject image obtained under the illumination of light sources with different spectral distributions, and the chromaticity coordinate values ​​corresponding to the subject are output.

Images