Color correction method, spectrum sensing device and imaging device

Abstract

The invention relates to a color correction method, a spectrum sensing device and an imaging device. The color correction method comprises the following steps: acquiring spectral data of current ambient light; based on the spectral data of the current ambient light and a spectral sensitivity function of the imaging equipment, calculating an original multi-channel response value of the imaging equipment for a specific reflectivity sample; acquiring a stimulus value of the specific reflectivity sample in a target color space; calculating a conversion mapping from the original multi-channel response value to a stimulus value in the target color space; and applying the conversion mapping to an image currently shot by the imaging device to convert the image to the target color space, and performing color correction on the image in the target color space. Therefore, the original multi-channel response value of the imaging equipment can be converted into the target color space based on the spectral data of the current ambient light, so that the color correction precision of the imaging equipment is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of optical processing, more particularly, to a color correction method, a spectral sensing device and an imaging device. BACKGROUND

[0002] Auto white balance module and color correction module are important links in the image processing flow of color digital camera, which convert the device-dependent detection values to device-independent reference chromaticity values under standard light source (such as CIE D65), so as to realize the color reproduction consistent with human eye perception.

[0003] At present, the white balance and color correction module in color digital camera usually uses several groups of white balance parameters and color correction matrix parameters calibrated in advance as correction basis. In the calibration stage, several standard light sources are usually used to calculate the conversion parameters between the original RGB response values of each color block of standard color card under the light source and the CIE1931 XYZ standard values under the reference light source (such as CIE D65), and the corresponding calibration parameters are stored in the built-in storage space of image signal processor.

[0004] In the use stage, the color correction module will select a group (or several groups for weighting) of calibration parameters from the several groups of parameters calibrated in advance as the correction parameters of the current image according to the light source color estimation result in the auto white balance module. The accuracy of the auto white balance module is very easy to be affected by the shooting scene by only estimating the light source color through the image content, so when there is a large difference between the light source color estimated by the auto white balance module and the true value, the color correction module will also be affected, using the wrong calibration parameters, so that the final imaging effect will have serious color problems.

[0005] At present, the color correction method through several groups of correction matrix parameters also has some problems in the selection of calibration light source. If the calibration light source is too few, the continuity of the color correction result cannot be guaranteed, at this time, once the light source color estimation result fluctuates, the image after color correction will also present obvious jump; and if the calibration light source is too much, since the color digital camera only represents the light source through three response values (RGB), the influence of metamerism is large, and the probability of wrong selection of calibration parameters will also be greatly increased.

[0006] Therefore, it is expected to provide an improved color correction scheme. SUMMARY

[0007] The embodiment of the present application provides a color correction method, a spectral sensing device and an imaging device, which can convert original multi-channel response values of the imaging device into a target color space based on spectral data of current ambient light, thereby improving color correction accuracy of the imaging device.

[0008] According to an aspect of the present application, a color correction method is provided, comprising: obtaining spectral data of current ambient light; calculating original multi-channel response values of an imaging device for a specific reflectance sample based on the spectral data of the current ambient light and a spectral sensitivity function of the imaging device; obtaining stimulus values of the specific reflectance sample in a target color space; calculating a conversion mapping from the original multi-channel response values to the stimulus values in the target color space; and applying the conversion mapping to an image currently captured by the imaging device to convert the image to the target color space and perform color correction on the image in the target color space.

[0009] In the color correction method, a wavelength range of the spectral data of the current ambient light contains a wavelength range of the spectral sensitivity function of the imaging device, and sampling points of the spectral data of the current ambient light are consistent with sampling points of the spectral sensitivity function of the imaging device.

[0010] In the color correction method, the specific reflectance sample is a Munsell reflectance sample M(λ).

[0011] In the color correction method, the original three-channel response values of the imaging device for the specific reflectance sample are:

[0012] r n =∫ Ω P c (λ)M n (λ)S (r) (λ)dλ

[0013] g n =∫ Ω P c (λ)M n (λ)S (g) (λ)dλ

[0014] b n =∫ Ω P c (λ)M n (λ)S (b) (λ)dλ

[0015] wherein P c (λ) is the spectral data of the current ambient light, M n (λ) represents a spectral reflectance ratio of the nth Munsell reflectance sample; S (i)(λ), i e {r, g, b} represents the spectral sensitivity function of the i-th channel of the imaging device; Ω is the wavelength range of the spectral response of the imaging device; for N Munsell samples, the original multi-channel response value matrix D of the imaging device is calculated, each row in the matrix D respectively corresponds to the response value of each channel of a Munsell color block.

[0016] In the above color correction method, the original multi-channel response value of the imaging device for the specific reflectance sample is:

[0017] a n =∫ Ω P c (λ)M n (λ)S (a) (λ)dλ

[0018] b n =∫ Ω P c (λ)M n (λ) (b) (λ)dλ

[0019] g n =∫ Ω P c (λ)M n (λ)S (g) (λ)dλ

[0020] r n =∫ Ω P c (λ)M n (λ)S (r) (λ)dλ

[0021] f n =∫ Ω P c (λ)M n (λ)S (f) (λ)dλ

[0022] …

[0023] t n =∫ Ω P c (λ)M n (λ)S (t) (λ)dλ

[0024] Where P c (λ) is the spectral data of the current ambient light, M n (λ) represents the spectral reflectance of the n-th Munsell reflectance sample; S (i)wherein, (λ), i∈{a, b, g, r, f, …, t} represents the spectral sensitivity function of the i-th channel of the imaging device; Ω is the wavelength range of the spectral response of the imaging device; for N Munsell samples, a raw multi-channel response value matrix D of the imaging device is calculated, each row in the matrix respectively corresponds to the response values of each channel of a Munsell color block, for example, the response values of A, B, G, R, F, …, T channels.

[0025] In the color correction method described above, obtaining the stimulus value of the specific reflectance sample in the target color space includes: in response to the light source of the imaging device being a predetermined light source, directly reading the stimulus value of the specific reflectance sample corresponding to the predetermined light source in the target color space; and in response to the light source of the imaging device not being a predetermined light source, obtaining the stimulus value of the specific reflectance sample in the target color space based on the spectral data of the light source and the standard observer color matching function of the target color space.

[0026] In the color correction method described above, it is determined whether the light source is a predetermined light source based on the spectral data of the light source.

[0027] In the color correction method described above, the wavelength range of the spectral sensitivity function of the imaging device is the same as the wavelength range of the standard observer color matching function of the target color space, and the sampling points of the spectral sensitivity function of the imaging device are consistent with the sampling points of the standard observer color matching function of the target color space.

[0028] In the color correction method described above, the light source is a CIE D65 light source, and the stimulus value in the target color space is a CIE1931 XYZ standard value, which is represented as:

[0029] X D65,n =∫ Ω P D6 (λ)M n (λ)V (x) (λ)dλ

[0030] D6,n =∫ Ω P D65 (λ)M n (λ)V (y) (λ)

[0031] Z D65,n =∫ Ω P D65 (λ)M n (λ)V (z) (λ)

[0032] wherein, P D65 () represents a standard CIE D65 light source spectrum, M n(λ) represents the spectral reflectance of the nth Munsell reflectance sample, V (j) (λ), j e {x, y, z} correspond to different channels of CIE 1931 standard observer color matching functions, and Ω is the wavelength range of CIE 1931 XYZ standard observer color matching functions. For N Munsell samples, the stimulus value matrix H of the target color space is calculated, and each row in the matrix H corresponds to the stimulus value of a Munsell color block.

[0033] In the above color correction method, for the original multi-channel response value matrix D of the imaging device and the stimulus value matrix H of the target color space, the pseudo-inverse method is used to calculate the conversion mapping C of D converted to H, taking the color difference as the optimization method of the objective function:

[0034] C = argmin ΔE(D·C, H)

[0035] Wherein, ΔE(D·C, H) represents a function for calculating the color difference between D·C and H.

[0036] According to another aspect of the present application, a spectral sensing device is provided, which is applied to the color correction method as described above, for obtaining spectral data.

[0037] According to still another aspect of the present application, an imaging device is provided, which comprises the spectral sensing device as described above, and applies the color correction method as described above for color correction.

[0038] The color correction method, spectral sensing device and imaging device provided by the embodiments of the present application can convert the original multi-channel response value of the imaging device into the target color space based on the spectral data of the current ambient light, thereby improving the color correction accuracy of the imaging device. BRIEF DESCRIPTION OF DRAWINGS

[0039] Various other advantages and benefits of the present application will become apparent to those of ordinary skill in the art, upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of preferred embodiments of the present application and are not meant to limit the present application. Obviously, other embodiments of the present application will be apparent to those of ordinary skill in the art upon reading and understanding the following specification. Moreover, in the accompanying drawings, like reference numerals refer to components throughout the several views, of which:

[0040] Figure 1 A schematic flow chart of a color correction method according to an embodiment of the present application is shown.

[0041] Figure 2 A schematic diagram of a specific example flow of a color correction method according to an embodiment of the present application is shown.

[0042] Figure 3 Fig. 1 illustrates a schematic configuration diagram of a spectral sensing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] In the following, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of embodiments of the present application, rather than all embodiments of the present application, and it should be understood that the present application is not limited to the described example embodiments.

[0044] Schematic method

[0045] Figure 1 Fig. 2 illustrates a schematic flow chart of a color correction method according to an embodiment of the present application.

[0046] As shown in Fig. 2, the color correction method according to an embodiment of the present application comprises the following steps. Figure 1

[0047] In step S110, spectral data of a current ambient light is acquired. Specifically, in the embodiments of the present application, the spectral data of the current ambient light can be acquired by various types of ambient light sensors, for example, a spectral sensing device, such as a spectral sensing device, the details of which will be described in detail below.

[0048] In step S120, based on the spectral data of the current ambient light and a spectral sensitivity function of an imaging device, an original multi-channel response value of the imaging device for a specific reflectance sample is calculated. Here, the imaging device can be, for example, a color camera, and the spectral sensitivity function is a function obtained by calibrating the imaging device, for example, a factory calibration of a color camera.

[0049] That is, in order to make the color camera have higher accuracy for the light source estimation result, the color calculation is performed by combining the spectral data of the current ambient light and the spectral sensitivity function of the color camera, and first, the original multi-channel response value, for example, the original RGB response value, of the color camera for the specific reflectance sample needs to be calculated.

[0050] Here, it should be noted that in the above step of acquiring the spectral data of the current ambient light, the spectral data of the current ambient light acquired by the spectral sensing device, for example, denoted as P c ​(λ), which wavelength range needs to cover at least the wavelength range of the spectral sensitivity function of the imaging device, for example, the wavelength range of the CIE1931 XYZ standard observer color matching function, for example, denoted as V(λ). And, the spectral data of the current ambient light needs to be interpolated or resampled, etc. as necessary, so that the sampling points are consistent with the CIE1931 XYZ standard observer color matching function V(λ).

[0051] That is, in the color correction method according to the embodiments of the present application, the wavelength range of the spectral data of the current ambient light contains the wavelength range of the spectral sensitivity function of the imaging device, and the sampling points of the spectral data of the current ambient light are consistent with the sampling points of the spectral sensitivity function of the imaging device.

[0052] And, in the embodiments of the present application, in order to make the mapping of colors as accurate as possible, the specific reflectance sample is a Munsell reflectance sample, for example, denoted as M(λ).

[0053] In this case, the original three-channel (RGB channel) response value of the specific reflectance sample under the current ambient light by the imaging device, for example, a color camera, is:

[0054] r n =∫ Ω P c (λ)M n (λ)S (r) (λ)dλ

[0055] g n =∫ Ω P c (λ)M n (λ)S (g) (λ)dλ

[0056] b n =∫ Ω P c (λ)M n (λ)S (b) (λ)dλ

[0057] where P c (λ) is the spectral data of the current ambient light, M n (λ) represents the spectral reflectance of the nth Munsell reflectance sample; S (i) (λ), i∈{r,g,b} represents the spectral sensitivity function of the i-th channel of the imaging device; Ω is the wavelength range of the spectral response of the imaging device; and for N Munsell samples, for example, (N=1560), the original multi-channel response value matrix D of the imaging device can be calculated, each row of which corresponds to the response values of each channel, for example, R, G, B channels, of a Munsell color block.

[0058] On the basis of the above, multi-channel, for example, multi-channel can be represented as:

[0059] a n =∫ Ω P c (λ)M n (λ)S (a) (λ)dλ

[0060] b n =∫ Ω P c (λ)M n (λ)S (b) (λ)dλ

[0061] g n =∫ Ω P c (λ)M n (λ)S (g) (λ)dλ

[0062] r n =∫ Ω P c (λ)M n (λ)S (r) (λ)dλ

[0063] f n =∫ Ω P c (λ)M n (λ)S (f) (λ)dλ

[0064] …

[0065] t n =∫ Ω P c (λ)M n (λ)S (t) (λ)dλ

[0066] where P c (λ) is the spectral data of the current ambient light, M n (λ) represents the spectral reflectance of the nth Munsell reflectance sample; S (i) (λ), i∈{a, b, g, r, f, …, t} represents the spectral sensitivity function of the i-th channel of the imaging device; Ω is the wavelength range of the spectral response of the imaging device; for N Munsell samples, for example (N=1560), the original multi-channel response value matrix D of the imaging device can be calculated, each row in the matrix corresponds to the respective channel of a Munsell color block, for example, the response values of A, B, G, R, F, …, T channels.

[0067] Step S130, obtaining the stimulus value of the specific reflectance sample in the target color space. For example, for the specific reflectance sample of the color camera, it is determined that the target color space is CIE1931 XYZ color space, and in this case, the stimulus value in the target color space is the tristimulus value in the CIE1931 XYZ color space.

[0068] Specifically, in the embodiment of the present application, the spectral data of the light source and the standard observer color matching function of the target color space, such as the CIE1931 standard observer color matching function, can be used to obtain the stimulus value of the specific reflectance sample in the target color space. And the spectral data of the light source can also be set according to the needs, for example, the light source can be CIE D65 light source. In this case, if the spectral data of the light source is fixed, the corresponding stimulus value can be calculated in advance and stored, and if the spectral data of the light source changes, the stimulus value needs to be recalculated.

[0069] That is, in the color correction method according to the embodiment of the present application, obtaining the stimulus value of the specific reflectance sample in the target color space includes: in response to the light source of the imaging device being a predetermined light source, directly reading the stimulus value of the specific reflectance sample in the target color space corresponding to the predetermined light source; and in response to the light source of the imaging device not being a predetermined light source, obtaining the stimulus value of the specific reflectance sample in the target color space based on the spectral data of the light source and the standard observer color matching function of the target color space.

[0070] And in the above color correction method, whether the light source is a predetermined light source is determined based on the spectral data of the light source.

[0071] Here, in the embodiment of the present application, in order to improve the color correction accuracy as much as possible, the spectral sensitivity function of the imaging device, such as S (i) (λ), i∈{r,g,b} should be sampled into the same wavelength range as the standard observer color matching function of the target color space, such as the CIE1931 XYZ standard observer color matching function, such as V (j) (λ), j∈{x,y,z}, and the sampling points should be consistent.

[0072] That is, in the color correction method according to the embodiment of the present application, the wavelength range of the spectral sensitivity function of the imaging device is the same as the wavelength range of the standard observer color matching function of the target color space, and the sampling points of the spectral sensitivity function of the imaging device are consistent with the sampling points of the standard observer color matching function of the target color space.

[0073] And, in the embodiments of the present application, the purpose of color correction is to convert the original multi-channel response values of the imaging device, such as the original RGB response values, to the stimulus values in the target color space under the light source spectrum, such as the CIE D65, for example, the CIE1931 XYZ standard values, where the CIE1931 XYZ standard values under the CIE D65 light source can be expressed as:

[0074] X D6,n =∫ Ω P d65 (λ)M n (λ)V (x) (λ)dλ

[0075] Y D65,n =∫ Ω P D6 (λ)M n (λ)V (y) (λ)dλ

[0076] Z D6,n =∫ Ω P D65 (λ)M n (λ)V (z) (λ)dλ

[0077] where P D65 (λ) represents the standard CIE D65 light source spectrum, M n (λ) represents the spectral reflectance of the nth Munsell reflectance sample, V (j) (λ), j∈{x, y, z} respectively correspond to different channels of the CIE1931 standard observer color matching function, and Ω is the wavelength range of the CIE1931 XYZ standard observer color matching function. For N Munsell samples, for example (N = 1560), the stimulus value matrix H of the target color space can be obtained, for example, the standard tristimulus value matrix, each row in the matrix respectively corresponds to the stimulus value of a Munsell color block, for example, the X, Y, Z tristimulus values.

[0078] S140, calculate the conversion mapping of the original multi-channel response values to the stimulus values in the target color space. For example, calculate the conversion mapping between the original RGB response values of the color camera and the CIE1931 XYZ tristimulus values, for example, denoted as C(), so that XYZ = C(RGB). And in the embodiments of the present application, the constraint conditions of the conversion mapping C() can be the root mean square error, the color difference, etc., and preferably, the color difference constraint under the light source spectrum is used. For multi-channel, for example, six channels, then XYZ = C(ABGRFT).

[0079] Specifically, after obtaining the original multi-channel response value matrix D of the imaging device and the stimulus value matrix H of the target color space, a pseudo-inverse method or other methods can be used to calculate the conversion mapping C from D to H, taking the color difference as the optimization method of the objective function. When taking the color difference between D·C and H as the optimization objective, C can be calculated by using a nonlinear optimization method such as BFGS, for example:

[0080] C = argmin ΔE (D·C, H)

[0081] wherein ΔE (D·C, H) represents a function for calculating the color difference between D·C and H, for example, CIEDE2000 color difference.

[0082] Here, the method for calculating the conversion mapping includes but is not limited to using a pseudo-inverse method, a least squares method, a nonlinear optimization method, etc.

[0083] S150, applying the conversion mapping to the image currently captured by the imaging device to convert the image to the target color space and perform color correction on the image in the target color space.

[0084] That is, since color correction can be performed by only the conversion mapping in the color correction method according to the embodiments of the present application, the traditional white balance module and color correction module of the imaging device are not needed.

[0085] For example, in the case where the conversion mapping C is calculated, the currently captured RGB image can be converted to the CIE1931 color space as a whole, for example, the target color space is the CIE1931 XYZ color space under D65 light source:

[0086] Q XYZ = Q RGB ·C

[0087] wherein Q RGB represents the captured RGB image, the number of rows is equal to the number of pixels, and the number of columns is equal to 3; Q XYZ represents the tristimulus value of the CIE1931 color space, the number of rows is equal to the number of pixels, and the number of columns is equal to 3.

[0088] For the numerical value to be finally output or stored, there is:

[0089]

[0090] wherein C XYZ2RGB is the transformation matrix from the CIE1931 XYZ color space to the target color space, for example, using sRGB as the target color space, there is:

[0091]

[0092] Figure 2 Fig. 1 illustrates a schematic diagram of a specific example flow of a color correction method according to an embodiment of the present application.

[0093] Thus, according to the color correction method of the embodiments of the present application, by combining the spectral data of the current ambient light with the spectral sensitivity function of the imaging device, the original multi-channel response value of the imaging device can be converted into the target color space. Since the spectral data of the ambient light directly participates in the calculation, the white balance and color correction process is realized without relying on specific light source and color card calibration, that is, the traditional white balance module and color correction module of the imaging device can be directly skipped to obtain the final result, reducing the calculation process. Moreover, since the uncertainty problem of characterizing the light source according to the RGB three values in the traditional calibration scheme is avoided, the metamerism of the same temperature and different spectrum light source can be effectively avoided, and the accuracy and precision of the color environment of the imaging device are improved.

[0094] Schematic spectral sensing device

[0095] As described above, in the color correction method according to the embodiments of the present application, a spectral sensing device is needed to obtain the spectral data, such as the spectral data of the current ambient light and / or the spectral data of the light source. Here, Figure 3 Fig. 2 illustrates a schematic configuration diagram of a spectral sensing device according to an embodiment of the present application. As shown in Fig. 2, Figure 3 As shown in Fig. 2, in the spectral sensing device according to the embodiments of the present application, the optical system is optional, which can be a lens assembly, a light homogenization assembly, etc. The light filtering structure is a broadband light filtering structure in the frequency domain or the wavelength domain. The light transmission spectrum of different wavelengths of each light filtering structure is not exactly the same. The light filtering structure can be a super surface, a photonic crystal, a nanocolumn, a multilayer film, a dye, a quantum dot, a MEMS (micro-electro-mechanical system), an FP etalon, a cavity layer, a waveguide layer, a diffraction element, etc. with light filtering or light splitting properties. For example, in the embodiments of the present application, the light filtering structure can be the light modulation layer in Chinese patent CN201921223201.2, that is, implemented as a kind of computing spectral chip.

[0096] The image sensor (i.e. light detector array) can be a CMOS image sensor (CIS), a CCD, an array light detector, etc. In addition, the optional data processing unit can be an MCU, a CPU, a GPU, an FPGA, an NPU, an ASIC, etc. processing unit, which can export the data generated by the image sensor to the outside for processing.

[0097] For example, taking the computing spectrum chip as an example, after the image sensor measures the light intensity information, the data is transmitted to the data processing unit for recovery calculation. The process is described as follows:

[0098] The intensity signal of the incident light at different wavelengths λ is denoted as x(λ), the transmission spectrum curve of the filter structure is denoted as T(λ), the filter (filter structure) has m groups of structure units, and the transmission spectrum of each group of structure units is different from each other. As a whole, the filter structure can be denoted as Ti(λ) (i = 1, 2, 3, …, m). Each group of structure units has a corresponding physical pixel below, which detects the light intensity bi modulated by the filter structure. In a specific embodiment of the present application, one physical pixel, i.e., one physical pixel corresponds to one group of structure units, but it is not limited thereto. In other embodiments, a plurality of physical pixels can correspond to one group of structure units. Therefore, in the spectrum sensing device according to the embodiment of the present application, a plurality of groups of structure units constitute a “spectrum pixel”. Further, the present application can use at least one spectrum pixel to restore an image. It should be noted that the number of effective transmission spectra (transmission spectra used for spectrum recovery, called effective transmission spectra) of the filter structure can be inconsistent with the number of structure units. The transmission spectrum of the filter structure is artificially set, tested, or calculated according to certain rules according to the identification or recovery requirements (for example, the transmission spectrum of each structure unit obtained by testing is the effective transmission spectrum), so the number of effective transmission spectra of the filter structure can be less than the number of structure units, or even more than the number of structure units. In this variant embodiment, a certain transmission spectrum curve is not necessarily determined by a group of structure units.

[0099] The relationship between the spectral distribution of the incident light and the measurement value of the image sensor can be represented by the following formula:

[0100] bi = ∫x(λ)*Ti(λ)*R(λ)dλ

[0101] After discretization, we get:

[0102] bi = Σ(x(λ)*Ti(λ)*R(λ))

[0103] where R(λ) is the response of the image sensor, denoted as:

[0104] Ai(λ) = Ti(λ)*R(λ),

[0105] Then the above formula can be extended to matrix form:

[0106]

[0107] Wherein, bi (i = 1, 2, 3, …, m) is the response of the image sensor after the light to be measured transmits the light filtering structure, and corresponds to the light intensity measurement value of the image sensor corresponding to the m structural units respectively. When one physical pixel corresponds to one structural unit, it can be understood as the light intensity measurement value corresponding to m “physical pixels”, which is a vector with a length of m. A is the system response to light of different wavelengths, which is determined by the transmittance of the light filtering structure and the quantum efficiency of the image sensor. A is a matrix, and each row vector corresponds to the response of a group of structural units to incident light of different wavelengths. Here, the incident light is discretely and uniformly sampled, and there are n sampling points. The column number of A is the same as the sampling point number of the incident light. Here, x (λ) is the light intensity of the incident light at different wavelengths λ, that is, the incident light spectrum to be measured.

[0108] In some embodiments, different from the above embodiments, the light filtering structure can be directly formed on the upper surface of the image sensor, such as quantum dots, nanowires, etc., which directly form a light filtering structure or material (nanowire, quantum dot, etc.) on the photosensitive area of the sensor, taking the light filtering structure as an example. At this time, it can be understood that the raw material of the image sensor is processed to form a light filtering structure on the upper surface of the raw material when the image sensor is processed. The transmittance spectrum and the response of the image sensor are integrated, that is, the response of the detector and the transmittance spectrum are the same curve. At this time, the relationship between the spectral distribution of the incident light and the light intensity measurement value of the image sensor can be represented by the following formula:

[0109] bi = Σ (x (λ) * Ri (λ))

[0110] That is, in this embodiment, the transmittance spectrum Ai (λ) = Ri (λ)

[0111] Further, it can also be a combination of the above two embodiments, that is, at least one light filtering structure is arranged on the image sensor with a light filtering structure to modulate the incident light. It can be understood that the image sensor (i.e. the array of light detectors) in the first embodiment can be replaced by the image sensor integrated with the light filtering structure in the second embodiment.

[0112] At this time, the relationship between the spectral distribution of the incident light and the light intensity measurement value of the image sensor can be represented by the following formula:

[0113] bi = ∫x (λ) * Ti (λ) * Ri (λ) dλ

[0114] Discretization is performed again to obtain:

[0115] bi = Σ (x (λ) * Ti (λ) * Ri (λ))

[0116] That is, in this embodiment, Ai(λ) = Ti(λ) * Ri(λ)

[0117] If a multi-channel spectral imaging is considered, the imaging principle of each spectral pixel of a spectral imaging device, for example, a snapshot spectral camera, can be expressed by the following equation:

[0118] AX = B

[0119] The spectral information of the incident light is reconstructed by the above calculation, i.e., the spectral data acquired by the spectral sensing can be acquired.

[0120] The above describes the basic principles of the present application in combination with specific embodiments, however, it is pointed out that the advantages, advantages, effects, etc. mentioned in the present application are only examples and not limitations, and these advantages, advantages, effects, etc. cannot be considered as the must-have of each embodiment of the present application. In addition, the above-mentioned specific details are only for the purpose of example and for the purpose of understanding, and the above-mentioned details do not limit the present application to the must-use above-mentioned specific details.

[0121] The block diagrams of the devices, apparatuses, equipment, systems involved in the present application are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagram. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, which mean "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0122] It is also pointed out that in the devices, equipment and methods of the present application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombination should be considered as equivalent solutions of the present application.

[0123] The above description of the disclosed aspects is provided so that any person skilled in the art can make or use the present application. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0124] The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the application to the forms disclosed herein. Although various example aspects and embodiments have been discussed above, those of ordinary skill in the art will appreciate a variety of modifications, alternatives, permutations, additions, and sub-combinations, which fall within the scope of the application.

Claims

1. A color correction method, comprising: obtaining spectral data of a current ambient light; calculating original multi-channel response values of an imaging device for a specific reflectance sample based on the spectral data of the current ambient light and a spectral sensitivity function of the imaging device; obtaining stimulus values of the specific reflectance sample in a target color space; calculating a conversion mapping from the original multi-channel response values to the stimulus values in the target color space; and applying the conversion mapping to an image currently captured by the imaging device to convert the image to the target color space and perform color correction on the image in the target color space. The spectral data of the current ambient light has a wavelength range that is the same as a wavelength range of the spectral sensitivity function of the imaging device, and has sampling points that are consistent with sampling points of the spectral sensitivity function of the imaging device.

2. The color correction method of claim 1, wherein, The specific reflectance sample is a Munsell reflectance sample M(λ).

3. The color correction method of claim 1, wherein, The original three-channel response values of the imaging device for the specific reflectance sample are:

4. The color correction method of claim 3, wherein, Ω is a wavelength range of spectral response of the imaging device; for N Munsell samples, a matrix D of original multi-channel response values of the imaging device is calculated, each row of the matrix D corresponding to response values of each channel of a Munsell patch. r n = ∫ Ω P c (λ)M n (λ)S (r) (λ)dλ g n = ∫ Ω P c (λ)M n (λ)S (g) (λ)dλ b n = ∫ Ω P c (λ)M n (λ)S (b) (λ)dλ where P c (λ) is spectral data of the current ambient light, M n (λ) represents the spectral reflectance of the nth Munsell reflectance sample; S (i) (λ), i ∈ {r, g, b} represents the spectral sensitivity function of the i-th channel of the imaging device; The original multi-channel response values of the imaging device for the specific reflectance sample are:

5. The color correction method of claim 3, wherein, The obtaining of the stimulus values of the specific reflectance sample in the target color space comprises: a n = ∫ Ω P c (λ)M n (λ)S (a) (λ)dλ b n = ∫ Ω P c (λ)M n (λ)S (b) (λ)dλ g n = ∫ Ω P c (λ)M n (λ)S (g) (λ)dλ r n = ∫ Ω P c (λ)M n (λ)S (r) (λ)dλ f n = ∫ Ω P c (λ)M n (λ)S (f) (λ)dλ … t n = ∫ Ω P c (λ)M n (λ)S (t) (λ)dλ where P c (λ) is the spectral data of the current ambient light, M n (λ) represents the spectral reflectance of the nth Munsell reflectance sample; S (i) (λ), i ∈ {a, b, g, r, f, …, t} represents the spectral sensitivity function of the i th channel of the imaging device; Ω is the wavelength range of the spectral response of the imaging device; for N Munsell samples, the original multi-channel response value matrix D of the imaging device is calculated, and each row in the matrix corresponds to each channel of a Munsell color block, respectively.

6. The color correction method of claim 3, wherein, in response to a light source of the imaging device being a predetermined light source, directly reading the stimulus values of the specific reflectance sample in the target color space corresponding to the predetermined light source; and in response to the light source of the imaging device not being the predetermined light source, obtaining the stimulus values of the specific reflectance sample in the target color space based on spectral data of the light source and a standard observer color matching function of the target color space. The determination of whether the light source is the predetermined light source is based on spectral data of the light source.

7. The color correction method of claim 6, wherein, The wavelength range of the spectral sensitivity function of the imaging device is the same as a wavelength range of the standard observer color matching function of the target color space, and the sampling points of the spectral sensitivity function of the imaging device are consistent with sampling points of the standard observer color matching function of the target color space.

8. The color correction method of claim 6, wherein, The light source is a CIE D65 light source, and the stimulus values in the target color space are CIE 1931 XYZ standard values, represented as:

9. The color correction method of claim 8, wherein, For the matrix D of original multi-channel response values of the imaging device and the matrix H of stimulus values of the target color space, a conversion mapping C of D converted to H is calculated using a pseudo-inverse method, with a color difference as an optimization method of an objective function: X D6，n = ∫ Ω P D65 (λ)M n (λ)V (x) (λ)dλ Y D65，n = ∫ Ω P D65 (λ)M n (λ)V (y) (λ)dλ Z D65，n = ∫ Ω P D6 (λ)M n (λ)V (z) (λ)dλ where P D65 (λ) represents the standard CIE D65 illuminant spectrum, M n (λ) represents the spectral reflectance of the nth Munsell reflectance sample, V (j) (λ), j e {x, y, z} correspond to different channels of the CIE 1931 standard observer color matching functions, and Ω is the wavelength range of the CIE 1931 XYZ standard observer color matching functions. For N Munsell samples, a stimulus value matrix H of the target color space is calculated, each row in the matrix H corresponding to the stimulus value of a Munsell patch.

10. The color correction method of claim 9, wherein, C = argmin ΔE(D · C, H) where ΔE(D · C, H) represents a function used to calculate a color difference between D · C and H.

11. A spectral sensing device applied to the color correction method of any one of claims 1 to 10, for obtaining spectral data.

12. An imaging device comprising the spectral sensing device of claim 11 and applying the color correction method of any one of claims 1 to 10 to perform color correction. ​

Images