A CMOS sensor color temperature detection method

Abstract

The application discloses a CMOS sensor color temperature detection method, comprising a first filter array, a photodiode array, a sampling and processing circuit and a second filter array, the first filter array and the second filter array are installed side by side above the photodiode array, the second filter array is located at the edge pixel of the first filter array, the photodiode array is connected and outputs a signal to the sampling and processing circuit, a space area is arranged between each group of the second filter array, after an external detection light source enters the second filter array and is filtered to remove other spectra, the external detection light source enters the photodiode array and is converted into an electric signal, and after sampling and processing by the sampling and processing circuit, the irradiation brightness of different spectra of the external detection light source is obtained, and the color temperature of the external detection light source is calculated.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for color temperature detection using a CMOS sensor. Background Technology

[0002] The human eye possesses color constancy, meaning it can automatically recognize various colors under different lighting conditions, unaffected by color temperature. However, CCD / CMOS image sensors lack this function, resulting in color casts in captured images under different color temperature lighting conditions. Objects appear bluish under high color temperature light and reddish or yellowish under low color temperature light. This necessitates the automatic white balance (AWB) function to make the colors of an image closer to the natural colors observed by the human eye.

[0003] Automatic white balance (AWB) is a technology used in imaging devices such as cameras to automatically adjust the colors of an image so that it can present an accurate white under different color temperature light sources.

[0004] Currently, there are two main AWB (Auto-Aspect Ratio) methods: the grayscale world method and the white block method, along with subsequent improved algorithms based on these two assumptions. The advantage of these two algorithms is their low cost, low complexity, and ease of implementation. However, these algorithms are based on assumptions, and for various complex lighting conditions in reality, it is often difficult to accurately assess the color temperature, leading to significant color distortion in the camera.

[0005] Currently, high-end mobile phones use dedicated color temperature sensors. These sensors measure the spectral distribution of a light source and calculate its color temperature based on the relationship between spectral energy distribution and color temperature. The camera's white balance parameters are then adjusted according to the color temperature value to ensure accurate color reproduction in the image. This method offers high precision but requires additional spectral measurement equipment, resulting in higher costs and more demanding hardware requirements. Summary of the Invention

[0006] In view of the above-mentioned shortcomings in the current image processing technology field, the present invention provides a CMOS sensor color temperature detection method, which can significantly improve the accuracy of color temperature detection in complex light source environments. Compared with adding an additional independent color temperature sensor, it can significantly reduce the cost of use and has good performance and cost advantages.

[0007] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0008] A CMOS sensor color temperature detection method for detecting the color temperature of an external detection light source, comprising:

[0009] The system includes a first filter array, a photodiode array, and a sampling and processing circuit. It also includes multiple sets of second filter arrays. The first and second filter arrays are mounted side by side above the photodiode array. The photodiode array is connected and outputs signals to the sampling and processing circuit. The second filter arrays are positioned at the edge pixels of the first filter array. A gap is provided between each set of second filter arrays.

[0010] An external detection light source is input into the second filter array to filter out other spectra;

[0011] An external detection light source that filters out other spectra is input to a photodiode array and converted into an electrical signal;

[0012] The electrical signal is input to the sampling and processing circuit for sampling and processing to obtain the irradiance of different spectra of the external detection light source;

[0013] The color temperature of the external detection light source is obtained by calculating the irradiance of different spectra of the external detection light source.

[0014] According to one aspect of the present invention, the first filter array is configured as a Bayer filter array, and the second filter array is configured as a 380-730nm multi-row bandpass filter array. After the external detection light source enters the second filter array and other spectra are filtered out, it enters the photodiode array and is converted into an electrical signal. After being sampled and processed by the sampling and processing circuit, the irradiance of different spectra of the external detection light source is obtained, and the color temperature of the external detection light source is calculated.

[0015] According to one aspect of the present invention, each group of the second filter array is configured with the same fixed number of rows and columns, the number of rows being at least 1 row and the number of columns being at least 1 column, and each group of the second filter array is provided with the same fixed interval, the same fixed interval being at least 5 nanometers.

[0016] According to one aspect of the present invention, a blank row is provided between the second filter array and the first filter array, or no blank row is provided.

[0017] According to one aspect of the invention, each group of the second filter array has an empty column, or no empty column is provided.

[0018] According to one aspect of the present invention, the irradiance of different spectra of the external detection light source is set to B(λ, T), the color temperature expressed in absolute temperature is set to T, the Planck constant is set to h, the speed of light in vacuum is set to c, and the Boltzmann constant is set to k, thus obtaining the irradiance of different spectra of the external detection light source as follows:

[0019]

[0020] According to one aspect of the present invention, the pixel array signal is further subjected to analog processing, analog-to-digital conversion, image processing color temperature estimation in sequence, and then output to the MIPI interface or DVP interface.

[0021] According to one aspect of the present invention, the signal output by the configuration register control timing control is sequentially processed by analog, converted from analog to digital, and estimated by image processing color temperature before being output to the MIPI interface or DVP interface, or directly estimated by image processing color temperature before being output to the MIPI interface or DVP interface.

[0022] According to one aspect of the present invention, the irradiance of different spectra of the external detection light source is obtained by using the least squares method to minimize the sum of squared errors between the calculated irradiance and the actual measured value, and the color temperature of the external detection light source is obtained.

[0023] According to one aspect of the present invention, the fitted color temperature is set to T. fit , wavelength λ i The obtained spectral irradiance is set to B. mea (λ i ), λ i For 36 wavelengths ranging from 380 to 730 nm, the least squares fitting formula is obtained as follows:

[0024] minΣ i [B(λ i ,T fit )-B mea (λ i )] 2 .

[0025] The advantages of this invention are as follows: It includes a Bayer filter array, a photodiode array, a sampling and processing circuit, and a 380-730nm multi-row bandpass filter array. The Bayer filter array and the 380-730nm multi-row bandpass filter array are mounted side-by-side above the photodiode array. The 380-730nm multi-row bandpass filter array is located at the edge pixels of the Bayer filter array. The photodiode array is connected to and outputs signals to the sampling and processing circuit. An isolation zone is set between each 380-730nm multi-row bandpass filter array. After the external detection light source enters the 380-730nm multi-row bandpass filter array and other spectra are filtered out, it enters the photodiode array and is converted into an electrical signal. After sampling and processing by the sampling and processing circuit, the irradiance of different spectra of the external detection light source is obtained, and the color temperature of the external detection light source is calculated. This invention can significantly improve the accuracy of color temperature detection in complex light source environments. Compared with adding an additional independent color temperature sensor, it can significantly reduce the cost of use, and has good performance and cost advantages. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the structure of a CMOS sensor color temperature detection method according to the present invention;

[0028] Figure 2 This is a schematic diagram of the circuit structure of a CMOS sensor color temperature detection method according to the present invention;

[0029] Figure 3 This is a structural diagram of an embodiment of the CMOS sensor color temperature detection method described in this invention;

[0030] Figure 4 This is a schematic diagram of the layout of a CMOS sensor color temperature detection method according to the present invention.

[0031] Explanation of reference numerals in the attached figures:

[0032] 1. Second filter array; 2. First filter array; 3. Photodiode array; 4. Sampling and processing circuit. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Example 1:

[0035] like Figures 1 to 4 As shown, a CMOS sensor supporting color temperature detection includes a first filter array 2, a photodiode array 3, a sampling and processing circuit 4, and a second filter array 1. In this embodiment, the first filter array 2 is a Bayer filter array, and the second filter array 1 is a 380-730nm multi-row bandpass filter array. The Bayer filter array and the 380-730nm multi-row bandpass filter array are mounted side-by-side above the photodiode array 3. The 380-730nm multi-row bandpass filter array is located at the edge pixels of the Bayer filter array. The photodiode array 3 is connected to and outputs signals to the sampling and processing circuit 4.

[0036] This embodiment, based on a traditional Bayer array CMOS sensor, replaces the original Bayer array with a multi-row bandpass filter array of 380-730nm at the edge pixels of the CMOS sensor. Based on cost and detection accuracy requirements, the spacing of the 380-730nm multi-row bandpass filter array is set to T nanometers, where T≥5. Smaller spacing results in more accurate color temperature calculations but higher costs; conversely, larger spacing reduces costs but lowers accuracy. In this embodiment, the spacing T is set to 10nm, requiring 36 bandpass filters at wavelengths of 380, 390, 400, 410…700, 710, 720, and 730nm.

[0037] like Figure 4 As shown, each spectrum is configured with M rows and N columns, where M ≥ 1 and N ≥ 1. To reduce crosstalk between adjacent different spectra, a gap is set between each 380-730nm multi-row bandpass filter array. There are m empty rows between the 380-730nm multi-row bandpass filter array and the Bayer array, where m ≥ 0; there are n empty columns between adjacent 380-730nm multi-row bandpass filter arrays, where n ≥ 0.

[0038] The radiance of different spectra can be obtained using Planck's law formula:

[0039]

[0040] Where B(λ,T) is the spectral radiance, T is the color temperature expressed in absolute temperature, h is Planck's constant, c is the speed of light in vacuum, and k is Boltzmann's constant.

[0041] The external detection light source passes through a lens and enters a multi-row bandpass filter array of 380-730nm. After filtering out other spectra, it enters a photodiode array 3, which converts the light into an electrical signal. This signal is then processed by a sampling and processing circuit 4 to obtain the irradiance B of different spectra of the external detection light source. mea (λ i By using optimization algorithms such as the least squares method to minimize the sum of squared errors between the theoretically calculated spectral radiance and the actual measured value, the color temperature of the external detection light source can be obtained.

[0042] The relevant formulas are as follows:

[0043] min∑ i [B(λ i ,T fit )-B mea (λ i ) 2

[0044] Among them, T fit It is the color temperature obtained through fitting, B mea (λ i ) is the wavelength λ iThe spectral irradiance measured at λ i It has 36 wavelengths ranging from 380 to 730 nm.

[0045] In this embodiment, the CMOS sensor output resolution is selected as 1920*1080, and the effective pixel count is set to 1920*1088. A multi-row bandpass filter array ranging from 380-730nm is set in the first two rows. Each wavelength filter is configured with 52 columns * 2 rows, with a wavelength interval of 10nm, for a total of 36 wavelengths. The total number of bandpass filter array columns is 52 * 36 = 1872. To prevent crosstalk between different spectra, an empty column is set between filters of different spectra, for a total of 36 - 1 = 35 empty columns. Six empty columns are set in front of the bandpass filter array, and seven empty columns are set behind it, for a total of 1920 columns. Since each wavelength filter is configured with 52 columns * 2 rows, the total photosensitive area for each wavelength spectrum is very large, which can significantly improve the measurement sensitivity.

[0046] The advantages of this invention are as follows: It includes a Bayer filter array, a photodiode array, a sampling and processing circuit, and a 380-730nm multi-row bandpass filter array. The Bayer filter array and the 380-730nm multi-row bandpass filter array are mounted side-by-side above the photodiode array. The 380-730nm multi-row bandpass filter array is located at the edge pixels of the Bayer filter array. The photodiode array is connected to and outputs signals to the sampling and processing circuit. An isolation zone is set between each 380-730nm multi-row bandpass filter array. After the external detection light source enters the 380-730nm multi-row bandpass filter array and other spectra are filtered out, it enters the photodiode array and is converted into an electrical signal. After sampling and processing by the sampling and processing circuit, the irradiance of different spectra of the external detection light source is obtained, and the color temperature of the external detection light source is calculated. This invention can significantly improve the accuracy of color temperature detection in complex light source environments. Compared with adding an additional independent color temperature sensor, it can significantly reduce the cost of use, and has good performance and cost advantages.

[0047] Example 2:

[0048] A CMOS sensor color temperature detection method is implemented based on a CMOS sensor supporting color temperature detection as described in Embodiment 1. It includes the following steps:

[0049] S1: Install the Bayer filter array and multiple 380-730nm multi-row bandpass filter arrays above the photodiode array 3, connect the photodiode array 3 and output signals to the sampling and processing circuit 4, and set an isolation area between each 380-730nm multi-row bandpass filter array;

[0050] S2: Input the external detection light source into a 380-730nm multi-row bandpass filter array and filter out other spectra;

[0051] S3: Input the external detection light source that filters out other spectra to the photodiode array 3 and convert it into an electrical signal;

[0052] S4: Input the electrical signal into the sampling and processing circuit 4 for sampling and processing to obtain the irradiance of different spectra of the external detection light source;

[0053] S5: The irradiance of different spectra of the external detection light source is calculated by fitting the data using the least squares method to minimize the sum of squared errors between the calculated irradiance and the actual measured value, thus obtaining the color temperature of the external detection light source.

[0054] The advantages of this invention are as follows: A Bayer filter array and multiple 380-730nm multi-row bandpass filter arrays are mounted above a photodiode array. The photodiode array is connected and outputs signals to a sampling and processing circuit. An isolation zone is set between each 380-730nm multi-row bandpass filter array. An external detection light source is input into the 380-730nm multi-row bandpass filter array, filtering out other spectra. The external detection light source with filtered-out spectra is input into the photodiode array and converted into an electrical signal. The electrical signal is input into the sampling and processing circuit for sampling processing to obtain the irradiance of different spectra of the external detection light source. The irradiance of different spectra of the external detection light source is fitted and calculated using the least squares method to minimize the sum of squared errors between the calculated irradiance and the actual measured value, thus obtaining the color temperature of the external detection light source. This significantly improves the accuracy of color temperature detection in complex lighting environments. Compared to adding an additional independent color temperature sensor, it significantly reduces usage costs, offering good performance and cost advantages.

[0055] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A CMOS sensor color temperature detection method for detecting the color temperature of an external detection light source, characterized in that, include: The circuit consists of a first filter array (2), a photodiode array (3), and a sampling and processing circuit (4). It also includes multiple sets of second filter arrays (1). The first filter array (2) and the second filter array (1) are mounted side-by-side above the photodiode array (3). The photodiode array (3) is connected and outputs a signal to the sampling and processing circuit (4). The second filter array (1) is positioned at the edge pixels of the first filter array (2). A gap is provided between each set of second filter arrays (1). The first filter array (2) is a Bayer filter array, and the second filter array (1) is a 380-730nm multi-row bandpass filter array. An external detection light source is input into the second filter array (1) to filter out other spectra; An external detection light source that filters out other spectra is input to a photodiode array (3) and converted into an electrical signal; The electrical signal is input into the sampling and processing circuit (4) for sampling and processing to obtain the irradiance of different spectra of the external detection light source; The color temperature of the external detection light source is obtained by calculating the irradiance of different spectra of the external detection light source.

2. The CMOS sensor color temperature detection method according to claim 1, characterized in that, After the external detection light source enters the second filter array (1) and filters out other spectra, it enters the photodiode array (3) and is converted into an electrical signal. After being sampled and processed by the sampling and processing circuit (4), the irradiance of different spectra of the external detection light source is obtained, and the color temperature of the external detection light source is calculated.

3. The CMOS sensor color temperature detection method according to claim 1, characterized in that, Each group of the second filter array (1) is set to the same fixed number of rows and columns, the number of rows being at least 1 row and the number of columns being at least 1 column. Each group of the second filter array (1) is provided with the same fixed interval, the same fixed interval being at least 5 nanometers.

4. The CMOS sensor color temperature detection method according to claim 1, characterized in that, There may be a blank line between the second filter array (1) and the first filter array (2), or there may be no blank line.

5. The CMOS sensor color temperature detection method according to claim 1, characterized in that, The second filter array (1) has empty columns between each group, or no empty columns are set.

6. The CMOS sensor color temperature detection method according to claim 1, characterized in that, The irradiance of the external detection light source for different spectra is set to Let the color temperature expressed in absolute temperature be T, Planck's constant be h, the speed of light in vacuum be c, and Boltzmann's constant be k. The irradiance of different spectra of the external detection light source is then obtained as follows: 。 7. The CMOS sensor color temperature detection method according to claim 1, characterized in that, It also includes sequentially performing analog processing, analog-to-digital conversion, and image processing color temperature estimation on the pixel array signal before outputting it to the MIPI interface or DVP interface.

8. The CMOS sensor color temperature detection method according to claim 7, characterized in that, It also includes sequentially performing analog processing, analog-to-digital conversion, and image processing color temperature estimation on the signals output by the configuration register control timing control before outputting them to the MIPI interface or DVP interface, or directly performing image processing color temperature estimation before outputting them to the MIPI interface or DVP interface.

9. The CMOS sensor color temperature detection method according to any one of claims 1 to 7, characterized in that, The irradiance of different spectra of the external detection light source is obtained by using the least squares method to minimize the sum of squared errors between the calculated irradiance and the actual measured value, and the color temperature of the external detection light source is obtained.

10. The CMOS sensor color temperature detection method according to claim 8, characterized in that, Set the fitted color temperature to , wavelength The obtained spectral irradiance is set to , For 36 wavelengths ranging from 380 to 730 nm, the least squares fitting formula is obtained as follows: 。

Images