Image sensor and color restoration method

Abstract

The application discloses an image sensor and a color restoration method, and relates to the technical field of image sensors, and aims to improve the photosensitive capacity of the image sensor in a dark light source environment and output accurate color image signals. The image sensor comprises a plurality of pixel units arranged repeatedly, and each pixel unit comprises at least two infrared light pixels of different wave band ranges; each infrared light pixel corresponds to a visible light pixel group; the visible light pixel group comprises red, blue and green pixels; the pixel unit comprises a photosensitive layer, a color filter layer and a lens layer; the color filter layer comprises red, blue and green color filters which are arranged above the photosensitive layer corresponding to the red, blue and green pixels respectively; the color filter layer is further provided with an infrared light filter above the photosensitive layer of the infrared light pixel; the number of the infrared light filter depends on the wave band range of the infrared light pixel; the lens layer is arranged above the color filter layer, and the lens layer comprises a plurality of lenses, and each pixel is provided with a lens.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image sensor and a color restoration method. Background Technology

[0002] With the development of the security industry, the use of surveillance equipment is increasing in various industries and applications, such as finance, public security, smart city initiatives, the judiciary, transportation, and power. The demand for surveillance systems is constantly rising, and the image quality presented by cameras directly determines the degree of scene reproduction in the monitored area. Especially in dangerous locations such as dark nighttime environments, there is a greater desire for the image information returned by surveillance equipment to reproduce or closely approximate the real scene. Currently, most night-vision surveillance devices on the market use image sensors with RGB-IR configurations adjusted from the traditional RGB Bayer structure. The current RGB-IR Bayer array structure includes red, green, and blue pixels, as well as infrared pixels. Because the photoelectric conversion devices in the red, green, and blue visible light pixels are more sensitive to infrared light, infrared cutoff filters are usually placed on the lenses above the visible light pixels to ensure more accurate color results. Infrared pixels, however, do not have a filter layer and are used to receive light from the visible light band to the infrared band.

[0003] In bright environments, surveillance equipment can continuously sense light and reconstruct a color image of the monitored area based on the received light. However, in dark environments, only infrared pixels in the equipment can receive ambient light, especially infrared illumination sources in dark conditions, for scene reconstruction. Currently, most surveillance equipment in dark environments can only reconstruct a black and white image of the monitored area, making it difficult to clearly and accurately reproduce the color features of the scene. Because the black and white images generated in dark environments lack color features, users find it difficult to accurately identify colors in the monitored area when viewing the imaging data of surveillance equipment in dark environments, such as the color of a particular object or the color of a person's clothing, thus reducing the scene reconstruction accuracy of the surveillance equipment. How to output color images after sensing in dark environments is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] This application provides an image sensor and a color restoration method to improve the light sensitivity in low-light environments and output accurate color image signals.

[0005] In a first aspect, this application provides an image sensor, comprising:

[0006] It includes a plurality of repeating pixel units, each pixel unit containing at least two infrared light pixels of different wavelength ranges, specifically:

[0007] Each of the infrared light pixels corresponds to at least one visible light pixel group; wherein the visible light pixel group includes red pixels, blue pixels, and green pixels;

[0008] The pixel unit contains a photosensitive layer, a color filter layer, and a lens layer;

[0009] The color filter layer includes a red color filter, a blue color filter, and a green color filter, which are respectively disposed above the photosensitive layer corresponding to the red pixel, blue pixel, and green pixel;

[0010] The color filter layer also includes an infrared light filter located above the photosensitive layer corresponding to the infrared light pixel; wherein, the number of infrared light filters depends on the wavelength range contained in the infrared light pixel within the pixel unit;

[0011] The lens layer is located above the color filter layer; wherein, the lens layer contains a plurality of lenses, and one lens is provided for each pixel.

[0012] This improvement to the image sensor structure, compared to traditional RGB Bayer arrays or pixel arrangements that incorporate the W full-band, introduces infrared light pixel groups of specific wavelengths while retaining the traditional RGB pixel arrangement, thus enriching the image sensor's photosensitive color gamut. As is known to those skilled in the art, red filters primarily allow red light to pass through, green filters primarily allow green light to pass through, and blue filters primarily allow blue light to pass through. The image sensor provided in this application also includes an infrared light filter, which can absorb and penetrate infrared light that is invisible within a preset wavelength range to the photosensitive layer corresponding to the infrared light pixels, thereby improving the image sensor's photosensitive color gamut.

[0013] In one implementation, an infrared cut-off filter layer is disposed above the photosensitive layer corresponding to each red pixel, blue pixel and green pixel in the visible light pixel group.

[0014] An infrared cut-off filter layer is used to block the penetration of infrared light. Compared with coating an infrared cut-off filter layer directly on the lens, the image sensor provided in this embodiment of the invention sets an infrared cut-off filter layer between the color filter and the photosensitive layer corresponding to each pixel. This can avoid a large loss of light intake caused by the wavelength limitation of the infrared cut-off filter layer before the light signal received by the lens reaches the photosensitive layer. The total amount of light intake of the image sensor is increased, which improves the photosensitivity of the image sensor and thus improves the imaging performance of the image sensor under low light conditions.

[0015] In one implementation, an infrared light filter is further disposed within the color filter layer above the photosensitive layer corresponding to the infrared light pixel, specifically:

[0016] The color filter layer includes at least two infrared color filters with different transmittance bands; wherein the wavelength difference between the maximum peak wavelength and the minimum peak wavelength in the infrared filter is at least 90 nanometers.

[0017] In one implementation, each lens in the image sensor is coated with a dual-pass filter layer to allow visible and infrared light to pass through to the photosensitive layer and to filter out light within a preset wavelength range.

[0018] In one implementation, the image sensor further includes a photoelectric converter, a data converter, and a color restoration database, specifically:

[0019] Each pixel in the pixel unit contains a photoelectric converter; wherein the photoelectric converter is used to convert light received in the pixel into electrical charge;

[0020] The data converter is used to read the accumulated charge of the photoelectric converter and generate a digital quantization value based on the read charge;

[0021] The color restoration database is used to restore the color of the surface of the object under test based on the generated digital quantization value; wherein, the color restoration database contains several sets of digital quantization value records of different coating layers and objects in bright and dark environments respectively, and the color restoration database records the digital quantization value of each pixel in the pixel unit as a recording unit.

[0022] Secondly, this application also provides a color restoration method, including:

[0023] The reflected light from the surface of the object to be measured is input into the image sensor as described above;

[0024] The sensing value of the image sensor is obtained; wherein, when the sensing value is lower than the visible light sensing reference value, the surface of the object to be tested is placed in a preset infrared light source environment, and the infrared light sensing ratio value of the first infrared light pixel and the second infrared light pixel of the image sensor is obtained; wherein, the first infrared light pixel is the infrared light pixel with the largest band range in the pixel unit, and the second infrared light pixel is the infrared light pixel with the smallest band range in the pixel unit.

[0025] The infrared light sensing ratio value is input into the color restoration database for traversal query to obtain the infrared sensing value record with the smallest deviation from the infrared light sensing ratio value in the color restoration database; wherein, the color restoration database contains several sets of digital quantization value records of different standard samples in bright and dark environments respectively.

[0026] The visible light sensing value record corresponding to the infrared light sensing ratio value is called, and the color of the surface of the object to be tested is restored according to the visible light sensing value record.

[0027] In this way, by applying the image sensor described above, while increasing the light intake and improving the photosensitivity of the image sensor, a color restoration database is also established to obtain the sensing values ​​of the reflected light from the surface of the object under test. When the sensing value is greater than the visible light sensing reference value, the color coordinates are directly calculated based on the sensing value to restore the light to be tested. When the sensing value is less than the visible light sensing reference value, it is determined that the image sensor cannot sense visible light or the effect is poor in the current environment. The surface of the object under test is then placed in a preset infrared light source environment, and sensing is performed again under the infrared light source. An infrared light sensing ratio value is generated and traversed and queried in the pre-established color restoration database to obtain the sensing value record with the smallest deviation from the infrared light sensing ratio value. The visible light sensing value record corresponding to the infrared light sensing ratio value record is called, and the color of the surface of the object under test is restored based on the visible light sensing value record. The color restoration method provided in this application can still output the imaging effect in a bright light source environment even when acquiring the light to be tested in a dark environment, further improving the photosensitivity and imaging performance in a dark light source environment.

[0028] In one implementation, the process of establishing the color restoration database specifically includes:

[0029] The image sensor was placed in a dark room environment, and a standard lighting source was placed in the dark room environment to simulate bright ambient lighting;

[0030] Different standard samples were placed sequentially under the standard lighting environment.

[0031] The image sensor records the visible light sensing values ​​of RGB pixels under the bright environment and different standard samples; wherein, the sensing values ​​are RGB visible light digital quantization values.

[0032] In one implementation, the process of establishing the color restoration database further includes:

[0033] After the image sensor generates a visible light sensing value record, the standard illumination source is turned off, and an infrared light source group is placed in the dark room environment to simulate dark environment lighting; wherein, the infrared light source group contains at least two infrared light sources of different wavelengths.

[0034] Generate an infrared light sensing ratio record corresponding to each visible light sensing value record in a dark environment; wherein, the infrared light sensing ratio record is the ratio of the digital quantization values ​​of the first infrared light pixel to the second infrared light pixel.

[0035] In one implementation, the standard sample is an object, animal fur, or plant leaves with a colorant coating on its surface.

[0036] In one implementation, the infrared light source group can be a continuous infrared light band or two infrared light bands.

[0037] In one implementation, the infrared light source group includes at least two infrared light sources of different wavelengths, specifically:

[0038] The half-wavelength band of the infrared light source group includes the peak value of the filter layer of the first infrared light pixel and the peak value of the filter layer of the second infrared light pixel.

[0039] In one implementation, the step of obtaining the infrared light sensing ratio value record with the smallest deviation from the infrared light sensing ratio value in the color restoration database, calling the visible light sensing value record corresponding to the infrared light sensing ratio value record, and restoring the color of the object surface according to the visible light sensing value record specifically includes:

[0040] Obtain the visible light sensing value record corresponding to the infrared light sensing ratio record;

[0041] The red, green and blue colors of the surface of the object under test in a bright environment are reconstructed one by one based on the RGB visible light digital quantization values ​​recorded in the visible light sensing data.

[0042] Thirdly, this application also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the color restoration method described above.

[0043] Fourthly, this application also provides a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the color restoration method as described above. Attached Figure Description

[0044] Figure 1 This is a partial cross-sectional view of an image sensor provided in an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of a first arrangement of pixel units provided in an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of a second arrangement of pixel units provided in an embodiment of the present invention;

[0047] Figure 4This is a schematic diagram of a third arrangement of pixel units provided in an embodiment of the present invention;

[0048] Figure 5 This is a schematic flowchart of a color restoration method provided in an embodiment of the present invention;

[0049] Figure 6 This is a schematic diagram illustrating the relationship between wavelength and reflectance of a pigment sample, provided by an embodiment of the present invention.

[0050] Figure 7 This is a schematic diagram of a visible light digital value provided in an embodiment of the present invention. Detailed Implementation

[0051] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0052] The terms "first" and "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0053] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0054] First, some of the terms used in this application will be explained to facilitate understanding by those skilled in the art.

[0055] (1) DN value: DN value (Digital Number) is the brightness value of a pixel in a remote sensing image, which is the gray value of the recorded ground object. It has no unit and is an integer value. The value is related to the radiometric resolution of the sensor, the emissivity of the ground object, the atmospheric transmittance and scattering rate, etc. The general term for pixel value is digital quantization value or DN value, which is usually used to describe pixel values ​​that have not yet been calibrated to meaningful units.

[0056] (2) D65 light source: also known as international standard artificial daylight, its color temperature is 6500K.

[0057] Example 1

[0058] This invention provides an image sensor comprising a plurality of repeatedly arranged pixel units, each pixel unit containing at least two infrared pixels of different wavelength ranges, specifically:

[0059] Each infrared pixel corresponds to at least one visible light pixel group; wherein the visible light pixel group includes red pixels, blue pixels, and green pixels; the pixel unit includes a photosensitive layer, a color filter layer, and a lens layer; the color filter layer includes a red color filter, a blue color filter, and a green color filter, respectively disposed above the photosensitive layer corresponding to the red pixel, blue pixel, and green pixel; the color filter layer also includes an infrared color filter IR located above the photosensitive layer corresponding to the infrared pixel; wherein the number of infrared color filters depends on the wavelength range included by the infrared pixels in the pixel unit; the lens layer 103 is located above the color filter layer; wherein the lens layer includes a plurality of lenses, with one lens disposed corresponding to each pixel.

[0060] In this embodiment of the invention, a pixel unit contains at least two infrared light pixels with different wavelength ranges. Preferably, if a pixel unit contains three or more infrared light pixels, and the pixel unit contains two infrared light pixels with different wavelength ranges, it is permissible for one infrared light pixel to have the same wavelength range as the aforementioned two infrared light pixels. Correspondingly, the number of infrared light filters depends on the wavelength ranges contained in the infrared light pixels within the pixel unit. One infrared light filter is provided for each infrared light pixel with a wavelength range. If the infrared light pixels have the same wavelength range, the same infrared light filter is used.

[0061] See Figure 1 , Figure 1This is a partial cross-sectional view of an image sensor provided in an embodiment of the present invention. The embodiment of the present invention provides a partial cross-sectional view of an image sensor, which is a schematic diagram of a structure where one infrared pixel corresponds to one group of visible light pixels. Wherein, 101 is a photosensitive layer, 102 is a color filter layer, and 103 is a lens layer. 1011, 1012, 1013, and 1014 are photosensitive layers corresponding to red, blue, green, and infrared pixels, respectively. The color filter layer 102 includes a red filter RF, a blue filter BF, a green filter GF, and an infrared filter IR. Preferably, the color filter layer 102 includes at least two infrared filters IR with different transmittance bands; wherein the wavelength difference between the maximum peak wavelength and the minimum peak wavelength in the infrared filter is at least 90 nm. Each filter is disposed above the photosensitive layer of the corresponding pixel. Each pixel also has a corresponding lens disposed above the color filter layer. The lower cutoff band of the red filter RF is approximately 575nm; the blue filter BF has a center band of approximately 440nm to 475nm and an upper cutoff band of approximately 550nm, while also allowing infrared light with wavelengths greater than 800nm ​​to pass through; the green filter GE has a center band of 520nm to 550nm and upper and lower cutoff bands of approximately 620nm and 460nm respectively, while also allowing infrared light with wavelengths greater than 700nm to pass through. The wavelength range of infrared (IR) light is between 720 nm and 1000 nm. The typical wavelength of blue light is 435 nm to 450 nm, the typical wavelength of green light is 492 nm to 577 nm, and the typical wavelength of red light is 622 nm to 760 nm. Therefore, a blue filter (BF) allows blue light and infrared light to pass through, a green filter (GF) allows green light and infrared light to pass through, and a red filter (RF) allows red light and infrared light to pass through. An infrared filter (IR) allows infrared light of a specific wavelength range to pass through, and the specific wavelength range can be selected according to requirements.

[0062] Lenses 1031, 1032, 1033, and 1034 are respectively disposed above the red filter RF, blue filter BF, green filter GF, and infrared filter IR. Each of the lenses 103 in the lens layer 102 is coated with a dual-pass filter layer to allow visible and infrared light to pass through to the filter layer 102, while filtering out light within a preset wavelength range. Preferably, wavelengths below 380nm, 680-800nm, and above 1400nm are all filtered out by the dual-pass filter layer, further concentrating and focusing light to improve photosensitive purity.

[0063] As a preferred embodiment of the present invention, an infrared cut-off filter layer 104 is further coated on the photosensitive layer corresponding to each red pixel, blue pixel and green pixel in the visible light pixel group, so as to block the transmission of infrared light while maintaining high transmittance of visible light.

[0064] See Figure 2 , Figure 2 This is a schematic diagram of a first arrangement of pixel units according to an embodiment of the present invention. Pixel unit P1 contains at least two infrared pixels of different wavelength ranges, and each infrared pixel corresponds to at least one visible light pixel group. In this embodiment, a pixel unit P1 contains two IR pixels of different wavelengths, namely IR1 and IR2, and each IR pixel corresponds to one complete RGB pixel. That is, when there are two IR pixels of different wavelengths in the infrared pixel group in pixel unit P1, pixel unit P1 also contains at least two complete visible light pixel groups composed of RGB pixels. See also... Figure 3 , Figure 3 This is a schematic diagram of a second arrangement of pixel units provided in an embodiment of the present invention. It should be noted that the pixel unit provided in this embodiment is illustrated using two infrared light pixels IR1 and IR2 of different wavelength bands as an example, but it is not intended to limit the arrangement of pixel units. When a pixel unit already contains two infrared light pixels IR1 and IR2 of different wavelength ranges, the pixel unit can contain not only infrared light pixels of different wavelength ranges, but also infrared light pixels of the same wavelength range as the aforementioned two infrared light pixels IR1 and IR2. Furthermore, any pixel can be used as the starting pixel of pixel unit P1, and the arrangement relationship between IR pixels and visible light pixels (RGB) is not limited. See also... Figure 4 , Figure 4 This is a schematic diagram of a third arrangement of pixel units provided in an embodiment of the present invention. Figure 4 This is a schematic diagram illustrating the arrangement of three different IR pixels, IR1, IR2, and IR3, within a pixel unit P1 according to an embodiment of the present invention. When a pixel unit contains more than two IR pixels, its composition structure is adjusted accordingly to ensure that each infrared pixel in the pixel unit is mixed with a complete GRB pixel.

[0065] In this embodiment of the invention, each pixel in each pixel group within a pixel unit includes a photoelectric converter, which can be a photodiode, used to convert light signals into electrical signals or charges. Further, the image sensor in this embodiment also includes a data converter, used to read the accumulated charge of the photoelectric converter and generate a digital quantization value based on the read charge. Specifically, the data converter can be an analog-to-digital converter. The image sensor provided in this embodiment also includes a color restoration database, used to restore the color of the surface of the object under test based on the generated digital quantization value; wherein, the color restoration database contains several sets of digital quantization value records for different coating layers and objects in bright and dark environments respectively, and the color restoration database records the digital quantization value of each pixel within the pixel unit, using one pixel unit as the recording unit. It should be noted that in this embodiment of the invention, one pixel unit is used as the object for generating digital quantization values, and the digital quantization value of each pixel unit is calculated sequentially according to the moving arrangement direction of the pixel units. Preferably, see... Figure 2 , Figure 2 The pixel unit P1 is arranged by shifting one pixel to the right. Let the pixel unit P1 enclosed by the dashed box be the initial pixel unit; then the pixel unit enclosed by the solid box is the arrangement after shifting the initial pixel unit one pixel to the right. As another embodiment of the present invention, see [link to relevant documentation]. Figure 3 , Figure 3 The middle pixel unit P1 is arranged by shifting down one pixel. It can be observed that... Figure 2 and Figure 3 The initial pixel unit structure is the same, but their arrangement direction can be completely different. Pixel unit P1 can also be arranged one pixel away from the top or one pixel away from the left. For the sake of brevity and convenience, it will not be elaborated here.

[0066] This invention provides an image sensor with an improved structure. Compared to traditional RGB Bayer arrays or pixel arrangements that incorporate the W full-band, this invention retains the traditional RGB pixel arrangement while introducing infrared light pixel groups of specific wavelengths, thus enriching the image sensor's photosensitive color gamut. As those skilled in the art know, red filters primarily allow red light to pass through, green filters primarily allow green light to pass through, and blue filters primarily allow blue light to pass through. The image sensor provided in this invention also includes an infrared light filter, which allows invisible infrared light within a preset wavelength range to be absorbed and penetrate to the photosensitive layer corresponding to the infrared light pixels, further improving the image sensor's photosensitive color gamut. An infrared cut-off filter layer is used to block the penetration of infrared light. Compared with coating an infrared cut-off filter layer directly on the lens, the image sensor provided in this embodiment of the invention sets an infrared cut-off filter layer between the color filter and the photosensitive layer corresponding to each pixel. This can avoid a large loss of light intake caused by the wavelength limitation of the infrared cut-off filter layer before the light signal received by the lens reaches the photosensitive layer. The total amount of light intake of the image sensor is increased, which improves the photosensitivity of the image sensor and thus improves the imaging performance of the image sensor under low light conditions.

[0067] Example 2

[0068] See Figure 5 , Figure 5 This is a flowchart illustrating a color restoration method provided in an embodiment of the present invention. The embodiment of the present invention provides a color restoration method, including steps 201 to 204, each step of which is detailed below:

[0069] Step 201: Input the reflected light from the surface of the object to be measured into the image sensor as described above;

[0070] Step 202: Obtain the sensing value of the image sensor; wherein, when the sensing value is lower than the visible light sensing reference value, the surface of the object to be tested is placed in a preset infrared light source environment, and the infrared light sensing ratio value of the first infrared light pixel and the second infrared light pixel of the image sensor is obtained; wherein, the first infrared light pixel is the infrared light pixel with the largest band range in the pixel unit, and the second infrared light pixel is the infrared light pixel with the smallest band range in the pixel unit;

[0071] Step 203: Input the infrared light sensing ratio value into the color restoration database for traversal query, and obtain the infrared sensing value record with the smallest deviation from the infrared light sensing ratio value in the color restoration database; wherein, the color restoration database contains several sets of digital quantization value records of different standard samples in bright and dark environments respectively.

[0072] Step 204: Call the visible light sensing value record corresponding to the infrared light sensing ratio value record, and restore the color of the surface of the object to be tested according to the visible light sensing value record.

[0073] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific structure and working process of the image sensor can be referred to the corresponding process in the aforementioned Embodiment 1, and will not be repeated here.

[0074] The process of establishing a color reproduction comparison database specifically includes: placing an image sensor in a darkroom environment and introducing a standard illumination source into the darkroom environment to simulate bright ambient lighting; sequentially introducing different standard samples under the standard illumination source environment; and recording the visible light sensing values ​​of the image sensor for RGB pixels under the bright environment and different standard samples; wherein, the sensing values ​​are RGB visible light digital quantization values. The absorption / reflection spectra of the same wavelength after passing through different dyes are generally consistent, with their waveform curve changes and characteristic peaks remaining almost unchanged, only with slight differences in intensity characteristics. Based on this characteristic, in this embodiment of the invention, a D65 light source is set in a darkroom to simulate a bright environment, and a pigment sample CB is introduced under this light source environment to obtain the visible light digital quantization values ​​sensed by the image sensor under bright ambient lighting. In one embodiment, the standard sample is an object with a pigment coating on its surface, animal fur, or plant leaves. Preferably, embodiments of the present invention further include establishing the sensing digital quantization value of the image sensor under various light source environments such as D50, D75, TL84, CWF, UV, F, A, U30, and HOR, and under various pigment samples such as CB, UB, and PhB. Based on the established image sensor's sensing digital quantization value record for visible light, a visible light sensing reference value for the image sensor can be determined. When the sensing value of the image sensor is greater than the visible light sensing reference value, the image sensor can directly reconstruct the light to be tested based on the sensed visible light digital quantization value. It should be noted that the visible light sensing value includes the red light digital quantization value, the blue light digital quantization value, and the green light digital quantization value. When the sensing value of the image sensor is lower than the visible light sensing reference value, it is determined that the brightness in the detection environment is low, and under normal detection conditions, it cannot be reconstructed by the image sensor.

[0075] Furthermore, the process of establishing the color restoration database also includes: after the image sensor generates a visible light sensing value record, the standard illumination source is turned off, and an infrared light source group is placed in a dark room environment to simulate dark environment illumination; an infrared light sensing ratio value record corresponding to each visible light sensing value record in the dark environment is generated; wherein, the infrared light sensing ratio value record is the ratio of the digital quantization values ​​of the first infrared light pixel and the second infrared light pixel. In this embodiment of the invention, an infrared light source group is set in a dark room environment, wherein the light source group contains at least two infrared light sources of different wavelengths. Preferably, the infrared light source group can be a continuous infrared light band or two infrared light bands. The infrared light source group contains at least two infrared light sources of different wavelengths, specifically: the half-band of the infrared light source group includes the filter layer peak value of the first infrared light pixel and the filter layer peak value of the second infrared light pixel. After each visible light sensing value record is generated, the illumination source is turned off, and the infrared light source group is turned on to simulate a dark environment. It should be noted that the detection in the bright environment and the dark environment is the same except for the light source. The visible light sensing value under each bright environment is generated to correspond to the infrared sensing value under the dark environment. As a preferred method of this embodiment of the invention, in order to enhance the imaging effect under the infrared light source group, the wavelength difference between the peak wavelength of the first band and the peak wavelength of the second band in the two infrared light filters is at least 90 nm.

[0076] See Figure 6 , Figure 6 This is a schematic diagram illustrating the relationship between wavelength and reflectance of a pigment sample, provided by an embodiment of the present invention. The horizontal axis represents wavelength (Wavelength), and the vertical axis represents reflectance (R%). Figure 6The data includes the reflectance changes of different wavelengths after passing through dyes PhB, UB, and CB. Preferably, the digital sensing values ​​of 850nm infrared light under CB, PhB, and UB dyes are 80.5, 87, and 70, respectively. The digital sensing values ​​of 1350nm infrared light under CB, PhB, and UB dyes are 38, 75.5, and 72, respectively. In this embodiment of the invention, the infrared light source group includes one 850nm and one 1350nm infrared light source. The pigment sample CB is placed in this light source environment, and the infrared sensing values ​​generated by the image sensor in this dark environment are obtained. The infrared sensing values ​​are the ratio of the two digital quantization values ​​of the infrared light in the infrared light source group. Specifically, after passing through dye CB, the digital quantization value sensed by the image sensor from the 850nm infrared light source is 80.5, and after passing through dye CB from the 1350nm infrared light source, the digital quantization value sensed by the image sensor is 72. Therefore, the generated infrared sensing value is recorded as CB_Color(x, y) = 80.5 / 38 = 2.118. Preferably, embodiments of the present invention can also generate infrared sensing value records for other different pigment sample cases. When the pigment sample is UB, the infrared sensing value record UB_Color (x, y) = 70 / 72 = 0.972; when the pigment sample is PhB, the infrared sensing value record PhB_Color (x, y) = 87 / 75.5 = 1.152. As a preferred embodiment of the present invention, the infrared light source in the light source group can also be set to a classic infrared light source, such as a 940nm and a 1350nm infrared light source. The infrared light source group can also contain three or more infrared light sources. Correspondingly, the pigment samples are also adjusted and replaced accordingly to expand the data records for different infrared light source groups and pigment samples.

[0077] In this embodiment of the invention, when the image sensor's sensing value of the light to be measured is less than the visible light sensing reference value, the light to be measured is placed in a preset infrared light source environment to obtain the infrared sensing value of the image sensor. This preset light source environment can be any infrared light source environment used by the color restoration database to generate infrared sensing data records. The infrared sensing value generated by the image sensor is obtained and input into the color restoration database for traversal querying. The infrared sensing value record with the smallest deviation between the color restoration database and the current image sensor's infrared sensing value is obtained, and the corresponding visible light sensing value record is called. Based on the RGB visible light digital quantization values ​​in the visible light sensing value record, the red, green, and blue colors of the light to be measured in a bright environment are reconstructed one by one. See also... Figure 7 , Figure 7This is a schematic diagram of visible light digital quantization values ​​provided in an embodiment of the present invention. The horizontal axis represents the proportion of the infrared light digital quantization value, and the vertical axis represents the range of the digital quantization value. The diagram sequentially includes the digital quantization values ​​of red, green, and blue light. In this embodiment, the image sensor reconstructs visible light using 8 bits, and the range of the digital quantization value DN is between 0 and 255. When the image sensor's sensing value is greater than the visible light sensing reference value, the red, green, and blue colors of the light being measured are directly reconstructed one by one based on the digital quantization value of each visible light ray in the sensing value.

[0078] In this embodiment of the invention, a photosensitive device is also provided, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the above-described color restoration method.

[0079] In this embodiment of the invention, a computer-readable storage medium is also provided. The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the aforementioned color restoration method. Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and these instruction segments describe the execution process of the computer program in the color restoration device.

[0080] The data collection device for the color reproduction device can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The data collection device for the color reproduction device may include, but is not limited to, a processor, memory, and display. Those skilled in the art will understand that the above components are merely examples of a color reproduction device and do not constitute a limitation on the color reproduction device. It may include more or fewer components than described above, or a combination of certain components, or different components. For example, the color reproduction device may also include input / output devices, network access devices, buses, etc.

[0081] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the color reproduction device, connecting all parts of the device via various interfaces and lines.

[0082] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the color reproduction device by running or executing the computer programs and / or modules stored in the memory and by calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, text conversion function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, text message data, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0083] If the color restoration device integrated module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals. This can be understood and implemented by those skilled in the art without any creative effort.

[0084] This invention provides a color restoration method that utilizes the image sensor described in Embodiment 1. While increasing the light intake and photosensitivity of the image sensor, it also establishes a color restoration database to obtain the sensor's sensing values ​​for the light to be tested. When the sensing value is greater than the visible light sensing reference value, color coordinate calculations are performed directly based on the sensing value to restore the light to be tested. When the sensing value is less than the visible light sensing reference value, it is determined that the image sensor cannot sense visible light or the effect is poor in the current environment. The light to be tested is then placed in a preset infrared light source environment, and sensing is performed again under the infrared light source. Infrared sensing values ​​are generated and traversed and queried in the pre-established color restoration database to obtain the sensing value record with the smallest deviation from the infrared sensing value. The corresponding visible light sensing value record is then called, and the light to be tested is restored based on the visible light sensing value record. This color restoration method provided by this invention can still output the imaging effect under bright light source environment even when acquiring the light to be tested in a dark environment, further improving the photosensitivity and imaging performance in dark light source environments.

[0085] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. An image sensor, characterized in that, It includes a plurality of repeating pixel units, each pixel unit containing at least two infrared light pixels of different wavelength ranges, specifically: Each of the infrared light pixels corresponds to at least one visible light pixel group; wherein the visible light pixel group includes red pixels, blue pixels, and green pixels; The pixel unit contains a photosensitive layer, a color filter layer, and a lens layer; The color filter layer includes a red color filter, a blue color filter, and a green color filter, which are respectively disposed above the photosensitive layer corresponding to the red pixel, blue pixel, and green pixel; The color filter layer also includes an infrared light filter located above the photosensitive layer corresponding to the infrared light pixel; wherein, the number of infrared light filters depends on the wavelength range contained in the infrared light pixel within the pixel unit; The lens layer is located above the color filter layer; wherein, the lens layer contains a plurality of lenses, and one lens is provided for each pixel; The image sensor also includes a photoelectric converter, a data converter, and a color restoration database, specifically: Each pixel in the pixel unit contains a photoelectric converter; wherein the photoelectric converter is used to convert light received in the pixel into electrical charge; The data converter is used to read the accumulated charge of the photoelectric converter and generate a digital quantization value based on the read charge; The color restoration database is used to restore the color of the surface of the object under test based on the generated digital quantization value; wherein, the color restoration database contains several sets of digital quantization value records of different coating layers and objects in bright and dark environments respectively, and the color restoration database records the digital quantization value of each pixel in the pixel unit as a recording unit.

2. An image sensor as described in claim 1, characterized in that, An infrared cut-off filter layer is provided above the photosensitive layer corresponding to each red, blue, and green pixel in the visible light pixel group.

3. An image sensor as described in claim 1, characterized in that, The color filter layer also includes an infrared light filter located above the photosensitive layer corresponding to the infrared light pixel, specifically: The color filter layer includes at least two infrared color filters with different transmittance bands; wherein the wavelength difference between the maximum peak wavelength and the minimum peak wavelength in the infrared filter is at least 90 nanometers.

4. An image sensor as described in claim 1, characterized in that, In the image sensor, each lens is coated with a dual-pass filter layer to allow visible light and infrared light to penetrate to the photosensitive layer, while filtering out light within a preset wavelength range.

5. A color restoration method, characterized in that, include: The reflected light from the surface of the object to be measured is input into the image sensor as described in any one of claims 1 to 4; The sensing value of the image sensor is obtained; wherein, when the sensing value is lower than the visible light sensing reference value, the surface of the object to be tested is placed in a preset infrared light source environment, and the infrared light sensing ratio value of the first infrared light pixel and the second infrared light pixel of the image sensor is obtained; wherein, the first infrared light pixel is the infrared light pixel with the largest band range in the pixel unit, and the second infrared light pixel is the infrared light pixel with the smallest band range in the pixel unit. The infrared light sensing ratio value is input into the color restoration database for traversal query to obtain the infrared sensing value record with the smallest deviation from the infrared light sensing ratio value in the color restoration database; wherein, the color restoration database contains several sets of digital quantization value records of different standard samples in bright and dark environments respectively. Call the infrared light sensing ratio value record and record the corresponding visible light sensing value record, and restore the color of the surface of the object to be tested based on the visible light sensing value record.

6. The color restoration method as described in claim 5, characterized in that, The process of establishing the color restoration database specifically includes: placing the image sensor in a dark room environment and placing a standard lighting source in the dark room environment to simulate bright ambient lighting; Different standard samples were placed sequentially under the standard lighting environment. The image sensor records the visible light sensing values ​​of RGB pixels under the bright environment and different standard samples; wherein, the sensing values ​​are RGB visible light digital quantization values.

7. The color restoration method as described in claim 6, characterized in that, The process of establishing the color restoration database also includes: After the image sensor generates a visible light sensing value record, the standard illumination source is turned off, and an infrared light source group is placed in the dark room environment to simulate dark environment lighting; wherein, the infrared light source group contains at least two infrared light sources of different wavelengths. Generate an infrared light sensing ratio record corresponding to each visible light sensing value record in a dark environment; wherein, the infrared light sensing ratio record is the ratio of the digital quantization values ​​of the first infrared light pixel to the second infrared light pixel.

8. The color restoration method as described in claim 5, characterized in that, The standard samples are objects with a colored coating on their surface, animal fur, or plant leaves.

9. A color restoration method as described in claim 7, characterized in that, The infrared light source group includes at least two infrared light sources of different wavelengths, specifically: The half-wavelength band of the infrared light source group includes the peak value of the filter layer of the first infrared light pixel and the peak value of the filter layer of the second infrared light pixel.

10. A color restoration method as described in claim 5, characterized in that, The step of obtaining the infrared light sensing ratio value record with the smallest deviation from the infrared light sensing ratio value in the color restoration database, calling the visible light sensing value record corresponding to the infrared light sensing ratio value record, and restoring the color of the surface of the object to be tested based on the visible light sensing value record specifically includes: Obtain the corresponding visible light sensing value record for the infrared light sensing ratio record; The red, green and blue colors of the surface of the object under test in a bright environment are reconstructed one by one based on the RGB visible light digital quantization values ​​recorded in the visible light sensing data.

Images