Backside illuminated image sensor and control method

Abstract

This invention relates to image sensor technology and discloses a back-illuminated image sensor and control method. The sensor includes a semiconductor substrate layer with a plurality of discretely arranged pixel units formed within it; a filter layer located above the semiconductor substrate layer, the filter layer including color filter units corresponding to the pixel units; a storage layer adjacent to the filter layer, the storage layer including storage units for storing color filter material; and a control layer connected to the storage layer and the filter layer. A control unit within the control layer controls the transfer of color filter material between the storage units and the color filter units, thereby altering the light transmittance characteristics of the color filter units. This invention significantly improves the image signal-to-noise ratio in black-and-white night vision mode by modulating the transmittance of the R, G, and B channels of the color filter, and compensates for local color deviations in the RGB daytime color image mode, making the overall image closer to human visual perception.

Description

Technical Field

[0001] This invention relates to image sensor technology, and more particularly to a back-illuminated image sensor and its control method. Background Technology

[0002] Current image sensor technologies primarily improve the near-infrared signal-to-noise ratio (SNR) by enhancing incident light collection in BSI CMOS technology, increasing the light-receiving surface in circuit design, and integrating materials more sensitive to near-infrared radiation. These methods achieve only minor improvements to the SNR of infrared night vision images. However, current CMOS color filters have fixed transmittance for their R, G, and B channels, making it difficult to effectively avoid localized color casts in mixed color temperature scenes; a trade-off between high and low color temperatures is only possible.

[0003] For example, the image sensor in the existing technology CN100407276C cannot effectively improve the signal-to-noise ratio of infrared night vision, and it cannot avoid local color distortion in scenarios such as mixed color temperatures. Summary of the Invention

[0004] This invention addresses the problem that existing back-illuminated image sensors can only be used for specific color temperatures or as a compromise between multiple color temperatures, and cannot avoid local color casts in scenarios with mixed color temperatures. It provides a back-illuminated image sensor and control method.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: A back-illuminated image sensor includes a semiconductor substrate layer, in which a plurality of discretely arranged pixel units are formed, and further includes: A filter layer located above a semiconductor substrate layer, the filter layer including color filter units corresponding to pixel units; A storage layer adjacent to the filter layer, the storage layer including storage units for storing color filter material; The control layer, which is connected to the storage layer and the filter layer, controls the transfer of the color filter material between the storage unit and the color filter unit through the control unit provided in the control layer, thereby changing the light transmission characteristics of the color filter unit.

[0006] Preferably, the color filter unit is made of transparent material. The color filter unit is designed to be transparent and hollow, allowing for free switching between effective color filtering and transparent states; Preferably, the filter material is a liquid. The filter material is designed to be liquid, which facilitates the material flowing out to fill the filter sheet and into the color storage chamber.

[0007] Preferably, it also includes a processing unit to acquire images collected by the sensor; Identify different color temperature regions in an image; Based on the target white balance gain in different color temperature regions, determine the target inflow rate of the liquid color filter material in the corresponding color filter unit. The control unit controls the amount of liquid filter material in the corresponding filter unit based on the target inflow rate.

[0008] Preferably, the processing unit control unit controls the amount of liquid filter material in the corresponding area of ​​the color filter unit based on the target inflow rate, including: Based on the first white balance gain of the first color temperature region in the acquired image, determine the first material inflow amount of the first type of color filter unit corresponding to the first color temperature region; Based on the second white balance gain of the second color temperature region in the image, determine the second material inflow amount of the second type of color filter unit corresponding to the second color temperature region.

[0009] Preferably, a microlens is also included, which is located above the color filter.

[0010] To address the aforementioned technical problems, the present invention also provides a control method for a back-illuminated image sensor, which includes the aforementioned image sensor; Control signals are generated based on the operating mode of the image sensor or the content of the acquired image; The control unit receives control signals, transfers the color filter material between the storage unit and the color filter unit, and changes the light transmission characteristics of the color filter unit.

[0011] Preferably, the image sensor's operating modes include RGB daytime color frame mode and black-and-white night vision mode.

[0012] Preferably, the image sensor operates in RGB daytime color mode. When in a mixed color temperature environment, including low color temperature areas and high color temperature areas, the white point in the low color temperature areas and high color temperature areas is statistically analyzed using a white balance algorithm.

[0013] Preferably, it also includes identifying at least a first color temperature region and a second color temperature region in the image; Determine the first target white balance gain corresponding to the first color temperature region and the second target white balance gain corresponding to the second color temperature region; Calculate the first material inflow amount of the color filter unit within the first color temperature region based on the first target white balance gain; Calculate the second material inflow amount of the color filter unit in the second color temperature region based on the second target white balance gain; The control signal is generated to drive the amount of filter material in the filter unit in the first color temperature region to approach the first material inflow amount, and to drive the amount of filter material in the filter unit in the second color temperature region to approach the second material inflow amount.

[0014] This invention, by adopting the above technical solutions, has significant technical effects: The storage unit of this invention is used to store liquid color filter material, enabling the color filter to become transparent during infrared night vision. The color filter unit is designed to be transparent and hollow, allowing for free switching between effective color filtering and transparent states. The color filter material of this invention is designed to be liquid, which facilitates the material to flow out and fill the color filter and flow into the color storage chamber; The present invention allows for adjustable flow rate of color filter material. For mixed color temperature scenarios, by adjusting the flow rate of color filter material, the transmittance of the R / G / B three channels in different color temperature regions is affected, thereby achieving global color matching the human eye's observation effect.

[0015] The back-illuminated image sensor designed in this invention significantly improves the signal-to-noise ratio of infrared night vision images, while the CMOS chip solves the problem of local color cast in device images, further significantly improving the signal-to-noise ratio of infrared night vision images and solving the problem of local color cast.

[0016] This invention allows the light-receiving surface to collect incident light more effectively, significantly improving its photosensitivity. For night vision images under commonly used 850 / 940nm infrared illumination, the color filters in current CMOS processes block a certain amount of infrared transmittance, reducing the signal-to-noise ratio of the image in infrared night vision mode.

[0017] This invention significantly improves the signal-to-noise ratio of images in black-and-white night vision mode by modulating the transmittance of the R, G, and B channels of the color filter, and compensates for local color deviations in the RGB daytime color image mode, making the overall image closer to the appearance observed by the human eye. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the image sensor distribution of the present invention.

[0019] Figure 2 This is a schematic diagram of the distribution of existing image sensors.

[0020] Figure 3 This is a schematic diagram showing the R / G and B / G distribution of each block in a mixed color temperature scene image.

[0021] Figure 4 It is the quantum efficiency spectrum of a traditional image sensor.

[0022] Figure 5 This is the quantum efficiency spectrum of the image sensor in black and white night vision mode according to the present invention.

[0023] Figure 6-1 The image is based on warm yellow curtains.

[0024] Figure 6-2 It is an image obtained by compromising on multiple color temperatures. Detailed Implementation

[0025] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0026] Example 1 A back-illuminated image sensor, Figure 1 The process includes a semiconductor substrate layer, within which multiple discretely arranged pixel units are formed, and further includes: A filter layer located above a semiconductor substrate layer, the filter layer including color filter units corresponding to pixel units; A storage layer adjacent to the filter layer, the storage layer including storage units for storing color filter material; The control layer, which is connected to the storage layer and the filter layer, controls the transfer of the color filter material between the storage unit and the color filter unit through the control unit provided in the control layer, thereby changing the light transmission characteristics of the color filter unit.

[0027] The color filter unit is made of transparent material.

[0028] The filter material is a liquid filter material.

[0029] It also includes a processing unit to acquire images collected by the sensor; Identify different color temperature regions in an image; Based on the target white balance gain in different color temperature regions, determine the target inflow rate of the liquid color filter material in the corresponding color filter unit. The control unit controls the amount of liquid filter material in the corresponding filter unit based on the target inflow rate.

[0030] The processing unit control unit controls the amount of liquid filter material in the corresponding area of ​​the color filter unit based on the target inflow rate, including: Based on the first white balance gain of the first color temperature region in the acquired image, determine the first material inflow amount of the first type of color filter unit corresponding to the first color temperature region; Based on the second white balance gain of the second color temperature region in the image, determine the second material inflow amount of the second type of color filter unit corresponding to the second color temperature region.

[0031] It also includes microlenses, which are located above the color filters.

[0032] Example 2 This embodiment is a control method for a back-illuminated image sensor, which includes the aforementioned image sensor; Control signals are generated based on the operating mode of the image sensor or the content of the acquired image; The control unit receives control signals, transfers the color filter material between the storage unit and the color filter unit, and changes the light transmission characteristics of the color filter unit.

[0033] The image sensor operates in two modes: RGB daytime color mode and black-and-white night vision mode.

[0034] Preferably, the image sensor operates in RGB daytime color mode. When in a mixed color temperature environment, including low color temperature areas and high color temperature areas, the white point in the low color temperature areas and high color temperature areas is statistically analyzed using a white balance algorithm.

[0035] It also includes identifying at least a first color temperature region and a second color temperature region in the image; Determine the first target white balance gain corresponding to the first color temperature region and the second target white balance gain corresponding to the second color temperature region; Calculate the first material inflow amount of the color filter unit within the first color temperature region based on the first target white balance gain; Calculate the second material inflow amount of the color filter unit in the second color temperature region based on the second target white balance gain; The control signal is generated to drive the amount of filter material in the filter unit in the first color temperature region to approach the first material inflow amount, and to drive the amount of filter material in the filter unit in the second color temperature region to approach the second material inflow amount.

[0036] Example 3 Based on the above embodiments, Figure 2 This is based on a traditional back-illuminated (BSI) CMOS design. This embodiment adds... Figure 2 The color filter corresponds to a color storage unit; the red color filter unit corresponds to a red storage unit, the green color filter unit corresponds to a green storage unit, and the blue color filter unit corresponds to a blue storage unit. Each storage unit is designed as an independent transparent storage space, and the color filter material is in liquid form to achieve the color filtering function.

[0037] When an image requires an RGB color frame, the corresponding color filter material flows out from the color storage chamber to fill the color filter, enabling Bayer data acquisition. When the image needs to enter infrared night vision mode, all the color filter material enters the color storage chamber, making the color filter transparent. At this time, all wavelengths of ambient light can pass through the filter layer, significantly increasing the infrared light transmittance compared to traditional CMOS chips.

[0038] When the device is in RGB daytime color frame mode, the default portion (e.g., 50%) of the color filter material flows into the color filter to achieve RGB color filtering. When in mixed color temperature (local high color temperature and local low color temperature), AWB performs white balance for specific color temperatures, while other color temperature areas achieve the final automatic white balance effect by modulating the inflow and outflow of the R / G / B three-channel color filter material.

[0039] Figure 3Each red dot represents the R / G and B / G values ​​measured on an 18% gray card under characteristic color temperatures such as D75, D65, D50, CWF, TL84, A, and H. The white balance algorithm creates a gray area based on these reference points. Areas in the image whose R / G and B / G values ​​fall within this gray area are considered white areas by human perception. In the mixed color temperature scene shown below, some white dots fall at high color temperatures, while others fall at low color temperatures.

[0040] The white balance algorithm performs AWB statistics on white points at low and high color temperatures separately.

[0041] Obtain the corresponding low color temperature white balance gains R1gain and B1gain, and high color temperature white balance gains R2gain and B2gain; where R1gain is the low color temperature red component gain, B1gain is the low color temperature blue component gain, R2gain is the high color temperature red component gain, and B2gain is the high color temperature blue component gain. R 低 R1gain=B 低 B1gain=G 低 ; R 高 R2gain=B 高 B2gain=G 高 ; Among them, R 低 B represents the red component in the low color temperature region. 低 For the blue component in the low color temperature region; G 低 For the green component in the low color temperature region; R 高 B represents the red component in the high-temperature region. 高 For the blue component in the high color temperature region; G 高 For the green component in the high color temperature region; for R 低 Red component and B in low color temperature region 低 Blue component in low color temperature region, G 低 Green component in low color temperature region, R 高 Red component in high-temperature region, B 高 Blue component and G in high color temperature region 高 The green component in the high color temperature region is obtained through Bayer data after photoelectric conversion on the image sensor. Assuming the image is collimated at a low color temperature, the global R / B channels are R1gain and B1gain respectively, and the white balance of the image in the low color temperature region (R=B=G) is normal.

[0042] High color temperature region R 高 R1gain≠B 高 B1gain≠G 高 In order to achieve R=B=G at high color temperatures, In the high color temperature region, the R and B channels need to perform gain operations similar to R2gain / R1gain and B2gain / B1gain.

[0043] Assume the transmittances yR, yG, and yB of the three channels R, G, and B of the color filter are related to the inflow rate x of the liquid color filter material, and: yR=f(x); yG=g(x); yB=k(x); The inflow rate x is assumed to be a by default. Then, the transmittance of the R and B channels under default conditions are f(a) and k(a), respectively. To achieve effective white balance at high color temperatures in mixed color temperature scenarios, the transmittance of the R and B channels is: F1(a) = f(a) * R2gain / R1gain; K1(a) = k(a) * B2gain / B1gain; Wherein, F1(a) is the effective white balance R channel transmittance of high color temperature in the mixed color temperature scene; K1(a) is the effective white balance B channel transmittance of high color temperature in the mixed color temperature scene; f(a) is the R channel transmittance in the default state; k(a) is the B channel transmittance in the default state. At this point, the inflow rates of liquid filter material in the R and B channels of the high color temperature zone are respectively: xR = f -1 (f(a)*R2gain / R1gain); xB=f -1 (k(a)* B2gain / B1gain); Where xR represents the inflow into the red R channel; and xB represents the inflow into the blue B channel. This allows for an effective white balance effect where R=G=B in the high color temperature region.

[0044] by Figure 4 Taking the quantum efficiency spectrum of a traditional CMOS image sensor as an example, the efficiency is approximately 0.3 at 850nm infrared and approximately 0.1 at 940nm infrared. Figure 5 For CMOS image sensors entering black-and-white night vision mode, the quantum efficiency spectrum in the 400nm~1000nm band can reach around 0.9, resulting in an improvement of more than 3 times for 850nm infrared and more than 9 times for 940nm infrared. For mixed color temperature scenarios, by modulating the amount of filter material flowing into different color temperature regions and controlling the transmittance of its R / G / B three channels, AWB can achieve accurate color reproduction in other color temperature regions even after calibration at a specific color temperature. This effectively solves the current problem that digital images cannot guarantee global color consistency with human visual perception in mixed color temperature scenarios. Figure 6-1Using warm yellow curtains as a white balance will make the walls appear noticeably bluish, while the brown floor color will be closer to the actual color. Figure 6-2 Using a compromise white balance with multiple color temperatures as a reference, the brown floor appears greenish, while the walls have a slight bluish tint, which can be subjectively improved.

Claims

1. A backside illumination image sensor comprising a semiconductor substrate layer, a plurality of pixel units arranged discretely formed within the semiconductor substrate, characterized by, Also includes: A filter layer located above a semiconductor substrate layer, the filter layer including color filter units corresponding to pixel units; A storage layer adjacent to the filter layer, the storage layer including storage units for storing color filter material; The control layer, which is connected to the storage layer and the filter layer, controls the transfer of the color filter material between the storage unit and the color filter unit through the control unit provided in the control layer, thereby changing the light transmission characteristics of the color filter unit.

2. The back-illuminated image sensor according to claim 1, characterized in that, The color filter unit is made of transparent material.

3. A back-illuminated image sensor according to claim 1, characterized in that, The filter material is a liquid filter material.

4. A back-illuminated image sensor according to claim 1, characterized in that, It also includes a processing unit to acquire images collected by the sensor; Identify different color temperature regions in an image; Based on the target white balance gain in different color temperature regions, determine the target inflow rate of the liquid color filter material in the corresponding color filter unit. The control unit controls the amount of liquid filter material in the corresponding filter unit based on the target inflow rate.

5. A back-illuminated image sensor according to claim 1, characterized in that, The processing unit control unit controls the amount of liquid filter material in the corresponding area of ​​the color filter unit based on the target inflow rate, including: Acquire the first white balance gain of the first color temperature region in the image, and determine the first material inflow amount of the first type of color filter unit corresponding to the first color temperature region; The second material inflow amount of the second type of color filter unit corresponding to the second color temperature region is determined by the second white balance gain in the second color temperature region of the image.

6. A back-illuminated image sensor according to claim 1, characterized in that, It also includes microlenses, which are located above the color filters.

7. A control method for a back-illuminated image sensor, characterized in that, Includes the image sensor as described in any one of claims 1 to 6; Control signals are generated based on the operating mode of the image sensor or the content of the acquired image; The control unit receives control signals, transfers the color filter material between the storage unit and the color filter unit, and changes the light transmission characteristics of the color filter unit.

8. The control method for a back-illuminated image sensor according to claim 7, characterized in that, The image sensor operates in two modes: RGB daytime color mode and black-and-white night vision mode.

9. The control method for a back-illuminated image sensor according to claim 8, characterized in that, The image sensor operates in RGB daytime color mode. When in a mixed color temperature environment, including low color temperature areas and high color temperature areas, the white balance algorithm is used to perform AWB statistics on the white points in the low color temperature areas and high color temperature areas.

10. The control method for a back-illuminated image sensor according to claim 7, characterized in that, It also includes identifying at least a first color temperature region and a second color temperature region in the image; Determine the first target white balance gain corresponding to the first color temperature region and the second target white balance gain corresponding to the second color temperature region; Calculate the first material inflow amount of the color filter unit within the first color temperature region based on the first target white balance gain; Calculate the second material inflow amount of the color filter unit within the second color temperature region based on the second target white balance gain; The control signal is generated to drive the amount of filter material in the filter unit in the first color temperature region to approach the first material inflow amount, and to drive the amount of filter material in the filter unit in the second color temperature region to approach the second material inflow amount.

Images