An image sensor with spectral detection and phase focusing function

Abstract

The application discloses an image sensor with spectrum detection and phase focusing functions, comprising a pixel array, a color filter array composed of multiple color filter units covering the pixel array, wherein the color filter units comprise first color filter units and second color filter units, each first color filter unit comprises at least one specific color filter, and each second color filter unit is composed of multiple basic color filters; wherein the color of the specific color filter is at least partially the same as that of the basic color filter, and the light signal passing through the specific color filter is used at least for spectrum measurement, and the light signal passing through the basic color filter is used at least for automatic focusing. By integrating the pixel array for spectrum measurement and phase focusing together, the precision of color restoration is improved, and the calculation difficulty of white balance coefficient and color conversion matrix is reduced without affecting the phase focusing and imaging functions.

Description

Technical Field

[0001] This invention relates to the field of image sensors, and more particularly to a pixel structure and an image sensor. Background Technology

[0002] Image sensors utilize the photoelectric conversion function of photoelectric devices in a pixel array to convert the light image on the photosensitive surface into an electrical signal proportional to the light image. This signal is then processed and stored by peripheral circuits to record image information.

[0003] Red-green-blue (RGB) imaging technology refers to acquiring the red, green, and blue spectral information of a target object within the visible light spectrum to generate an image of the object. Hyperspectral or multispectral imaging technologies have more spectral bands, enabling the acquisition of light signals from the target object across a wider range of wavelengths. They offer advantages such as high spectral resolution and a broad spectral range, resulting in images with richer information. To present an image easily viewed by the human eye, after acquiring the spectral information of the target object using a hyperspectral or multispectral filter array, an RGB image needs to be output. Therefore, a solution that uses a hyperspectral or multispectral filter array to acquire the light signal of the target object while being compatible with RGB imaging technology is urgently needed.

[0004] Common color filter arrays consist of identical color filter units, each containing three or four color filters. This means the array has three or four sampling points. However, in real life, even a tiny change in light intensity across any spectral band can cause a change in color perception in biological vision. Therefore, commonly used image sensors are only capable of providing a rough and simple color restoration. In other words, conventional image sensors reproduce a relatively sparse spectrum, leading to numerous metamerism issues and posing challenges to the calculation of white balance coefficients and color restoration matrices. Furthermore, current spectral measurements require a separate image sensor chip to acquire spectral information, which increases cost and computational complexity. Summary of the Invention

[0005] Based on the problems described above, the image sensor proposed in this invention, which integrates spectral detection and phase focusing into the same color filter array, can improve the accuracy of color reproduction and reduce the calculation difficulty of white balance coefficient and color conversion matrix without affecting phase focusing and imaging functions.

[0006] In a first aspect, the present invention proposes an image sensor structure, the image sensor comprising: a pixel array and a color filter array consisting of a plurality of color filter units covering the pixel array, the plurality of color filter units including at least one first color filter unit and at least one second color filter unit, each first color filter unit including at least one specific color filter, each second color filter unit consisting of a plurality of basic color filters; wherein the color of the specific color filter is at least partially the same as the color of the basic color filter, the light signal passing through the specific color filter is used at least for spectral measurement, and the light signal passing through the basic color filter is used at least for autofocus.

[0007] Furthermore, the featured color filter in each of the first color filter units is repeated periodically.

[0008] Furthermore, the coverage area of ​​a single specific color filter and a single basic color filter is the same.

[0009] Furthermore, the colors of the basic color filter include red, green, and blue; the colors of the specific color filter include one or more of red, green, and blue, as well as one or more of cyan, magenta, yellow, white, gray, indigo, amber, and dark red; and there is at least one basic color filter between adjacent specific color filters.

[0010] Furthermore, the light signal passing through the specific color filter can be transmitted separately.

[0011] Furthermore, the total number of the specific color filters accounts for 3-6% of the total number of the color filter array.

[0012] Furthermore, the photodiode of the image sensor is a four-quadrant photodiode or a dual photodiode.

[0013] Secondly, this application also provides a method for operating the above-mentioned image sensor, specifically including: when the image sensor is not used for shooting, the image sensor does not collect the light signal passing through the basic color filter but only collects the light signal passing through the specific color filter; when the image sensor is used for shooting, the image sensor collects the light signal passing through the basic color filter, while the light signal data passing through the specific color filter is output separately for white balance calibration.

[0014] Furthermore, the working method also includes: acquiring and transmitting light signal data through the specific color filter in a 2×2 pixel merging pattern.

[0015] Furthermore, when the image sensor is used for shooting, the first light signal data passing through the basic color filter can be used for image generation, while the second light signal data passing through the specific color filter is treated as a bad pixel for correction; the first light signal data can also be used for focusing.

[0016] Compared with the prior art, the technical solution of the present invention has the following beneficial effects: The image sensor with spectral detection and phase focusing functions proposed in the present invention integrates the pixel arrays of spectral measurement and phase focusing, thereby improving the accuracy of color reproduction and reducing the calculation difficulty of white balance coefficient and color conversion matrix without affecting the phase focusing and imaging functions. Attached Figure Description

[0017] The accompanying drawings, which form part of this specification, are used to further understand the invention. The drawings illustrate embodiments of the invention and, together with the specification, serve to explain the principles of the invention.

[0018] Figure 1 This is a schematic diagram of the structure of an image sensor filter unit in the prior art.

[0019] Figure 2 This is a schematic diagram of the structure of a multispectral image sensor unit in the prior art.

[0020] Figure 3 This is a schematic diagram of a filter unit structure according to an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of a pixel structure according to an embodiment of the present invention. Detailed Implementation

[0022] The following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0023] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the present invention.

[0024] like Figure 1 As shown, a typical image sensor includes a pixel array and a color filter array covering the pixel array. The color filter array comprises 8×8 color filter units of equal area, with 64 color filters arranged in 8 rows and 8 columns. Each color filter unit includes four primary color filters, arranged in 2 rows and 2 columns. The four primary color filters are 3-channel RGB.

[0025] like Figure 2The diagram shows a common color filter array structure with multispectral measurement capabilities. The color filter unit has 9 channels, including multiple color channels such as R, G, B, C, M, Y, IR, and W (red, green, blue, cyan, magenta, yellow, and near-infrared). The W channel is a white filter, which may or may not have a microlens.

[0026] In summary, it can be seen that the color filter arrays commonly used for phase detection autofocus or imaging and those used for spectral measurement differ significantly in color arrangement and structure. Therefore, current mobile phone modules generally use image sensors with phase detection autofocus for imaging, while adding an extra image sensor chip for spectral measurement occupies the space of a separate camera, thus increasing manufacturing costs. Furthermore, image sensors with only phase detection autofocus or imaging capabilities can only perform rough and simple color reproduction, leading to color distortion and posing challenges to the calculation of white balance coefficients and color conversion matrices.

[0027] To address the aforementioned technical problems, in a first aspect, embodiments of the present invention propose an image sensor with spectral detection and phase focusing functions. The image sensor includes a pixel array, a color filter array, and a microlens array. The color filter array includes one or more first color filter units and one or more second color filter units. The first color filter unit includes at least one basic color filter and at least one specific color filter. The second color filter unit includes multiple basic color filters, wherein the specific color filter is at least partially the same color as the basic color filter.

[0028] A specific color filter can be used to filter incident light to obtain a first light signal, which can be used for spectral measurement. When the color of the specific color filter is the same as that of the basic color filter, the first light signal passing through the specific color filter can also be used for phase focusing or imaging.

[0029] A primary color filter is used to filter the incident light to obtain a second light signal, which can be used for phase focusing or imaging.

[0030] The pixel array includes multiple pixels. The pixel covered by a specific color filter is called the first pixel, and the pixel covered by a basic color filter is called the second pixel. The first pixel receives a first light signal and obtains a first electrical signal, and the second pixel receives a second light signal and obtains a second electrical signal.

[0031] The readout circuit can read the first electrical signal from the first pixel and the second electrical signal from the second pixel.

[0032] The image processor acquires a first electrical signal and a second electrical signal, determines first spectral measurement data for the current frame based on the first electrical signal, and determines fragrance focus data for the current frame based on the second electrical signal. The image processor can also determine the white balance coefficient and / or color and transformation matrix for the current frame based on the first spectral data and phase focus data.

[0033] The data from the signals output by the image sensor can be stored in a memory or displayed on the screen of an electronic device, including mobile phones, smartphones, tablets, phablets, or personal computers. In addition to the imaging devices mentioned above, those skilled in the art should understand that the techniques disclosed herein can also be applied to other electronic devices with imaging capabilities.

[0034] The technical solution provided by this invention, which integrates spectral measurement and phase focusing functions into the same color filter array, can improve the color reproduction accuracy of the image sensor, reduce the number of image sensor chips, and reduce costs compared to existing image sensors that only have phase focusing functions. At the same time, it reduces the calculation difficulty of white balance coefficient and color conversion matrix.

[0035] Specifically, such as Figure 3 The diagram shows a schematic of the image sensor structure provided in an embodiment of this application. The image sensor includes a pixel array and a color filter array covering the pixel array. The color filter array shown in the diagram includes 8×8 color filter units of equal area, with 64 color filter units arranged in an 8x8 grid. The color filter unit array includes a first color filter unit 101 and a second color filter unit 102. The first color filter unit 101 includes multiple basic color filters and multiple specific color filters. The specific color filters specifically include colors such as R, G, B, C, M, Y, and W, while the basic color filters include the three colors RGB. The second color filter unit 102 includes four basic color filters (RGGB), arranged in a 2x2 grid. The four basic color filters are 3-channel RGB. In some other embodiments, the specific color filters may also include gray.

[0036] like Figure 3As shown, the color filter units are arranged in an 8×8 pattern as a periodic array, with 8n multispectral channels in each period. In this embodiment, n=2. In other embodiments, n can also be a positive integer such as 1, 3, 4, etc. The thick boxes in the figure represent specific color filters, and the other boxes represent basic color filters. It can be seen from the figure that at least some of the specific color filters have the same color as the basic color filters. This is to avoid the image sensor chip discarding too many non-RGB channels as bad pixels when outputting images in normal mode, causing a large loss of sampling points and affecting the quality of phase focusing or imaging. In this embodiment, 5 non-RGB color filters and 3 RGB color filters are used as specific color filters out of 256 color filters. Other arrangements and combinations are possible in other embodiments, as long as the number of non-RGB color filters is 3-6% of the total number of color filters.

[0037] like Figure 4 As shown, in some other embodiments, the pixel unit corresponding to each specific color filter or each basic color filter unit can be a four-quadrant photodiode or a dual photodiode structure or a combination thereof, which will not be listed here. In some other embodiments not shown, each specific color filter and each basic color filter have the same area size.

[0038] This application also provides a method for operating the above-mentioned image sensor. Specifically, the image sensor may have multiple shooting modes.

[0039] When the image sensor is used for shooting, the image sensor chip does not output data from the conventional channels, but only outputs multispectral channel data in a merged 2×2 mode, which is used as multispectral data to assist in color temperature determination. That is, it only outputs the data of the first electrical signal generated by the first light signal after passing through the first color filter unit, and uses the image sensor for spectral measurement.

[0040] When the image sensor is used for shooting, the conventional channels of the image sensor chip normally output RGB data for phase focusing or imaging, while the data from the multispectral channels is discarded or corrected as bad pixels. At this time, the conventional channels can be used for phase focusing or imaging, and the data from the multispectral channels is output separately in the same output mode as the conventional channels for white balance calibration. Specifically, the light signals from the RGB channels in the first color filter unit and all filter channels in the second color filter unit are collected and output as corresponding electrical signals for phase focusing or imaging, and the light signals from all filter channels in the first color filter unit are collected and output for spectral measurement and white balance calibration. In other words, the image sensor of this invention, with spectral detection and phase focusing functions, can multiplex a portion of the RGB filter channel data in the first color filter unit, which can be used for both phase focusing or imaging and spectral measurement.

[0041] In summary, the image sensor proposed in this invention improves the accuracy of color reproduction and reduces the computational difficulty of white balance coefficient and color conversion matrix by integrating the pixel arrays for spectral measurement and phase focusing, without affecting the phase focusing and imaging functions.

[0042] The basic concepts have been described above. It is clear that the detailed disclosure above is merely illustrative and does not constitute a limitation of the present invention. Although not explicitly stated herein, various modifications, improvements, and corrections may be made to the present invention by those skilled in the art. Such modifications, improvements, and corrections are suggested in this invention and therefore remain within the spirit and scope of the exemplary embodiments of the present invention.

[0043] It should be understood that the embodiments described in this invention are merely illustrative of the principles of the invention. Other modifications may also fall within the scope of this invention. Therefore, alternative configurations of the embodiments of this invention are considered as examples and not limitations, and are regarded as consistent with the teachings of this invention. Accordingly, the embodiments of this invention are not limited to those explicitly described and illustrated herein.

Claims

1. An image sensor, characterized in that, include: A pixel array and a color filter array covering the pixel array, wherein: The color filter array includes multiple color filter units, and the multiple color filter units include at least one first color filter unit and at least one second color filter unit; Each of the first color filter units includes at least one specific color filter, and each of the second color filter units consists of a plurality of basic color filters; The color of the specific color filter is at least partially the same as the color of the basic color filter, the light signal passing through the specific color filter is used at least for spectral measurement, and the light signal passing through the basic color filter is used at least for autofocus.

2. The image sensor as described in claim 1, characterized in that, The featured color filter in each of the first color filter units repeats periodically.

3. The image sensor as described in claim 1, characterized in that, The coverage area of ​​a single specific color filter and a single basic color filter is the same.

4. The image sensor as described in claim 1, characterized in that, The basic color filter includes red, green, and blue colors; The specific color filter's color includes one or more of red, green, and blue, as well as one or more of cyan, magenta, yellow, white, gray, indigo, amber, and dark red; There is at least one basic color filter between adjacent specific color filters.

5. The image sensor as described in claim 1, characterized in that, Light signals passing through the specific color filter can be transmitted separately.

6. The image sensor as claimed in claim 1, characterized in that, The total number of the specific color filters accounts for 3-6% of the total number of color filters.

7. The image sensor as claimed in claim 1, characterized in that, The photodiode of the image sensor is a four-quadrant photodiode or a dual photodiode.

8. A method for operating an image sensor as described in claim 1, characterized in that, When the image sensor is not used for taking pictures, the image sensor does not collect the light signal passing through the basic color filter but only collects the light signal passing through the specific color filter; When the image sensor is used for shooting, the image sensor acquires the light signal passing through the basic color filter, while the light signal data passing through the specific color filter is output separately for white balance calibration.

9. The working method as described in claim 8, characterized in that, The light signal data passing through the specific color filter is acquired and transmitted in a 2×2 pixel merge pattern.

10. The working method as described in claim 8, characterized in that, When the image sensor is used to take a picture, the first light signal data passing through the basic color filter can be used for image generation, while the second light signal data passing through the specific color filter is treated as a bad pixel for correction; the first light signal data can also be used for focusing.

Images