Image sensor and imaging device

Abstract

The present application provides an image sensor and an imaging device. The image sensor includes a plurality of pixel units. The plurality of pixel units are divided into a first pixel group (23), a second pixel group (24), a third pixel group (25), and a fourth pixel group (26) arranged in an array. The second pixel group (24) and the third pixel group (25) are arranged diagonally. The first pixel group (23) includes a first pixel unit and a fourth pixel unit. The second pixel group (24) includes a second pixel unit and a fourth pixel unit. The third pixel group (25) includes a second pixel unit and a fourth pixel unit. The fourth pixel group (26) includes a third pixel unit and a fourth pixel unit. The first pixel unit, the second pixel unit, and the third pixel unit are monochrome pixel units used to receive light of different colors, respectively. The fourth pixel unit is a white pixel unit that receives white light. In the above technical solution, a white pixel unit is introduced into each pixel group, and the sensitivity of the image sensor is improved by using the white pixel unit, thereby improving the image resolution and signal-to-noise ratio.

Description

[Technical field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority to Chinese Patent Application No. 202011384707.9, entitled “IMAGE SENSOR AND IMAGING APPARATUS,” filed with the China National Intellectual Property Office on November 30, 2020, which is incorporated herein by reference in its entirety.

[0002] The present application relates to the field of imaging technology, and in particular to image sensors and imaging devices. [Background technology]

[0003] Traditional pixel arrays in image sensors usually use a cross arrangement of red, green, and blue pixels. With the development of imaging technology, a brand new pixel arrangement technique has emerged, in which four adjacent pixels of the same color are arranged together to form a pixel with an area four times that of the original pixel. Therefore, this technique is also called "Quad Bayer" technique.

[0004] In recent years, the "Quad Bayer" technology has been widely used in mobile phone cameras. It can obtain pixels with an area four times larger than the original pixel area, which can improve the sensitivity in low-light shooting environments and improve the signal-to-noise ratio of the image. Finally, a bright and low-noise image is produced.

[0005] Currently, image sensors using a form of "Quad Bayer" arrangement are widely used. In the default 10 megapixel output mode, through Quad Bayer processing, four adjacent pixels of the same color are sampled simultaneously to obtain an image with four times the sensitivity, 1 / 4 resolution, a single pixel size of 2 μm, and 10 megapixels. This implements high sensitivity in a small high-pixel image sensor. In the 40 megapixel full resolution output mode, a re-mosaic algorithm is used to rearrange the pixels into a conventional pixel array format. After image signal processing, a 40 megapixel high-resolution full pixel image is output.

[0006] However, in the case of current pixel arrays having a "Quad Bayer" array configuration, images output at quarter resolution have significantly improved signal-to-noise ratios but significantly reduced resolution compared to images output in conventional pixel arrays, and results output at full resolution have reduced resolution compared to results in conventional pixel array formats that do not require the pixels to be staggered. Summary of the Invention

[0007] The present application provides an image sensor and an imaging device to improve the resolution of the image sensor and improve the imaging effect of the image sensor.

[0008] According to a first aspect, an image sensor is provided. The image sensor includes a plurality of pixel units. The plurality of pixel units are divided into a first pixel group, a second pixel group, a third pixel group, and a fourth pixel group arranged in an array. The second pixel group and the third pixel group are arranged in a diagonal direction. The first pixel group includes a first pixel unit and a fourth pixel unit. The second pixel group includes a second pixel unit and a fourth pixel unit. The third pixel group includes a second pixel unit and a fourth pixel unit. The fourth pixel group includes a third pixel unit and a fourth pixel unit. The third pixel unit is a monochrome pixel unit used to receive light of a different color. The fourth pixel unit is a white pixel unit that receives white light. In the above technical solution, a white pixel unit is introduced into each pixel group, and the sensitivity of the image sensor is improved by using the white pixel unit. This improves the resolution and signal-to-noise ratio of the image.

[0009] In a specific implementable solution, each of the first pixel group, the second pixel group, the third pixel group, and the fourth pixel group includes four pixel units arranged in an array, and among the four pixel units in each pixel group, at least one pixel unit is the white pixel unit, and the remaining pixel units are monochrome pixel units used to receive light of the same color. Different proportions of white pixel units can be selected in the pixel groups.

[0010] In a specific possible solution, each pixel group includes three monochrome pixel units and one white pixel unit that are used to receive light of the same color. By using one white unit and three monochrome pixel units, the sensitivity of the image sensor is improved and the color information obtained is reliable.

[0011] In a particular implementable solution, in each of two adjacent pixel groups, a white pixel unit located in one pixel group and a white pixel unit located in the other pixel group are separated by another pixel unit.

[0012] In a particular implementable solution, each pixel group includes two monochrome pixel units used to receive light of the same color and two white pixel units, the two white pixel units being arranged in a diagonal direction.

[0013] In a specific implementable solution, the size of the fourth pixel unit is smaller than the size of the first pixel unit, the size of the second pixel unit, and the size of the third pixel unit. By reducing the size of the white pixel unit, the exposure of the white pixel unit is reduced, the sensitivity of the white pixel unit is reduced, the saturation of the white pixel unit is delayed, other pixel units are more fully exposed, and the number of image color signals is increased.

[0014] In a specific possible solution, the white pixel unit is wrapped with a metal shielding layer, and the size of the white pixel unit wrapped with the metal shielding layer is equal to the size of the monochrome pixel unit. By disposing the metal shielding layer, the size of the fourth pixel unit is reduced.

[0015] In a specific possible solution, the image sensor further includes an optical filtering module disposed on a light-entering side of the plurality of pixel units, the optical filtering module filtering out noise.

[0016] In a specific possible solution, the image sensor further includes a lens used for converging light, the lens being located at the light-entering side of the plurality of pixel units. The lens for converging light increases light capture.

[0017] In a specific possible solution, a light filtering module is further included, the light filtering module being disposed at the light input side of the plurality of pixel units, thereby reducing noise interference and improving imaging effect.

[0018] In a specific implementable solution, the optical filtering module is a dual passband optical filtering module, and the dual passband optical filtering module can transmit the white light and the infrared light, and the noise interference is reduced and the imaging effect is improved.

[0019] In a specific possible solution, the monochrome pixel unit is coated with a filter layer to block the infrared light, and the white pixel unit receives the white light and the infrared light, which is used to increase the brightness value and improve the imaging effect.

[0020] In a specific implementable solution, the method further includes an image processing module, which performs an interpolation process on the first color light received by the monochrome pixel unit in each pixel group to obtain a second color light corresponding to a pixel unit at the position of the white pixel unit in the pixel group; performs a Quad Bayer process on the four pixel units in each pixel group based on the first color light and the second color light to obtain a Quad Bayer image; obtains luminance information based on the white light received by the white pixel unit in each pixel group; and combines the Quad Bayer image formed by each pixel group with the luminance information corresponding to the pixel group to obtain a 1 / 4 resolution image. A high-quality 1 / 4 resolution image is obtained.

[0021] In a specific possible solution, the image processing module is further configured to: re-mosaic a pixel array to obtain an image; perform an interpolation process on the white light received by each white pixel unit to obtain luminance information corresponding to each pixel unit; and combine the image obtained through the re-mosaic process and the luminance information corresponding to each pixel unit to obtain a full resolution image. A high quality full resolution image is obtained.

[0022] According to a second aspect, an imaging device is provided, the imaging device includes a housing and the above-mentioned image sensor disposed in the housing. In the above technical solution, a white pixel unit is introduced into each pixel group, and the sensitivity of the image sensor is improved by using the white pixel unit, thereby improving the resolution and signal-to-noise ratio of the image. [Brief description of the drawings]

[0023] [Figure 1] FIG. 1 is a schematic diagram of an application scenario of an image sensor according to an embodiment of the present application;

[0024] [Diagram 2] FIG. 1 is a schematic diagram of a pixel array of an image sensor in the prior art.

[0025] [Diagram 3] FIG. 1 is a schematic diagram of a structure of an image sensor according to an embodiment of the present application.

[0026] [Figure 4] FIG. 1 is a schematic diagram of a pixel array according to an embodiment of the present application;

[0027] [Diagram 5] FIG. 2 is a schematic diagram of illumination of light onto a pixel array according to an embodiment of the present application.

[0028] [Figure 6] FIG. 2 is a circuit diagram of a color information collecting unit according to an embodiment of the present application.

[0029] [Figure 7] FIG. 2 is a circuit diagram of a luminance information collecting unit according to an embodiment of the present application.

[0030] [Figure 8] 1 is a flow chart for forming a high quality quarter resolution image by an image sensor according to an embodiment of the present application.

[0031] [Figure 9] 1 is a flow chart of the formation of a high quality full resolution image by an image sensor according to one embodiment of the present application.

[0032] [Figure 10] FIG. 1 is a schematic diagram of the resolution of a 1 / 4 resolution image of an image sensor in the prior art.

[0033] [Figure 11] FIG. 2 is a schematic diagram of the resolution of a quarter resolution image of an image sensor according to an embodiment of the present application.

[0034] [Figure 12] FIG. 1 is a schematic diagram of the resolution of a full resolution image of an image sensor in the prior art.

[0035] [Figure 13] FIG. 2 is a schematic diagram of the resolution of a full resolution image of an image sensor according to an embodiment of the present application.

[0036] [Figure 14] FIG. 2 is a schematic diagram of another pixel array according to an embodiment of the present application.

[0037] [Figure 15] FIG. 2 is a schematic diagram of another pixel array according to an embodiment of the present application.

[0038] [Figure 16] FIG. 2 is a schematic diagram of illumination of light onto a pixel array according to an embodiment of the present application.

[0039] [Figure 17] 6 is a light reception curve of the pixel array shown in FIG. 5.

[0040] [Figure 18] 16 is a light reception curve of the pixel array shown in FIG.

[0041] [Figure 19]FIG. 1 is a schematic diagram of resolution improvement by using a pixel array according to an embodiment of the present application.

[0042] [Figure 20] FIG. 2 is a schematic diagram of another pixel array according to an embodiment of the present application.

[0043] [Figure 21] FIG. 13 is a diagram of light incidence on another pixel array according to an embodiment of the present application.

[0044] [Figure 22] FIG. 2 is a schematic diagram of another image sensor structure according to an embodiment of the present application.

[0045] [Diagram 23] FIG. 2 is a schematic diagram of another pixel array according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0046] The embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0047] The application scenario of the image sensor provided in the embodiment of the present application is described first. The image sensor provided in the embodiment of the present application is used in an electronic device, for example, a general device with a photographing function, for example, a mobile phone, a tablet computer, or a wearable electronic device. Of course, the method may be applied to another type of electronic device with a photographing function. As shown in FIG. 1, when the image sensor 2 provided in the embodiment of the present application is used in a mobile terminal, the mobile terminal includes a housing 1 and an image sensor 2 disposed in the housing 1. The image sensor 2 is disposed on a main board 3 and electrically connected to the main board 3. When light from a target object is emitted to the image sensor 2, the image sensor 2 can convert the light signal into an electrical signal and perform image processing.

[0048] An image sensor in the prior art is shown in FIG. 2. The image sensor includes a plurality of pixel units. The plurality of pixel units are arranged in an array. A 4*4 array is used as an example. The plurality of pixel units are divided into four pixel groups, namely a first pixel group 4, a second pixel group 5, a third pixel group 6, and a fourth pixel group 7. The first pixel group 4 includes four pixel units used to receive red light (represented by R in the drawings). The second pixel group 5 includes four pixel units used to receive green light (represented by Gr in the drawings). The third pixel group 6 includes four green light (represented by Gb in the drawings) pixel units. The fourth pixel group 7 includes four blue light (represented by B in the drawings) pixel units. However, when the above technical solution is used, each pixel group in the image sensor in the prior art only collects a monochromatic light (red light, green light, or blue light) of a target object. As a result, in an environment with poor white light illumination, the light sensitivity of the image sensor is poor, thereby reducing image quality. Therefore, an embodiment of the present application provides an image sensor. The image sensor is described in detail below with reference to certain accompanying drawings and embodiments.

[0049] 3 is a schematic diagram of the structure of an image sensor according to an embodiment of the present application. The image sensor provided in this embodiment of the present application includes a lens 10, a pixel array 20, and an image processing module 30 arranged along an optical path. The pixel array 20 includes a color filter layer 21 and an image acquisition module 22. After the light irradiated from a target object passes through the lens 10 and the color filter layer 21 successively, the light is irradiated to the image acquisition module 22. The image acquisition module 22 converts the optical signal into a digital signal. The image processing module 30 performs image processing according to the digital signal of the image acquisition module 22 to form an image.

[0050] It should be understood that the lens 10 is an arbitrary modular structure. When the image sensor is arranged, the lens 10 may be selectively arranged based on actual requirements, which is not specifically limited in the embodiment of the present application.

[0051] Also refer to FIG. 4. The pixel array includes a plurality of pixel units. The plurality of pixel units are arranged in M ​​rows and N columns. M and N are both integers greater than 2. The plurality of pixel units include pixel units of different types. For example, the aforementioned pixel units can be classified into a first pixel unit, a second pixel unit, a third pixel unit, and a fourth pixel unit based on the types. There are a plurality of first pixel units, a plurality of second pixel units, a plurality of third pixel units, and a plurality of fourth pixel units. The sizes of the first pixel unit, the second pixel unit, the third pixel unit, and the fourth pixel unit are the same.

[0052] The first pixel unit, the second pixel unit, and the third pixel unit are monochrome pixel units that can receive different color lights and transmit monochromatic light (red light, green light, and blue light) of a target object. For example, the first pixel unit, the second pixel unit, and the third pixel unit can be a red pixel unit that receives red light, a green pixel unit that receives green light, and a blue pixel unit that receives blue light, respectively. Alternatively, the first pixel unit, the second pixel unit, and the third pixel unit are a cyan pixel unit, a magenta pixel unit, and a yellow pixel unit, respectively. Of course, the above are only two specific examples of the first pixel unit, the second pixel unit, and the third pixel unit. The first pixel unit, the second pixel unit, and the third pixel unit provided in the embodiments of the present application can receive other color lights based on imaging requirements.

[0053] The fourth pixel unit is a white light pixel unit that can receive white light. When the white light is irradiated onto the fourth pixel unit, the white light can be received by the image acquisition module in the fourth pixel unit.

[0054] Please refer to FIG. 4 again. A 4*4 array is used as an example. A plurality of pixel units are divided into four pixel groups. The four pixel groups are a first pixel group 23, a second pixel group 24, a third pixel group 25, and a fourth pixel group 26. The four pixel groups are arranged in an array. The first pixel group 23 and the second pixel group 24 are located in the same row. The third pixel group 25 and the fourth pixel group 26 are located in the same row. The third pixel group 25 and the second pixel group 24 are arranged in a diagonal direction.

[0055] The first pixel group 23 includes three red pixel units and one white pixel unit. In FIG. 4, R is used to represent the red light received by the red pixel unit, and W is used to represent the white light received by the white pixel unit. The second pixel group 24 includes three green pixel units and one white pixel unit. In FIG. 4, Gr is used to represent the green light received by the green pixel unit in the second pixel group. The third pixel group 25 includes three green pixel units and one white pixel unit. In FIG. 4, Gb is used to represent the green light received by the green pixel unit in the third pixel group. The fourth pixel group 26 includes three blue pixel units and one white pixel unit. In FIG. 4, B represents the blue light received by the blue pixel unit. It should be understood that the above Gr and Gb represent green light separately. To distinguish the green light received by different green pixel units in the second pixel group 24 and the third pixel group 25, Gr and Gb are used to represent the green light separately.

[0056] As mentioned above, referring to FIG. 3, the pixel array 20 includes a color filter layer 21 and an image collection module 22. The pixel array 20 includes a first pixel unit, a second pixel unit, a third pixel unit, and a fourth pixel unit. See FIG. 5. The first pixel unit includes a first filter area 211 and a first color information collection unit 221. The second pixel unit includes a second filter area 212 and a second color information collection unit 222. The third pixel unit includes a third filter area 213 and a third color information collection unit 223. The fourth pixel unit includes a fourth filter area 214 and a luminance information collection unit 224. It should be noted that the first filter area 211, the second filter area 212, the third filter area 213, and the fourth filter area 214 are all located in the color filter layer 21. The first color information collecting unit 221 , the second color information collecting unit 222 , the third color information collecting unit 223 and the luminance information collecting unit 224 are all located in the image collecting module 22 .

[0057] In an optional solution, the pixel array further includes a microlens group. The first pixel unit may further include a first microlens 231. By using the first microlens 231, the light can be focused into the first filter area 211. Similarly, the second pixel unit may further include a second microlens 232. The third pixel unit may further include a third microlens 233. The fourth pixel unit may further include a fourth microlens 234.

[0058] When receiving different light, the first pixel unit, the second pixel unit, the third pixel unit, and the fourth pixel unit filter the light by using the filter area located in the corresponding pixel unit. For example, if the white light is a light including red, green, and blue, the first filter area 211 of the first pixel unit can transmit only red light and block other color lights. The second filter area 212 of the second pixel unit can transmit only green light and block other color lights. The third filter area 213 of the third pixel unit can transmit only blue light and block other color lights. The fourth filter area 214 of the fourth pixel unit can transmit only white light (red, green, and blue). The color light (red, green, and blue light) related to the color information of the imaging of the target object is obtained by using the filter areas of the first pixel unit to the third pixel unit, and the white light related to the brightness information of the imaging of the target object is obtained by using the filter area of ​​the fourth pixel unit. The first pixel unit, the second pixel unit, the third pixel unit and the fourth pixel unit have the same size, which means that the aperture size of the filter area in all pixel units is the same. In the embodiment of the present application, a rectangular filter area is used as the filter area of ​​each pixel unit.

[0059] After passing through the corresponding filter area, the light is illuminated to the image collecting module 22 for further processing (see FIG. 3). The image collecting module 22 can obtain the light signal including the color and brightness of the target object, convert the light signal into a voltage signal, and then convert the light signal into a digital signal. Referring to the structure shown in FIG. 5, the first color information collecting unit 221 of the first pixel unit receives red light, the second color information collecting unit 222 of the second pixel unit receives green light, and the third color information collecting unit 223 of the third pixel unit receives blue light. The brightness information collecting unit 224 of the fourth pixel unit receives white light.

[0060] FIG. 6 is a circuit diagram of a color information collecting unit according to an embodiment of the present application. The first color information collecting unit, the second color information collecting unit, and the third color information collecting unit are all conventional four-transistor pixel sensors. Each color information collecting unit includes a transmission transistor MTG and a floating diffusion area FD. A control terminal of the transmission transistor MTG receives a control signal. An input terminal of the transmission transistor MTG is connected to a photodiode PD. An output terminal of the transmission transistor MTG outputs a voltage signal when the control terminal of the transmission transistor MTG receives a control signal. The transmission transistor MTG transfers the voltage signal of the photodiode PD to the floating diffusion area FD for storage. The photo-generated carriers accumulated in the photodiode PD are transferred to the floating diffusion area FD and converted into a voltage signal. After being converted into a digital signal by a digital-to-analog conversion ADC, the digital signal is sent to the color information processing unit. A plurality of first pixel units, a plurality of second pixel units, and a plurality of third pixel units are provided, each pixel unit includes a photodiode PD used to receive a light signal. Other circuit components shown in FIG. 6 are conventional electronic components in a four-transistor pixel sensor. The details will not be explained again here.

[0061] FIG. 7 is a circuit diagram of a luminance information collecting unit according to an embodiment of the present application. The structure of the luminance information collecting unit is the same as that of each color information collecting unit. The difference between the two is only in the light received. Therefore, the acquisition principle of the luminance information collecting unit is the same as that of the color information collecting unit. Here, the details are not described again. A fourth pixel unit (white pixel unit) is introduced into the pixel array. The white pixel unit allows three different wavelengths of light, namely red light, green light, and blue light, to pass through, so that the brightness of the white pixel unit is higher than that of other pixel units that receive monochromatic light (for example, the first pixel unit that receives red light and the second pixel unit that receives green light). Therefore, the brightness of the image output by the image sensor is higher than that of the image output by the image sensor that only uses monochromatic light to obtain luminance information in the prior art.

[0062] See Fig. 3. The image processing module 30 includes a color image processing unit, a luminance information processing unit, and a color fusion image unit. The color image processing unit is configured to process the color information acquired by the image acquisition module 22. The luminance information processing unit is configured to process the luminance information acquired by the image acquisition module 22. The color fusion image unit is configured to fuse the processed color information and luminance information to acquire a color image.

[0063] When the image processing module 30 processes the light received by the pixel array, one image processing method is to form a high-quality 1 / 4 resolution image, and the other is to form a high-quality full resolution image. The two methods are described one by one below.

[0064] To facilitate understanding of the principle of image formation by the image sensor provided in the embodiment of the present application, the interpolation process is first described. The interpolation process refers to predicting unknown data from known data. Image interpolation refers to predicting the value of a given pixel unit based on information about pixel units around the pixel unit. Common interpolation algorithms can be classified into two types: adaptive and non-adaptive. In adaptive interpolation algorithms, changes can be made based on the content of the interpolation (sharp edges or smooth textures). In non-adaptive algorithms, all pixel units are processed in the same way. Non-adaptive algorithms include nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, spline interpolation, etc.

[0065] As shown in the flow chart shown in FIG. 8, for ease of explanation, the output signals (digital signals) obtained by performing photoelectric conversion on the light received in the pixel units are named separately. The output signals converted from the red light received by the three first pixel units in the first pixel group are represented by R1, R2, and R3, respectively. The output signal converted from the white light received by the fourth pixel unit is represented by W1. The output signals converted from the green light received by the three second pixel units in the second pixel group are represented by Gr1, Gr2, and Gr3, respectively. The output signal converted from the white light received by the fourth pixel unit is represented by W2. The output signals converted from the green light received by the three second pixel units in the third pixel group are represented by Gb1, Gb2, and Gb3, respectively. The output signal converted from the white light received by the fourth pixel unit is represented by W3. The output signals converted from the blue light received by the three first pixel units in the fourth pixel group are denoted as B1, B2, and B3, respectively. The output signal converted from the white light received by the fourth pixel unit is denoted as W1. The output signals R1, R2, R3, Gr1, Gr2, Gr3, Gb1, Gb2, Gb3, B1, B2, and B3 are color information of the target object. The output signals W1, W2, W3, and W4 are luminance information of the target object.

[0066] When a high-quality 1 / 4 resolution image is formed, referring to FIG. 3, the pixel array 20 converts the received light into an optical signal. The optical signal enters the image processing module 30 for processing. The image processing module 30 obtains the first color light received by the monochrome pixel units (the first pixel unit, the second pixel unit, and the third pixel unit) in each pixel group; performs an interpolation process on the first color light received by the monochrome pixel units in each pixel group to obtain a second color light corresponding to the pixel unit at the position of the white pixel unit (the fourth pixel unit) in the pixel group through the interpolation process; performs a Quad Bayer process on the four pixel units in each pixel group according to the obtained first color light and second color light to obtain a Quad Bayer image; obtains brightness information according to the white light received by the white pixel unit in each pixel group; and combines the Quad Bayer image formed by each pixel group with the brightness information corresponding to the pixel group to obtain a 1 / 4 resolution image.

[0067] Specifically, the color image processing unit obtains an original image of the pixel array by using the color information collecting unit and the luminance information collecting unit, and performs an interpolation process at a pixel unit position corresponding to the fourth pixel unit in the original image to obtain color information of the pixel position of the fourth pixel unit. As an example, the fourth pixel unit in the first pixel group is used. The color information R4 of the pixel position of the fourth pixel unit can be obtained by performing an interpolation process on the known R2 and R3. R4=(R2+R3) / 2 Quad Bayer processing is performed on R1, R2, R3, and R4 to obtain the output signal R after the Quad Bayer processing is performed. That is, R=(R1+R2+R3+R4) / 4. Similarly, the same calculation is performed on the second pixel group, the third pixel group, and the fourth pixel group, respectively, to obtain Gr=(Gr1+Gr2+Gr3+Gr4) / 4, Gb=(Gb1+Gb2+Gb3+Gb4) / 4, and B=(B1+B2+B3+B4) / 4. In this way, a Quad Bayer image is obtained.

[0068] The R, Gr, Gb, and B in the resulting Quad Bayer image are respectively combined with four fourth pixel units W1, W2, W3, and W4 to obtain a high-quality 1 / 4 resolution image.

[0069] As shown in the flowchart shown in FIG. 9, when a high-quality full-resolution image is formed, the image processing module is further configured to perform a re-mosaic process on the output signal of the pixel array (the output signal of each pixel unit) to obtain an image; perform an interpolation process on the white light received by each white pixel unit to obtain brightness information corresponding to each pixel unit (including each white pixel unit and each monochrome pixel unit); and combine the image obtained through the re-mosaic process with the brightness information corresponding to each pixel unit to obtain a full-resolution image.

[0070] Specifically, the color image processing unit obtains an original image by using the color information collecting unit and the luminance information collecting unit, and performs an interpolation process on the color pixel corresponding to the position of the fourth pixel unit in the original image to obtain the monochromatic information of the pixel position of the fourth pixel unit. As an example, the fourth pixel unit in the first pixel group is used. The monochromatic information R4 of the pixel position of the fourth pixel unit is obtained by performing an interpolation process on the known R1, R2, and R3. Similarly, R4=(R2+R3) / 2, Gr=(Gr2+Gr3) / 2, Gb=(Gb2+Gb3) / 2, and B=(B2+B3) / 2. The color image processing unit performs re-mosaic and demosaic operations on the original image based on the obtained monochrome information of each pixel unit in the pixel array.

[0071] The luminance information processing unit is configured to perform an interpolation process on the obtained output signals W1, W2, W3, and W4. In the pixel array with M rows and N columns, after the output signals W1, W2, W3, and W4 are obtained, the luminance information corresponding to each pixel unit in the M rows and N columns of the pixel array is obtained, in which the position and output signal of the fourth pixel unit are substituted into the interpolation process.

[0072] The demosaiced image is fused with the luminance information to obtain a high quality full resolution image.

[0073] From the above description, it can be understood that in the pixel array provided in the embodiment of the present application, a fourth pixel unit (white pixel unit) is introduced into the pixel array to receive white light. The white pixel unit allows three different wavelengths of light, i.e., red light, green light, and blue light, to pass through, so that the brightness of the white pixel unit is higher than that of other pixel units that receive monochromatic light (for example, the first pixel unit that receives red light and the second pixel unit that receives green light). Therefore, the brightness of the image output by the image sensor is higher than that of the image output by the image sensor that only uses monochromatic light to obtain brightness information in the prior art. This can improve the image resolution (the ability of the image sensor to capture image details) and the signal-to-noise ratio (referring to the ratio of signal to noise in the image sensor).

[0074] Figure 10 and Figure 11 show a comparison between high quality 1 / 4 resolution image and the output effect of the resolution of the image sensor in the prior art. There is no texture information in the circle in Figure 10. However, there is obvious texture information in the circle in Figure 11. Therefore, the image sensor provided in the embodiment of the present invention can be greatly improved in terms of resolution.

[0075] Figure 12 and Figure 13 show a comparison between the output effect of high-quality full resolution image and the resolution of the image sensor in the prior art. There is no texture information in the circle in Figure 12. However, there is obvious texture information in the circle in Figure 13. Therefore, the image sensor provided in the embodiment of the present invention can be greatly improved in terms of resolution. In addition, the signal-to-noise ratio of the image sensor provided in the embodiment of the present application is 3 dB higher than that of the image sensor in the prior art.

[0076] From the above description, it can be seen that compared with image sensors in the prior art, after the introduction of the fourth pixel unit, the image sensor provided in the embodiment of the present application increases light capture, improves the resolution and signal-to-noise ratio of the image sensor in the output format of a high-quality full-resolution image, and improves the resolution in the output format of a high-quality 1 / 4 resolution image.

[0077] Figure 14 shows another modified structure based on the pixel array shown in Figure 4. In the image sensor shown in Figure 14, the number of fourth pixel units introduced into each pixel group is increased. Each of the first pixel group 23, the second pixel group 24, the third pixel group 25 and the fourth pixel group 26 includes two fourth pixel units. In an optional solution, the two fourth pixel units are arranged in a diagonal direction.

[0078] As more fourth pixel units are introduced, the light capture increases, improving the resolution and signal-to-noise ratio of the image sensor in a high-quality full resolution image output format, and further synchronously improving the resolution and signal-to-noise ratio of the image sensor in a high-quality 1 / 4 resolution image output format.

[0079] It should be understood that Figure 14 only shows one example of a specific arrangement method of the pixel array. In the embodiment of the present application, the arrangement order of the pixel units in different pixel groups is not particularly limited. The fourth pixel unit in the different pixel groups may be adjacent or non-adjacent.

[0080] Figure 15 shows another modified structure based on the pixel array shown in Figure 5. An embodiment of the present application provides another arrangement method of the pixel array. In the pixel array shown in Figure 15, the size of the fourth pixel unit is smaller than the size of the first pixel unit, the size of the second pixel unit, and the size of the third pixel unit.

[0081] FIG. 16 is a schematic diagram of the arrangement of different types of pixel units. For the reference symbols in FIG. 16, please refer to the same symbols in FIG. 5. The fourth pixel unit is wrapped by a metal shield layer 215. The metal shield layer 215 wraps around the fourth filter area 214 of the fourth pixel unit. The size of the white pixel unit wrapped by the metal shield layer 215 is equal to the size of the monochrome pixel unit (the first pixel unit, the second pixel unit, or the third pixel unit). In FIG. 16, the fourth filter area 214 of the fourth pixel unit is wrapped by the metal shield layer 215. Specifically, when wrapping, only a part of the fourth filter area 214 may be wrapped, or the entire fourth filter area 214 may be wrapped. For example, the metal shield layer 215 has a frame-like structure, and the fourth filter area 214 is located within the frame-like structure, so that the opening area of ​​the fourth filter area 214 is smaller than the opening areas of the first filter area 211, the second filter area 212, and the third filter area 213.

[0082] In an optional solution, the aperture area of ​​the fourth pixel unit is 0.8 to 0.9 times that of the first pixel unit, for example, the aperture area of ​​the fourth pixel unit is 0.8, 0.85, or 0.9 times that of the first pixel unit.

[0083] In order to facilitate understanding of the influence of reducing the size of the fourth pixel unit on another type of pixel unit, the following description is provided with reference to Fig. 17 and Fig. 18. Fig. 17 is a schematic diagram of the light received by different pixel units when the first pixel unit, the second pixel unit, the third pixel unit, and the fourth pixel unit have the same size. Fig. 18 is a schematic diagram of the light received by different pixel units when the size of the fourth pixel unit is smaller than that of the first pixel unit, the second pixel unit, and the third pixel unit. W is the white light received by the fourth pixel unit, R is the red light received by the first pixel unit, G is the green light received by the second pixel unit, and B is the blue light received by the fourth pixel unit.

[0084] The reference line a in FIG. 17 and the reference line b in FIG. 18 are compared. The reference line a and the reference line b respectively represent the same exposure time. Since the first pixel unit, the second pixel unit, and the third pixel unit receive monochromatic light, and the fourth pixel unit receives white light (receives red light, blue light, and green light simultaneously), the fourth pixel unit can receive more light in the same period, and the efficiency of receiving monochromatic light by the other pixel units is lower than the efficiency of receiving white light by the fourth pixel unit. By comparing FIG. 17 with FIG. 18, it can be understood that when the first pixel unit to the fourth pixel unit have the same size, the light received by the other pixel units is low when the white light received by the fourth pixel unit tends to be saturated. However, after the size of the fourth pixel unit shown in FIG. 18 is reduced, it is equivalent to the efficiency of receiving white light by the fourth pixel unit being reduced. Therefore, the other pixel units may obtain more monochromatic light, so that when the white light received by the fourth pixel unit tends to be saturated, the other pixel units may obtain more monochromatic light, making the light representing the color of the target object in the output signal of the pixel array more abundant.

[0085] After the size of the fourth pixel unit provided in the embodiment of the present application is reduced, the resolution can be further improved. With reference to the schematic diagram shown in FIG. 19, the term "line pair" that appears below is first explained. "Line pair" is a special term for film, lens, and other film photography fields. Line pairs per millimeter is generally a unit of resolution, which means how many line pairs an instrument can distinguish within one millimeter. The larger the number of line pairs that can be distinguished, the smaller the width of each line that can be distinguished, and the higher the resolution. Line pairs are a factor that reflects image resolution. The higher the image resolution, the more line pairs there are. Pixel 1, pixel 2, pixel 3, pixel 4, and pixel 5 in FIG. 19 represent pixel units of different sizes.

[0086] A sampling test is performed by using the line pair shown in FIG. 19 as a reference. One black line and one white line in the row shown in FIG. 19 form one line pair. The black lines and white lines are arranged alternately to form five line pairs. For example, the width of pixel 1 is equal to the width of one line pair (the width of pixel 1 is equal to the width of one black line and one white line). Since the width of pixel 1 is equal to the width of the line pair, the line pair cannot be displayed in pixel 1. Therefore, the resolution of the sampling result is 0, and the resolution is determined by using MTF (Modulation Transfer Function). The width of pixel 2 is half the width of one line pair (the width of one black line). In this case, when pixel 2 has one-to-one correspondence with the position of the line pair, the resolution of pixel 2 can reach 100. Pixel 3 is a pixel whose size is the same as that of pixel 2. However, the position of pixel 3 and the line pair are shifted by 1 / 2 of the pixel width, and the resolution of the sampling result of pixel 3 is 0. For pixel 4, the size of pixel 5 is 1 / 4 of the width of the line pair. Regardless of the sampling position and sampling interval, pixel 4 and pixel 5 can both reach the sampling resolution 100. For example, pixel 4 and pixel 5 are shifted by 1 / 2 pixel to obtain pixel 4' and pixel 5', and the sampling resolution 100 can still be reached. From the comparison of pixel 1, pixel 2, pixel 3, pixel 4, and pixel 5, it can be understood that when the size (width) of the pixel is smaller, the line pair is better reflected, and the resolution of the corresponding image is higher. Therefore, after the size of the fourth pixel unit is smaller, the resolution of the image can be improved.

[0087] From the above description, it can be understood that in the pixel array provided in the embodiment of the present application, the efficiency of receiving light by the first pixel unit, the second pixel unit, and the third pixel unit is much smaller than the efficiency of receiving light by the fourth pixel unit, so that in the same exposure time, the output signal of the fourth pixel unit is always higher than that of the first pixel unit, the second pixel unit, and the third pixel unit. In this case, the brightness of the image is higher and the color is slightly brighter. Therefore, the size of the fourth pixel unit is smaller than that of another type of pixel unit, the exposure of the fourth pixel unit can be reduced, and the saturation of the fourth pixel unit is delayed, so that the first pixel unit, the second pixel unit, and the third pixel unit are more fully exposed, and the number of image color signals is increased. In addition, the size of the fourth pixel unit can be reduced, and the image resolution can be further improved.

[0088] FIG. 20 shows another pixel array according to an embodiment of the present application. In the pixel array shown in FIG. 20, the sizes of the first pixel unit, the second pixel unit, the third pixel unit, and the fourth pixel unit are reconfigured based on the concept of reducing the size of the fourth pixel unit to improve image resolution. The first pixel group includes two red pixel units and two white pixel units. In FIG. 20, R is used to represent the red light received by the red pixel unit, and W is used to represent the white light received by the white pixel unit. The second pixel group includes two green pixel units and two white pixel units. In FIG. 20, Gr is used to represent the green light received by the green pixel unit in the second pixel group. The third pixel group includes two green pixel units and two white pixel units. In FIG. 20, Gb is used to represent the green light received by the green pixel unit in the third pixel group. The fourth pixel group includes two blue pixel units and two white pixel units. In Fig. 20, B represents the blue light received by the blue pixel unit. It should be understood that Gr and Gb above represent the green light separately. Gr and Gb are used to represent the green light separately to distinguish the green light received by the different green pixel units in the second pixel group and the third pixel group.

[0089] The first pixel unit, the second pixel unit, and the third pixel unit have the same size, and the size of the fourth pixel unit is smaller than that of the first pixel unit. Compared with the fourth pixel unit shown in Fig. 4, the size of the fourth pixel unit shown in Fig. 20 is 0.8 to 0.9 times that of the fourth pixel unit shown in Fig. 4, for example, 0.8 times, 0.85 times, or 0.9 times. The sizes of the first pixel unit, the second pixel unit, and the third pixel unit may be increased to 1 to 4 times that of the same pixel unit shown in Fig. 4, for example, 1 time, 2 times, 3 times, 4 times, or other different sizes.

[0090] Please continue to refer to FIG. 20. In the row and column directions of the pixel array, two monochrome pixel units are separated by one white pixel unit to prevent the monochrome pixel units with increased size from affecting each other's light capture. FIG. 21 is a schematic diagram of the light capture of different pixel units. It can be seen from FIG. 21 that when each of the first pixel unit, the second pixel unit, and the fourth pixel unit includes a microlens, the size of the first microlens 231 is larger than the size of the fourth microlens 234, so that more light can be focused on the first filter area 211 and the light irradiated on the fourth filter area 214 is reduced. In addition, the size of the second microlens 232 is also larger than the size of the fourth microlens 234, so that more light can be focused on the second filter area 212 and the light irradiated on the fourth filter area 214 is reduced. The third pixel unit may be arranged in the same manner as the first pixel unit and the second pixel unit. The details will not be explained again here.

[0091] From the above description, it can be seen that by changing the size of the microlens, the efficiency of receiving light by the first pixel unit, the second pixel unit, and the third pixel unit can be increased, and the efficiency of receiving light by the fourth pixel unit can be reduced, so that the first pixel unit, the second pixel unit, and the third pixel unit are more fully exposed, and the number of image color signals is increased. In addition, the size of the fourth pixel unit can be reduced, and the image resolution can be further improved. The principle is the same as that of the pixel array shown in FIG. 15. Here, the details will not be described again.

[0092] FIG. 22 shows a variation based on the image sensor shown in FIG. 3. The image sensor shown in FIG. 22 adds an optical filtering module 40. After being focused by the lens 10, the mixed incident light passes through the optical filtering module 40 and is then illuminated onto the pixel array 20. The optical filtering module 40 is disposed on the light-incoming side of the pixel array 20 to filter out noise in the mixed incident light of the composite band, and only keeps the light that can pass through the pixel array 20 (i.e., the light that is emitted from the target object and used for imaging) to improve the imaging effect. For example, when the light required by the pixel array 20 is the light of the white light band and the light of the infrared light band, the optical filtering module 40 can be a dual passband optical filtering module. The two passbands that can pass through the dual passband optical filtering module are the white light band and the infrared light band, respectively. When the infrared light band corresponding to the pixel array 20 is a specific narrowband infrared light band, the dual passband optical filtering module can transmit the light of the specific narrowband infrared light band correspondingly. After the dual passband optical filtering module is used to transmit white light and infrared light and block noise in other frequency bands, the impact of noise on the imaging of the image sensor can be reduced, the optical interference and noise present in the operation of subsequent modules can be reduced, and the imaging quality of the image sensor can be improved.

[0093] Referring to FIG. 23, the monochrome pixel units (first pixel unit, second pixel unit, and third pixel unit) in the pixel array are coated with a filter layer that blocks infrared light, and the white pixel unit (fourth pixel unit) receives white light and infrared light. In FIG. 23, a shadow is used to represent the filter layer, and the first pixel unit, the second pixel unit, and the third pixel unit are all coated with a filter layer made of an infrared light blocking material. In this way, the first pixel unit, the second pixel unit, and the third pixel unit only sense the red, green, and blue components in white light. This ensures color accuracy. However, since the fourth pixel unit senses white light and infrared light, the infrared light can be used as a luminance channel in very low illumination. This greatly improves the light sensitivity performance of the image sensor and improves the signal-to-noise ratio of the image.

[0094] An embodiment of the present application further provides an imaging device. The imaging device is an electronic device, for example, a general device with a photographing function, for example, a mobile phone, a tablet computer, or a wearable electronic device. Of course, the electronic device may be another type of electronic device with a photographing function. The imaging device includes a housing and any of the above-mentioned image sensors disposed in the housing. In the above technical solution, a white pixel unit is introduced into each pixel group, and the sensitivity of the image sensor is improved by using the white pixel unit. This improves the resolution and signal-to-noise ratio of the image.

[0095] It is obvious that a person skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. As long as these modifications and variations of the present application fall within the scope of protection defined by the following claims and their equivalent technologies, the present application is intended to include these modifications and variations.

Claims

1. a plurality of pixel units, the plurality of pixel units having a first pixel group, a second pixel group, a third pixel group, and a fourth pixel group arranged in an array, the second pixel group and the third pixel group being arranged in an oblique direction; the first pixel group includes a first pixel unit and a fourth pixel unit, the second pixel group includes a second pixel unit and a fourth pixel unit, the third pixel group includes a second pixel unit and a fourth pixel unit, and the fourth pixel group includes a third pixel unit and a fourth pixel unit; the first pixel unit, the second pixel unit, and the third pixel unit are monochrome pixel units respectively used for receiving light of different colors, and the fourth pixel unit is a white pixel unit for receiving white light; The image processing module further includes: an image processing module that performs an interpolation process on a first color light received by the monochrome pixel unit in each pixel group to obtain a second color light corresponding to a pixel unit at a position of the white pixel unit in the pixel group; performs a Quad Bayer process on four pixel units in each pixel group based on the first color light and the second color light to obtain a Quad Bayer image; obtains luminance information based on the white light received by the white pixel unit in each pixel group; and combines the Quad Bayer image formed by each pixel group with the luminance information corresponding to the pixel group to obtain a ¼ resolution image. the image processing module re-mosaicking the pixel array to obtain an image; performing an interpolation process on the white light received by each white pixel unit to obtain luminance information corresponding to each pixel unit; The image obtained through the re-mosaic process and the luminance information corresponding to each pixel unit are combined to obtain a full resolution image. The method is further configured as follows: Image sensor.

2. each of the first pixel group, the second pixel group, the third pixel group, and the fourth pixel group includes four pixel units arranged in an array; Among the four pixel units in each pixel group, at least one pixel unit is the white pixel unit, and the remaining pixel units are monochrome pixel units used to receive light of the same color. The image sensor of claim 1 .

3. The image sensor of claim 2 , wherein each pixel group includes three monochrome pixel units and one white pixel unit that are used to receive light of the same color.

4. The image sensor of claim 3 , wherein in each of two adjacent pixel groups, a white pixel unit located in one pixel group and a white pixel unit located in the other pixel group are separated by another pixel unit.

5. 3. The image sensor of claim 2, wherein each pixel group includes two monochrome pixel units used to receive light of the same color and two white pixel units, the two white pixel units being arranged in a diagonal direction.

6. The image sensor of claim 1 , wherein a size of the white pixel unit is smaller than a size of the monochrome pixel unit.

7. The image sensor of claim 6 , wherein the white pixel unit is encapsulated in a metal shielding layer, and a size of the white pixel unit encapsulated in the metal shielding layer is equal to the size of the monochrome pixel unit.

8. The image sensor of claim 1 , further comprising a lens used to focus light, said lens being located on a light-entering side of said plurality of pixel units.

9. The image sensor of claim 1 , further comprising a light filtering module disposed on a light input side of the plurality of pixel units.

10. The image sensor of claim 9 , wherein the optical filtering module is a dual passband optical filtering module, the dual passband optical filtering module capable of transmitting the white light and infrared light.

11. 11. The image sensor of claim 9 or 10, wherein the monochrome pixel unit is coated with a filter layer that blocks infrared light, and the white pixel unit receives the white light and the infrared light.

12. An imaging device comprising: a housing; and an image sensor according to claim 1 disposed within the housing.

Images