Image sensor and imaging device

Abstract

The application provides an image sensor and an imaging device. The image sensor comprises a microlens array, a color filter array and a photosensitive pixel array. The microlens array comprises a plurality of first lens units and at least one second lens unit. The color filter array comprises a plurality of first color filter units and at least one second color filter unit. The transmittance of the first color filter units is different from that of the second color filter units. The first lens units are arranged correspondingly to the first color filter units, and the second lens units are arranged correspondingly to the second color filter units. The photosensitive pixel array comprises a plurality of pixel units. The first color filter units cover at least one pixel unit, and the second color filter units cover at least one pixel unit. Light signals passing through the first lens units and the first color filter units form first electric signals. Light signals passing through the second lens units and the second color filter units are used to form at least second electric signals. An image signal is formed based on the first electric signals and the second electric signals.

Description

Technical Field

[0001] This utility model relates to the field of image sensing technology, and in particular to an image sensor and imaging device. Background Technology

[0002] CMOS image sensors (CIS) are already ubiquitous in our daily lives, from smartphones to cars, security cameras, robots, and AR / VR entertainment devices. This trend is driven by strong demand for smart, connected, and autonomous consumer products as we transition into the Internet of Things (IoT) era. Thanks to advancements in manufacturing processes, CIS has evolved from front-illuminated to back-illuminated structures, significantly improving sensitivity, reducing optical crosstalk, and enhancing final image quality with smaller pixels.

[0003] Typically, CMOS image sensors use organic color filters of red, green, and blue for light filtering, and then rely on backend algorithms to reproduce the colors of the image. However, existing image sensors using only these three organic color filters have limited spectral richness, failing to meet user requirements. Furthermore, this method of reproducing object colors based solely on three colors suffers from insufficient color gamut and significant susceptibility to lighting conditions. Using a multispectral camera with more color channels to achieve higher spectral resolution is a feasible solution. This typically involves using a separate multispectral sensor to acquire spectral information and provide it to other image sensors used for imaging, resulting in better color reproduction performance. This approach requires at least one additional multispectral sensor. However, today's consumer products demand increasingly smaller and more integrated image sensors. Therefore, using a separate multispectral sensor presents drawbacks such as high cost and difficulty in miniaturization. Moreover, existing image sensors cannot simultaneously provide multispectral measurement and focusing capabilities while maintaining a miniaturized design. Utility Model Content

[0004] In view of this, the present invention provides an image sensor that increases the richness of transmittance and can meet user needs.

[0005] An image sensor includes a microlens array, a color filter array, and a photosensitive pixel array arranged sequentially along the incident light direction;

[0006] The microlens array includes multiple first lens units and at least one second lens unit;

[0007] The color filter array includes a plurality of first color filter units and at least one second color filter unit, wherein the transmittance of each first color filter unit is different from that of the second color filter unit;

[0008] The first lens unit is configured correspondingly to the first color filter unit, and the second lens unit is configured correspondingly to the second color filter unit;

[0009] The photosensitive pixel array includes a plurality of pixel units, wherein the first color filter unit covers at least one of the pixel units, and the second color filter unit covers at least one of the pixel units;

[0010] The light signals passing through the first lens unit and the first color filter unit are used to form at least a first electrical signal, and the light signals passing through the second lens unit and the second color filter unit are used to form at least a second electrical signal, so as to form an image signal based on the first electrical signal and the second electrical signal.

[0011] Optionally, the first color filter unit covers at least one of the pixel units, and the second color filter unit covers at least two of the pixel units; the light signal passing through the first lens unit and the first color filter unit is used for imaging, and the light signal passing through the second lens unit and the second color filter unit is used for spectral measurement and / or focusing.

[0012] Optionally, the image sensor further includes a grid, which is configured to correspond to the color filter array and form a light-transmitting area. The light-transmitting area includes a plurality of first light-transmitting areas and at least one second light-transmitting area. The first color filter unit is configured to correspond to the first light-transmitting area, and the second color filter unit is configured to correspond to the second light-transmitting area.

[0013] Optionally, the projection of the second light-transmitting area covers at least two of the pixel units and forms at least a first type of pixel unit group and a second type of pixel unit group, so as to at least realize the acquisition of the first phase information and the second phase information; and / or, the area between the pixel units corresponding to the second light-transmitting area is further provided with an overflow portion, located between adjacent pixel units.

[0014] Optionally, the projection of the first light-transmitting area covers one of the pixel units, and the projection of the second light-transmitting area covers two or four of the pixel units; or, the projection of the first light-transmitting area covers four of the pixel units, and the projection of the second light-transmitting area covers four of the pixel units; or, the projection of the first light-transmitting area covers two of the pixel units, and the projection of the second light-transmitting area covers two of the pixel units.

[0015] Optionally, the microlens array includes at least two adjacent second lens units, and the color filter array includes at least two second color filter units with the same transmittance. The two second color filter units are arranged adjacent to each other, and each second lens unit is arranged corresponding to each second color filter unit. The light signals passing through each second lens unit and each second color filter unit are used for spectral measurement and / or focusing.

[0016] Optionally, the color filter array further includes a third color filter unit, which covers at least one pixel unit and has a transmittance different from that of each first color filter unit. The light signal passing through the third color filter unit is used for at least spectral measurement. A combination of adjacent second color filter units and the third color filter units surrounding the second color filter units is defined as a color filter unit group. The image sensor includes at least one of the color filter unit groups.

[0017] Optionally, the image sensor includes a plurality of color filter unit groups, each of which has a different transmittance; and / or, the microlens array includes a third lens unit, which is correspondingly disposed with the third color filter unit; and / or, the color filter array includes at least four third color filter units with different transmittances, each of which has a transmittance different from that of the first color filter unit, each of which covers at least one pixel unit, and the light signals passing through each of the second lens units and each of the second color filter units are used for focusing, and the light signals passing through each of the third color filter units are used for spectral measurement; or, two of the third color filter units are located on opposite sides of a second color filter unit, and the other two of the third color filter units are located on opposite sides of the second color filter unit.

[0018] Optionally, the image output mode of the image sensor includes a first mode and a second mode, wherein:

[0019] In the first mode, the image sensor is controlled to acquire different data images. The first data image corresponds to the first electrical signal read out by the pixel unit corresponding to each of the first color filter units and the second electrical signal read out by the pixel unit corresponding to each of the second color filter units, and the pixel unit corresponding to each of the third color filter units does not read out an electrical signal; the second data image corresponds to the third electrical signal read out by the pixel unit corresponding to each of the third color filter units.

[0020] In the second mode, each pixel unit reads out an electrical signal individually, including an electrical signal for imaging, an electrical signal for focusing, and an electrical signal for spectral measurement.

[0021] Optionally, the image sensor includes a first mode and a second mode, wherein:

[0022] In the first mode, the image sensor is controlled to output images in two frames. In the first frame, the first electrical signal for imaging is read from the pixel unit corresponding to each of the first color filter units, and the second electrical signal for focusing is read from the pixel unit corresponding to each of the second color filter units. The pixel unit corresponding to each of the third color filter units does not read out any electrical signal. In the second frame, the third electrical signal for spectral measurement is read from the pixel unit corresponding to each of the third color filter units.

[0023] In the second mode, each pixel unit reads out an electrical signal individually, including an electrical signal for imaging, an electrical signal for focusing, and an electrical signal for spectral measurement.

[0024] Optionally, the pixel unit has a shared structure, and the adjacent second color filter unit corresponding pixel unit forms at least a first shared structure and a second shared structure with the adjacent third color filter unit corresponding pixel unit, wherein the shared structure corresponds to four pixel units, including a central output element and floating diffusion nodes located on both sides of the output element, and the floating diffusion nodes are electrically connected to the output element.

[0025] Optionally, the color filter array includes multiple color units, each color unit including a first color subunit based on a first color filter unit and a second color subunit based on a second color filter unit or a combination of the second and first color filter units, wherein the first and second color subunits are arranged according to a preset period; or, the color filter array includes multiple color units, each color unit including the second and first color filter units, wherein the color units are arranged according to a preset period.

[0026] Optionally, the first color filter unit in the first color subunit includes an RGGB / RYYB Bayer array, the second color subunit includes a first color filter unit and a second color filter unit, and the first color filter unit includes an RG / RY color filter unit; or the first color filter unit includes a four-Bayer color filter unit based on RGG / RYY; or the first color filter unit includes a four-Bayer color filter unit based on RG / RY; or the first color filter unit includes an RGG / RYY color filter unit.

[0027] Optionally, the color filter array includes a plurality of second color filter units, each of which has a different transmittance; and / or, the pixel unit includes at least one photoelectric conversion element for converting light signals into electrical signals; and / or, the second color filter units are spaced apart between the first color filter units; and / or, the pixel unit density corresponding to the second color filter unit does not exceed 25%; and / or, the color filter array has at least two sub-color filter layers, wherein the first color filter unit and the second color filter unit are located in different sub-color filter layers.

[0028] This application also relates to an imaging device, including the image sensor described above and a control method for the image sensor described above.

[0029] The image sensor of this invention uses a first color filter unit and a second color filter unit with different transmittance to filter light, and can form an image signal based on the first electrical signal and the second electrical signal, which increases the richness of transmittance and can meet user needs. Attached Figure Description

[0030] Figure 1 This is a partial top view of the image sensor according to the first embodiment of this application.

[0031] Figure 2 This is a cross-sectional structural diagram of the image sensor according to the first embodiment of this application.

[0032] Figure 3 This is a schematic diagram of the pixel circuit structure of the image sensor of this application.

[0033] Figure 4a This is a schematic diagram of a color unit according to an embodiment of this application.

[0034] Figure 4b This is a schematic diagram of a color unit according to another embodiment of this application.

[0035] Figure 5 This is a partial top view of the image sensor according to the second embodiment of this application.

[0036] Figure 6 This is a cross-sectional structural diagram of the image sensor according to the second embodiment of this application.

[0037] Figure 7 This is a partial top view of the image sensor according to the third embodiment of this application.

[0038] Figure 8 This is a cross-sectional structural schematic diagram of the image sensor according to the third embodiment of this application.

[0039] Figure 9 This is a schematic diagram of the shared structure of pixel units in the third embodiment of this application.

[0040] Figure 10 This is a partial top view of the image sensor according to the fourth embodiment of this application.

[0041] Figure 11 This is a cross-sectional structural schematic diagram of the image sensor according to the fourth embodiment of this application. Detailed Implementation

[0042] The following specific embodiments illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification.

[0043] In the following description, reference is made to the accompanying drawings, which illustrate several embodiments of the present application. It should be understood that other embodiments may also be used, and changes in mechanical composition, structure, electrical and operational aspects may be made without departing from the spirit and scope of the present application. The following detailed description should not be considered limiting; the terminology used herein is for describing particular embodiments only and is not intended to limit the present application.

[0044] Although the terms first, second, etc., are used in some instances to describe various elements herein, these elements should not be limited by these terms. These terms are only used to distinguish one element from another.

[0045] Furthermore, as used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms “comprising,” “including,” indicate the presence of a feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms “or” and “and / or” as used herein are interpreted as inclusive, or mean any one or any combination thereof. Thus, “A, B, or C” or “A, B, and / or C” means “any one of: A; B; C; A and B; A and C; B and C; A, B, and C.” Exceptions to this definition arise only when combinations of elements, functions, steps, or operations are inherently mutually exclusive in some way.

[0046] First Embodiment

[0047] Figure 1 This is a partial top view schematic diagram of the image sensor according to the first embodiment of this application. Figure 2 This is a cross-sectional structural schematic diagram of the image sensor according to the first embodiment of this application, as shown below. Figure 1 and Figure 2 As shown, the image sensor includes a microlens array 11, a color filter array 12, and a photosensitive pixel array 13 arranged sequentially along the incident light direction;

[0048] The microlens array 11 includes a plurality of first lens units 111 and at least one second lens unit 112;

[0049] The color filter array 12 includes a plurality of first color filter units 121 and at least one second color filter unit 122, wherein the transmittance of each first color filter unit 121 is different from the transmittance of the second color filter unit 122;

[0050] Each first lens unit 111 is correspondingly arranged with each first color filter unit 121, and the second lens unit 112 is correspondingly arranged with the second color filter unit 122;

[0051] The photosensitive pixel array 13 includes a plurality of pixel units 131, each first color filter unit 121 covers at least one pixel unit 131, and each second color filter unit 122 covers at least one pixel unit 131;

[0052] The light signal passing through the first lens unit 111 and the first color filter unit 121 is used to form at least a first electrical signal, and the light signal passing through the second lens unit 112 and the second color filter unit 122 is used to form at least a second electrical signal, so as to form an image signal based on the first electrical signal and the second electrical signal.

[0053] The image sensor of this application uses a first color filter unit 121 and a second color filter unit 122 with different transmittances for light filtering, and can form an image signal based on the first electrical signal and the second electrical signal, which increases the richness of transmittance and can meet user needs.

[0054] In one implementation, the first color filter unit 121 covers at least one pixel unit 131, and the second color filter unit 121 covers at least two pixel units 131; the light signal passing through the first lens unit 121 and the first color filter unit 121 is used for imaging, and the light signal passing through the second lens unit 112 and the second color filter unit 122 is used for spectral measurement and / or focusing.

[0055] Specifically, the light signal passing through the first lens unit 111 and the first color filter unit 121 is used for at least imaging, and the light signal passing through the second lens unit 112 and the second color filter unit 122 is used for at least spectral measurement and / or focusing. In this embodiment, the image sensor can be used in electronic devices, including digital cameras, cameras, medical imaging equipment, security monitoring, automotive electronics, mobile phones, machine vision, and other devices; the image sensor can be an active pixel sensor in complementary metal-oxide-semiconductor (CMOS) technology, or the image sensor can also be a charge-coupled device (CCD), but is not limited thereto.

[0056] In this embodiment, the image sensor can perform spectral measurements to achieve color correction and autofocus while realizing normal imaging functions. It does not require a separate multispectral sensor and has a smaller size and higher integration compared to existing image sensors.

[0057] Furthermore, this application can share multispectral pixels with focusing pixels (for example, the shared pixel is the pixel corresponding to the second color filter unit 122), and under certain algorithm correction capabilities, it can ensure that the shared pixel has a higher density (the algorithm correction capability is limited, and if independent multispectral pixels and independent focusing pixels are set, the setting density of these two types of pixels will inevitably be limited).

[0058] Optionally, the multiple first color filter units 121 include multiple red (R) color filter units, multiple green (G) color filter units, and multiple blue (B) color filter units. For example, a red color filter unit R, two green color filter units G, and a blue color filter unit B arranged adjacently can be combined to form a color filter unit, constituting a multi-Bayer array arrangement. That is, in one pixel region, four photoelectric conversion elements PD correspond to the same color filter material, thus four pixel regions correspond to sixteen photoelectric conversion elements PD to form a Bayer structure. Of course, in other implementations, a pixel region can also have color filter units of different colors, with different photoelectric conversion elements PD corresponding to different color filter units, and one pixel region forming a Bayer array.

[0059] Optionally, the color of the second color filter unit 122 is different from the color of each of the first color filter units 121. For example, the second color filter unit 122 is yellow, purple, brown, cyan, magenta, etc. Therefore, the transmittance of the second color filter unit 122 is different from the transmittance of the first color filter unit 121.

[0060] Optionally, when the light signal passing through the second lens unit 112 and the second color filter unit 122 is used for spectral measurement, the second color filter unit 122 is used to filter the incident light (so that light rays within the passband of the primary color filter pass through). When the light signal passing through the second lens unit 112 and the second color filter unit 122 is used for focusing, the light passing through the two pixel units 131 generates a phase difference, thereby determining the position of the focal point. Of course, the light signal passing through the second lens unit 112 and the second color filter unit 122 can be used simultaneously for spectral measurement and phase focusing.

[0061] Optionally, the first lens unit 111 and the second lens unit 112 of the microlens array 11 are convex lenses used to converge light rays.

[0062] Optionally, such as Figure 1 and Figure 2As shown, the image sensor also includes a grid 14, which is configured to correspond to the color filter array 12 and form a light-transmitting area. The light-transmitting area includes multiple first light-transmitting areas and at least one second light-transmitting area. Each first color filter unit 121 is configured to correspond to its respective first light-transmitting area, and each second color filter unit 122 is configured to correspond to its respective second light-transmitting area. In this embodiment, adjacent first color filter units 121 are separated by a grid 14 to prevent light crosstalk; the area covered by the second color filter unit 122 has no grid 14 structure. Specifically, the grid 14 can be configured in the same layer as the color filter units, which is equivalent to the grid 14 being embedded between the color filter units, or the grid 14 and the color filter units can be located in different material layers to achieve their respective functions.

[0063] Optionally, such as Figure 1 and Figure 2 As shown, the projection of the second light-transmitting area covers at least two pixel units 131, forming at least a first type pixel unit group and a second type pixel unit group, so as to at least achieve the acquisition of first phase information and second phase information. For example, the first type pixel unit group and the second type pixel unit group can be left and right pixel unit groups, so the corresponding first phase information and second phase information acquired are left phase information and right phase information. The first type pixel unit group and the second type pixel unit group can be upper and lower pixel unit groups, so the corresponding first phase information and second phase information acquired are upper phase information and lower phase information. Of course, it can also be other phase information that can achieve focusing.

[0064] Optionally, such as Figure 1 and Figure 2 As shown, the projection of each first light-transmitting area covers one pixel unit 131, and the projection of each second light-transmitting area covers two pixel units 131.

[0065] In this embodiment, the size of the first light-transmitting area matches the size of the first color filter unit 121, and the size of the second light-transmitting area matches the size of the second color filter unit 122.

[0066] Optionally, the size of the second lens unit 112 is larger than the size of the first lens unit 111, for example, the size of the second lens unit 112 is twice that of the first lens unit 111.

[0067] In other embodiments, the projection of the first light-transmitting area covers two pixel units 131, and the projection of the second light-transmitting area covers two pixel units 131; or, the projection of the first light-transmitting area covers four pixel units 131, and the projection of the second light-transmitting area covers four pixel units 131.

[0068] In this embodiment, the size of the first light-transmitting area matches the size of the first color filter unit 121, and the size of the second light-transmitting area matches the size of the second color filter unit 122.

[0069] Optionally, the color filter array 12 includes a plurality of second color filter units 122, each of which has a different transmittance.

[0070] Optionally, the array of multiple second color filter units 122 in the color filter array 12 includes multiple second color filter unit blocks, which are arranged periodically to form a second color filter unit array, and each second color filter unit block includes multiple second color filter units 122.

[0071] In this embodiment, each second color filter unit 122 has a different color. For example, the color filter array 12 includes at least one yellow second color filter unit 122, at least one purple second color filter unit 122, at least one brown second color filter unit 122, at least one cyan second color filter unit 122, and at least one magenta second color filter unit 122; or, each second color filter unit 122 in a second color filter unit block has a different color. For example, the second color filter unit block includes at least one yellow second color filter unit 122, at least one purple second color filter unit 122, at least one brown second color filter unit 122, at least one cyan second color filter unit 122, and at least one magenta second color filter unit 122.

[0072] Optionally, an overflow portion (not shown in the figure) is also provided in the area between the pixel units 131 corresponding to the second light-transmitting area. The overflow portion is located between adjacent pixel units 131. The overflow portion can be formed as an electron flow channel between adjacent pixel units 131, which is beneficial to realize the flow of electrons in adjacent pixel units 131 corresponding to the same color filter when they are overexposed. It is beneficial to balance the focusing performance and image data performance. For example, the overflow portion can be formed by the incomplete isolation of the isolation structures between adjacent pixel units 131. For adjacent pixel units 131 in a light-transmitting area, the opening of part of the deep trench isolation structure forms the overflow portion, so that electrons can flow based on the overflow portion.

[0073] Optionally, Figure 3 This is a schematic diagram of the pixel circuit structure of the image sensor of this application, as shown below. Figure 2 and Figure 3 As shown, each pixel unit 131 includes at least one photoelectric conversion element PD, which is used to convert optical signals into electrical signals.

[0074] Optionally, such as Figure 3As shown, each pixel unit 131 also includes a transmission transistor TX, a floating diffusion region FD, a reset transistor RST, and a source follower transistor SF corresponding to the photoelectric conversion element PD. The transmission transistor TX is connected between the photoelectric conversion element PD and the floating diffusion region FD. The source follower transistor SF is used to output the electrical signal of the floating diffusion region FD. The reset transistor RST is used to reset the floating diffusion region FD. In other implementations, each pixel unit 131 also includes a gain control transistor DCG, coupled to the floating diffusion region FD. It can be placed between the reset transistor RST and the floating diffusion region FD, and the capacitance of the corresponding floating diffusion region FD can be adjusted by turning the gain control transistor DCG on and off.

[0075] Optionally, each pixel unit 131 also forms a selection transistor RS, which is used to select the electrical signal output by the source follower transistor SF to the column line (Pixel out).

[0076] Optionally, the photosensitive pixel array 13 is formed on a semiconductor substrate, for example, by doping the semiconductor substrate, such as by plasma implantation, to form the source and drain terminals of a photoelectric conversion element PD, a floating diffusion region FD, a source follower transistor SF, a reset transistor RST, and a select transistor RS. The photoelectric conversion element PD can be a photodiode, and each transistor in the pixel circuit can be an NMOS transistor.

[0077] Optionally, the image sensor also includes a readout circuit and a control circuit connected to the photosensitive pixel array 13. The functional logic unit is connected to the readout circuit, and the readout circuit and control circuit are connected to a status register to control the pixel array. The photosensitive pixel array 13 includes multiple pixel units 131 arranged in rows and columns. The pixel signals output by the photosensitive pixel array 13 are output to the readout circuit via column lines. In specific implementations, the readout circuit may include an analog-to-digital converter (ADC) circuit and other circuits. In some applications, after the pixels acquire image data, they are read out using the readout mode specified by the status register and then transmitted to the functional logic unit. In some applications, the status register may contain a programmed selection system to determine whether the readout system uses a rolling shutter mode or a global shutter mode for exposure readout. The functional logic unit may store raw image data or image data after image processing. In some implementations, the readout circuit may read out one row of image data at a time along the readout column lines; of course, other methods may also be used to read out image data. The operation of the control circuit can be determined by the current setting of the status register. For example, the control circuit generates a shutter signal to control image acquisition. In some applications, this shutter signal can be a global exposure signal, which allows all pixels of the photosensitive pixel array 13 to acquire their image data simultaneously through a single acquisition window. In other applications, this shutter signal can also be a rolling exposure signal, which allows the pixels of each pixel row of the photosensitive pixel array 13 to continuously perform exposure reading operations through the acquisition window.

[0078] Optionally, Figure 4a This is a schematic diagram of a color unit according to an embodiment of this application. Figure 4b This is a schematic diagram of a color unit according to another embodiment of this application, as shown below. Figure 4a and Figure 4b As shown, the color filter array 12 includes multiple color units. Each color unit includes a first color sub-unit based on a first color filter unit 121 and a second color sub-unit based on a second color filter unit 122 or a combination of the second color filter unit 122 and the first color filter unit 121. The first and second color sub-units are arranged according to a preset period. In this embodiment, Figure 4a The density of the second color filter unit 122 is 1 / 64, wherein the second color filter unit 122 replaces the original blue B color filter unit; Figure 4b The density of the second color filter unit 122 is 1 / 16, and the second color filter unit 122 replaces the original blue B color filter unit.

[0079] Optionally, the color filter array 12 includes multiple color units, each including a second color filter unit 122 and a first color filter unit 121, arranged according to a preset period. That is, in the arrangement period of the pixel array of the image sensor, the first color filter unit 121 and the second color filter unit 122 exist in each period (each color unit). Thus, in this application, the design based on the second color filter unit 122 can ensure a relatively higher density and enrich image information under certain algorithm correction capabilities.

[0080] Optionally, the first color filter unit 121 in the first color subunit includes an RGGB / RYYB Bayer array, and the second color subunit includes the first color filter unit 121 and the second color filter unit 122. The first color filter unit 121 in the second color subunit includes an RG / RY color filter unit, or the first color filter unit 121 includes an RGG / RYY×4 color filter unit; or the first color filter unit 121 includes an RG / RY×4 color filter unit; or the first color filter unit 121 includes an RGG / RYY color filter unit. Wherein, the first color filter unit 121 including an RGGB / RYYB Bayer array means that the first color filter unit 121 includes an RGGB Bayer array, or the first color filter unit 121 includes an RYYB Bayer array. Of course, the first color filter unit 121 may also include both RGGB and RYYB Bayer arrays. In this example, RG / RY and RGG / RYY represent similar meanings. In addition, in other implementations, the first color filter unit 121 may also include other existing arrangements of conventional image sensor imaging in the prior art, such as adding white pixels to the array.

[0081] Specifically, in one example, the first color subunit may include multiple first color filter units 121 arranged in an RGGB Bayer array. The second color subunit includes first color filter units 121 and second color filter units 122. The first color filter units 121 in the second color subunit include RG color filter units, and the second color filter units 122 include yellow color filter units. This is equivalent to yellow and green units replacing the GB units in the RGGB array to form a replaced Bayer array. The first and second color subunits together form a periodically arranged pixel array. Each color filter unit may correspond to one pixel unit 131, and each first color filter unit 121 may correspond to one first lens unit 111. See [reference needed]. Figure 1 As shown.

[0082] Specifically, in another example, the first color subunit may include multiple first color filter units 121, which are arranged in a four-Bayer array of RGGB, i.e., four first color filter units 121 of the same color are arranged adjacently as a group, and sixteen color filter units form a Bayer structure. Further, the second color subunit includes a group of second color filter units 122 and three groups of first color filter units 121, i.e., the first color filter unit 121 includes RGG×4 color filter units. Alternatively, the first color filter unit 121 may include RG×4 color filter units, and the second color filter unit 122 may include yellow color filter units. See [reference needed]. Figure 5 As shown.

[0083] Specifically, in another example, the first color subunit may include multiple first color filter units 121 arranged in an RGGB Bayer array. The second color subunit may include first color filter units 121 and second color filter units 122. The first color filter units 121 in the second color subunit may include RGG color filter units, and the second color filter units 122 may include yellow color filter units. In this case, one first color filter unit corresponds to one first lens unit 111, and one first color filter unit 121 corresponds to two pixel units 131 (or four pixel units 131). One second color filter unit 122 corresponds to one second lens unit 112, and one second color filter unit 122 corresponds to two pixel units 131 (or four pixel units 131). That is, the first color filter unit 121 includes RGG color filter units. See [link to previous section]. Figure 10 As shown.

[0084] Optionally, the second color filter units 122 are spaced apart from the first color filter units 121. In this example, the spaced-apart second color filter units 122 are equivalent to replacing normal pixels in a non-adjacent manner, which is beneficial to the distribution of multispectral colors and algorithm processing.

[0085] Optionally, the pixel unit 131 corresponding to the second color filter unit 122 has a density not exceeding 25%. That is, the multispectral pixel uses the R / G / B positions that can be occupied by normal pixel arrangement, and its pixel density does not exceed 25%, for example, it can be 6%, etc. The multispectral pixel acquires multispectral images with lower spatial resolution, and the multispectral image can be extracted by an algorithm for image color correction. At the same time, the signal of the multispectral pixel can be corrected by an algorithm and then imaged with other normal RGB pixels.

[0086] Optionally, the color filter array 12 has at least two sub-color filter layers, with the first color filter unit 121 and the second color filter unit 122 located in different sub-color filter layers.

[0087] Specifically, in this example, the color filter array of the pixel array is designed using a combined filtering approach. Traditional color pixels, such as the RGGB unit corresponding to the first color filter unit 121, are fabricated in the first layer, while the second color filter unit 122, representing multispectral and / or focusing, is fabricated in a second layer different from the first layer, such as the second layer being located above the first layer. The color filter array 12 is formed based on the first and second layers together. The color filter layer of the multispectral pixel can be an organic material, a multilayer thin film structure, or a combination of both.

[0088] Second Embodiment

[0089] Figure 5 This is a partial top view schematic diagram of the image sensor according to the second embodiment of this application. Figure 6 This is a cross-sectional structural diagram of the image sensor according to the second embodiment of this application, as shown below. Figure 5 and Figure 6 As shown, the structure and function of the image sensor in this embodiment are largely the same as those in the first embodiment, except that the projection of the second light-transmitting area covers a different number of pixel units 131. In this embodiment, the projection of each first light-transmitting area covers one pixel unit 131, and the projection of each second light-transmitting area covers four pixel units 131. In this embodiment, the size of the first light-transmitting area matches the size of the first color filter unit 121, the size of the second light-transmitting area matches the size of the second color filter unit 122, and the size of the second color filter unit 122 matches the sum of the sizes of the four first color filter units 121.

[0090] Optionally, the size of the second lens unit 112 is larger than the size of the first lens unit 111, for example, the size of the second lens unit 112 is 4 times that of the first lens unit 111.

[0091] Optionally, two adjacent first color filter units 121 are separated by a grid 14 to prevent light crosstalk; the area covered by the second color filter unit 122 has no grid 14 structure.

[0092] Third Embodiment

[0093] Figure 7 This is a partial top view schematic diagram of the image sensor according to the third embodiment of this application. Figure 8 This is a cross-sectional structural schematic diagram of the image sensor according to the third embodiment of this application, as shown below. Figure 7 and Figure 8 As shown, the structure and function of the image sensor in this embodiment are largely the same as those in the above embodiments, except that the number of second lens units 112 and the color filter array 12 also include a third color filter unit 123. Specifically, as Figure 7 and Figure 8As shown, the microlens array 11 includes at least two adjacent second lens units 112, and the color filter array 12 includes at least two second color filter units 122 with the same transmittance. The two second color filter units 122 are arranged adjacent to each other, and each second lens unit 112 is correspondingly arranged with each second color filter unit 122. The light signals passing through each second lens unit 112 and each second color filter unit 122 are used for spectral measurement and / or focusing.

[0094] In this embodiment, each second color filter unit 122 covers at least two pixel units 131; the colors of two adjacent second color filter units 122 are the same, for example, both are yellow or other colors.

[0095] Optionally, such as Figure 7 As shown, the grille 14 includes at least two second light-transmitting areas, and each second color filter unit 122 is provided corresponding to each second light-transmitting area; the area covered by each second color filter unit 122 has no grille 14 structure; there is a grille 14 structure between two adjacent second color filter units 122.

[0096] Optionally, such as Figure 7 and Figure 8 As shown, the third color filter unit 123 covers at least one pixel unit 131, and its transmittance is different from that of each of the first color filter units 121. The light signal passing through the third color filter unit 123 is used for at least spectral measurement. A combination of adjacent second color filter units 122 and the third color filter units 123 surrounding the second color filter unit 122 is defined as a color filter unit group. The image sensor includes at least one color filter unit group, such as... Figure 7 The dashed box in the middle indicates a color filter unit group.

[0097] In other embodiments, the image sensor includes multiple color filter units, each with a different transmittance.

[0098] Optionally, the microlens array 11 includes a third lens unit 113, which is correspondingly configured with the third color filter unit 123.

[0099] In other embodiments, the light-incident side of each third color filter unit 123 may not be provided with a lens, for example, the third lens unit 113 may not be provided.

[0100] Optionally, such as Figure 7 and Figure 8As shown, the color filter array 12 includes at least four third color filter units 123, each with a different transmittance. The transmittance of each third color filter unit 123 differs from that of each first color filter unit 121. Each third color filter unit 123 covers at least one pixel unit 131. The light signals passing through each second lens unit 112 and each second color filter unit 122 are used for focusing, and the light signals passing through each third color filter unit 123 are used for spectral measurement. In this embodiment, the image sensor separates the multispectral pixels from the focusing pixels; that is, the pixels corresponding to each second color filter unit 122 are used as focusing pixels, and the pixels corresponding to each third color filter unit 123 are used as multispectral pixels. Of course, the pixels corresponding to the second color filter unit 122 can also be used for multispectral acquisition simultaneously, such as the second color filter unit 122 corresponding to yellow.

[0101] Optionally, each third color filter unit 123 may have a different color, for example, the four third color filter units 123 may have the colors of purple, brown, cyan and magenta respectively, but are not limited to this.

[0102] Optionally, the size of the second lens unit 112 is larger than the size of the first lens unit 111 and the third lens unit 113. For example, the size of the second lens unit 112 is twice that of the first lens unit 111, and the size of the second lens unit 112 is twice that of the third lens unit 113.

[0103] Optionally, the size of the first lens unit 111 is the same as the size of the third lens unit 113.

[0104] Optionally, such as Figure 7 As shown, two third color filter units 123 are located on opposite sides of a second color filter unit 122, and two other third color filter units 123 are located on opposite sides of another second color filter unit 122. Thus, the four third color filter units 123 are divided into two groups located on opposite sides of the second color filter unit 122 group formed by two adjacent second color filter units 122.

[0105] Optionally, in one implementation, the pixel array is arranged in a four-Beyer array configuration. Specifically, four adjacent first color filter units 111 of the same color form a group, and sixteen first color filter units form a four-Beyer array. Two second color filter units 122 are arranged adjacently, with each second color filter unit 122 corresponding to two pixel units 131. Four third color filter units 123 are located on either side of the two adjacent second color filter units 122. It can be understood that the color filter unit group formed by the second color filter units 122 and the third color filter units 123 effectively replaces one group (G) and one group (B) in the RGGB four-Beyer array formed by the first color filter units 121. Alternatively, in other examples, it could also be equivalent to replacing one group (G) and one group (R) in the RGGB four-Beyer array formed by the first color filter units 121.

[0106] Optionally, the image output mode of the image sensor includes a first mode and a second mode, wherein:

[0107] In the first mode, the image sensor is controlled to acquire different data images. The first data image corresponds to the first electrical signal read by the pixel unit 131 corresponding to each first color filter unit 121 and the second electrical signal read by the pixel unit 131 corresponding to each second color filter unit 122, and no electrical signal is read by the pixel unit 131 corresponding to each third color filter unit 123. The second data image corresponds to the third electrical signal read by the pixel unit 131 corresponding to each third color filter unit 123.

[0108] In the second mode, each pixel unit 131 reads out an electrical signal individually, including an electrical signal for imaging, an electrical signal for focusing, and an electrical signal for spectral measurement.

[0109] Optionally, the operation of the image sensor includes a first mode (Summing mode) and a second mode (Fullsize mode), wherein:

[0110] In the first mode, the image sensor outputs images in two frames. In the first frame, the first electrical signal is read from the pixel unit 131 corresponding to each first color filter unit 121, and the second electrical signal is read from the pixel unit 131 corresponding to each second color filter unit 122. The pixel unit 131 corresponding to each third color filter unit 123 does not read out any electrical signal. In the second frame, the third electrical signal is read from the pixel unit 131 corresponding to each third color filter unit 123.

[0111] Specifically, in one implementation of the first mode, the image sensor is controlled to acquire different data images, namely a first frame image and a second frame image. In the first frame image, the first signal corresponding to the first color filter unit 121 may be read out in a merged manner, the second signal corresponding to the second color filter unit 122 may be read out in a merged manner, and the electrical signals of the pixel units 131 corresponding to each third color filter unit 123 may not be read out. In the second frame image, the third electrical signals of the pixel units 131 corresponding to the third color filter unit 123 may be read out respectively.

[0112] In one implementation, in the first frame image, the first electrical signal corresponding to the first color filter unit 121 is read out by merging, and the second electrical signal corresponding to the second color filter unit 122 is read out by merging. For example, the first electrical signal corresponding to the first color filter unit 121 can be the result of merging the four electrical signals corresponding to four pixel units 131, as shown in the following figure. Figure 7As shown, four first color filter units 121 corresponding to the same color are combined into one signal output. There are two second color filter units 122, which are arranged adjacently. The four pixel units 131 corresponding to the two second color filter units 122 are divided into two groups and read out from different nodes simultaneously. See [reference] Figure 9 The second electrical signal can be read from different output transistors. The signal of the pixel unit 131 corresponding to the third color filter unit 123 in the same color filter unit group is not read out. At this time, the second electrical signal can be used for phase focusing, while the first electrical signal can be used for imaging. Of course, in other implementations, the signals of the two pixel units 131 can be combined, or other numbers of data frames can be used for reading out.

[0113] In the second mode, each pixel unit 131 reads out an electrical signal individually, including an electrical signal for imaging, an electrical signal for focusing, and an electrical signal for spectral measurement.

[0114] Specifically, in one implementation of this second mode, the signals corresponding to all pixel units 131 in the photosensitive pixel array 13 are read out respectively, so that the first signal based on the pixel can be used for ordinary imaging; the second signal can be used for ordinary imaging and / or focusing, or the second signal can be used for at least one of ordinary imaging, focusing and multispectral acquisition; and the third signal can be used for ordinary imaging and / or multispectral acquisition.

[0115] Optionally, Figure 9 This is a schematic diagram of the shared structure of pixel units in the third embodiment of this application, as shown below. Figure 9 As shown, pixel unit 131 has a shared structure, and the pixel unit 131 corresponding to the adjacent second color filter unit 122 and the pixel unit 131 corresponding to the adjacent third color filter unit 123 respectively form at least a first shared structure and a second shared structure, wherein the circuit configuration of each shared structure is the same, such as Figure 9 As shown in the dotted line frame, the shared structure corresponds to four pixel units 131, including a central output element and floating diffusion nodes on both sides of the output element. The floating diffusion nodes are electrically connected to the output element. As shown in the figure, the central output element SF corresponds to the floating diffusion nodes FD1 and FD2 on both sides. In addition, the shared structure also includes a shared selection transistor RS and a reset transistor RST.

[0116] Specifically, in this example, the structure group composed of the two second color filter units 122 is formed with the third color filter units on both sides to form two different shared structures. This is conducive to symmetrical arrangement and improves the accuracy of signal acquisition. The focus pixel is located in the center, which can help improve the accuracy of the focus signal.

[0117] Fourth embodiment

[0118] Figure 10 This is a partial top view schematic diagram of the image sensor according to the fourth embodiment of this application. Figure 11 This is a cross-sectional structural schematic diagram of the image sensor according to the fourth embodiment of this application, as shown below. Figure 10 and Figure 11 As shown, the structure and function of the image sensor in this embodiment are largely the same as those in the first embodiment, except that the projection of the first and second light-transmitting areas covers different numbers of pixel units 131. In this embodiment, the projection of each first light-transmitting area covers four pixel units 131, and the projection of each second light-transmitting area covers four pixel units 131.

[0119] Optionally, the size of the first light-transmitting area matches the size of the first color filter unit 121, the size of the second light-transmitting area matches the size of the second color filter unit 122, and the size of the second color filter unit 122 is equal to the size of the first color filter unit 121.

[0120] Optionally, the size of the second lens unit 112 is equal to the size of the first lens unit 111.

[0121] Fifth Embodiment

[0122] This application also relates to a control method for an image sensor, the control method being applied to the aforementioned image sensor, the control method comprising:

[0123] The light signal passing through the first lens unit 111 and the first color filter unit 121 is photoelectrically converted to obtain a first electrical signal, and the first electrical signal is used to generate an image signal.

[0124] The light signal passing through the second lens unit 112 and the second color filter unit 122 is photoelectrically converted to obtain a second electrical signal, which is then used for focusing and / or spectral measurement.

[0125] Optionally, the signal acquisition method of pixel unit 131 specifically includes at least one of the following methods:

[0126] The first electrical signal and the second electrical signal are acquired based on the same frame;

[0127] The first and second electrical signals are read out either by combining them or by reading them separately.

[0128] The color filter array 12 also includes a third color filter unit 133, which corresponds to the control mode of the image sensor, including a first mode (Summing mode) and a second mode (Fullsize mode), wherein:

[0129] In the first mode, the image sensor outputs images in two frames. In the first frame, the first electrical signal for imaging is read out by the pixel unit 131 corresponding to each first color filter unit 121, and the second electrical signal for focusing is read out by the pixel unit 131 corresponding to each second color filter unit 122. The pixel unit 131 corresponding to each third color filter unit 123 does not read out any electrical signal. In the second frame, the third electrical signal for spectral measurement is read out by the pixel unit 131 corresponding to each third color filter unit 123.

[0130] In the second mode, each pixel unit 131 reads out an electrical signal individually, including an electrical signal for imaging, an electrical signal for focusing, and an electrical signal for spectral measurement.

[0131] Sixth Embodiment

[0132] This application also relates to an imaging device, including the image sensor of the above embodiments and the control method of the image sensor described above.

[0133] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. An image sensor, characterized in that, It includes a microlens array, a color filter array, and a photosensitive pixel array arranged sequentially along the incident light direction; The microlens array includes multiple first lens units and at least one second lens unit; The color filter array includes a plurality of first color filter units and at least one second color filter unit, wherein the transmittance of each first color filter unit is different from that of the second color filter unit; The first lens unit is configured correspondingly to the first color filter unit, and the second lens unit is configured correspondingly to the second color filter unit; The photosensitive pixel array includes a plurality of pixel units, wherein the first color filter unit covers at least one of the pixel units, and the second color filter unit covers at least one of the pixel units; The light signals passing through the first lens unit and the first color filter unit are used to form at least a first electrical signal, and the light signals passing through the second lens unit and the second color filter unit are used to form at least a second electrical signal, so as to form an image signal based on the first electrical signal and the second electrical signal.

2. The image sensor as described in claim 1, characterized in that, The first color filter unit covers at least one of the pixel units, and the second color filter unit covers at least two of the pixel units; the light signal passing through the first lens unit and the first color filter unit is used for imaging, and the light signal passing through the second lens unit and the second color filter unit is used for spectral measurement and / or focusing.

3. The image sensor as described in claim 2, characterized in that, The image sensor also includes a grid, which is configured to correspond to the color filter array and form a light-transmitting area. The light-transmitting area includes a plurality of first light-transmitting areas and at least one second light-transmitting area. The first color filter unit is configured to correspond to the first light-transmitting area, and the second color filter unit is configured to correspond to the second light-transmitting area.

4. The image sensor as described in claim 3, characterized in that, The projection of the second light-transmitting area covers at least two of the pixel units, and forms at least a first type of pixel unit group and a second type of pixel unit group, so as to at least realize the acquisition of the first phase information and the second phase information; And / or, the area between the pixel units corresponding to the second light-transmitting area is further provided with an overflow portion, located between adjacent pixel units.

5. The image sensor as described in claim 4, characterized in that, The projection of the first light-transmitting area covers one pixel unit, and the projection of the second light-transmitting area covers two or four pixel units; or, the projection of the first light-transmitting area covers four pixel units, and the projection of the second light-transmitting area covers four pixel units; or, the projection of the first light-transmitting area covers two pixel units, and the projection of the second light-transmitting area covers two pixel units.

6. The image sensor as described in claim 2, characterized in that, The microlens array includes at least two adjacent second lens units, and the color filter array includes at least two second color filter units with the same transmittance. The two second color filter units are arranged adjacent to each other, and each second lens unit is arranged corresponding to each second color filter unit. The light signals passing through each second lens unit and each second color filter unit are used for spectral measurement and / or focusing.

7. The image sensor as described in claim 6, characterized in that, The color filter array further includes a third color filter unit, which covers at least one pixel unit and has a transmittance different from that of each first color filter unit. The light signal passing through the third color filter unit is used for at least spectral measurement. A combination of adjacent second color filter units and the third color filter units surrounding the second color filter units is defined as a color filter unit group. The image sensor includes at least one of the color filter unit groups.

8. The image sensor as claimed in claim 7, characterized in that, The image sensor includes multiple color filter unit groups, each with a different transmittance; and / or, the microlens array includes a third lens unit, which is correspondingly arranged with the third color filter unit; and / or, the color filter array includes at least four third color filter units with different transmittances, each of which has a transmittance different from that of the first color filter unit, each of which covers at least one pixel unit, and the light signals passing through each of the second lens units and the second color filter units are used for focusing, and the light signals passing through each of the third color filter units are used for spectral measurement; or, two of the third color filter units are located on opposite sides of a second color filter unit, and the other two of the third color filter units are located on opposite sides of the second color filter unit.

9. The image sensor as claimed in claim 7, characterized in that, The image sensor outputs images in two modes: a first mode and a second mode. In the first mode, the image sensor is controlled to acquire different data images. The first data image corresponds to the first electrical signal read out by the pixel unit corresponding to each of the first color filter units and the second electrical signal read out by the pixel unit corresponding to each of the second color filter units, and the pixel unit corresponding to each of the third color filter units does not read out an electrical signal; the second data image corresponds to the third electrical signal read out by the pixel unit corresponding to each of the third color filter units. In the second mode, each pixel unit reads out an electrical signal individually, including an electrical signal for imaging, an electrical signal for focusing, and an electrical signal for spectral measurement.

10. The image sensor according to any one of claims 7-9, characterized in that, The pixel unit has a shared structure, and the adjacent second color filter unit corresponding pixel unit forms at least a first shared structure and a second shared structure with the adjacent third color filter unit corresponding pixel unit. The shared structure corresponds to four pixel units, including a central output element and floating diffusion nodes on both sides of the output element. The floating diffusion nodes are electrically connected to the output element.

11. The image sensor as claimed in claim 1, characterized in that, The color filter array includes multiple color units, each color unit comprising a first color subunit based on a first color filter unit and a second color subunit based on a second color filter unit or a combination of the second and first color filter units, wherein the first and second color subunits are arranged according to a preset period; or, the color filter array includes multiple color units, each color unit comprising the second and first color filter units, wherein the color units are arranged according to a preset period.

12. The image sensor as claimed in claim 11, characterized in that, The first color filter unit in the first color subunit includes an RGGB / RYYB Bayer array, and the second color subunit includes a first color filter unit and a second color filter unit. The first color filter unit includes an RG / RY color filter unit; or the first color filter unit includes a four-Bayer color filter unit based on RGG / RYY; or the first color filter unit includes a four-Bayer color filter unit based on RG / RY; or the first color filter unit includes an RGG / RYY color filter unit.

13. The image sensor as claimed in claim 1, characterized in that, The color filter array includes a plurality of second color filter units, each of which has a different transmittance; and / or, the pixel unit includes at least one photoelectric conversion element for converting light signals into electrical signals; and / or, the second color filter units are spaced apart between the first color filter units; and / or, the pixel unit density corresponding to the second color filter unit does not exceed 25%. And / or, the color filter array has at least two sub-color filter layers, with the first color filter unit and the second color filter unit located in different sub-color filter layers.

14. An imaging device, characterized in that, The image sensor included in any one of claims 1 to 13.

Images