Image sensor and electronic device

Abstract

The invention discloses an image sensor and electronic equipment, and belongs to the technical field of sensors. The image sensor comprises a pixel array and a color filter array, the pixel array comprises at least one multispectral unit, each multispectral unit comprises at least two first-class pixels used for image imaging and at least two second-class pixels used for multispectral imaging, each multispectral unit is composed of at least one multispectral element, and the multispectral elements correspond to four basic color channels. Each basic color channel corresponds to 16 pixels distributed in a 4 * 4 array in the pixel array, and each filtering element in the filtering unit corresponds to one pixel of the pixel array; the color filter array comprises at least one filter unit arranged opposite to the at least one multispectral unit, the filter unit comprises at least two first-class filter elements and at least two second-class filter elements, and the first-class filter elements are arranged opposite to the first-class pixels; the second type of filtering elements and the second type of pixels are arranged oppositely.

Description

Technical Field

[0001] This application belongs to the technical field of camera modules, and particularly relates to an image sensor and an electronic device. Background Art

[0002] Traditional red, green, and blue (RGB) imaging technology generates images by collecting red, green, and blue light signals of a target object in the visible light spectrum. However, this technology has limitations in certain scenarios because it can only provide limited spectral information and cannot achieve better imaging effects. To address this problem, multispectral imaging technology has emerged. It can collect light signals of a target object in different spectral bands, greatly improving the spectral resolution and spectral range of images. This technology can provide more information than RGB imaging technology, thereby achieving better imaging effects.

[0003] Currently, common multispectral sensors all require an additional camera to obtain light signals in more spectral bands, which increases the cost, power consumption, and structural performance requirements of the shooting device. Moreover, the field of view between the additional multispectral camera and the imaging camera is not naturally aligned, and subsequent algorithms need to be optimized. Summary of the Invention

[0004] The objective of the embodiments of this application is to provide an image sensor and an electronic device that can achieve better imaging effects without increasing the device configuration.

[0005] In a first aspect, the embodiments of this application provide an image sensor, which includes: a pixel array; the pixel array includes at least one multispectral unit, the multispectral unit includes at least two first-type pixels for image imaging and at least two second-type pixels for multispectral imaging, each multispectral unit is composed of at least one multispectral element, the multispectral element corresponds to 4 basic color channels, each basic color channel corresponds to 16 pixels distributed in a 4×4 array in the pixel array, and each filter element in the filter unit corresponds to one pixel in the pixel array; the color filter array includes at least one filter unit disposed opposite to the at least one multispectral unit, the filter unit includes at least two first-type filter elements and at least two second-type filter elements, the first-type filter elements are disposed opposite to the first-type pixels; the second-type filter elements are disposed opposite to the second-type pixels; wherein, the transmission wavelength range of the first-type filter elements is different from that of the second-type filter elements, the at least two second-type filter elements are composed of filter elements of at least one transmission wavelength range, and at least two second-type filter elements included in each transmission wavelength range of filter elements are distributed in an M×N array, where both M and N are integers greater than 0 and less than or equal to 2.

[0006] In a second aspect, an embodiment of the present application provides an electronic device, which includes an image sensor as described in the first aspect.

[0007] According to the image sensor of the embodiment of the present application, the image sensor includes: a pixel array; the pixel array includes at least one multispectral unit, the multispectral unit includes at least two first-type pixels for image imaging and at least two second-type pixels for multispectral imaging, each multispectral unit is composed of at least one multispectral element, the multispectral element corresponds to 4 basic color channels, each basic color channel corresponds to 16 pixels distributed in a 4×4 array in the pixel array, and each filter element in the filter unit corresponds to one pixel in the pixel array; the color filter array includes at least one filter unit disposed opposite to the at least one multispectral unit, the filter unit includes at least two first-type filter elements and at least two second-type filter elements, the first-type filter elements are disposed opposite to the first-type pixels; the second-type filter elements are disposed opposite to the second-type pixels; wherein, the transmission wavelength range of the first-type filter elements is different from the transmission wavelength range of the second-type filter elements, the at least two second-type filter elements are composed of filter elements of at least one transmission wavelength range, and at least two second-type filter elements included in each filter element of each transmission wavelength range are distributed in an M×N array, and both M and N are integers greater than 0 and less than or equal to 2. In this way, in the image sensor, the multispectral unit includes first-type pixels for image imaging and at least one second-type pixel for multispectral imaging, the filter unit includes first-type filter elements disposed opposite to the first-type pixels and second-type filter elements disposed opposite to the second-type pixels, the image sensor realizes image imaging through the color channels corresponding to the first-type filter elements, and, identifies multispectral information through the color channels corresponding to the second-type filter elements to realize multispectral imaging, so that the image sensor can take into account both the image imaging function and the multispectral imaging function, without setting an additional multispectral sensor to identify multispectral information, and achieves better imaging effects without increasing the device configuration. At the same time, the pixels of each basic color channel in each multispectral element are distributed in a 4×4 array, so that each multispectral element can include 64 pixels, which can cover more types of color information, thereby providing higher-resolution color data in the image processing process. In addition, the pixels of each basic color channel are distributed in a 4×4 array, which can increase the adjustable position space of the first-type pixels and the second-type pixels, so as to have a more flexible distribution method, and further achieve more accurate color reconstruction and high-resolution output in the image processing. Description of the Drawings

[0008] Figure 1 is a schematic structural diagram of an image sensor provided by some embodiments of the present application; Figure 2 It is a schematic arrangement diagram of multi - spectral filtering provided by some embodiments of the present application; Figure 3 It is a schematic arrangement diagram of a Bayer array provided by some embodiments of the present application; Figure 4 It is a schematic arrangement diagram of a multi - spectral element provided by some embodiments of the present application; Figure 5 It is a schematic structural diagram of an image sensor provided by some embodiments of the present application; Figure 6a It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 6b It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 6c It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 7a It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 7b It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 7c It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 7d It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 7e It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 8a It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 8b It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 8c It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 8d It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 8e It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 9a It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 9b It is a schematic arrangement diagram of a second - type filter element provided by some embodiments of the present application; Figure 9cIt is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 9d It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 9e It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10a It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10b It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10c It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10d It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10e It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10f It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10g It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10h It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10i It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10j It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10k It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 10l It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11a It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11b It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11c It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11d It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11eIt is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11f It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11g It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11h It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11i It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11j It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11k It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 11l It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 12a It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 12b It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 12c It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 12d It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 12e It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 12f It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 12g It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 12h It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 12i It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 13a It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 13b It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 13cIt is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 13d It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 14a It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 14b It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 14c It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 14d It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 15a It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 15b It is a schematic diagram of different transmission wavelength ranges provided by some embodiments of the present application; Figure 16 It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 17a It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 17b It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 17c It is a schematic diagram of the arrangement of the second type of filter element provided by some embodiments of the present application; Figure 18 It is a schematic diagram of the arrangement of the optical unit provided by some embodiments of the present application; Figure 19 It is a schematic diagram of the structure of the image sensor provided by some embodiments of the present application; Figure 20 It is a schematic diagram of the structure of the electronic device provided by some embodiments of the present application. Detailed implementation manners The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary only for explaining the present invention and should not be construed as limiting the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts belong to the scope of protection of the present application.

[0009] The terms "first" and "second" in the description and claims of this application may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more. In addition, "and / or" in the description and claims means at least one of the connected objects, and the character " / ", generally represents an "or" relationship between the associated objects before and after.

[0010] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0011] The terms used in the embodiments of this application are only used to explain the specific embodiments of this application, and are not intended to limit this application.

[0012] The following is an explanatory note on the terms involved in the embodiments of this application. Multispectral sensor: A multispectral sensor is a sensor that can simultaneously capture spectral information in multiple wavelength ranges. Its principle is to use optical filters or spectroscopic elements to separate and capture light from different wavelength ranges, so as to realize the acquisition and analysis of spectral information in multiple wavelengths. Traditional color cameras use three sensors to capture light in the red, green, and blue wavelength ranges respectively, while multispectral imaging sensors can capture a wider wavelength range, usually including visible light, infrared, and ultraviolet spectra. These sensors can use multiple narrowband filters or spectroscopic elements, such as prisms or gratings, to separate light of different wavelengths, and then transmit them to the corresponding pixel arrays respectively. By simultaneously capturing spectral information in multiple wavelength ranges, multispectral imaging sensors can provide richer spectral information, and thus can be applied to many fields, such as agriculture, environmental monitoring, geological exploration, etc.

[0013] Binocular stereo vision: Binocular stereo vision is an important form of machine vision. It is a method of obtaining three-dimensional geometric information of an object from multiple images based on the principle of parallax. A binocular stereo vision system generally uses two cameras to simultaneously obtain two digital images of the object to be measured from different angles, or a single camera to obtain two digital images of the object to be measured from different angles at different times, and restores the three-dimensional geometric information of the object based on the principle of parallax to reconstruct the three-dimensional contour and position of the object. Binocular stereo vision systems have broad application prospects in the field of machine vision.

[0014] Phase Detection Auto Focus (PDAF): PDAF achieves fast focusing by detecting the phase difference through dedicated pixels on the sensor. These pixels are arranged in pairs, capturing light from the left and right sides of the lens respectively to form two signals. When the object to be photographed is out of focus, the positions of the two signals on the sensor will shift. By calculating the offset, the out-of-focus direction and distance can be determined. Based on this, the system directly drives the lens motor to move to the in-focus position without repeated adjustment, significantly improving the focusing speed. Compared with traditional contrast focusing that needs to scan to find the point of maximum contrast, PDAF has significant advantages in dynamic tracking focus scenarios and is widely used in single-lens reflex cameras and smartphones. However, it is easily limited in low-light or low-contrast environments, so it is often combined with contrast focusing to form a hybrid system, taking into account both speed and accuracy.

[0015] Auto White Balance (AWB): AWB refers to the function of the camera to automatically correct the color cast of the image. Its core goal is to identify the color temperature of the light source by analyzing the color distribution of the scene, adjust the gain ratio of the red and blue channels of the sensor, offset the color cast, and make white objects still appear neutral white under different light sources, thus restoring the true color.

[0016] Pinned Photodiode Pixel (PPD) image sensor: The PPD image sensor is a pixel design widely used in CMOS image sensors. It improves the structure of the traditional floating diffusion region photodiode and solves problems such as dark current and full well capacity. As Figure 1 shown, it is a schematic diagram of the PPD image sensor, and the working method can include the following steps: 1. Exposure. The electron-hole pairs generated by light irradiation will be separated due to the existence of the PPD electric field. Electrons move to the n region, and holes move to the p region.

[0017] 2. Reset. At the end of exposure, activate RST to reset the readout region, that is, the n+ region, to a high level.

[0018] 3. Reset level readout. After resetting, read out the reset level, which includes the offset noise, 1 / f noise of the operational amplifier, and the kTC noise introduced by resetting, and store the read signal in the first capacitor.

[0019] 4. Charge transfer. Activate TX to completely transfer the charge from the photosensitive region to the n+ region for readout, that is, perform charge transfer.

[0020] 5. Signal level reading. Read the voltage signal of the n+ region into the second capacitor. The voltage signal includes: the signal generated by photoelectric conversion, the offset generated by the operational amplifier, 1 / f noise, and the kTC noise introduced by reset.

[0021] 6. Signal output. Subtract the signals stored in the two capacitors. For example, by using CDS, the main noise in the Pixel can be eliminated. The obtained signal is amplified analogously and then sampled by the ADC, and then the digital signal can be output.

[0022] Next, in conjunction with the accompanying drawings, the image sensor and the electronic device provided by the embodiments of the present application will be described in detail through specific embodiments and their application scenarios.

[0023] Figure 1 An image sensor provided by an embodiment of the present application is shown. Figure 2 A schematic diagram of a multispectral unit provided by an embodiment of the present application is shown. As Figure 1 and Figure 2 shown, the image sensor includes: a pixel array 100 and a color filter array 200; The pixel array 100 includes at least one multispectral unit 110. The multispectral unit 110 includes at least two first-type pixels 111 for image imaging and at least two second-type pixels 112 for multispectral imaging. Each multispectral unit 110 is composed of at least one multispectral element 120. The multispectral element 120 corresponds to 4 basic color channels. Each basic color channel 121 corresponds to 16 pixels distributed in a 4×4 array in the pixel array 100. Each filter element in the filter unit 210 corresponds to one pixel of the pixel array 100.

[0024] Among them, the first-type pixels 111 may include visible light pixels for image imaging. The visible light pixels are used to collect color information in the visible light range and may include other pixels such as R pixels, G pixels, GR pixels, GB pixels, B pixels, and W pixels. The second-type pixels 112 may include visible light pixels and non-visible light pixels for multispectral imaging. Among them, the non-visible light pixels include: other pixels such as infrared pixels and ultraviolet pixels.

[0025] Continue to refer to Figure 2, each of the multispectral units 110 is composed of at least one multispectral element 120, and the multispectral element 120 corresponds to the pixels of 4 basic color channels. The pixels of each basic color channel 121 are distributed in a 4×4 array. Among them, the multispectral element 120 refers to the smallest imaging group for imaging, and the value of the product of 4 and 4 refers to the total number of the same pixels in the multispectral element 120. Usually, at least one red filter element corresponding to R pixels, one green filter element corresponding to G pixels, and one blue filter element corresponding to B pixels are included in a multispectral element 120. Exemplarily, as Figure 3 shown, it is a schematic diagram corresponding to a 4×4 Bayer array. In the embodiment of the present application, 0 pixels or some pixels or all pixels in a multispectral element 120 can be replaced by second-type pixels. Exemplarily, as Figure 4 shown, it is a schematic diagram corresponding to a 4×4 multispectral element 120 provided in the embodiment of the present application. Among them, the solid circles or ellipses are used to represent that at least two second-type filter elements transmit the optical signals within the same transmission wavelength range, which will not be elaborated later. Two Gr pixels in the multispectral element 120 are replaced by second-type pixels 112 corresponding to the filter element F1, two Gr pixels are replaced by second-type pixels 112 corresponding to the filter element F2, two Gr pixels are replaced by second-type pixels 112 corresponding to the filter element F3, two Gr pixels are replaced by second-type pixels 112 corresponding to the filter element F4, two Gb pixels are replaced by second-type pixels 112 corresponding to the filter element F5, two Gb pixels are replaced by second-type pixels 112 corresponding to the filter element F6, two Gb pixels are replaced by second-type pixels 112 corresponding to the filter element F7, and two Gb pixels are replaced by second-type pixels 112 corresponding to the filter element F8. It can be understood that each basic color channel in the multispectral element 120 includes 4×4 = 16 pixels, and each multispectral element includes a total of 64 pixels. Exemplarily, continue to refer to Figure 2 , the first basic color channel is the channel corresponding to R pixels. In this channel, there are 16 R pixels in total; the second basic color channel is the channel corresponding to Gr pixels. This channel includes 8 Gr pixels and 8 second-type pixels. It should be noted that the multispectral element 120 in the embodiment of the present application refers to inserting 0 or more second-type pixels in the case of including the three primary colors of R, G, and B. Therefore, the basic color channels corresponding to the multispectral element 120 can be divided into R channels, Gr channels, Gb channels, and B channels. It should be noted that the Gr channel and the Gb channel belong to the G channel. In the example of the present application, the G channel is divided into the Gr channel and the Gb channel. When inserting the second-type pixels, it is inserted based on these 4 basic color channels. The third basic color channel is the channel corresponding to Gb pixels. This channel includes 8 Gb pixels and 8 second-type pixels; the fourth basic color channel is the channel corresponding to B pixels. This channel includes 16 B pixels.

[0026] The color filter array 200 includes at least one filter unit 210 disposed opposite to the at least one multispectral unit 100. The filter unit 210 includes at least two first-type filter elements 211 and at least two second-type filter elements 212. The first-type filter elements 211 are disposed opposite to the first-type pixels 111; the second-type filter elements 212 are disposed opposite to the second-type pixels 112.

[0027] It can be understood that the pixel array 100 may include at least one multispectral unit 110, the color filter array 200 may include at least one filter unit 210, one multispectral unit 110 corresponds to one filter unit 210, and the number of multispectral units 110 is the same as the number of filter units 210. Moreover, one pixel in one multispectral unit 110 corresponds to one filter element in one filter unit 210. As Figure 5 shown, the pixel array 100 and the color filter array 200 may be stacked. The pixel array 100 may include multiple pixels, and the color filter array 200 may include multiple filter elements. One pixel corresponds to one filter element. That is to say, each filter element can only capture an optical signal within a specific range, and each pixel stores an optical intensity value separately in each range.

[0028] In the embodiment of the present application, one multispectral unit 110 in the pixel array 100 is composed of a first-type pixel 111 for image imaging and a second-type pixel 112 for multispectral imaging. Correspondingly, one filter unit 210 in the color filter array 200 is composed of a first-type filter element 211 corresponding to the first-type pixel 111 and a second-type filter element 212 corresponding to the second-type pixel 112. In this way, the image sensor realizes image imaging through the color channel corresponding to the first-type filter element 211, and identifies multispectral information through the color channel corresponding to the second-type filter element 212 to realize multispectral imaging, so that the image sensor can take into account both the image imaging function and the multispectral imaging function, without setting an additional multispectral sensor to identify multispectral information, reducing the hardware power consumption.

[0029] Wherein, the transmission wavelength range of the first-type filter element 211 is different from the transmission wavelength range of the second-type filter element 212. The at least two second-type filter elements 212 are composed of filter elements with at least one transmission wavelength range. At least two second-type filter elements 212 included in each filter element with a transmission wavelength range are distributed in an M×N array, and both M and N are integers greater than 0 and less than or equal to 2. Exemplarily, as Figure 6a shown, it is an arrangement of 1×2. As Figure 6b shown, it is an arrangement of 2×1. AsFigure 6c As shown, it is a 2×2 arrangement.

[0030] In the embodiments of the present application, regarding the at least two second type of filter elements 212 being composed of filter elements of at least one transmission wavelength range, it can be understood that in order to construct a color image, an image sensor needs to capture optical signals from different wavelengths. Each filter element selects to transmit optical waves within a specific range of wavelengths. Although these wavelength ranges are different, by combining the optical signals of at least three wavelength ranges, the color of the image will be able to be reconstructed. Among them, the at least three include the two transmission wavelength ranges corresponding to at least two first type of filter elements and at least one transmission wavelength range corresponding to at least two second type of filter elements. The intensity of the light captured by each filter element can be regarded as part of the information of the color. No matter which three colors are used, for example, RGB, red-yellow-blue, etc., they can be combined into any other color, that is, the combination of the optical signals of the three wavelength ranges can give sufficient information to generate any color, and thus a complete color image can be reconstructed, so as to effectively capture a variety of colors or specific spectral bands, and further provide richer and more accurate information in subsequent image processing.

[0031] According to an embodiment of the present application, an image sensor includes: a pixel array; the pixel array includes at least one multispectral unit, the multispectral unit includes at least two first-type pixels for image imaging and at least two second-type pixels for multispectral imaging, each multispectral unit is composed of at least one multispectral element, the multispectral element corresponds to 4 basic color channels, each basic color channel corresponds to 16 pixels distributed in a 4×4 array in the pixel array, and each filter element in the filter unit corresponds to one pixel of the pixel array; the color filter array includes at least one filter unit disposed opposite to the at least one multispectral unit, the filter unit includes at least two first-type filter elements and at least two second-type filter elements, the first-type filter elements are disposed opposite to the first-type pixels; the second-type filter elements are disposed opposite to the second-type pixels; wherein, the transmission wavelength range of the first-type filter elements is different from the transmission wavelength range of the second-type filter elements, the at least two second-type filter elements are composed of filter elements of at least one transmission wavelength range, and at least two second-type filter elements included in each filter element of each transmission wavelength range are distributed in an M×N array, and both M and N are integers greater than 0 and less than or equal to 2. In this way, in the image sensor, the multispectral unit includes first-type pixels for image imaging and at least one second-type pixel for multispectral imaging, the filter unit includes first-type filter elements disposed opposite to the first-type pixels and second-type filter elements disposed opposite to the second-type pixels, the image sensor realizes image imaging through the color channels corresponding to the first-type filter elements, and, identifies multispectral information through the color channels corresponding to the second-type filter elements to realize multispectral imaging, so that the image sensor can take into account both the image imaging function and the multispectral imaging function, without setting an additional multispectral sensor to identify multispectral information, and achieves better imaging effects without increasing the device configuration. At the same time, the pixels of each basic color channel in each multispectral element are distributed in a 4×4 array, so that each multispectral element can include 64 pixels, which can cover more types of color information, thereby providing higher-resolution color data in the image processing process. In addition, the pixels of each basic color channel are distributed in a 4×4 array, which can increase the adjustable position space of the first-type pixels and the second-type pixels, so as to have a more flexible distribution method, and further achieve more accurate color reconstruction and high-resolution output in the image processing.

[0032] In some embodiments of the present application, M is 1 and N is 2; the multispectral unit 110 includes two second-type pixels distributed in an array, and the filter unit 210 includes two second-type filter elements distributed in an array, and the two second-type pixels and the two second-type filter elements are arranged in one-to-one correspondence.

[0033] Exemplarily, as Figure 7a shown, taking the multi-spectral unit 110 including a multi-spectral element 120 as an example, the multi-spectral unit 110 includes two second-type pixels distributed in an array, and the filter unit 210 also includes two second-type filter elements distributed in an array. The two second-type pixels and the two second-type filter elements are arranged in one-to-one correspondence. It should be noted that the two second-type pixels distributed in an array can be arbitrarily distributed in the multi-spectral element 120. Exemplarily, as Figure 7b 、 7c 、7d, and 7e shown. In addition, F1 shown in 7a, 7b, 7c, 7d, and 7e refers to the two second-type filter elements corresponding to the two second-type pixels, and will not be elaborated when it appears in subsequent drawings.

[0034] In some embodiments of the present application, M is 2 and N is 1; the multi-spectral unit 110 includes two second-type pixels distributed in an array, and the filter unit 210 includes two second-type filter elements distributed in an array. The two second-type pixels and the two second-type filter elements are arranged in one-to-one correspondence.

[0035] Exemplarily, as Figure 8a shown, taking the multi-spectral unit 110 including a multi-spectral element 120 as an example, the multi-spectral unit 110 includes two second-type pixels distributed in an array, and the filter unit 210 also includes two second-type filter elements distributed in an array. The two second-type pixels and the two second-type filter elements are arranged in one-to-one correspondence. It should be noted that the two second-type pixels distributed in an array can be arbitrarily distributed in the multi-spectral element 120. Exemplarily, as Figure 8b 、 8c 、8d, and 8e shown.

[0036] In some embodiments of the present application, M is 2 and N is 2; the multi-spectral unit 110 includes four second-type pixels distributed in an array, and the filter unit 210 includes four second-type filter elements distributed in an array. The four second-type pixels and the four second-type filter elements are arranged in one-to-one correspondence.

[0037] Exemplarily, as Figure 9a shown, taking the multi-spectral unit 110 including a multi-spectral element 120 as an example, the multi-spectral unit 110 includes two second-type pixels distributed in an Two second - type pixels distributed in an array, and the filter unit 210 also includes two second - type filter elements distributed in an array. The two second - type pixels and the two second - type filter elements are arranged in one - to - one correspondence. It should be noted that the two second - type pixels distributed in an array can be arbitrarily distributed in the multispectral element 120. Exemplarily, as shown in 、 、9d, 9e. Figure 9b 、 9c 、9d, 9e.

[0038] In some embodiments of the present application, each basic color channel corresponds to a 4×4 array; the multispectral element 120 includes two second - type pixels; the two second - type pixels are oppositely arranged with two second - type filter elements 212 in a transmission wavelength range; when the two second - type pixels correspond to the same basic color channel, the two second - type pixels are located at any two adjacent positions in the 4×4 array of the same basic color channel.

[0039] It can be understood that one multispectral element corresponds to 4 basic color channels, and each basic color channel corresponds to 16 pixels distributed in a 4×4 array. Therefore, when the two second - type pixels correspond to the same basic color channel, the two second - type pixels can be located at any two adjacent positions in the 4×4 array of the same basic color channel.

[0040] Exemplarily, when the multispectral unit 110 includes two second - type pixels distributed in an array, the two second - type pixels can have the distribution modes shown in Figure 10a 、 Figure 10b 、 Figure 10c 、 Figure 10d 、 Figure 10e 、 Figure 10f 、 Figure 10g 、 Figure 10h 、 Figure 10i 、 Figure 10j 、 Figure 10k 、 Figure 10l in the basic color channel. When the multispectral unit 110 includes two second - type pixels distributed in an array, the two second - type pixels can have the distribution modes shown in Figure 11a 、 Figure 11b 、 Figure 11c 、 Figure 11d 、 Figure 11e 、 Figure 11f 、 Figure 11g 、 Figure 11h 、 Figure 11i 、 Figure 11j 、 Figure 11k 、 Figure 11lThe distribution shown in []. It should be noted that in the above example, the R channel is used as an illustrative example, and the arrangement of other channels is similar to that of the R channel, which will not be elaborated here.

[0041] In some embodiments of the present application, each basic color channel corresponds to a 4×4 array; the multispectral element (120) includes four second-type pixels; the four second-type pixels are oppositely arranged with four second-type filter elements (212) of a transmission wavelength range; when the four second-type pixels correspond to the same basic color channel, the four second-type pixels are located at any four adjacent and centrosymmetric positions in the 4×4 array of the same basic color channel.

[0042] It can be understood that one multispectral element corresponds to 4 basic color channels, and each basic color channel corresponds to 16 pixels distributed in a 4×4 array. Therefore, when the four second-type pixels correspond to the same basic color channel, the four second-type pixels can be located at any four adjacent and centrosymmetric positions in the 4×4 array of the same basic color channel. Exemplarily, the four second-type pixels in this basic color channel can have Figure 12a , Figure 12b , Figure 12c , Figure 12d , Figure 12e , Figure 12f , Figure 12g , Figure 12h , Figure 12i The distribution shown in []. It should be noted that in the above example, the R channel is used as an illustrative example, and the arrangement of other channels is similar to that of the R channel, which will not be elaborated here.

[0043] In some embodiments of the present application, each basic color channel corresponds to a 4×4 array; the multispectral element 120 includes at least two second-type pixels; the at least two second-type pixels are oppositely arranged with at least two second-type filter elements 212 of a transmission wavelength range; when the at least two second-type pixels correspond to two adjacent basic color channels on the left and right, the at least two second-type pixels are symmetric about the y-axis and adjacent; the y-axis is the vertical symmetry axis of the 4 basic color channels in the multispectral element 120.

[0044] Exemplarily, taking the multispectral unit 110 including two second-type pixels distributed in an array as an example, as shown in Figure 13aAs shown, the second type of pixels with 8 transmission wavelength ranges correspond to two adjacent basic color channels on the left and right. Among them, the second type of pixels corresponding to the filter elements F1, F2, F3, and F4 correspond to the adjacent R channel and Gr channel on the left and right, and the second type of pixels corresponding to the filter elements F5, F6, F7, and F8 correspond to the adjacent Gb channel and B channel on the left and right. The second type of pixels with each transmission wavelength range are symmetric about the y-axis and adjacent to each other, where the adjacent distribution means that the two second type of pixels corresponding to each transmission wavelength range are adjacent to each other.

[0045] Taking the multi-spectral unit 110 including four second type of pixels distributed in an Figure 13b array as an example, as Figure 13c shown, the second type of pixels with 4 transmission wavelength ranges correspond to two adjacent basic color channels on the left and right. Among them, the second type of pixels corresponding to the filter elements F1 and F2 correspond to the adjacent R channel and Gr channel on the left and right, and the second type of pixels corresponding to the filter elements F3 and F4 correspond to the adjacent Gb channel and B channel on the left and right. The second type of pixels with each transmission wavelength range are symmetric about the y-axis and adjacent to each other. As Figure 13c 、 13d shown, the second type of pixels corresponding to the filter elements F1 and F2 are symmetric about the y-axis and adjacent to each other.

[0046] In some embodiments of the present application, each basic color channel corresponds to a 4×4 array; the multi-spectral element 120 includes at least two second type of pixels; the at least two second type of pixels are disposed opposite to at least two second type of filter elements 212 with one transmission wavelength range; when the at least two second type of pixels correspond to two adjacent basic color channels up and down, the at least two second type of pixels are symmetric about the x-axis and adjacent to each other; the x-axis is the horizontal symmetry axis of the 4 basic color channels in the multi-spectral element 120.

[0047] Exemplarily, taking the multi-spectral unit 110 including two second type of pixels distributed in an Figure 14a array as an example, as

[0048] shown, the second type of pixels with 8 transmission wavelength ranges correspond to two adjacent basic color channels up and down. Among them, the second type of pixels corresponding to the filter elements F1, F2, F3, and F4 correspond to the adjacent R channel and Gb channel up and down, and the second type of pixels corresponding to the filter elements F5, F6, F7, and F8 correspond to the adjacent Gr channel and B channel up and down. The second type of pixels with each transmission wavelength range are symmetric about the x-axis and adjacent to each other.

[0048] Taking the multi-spectral unit 110 including four second type of pixels distributed in an Figure 14bAs shown, the second type of pixels in the four transmission wavelength ranges correspond to two adjacent basic color channels above and below. Among them, the second type of pixels corresponding to the filter elements F1 and F2 correspond to the adjacent R channel and Gb channel above and below, and the second type of pixels corresponding to the filter elements F3 and F4 correspond to the adjacent Gr channel and B channel above and below. The second type of pixels in each transmission wavelength range are symmetric about the x-axis and are adjacent to each other. As Figure 14c , 14d shown, the second type of pixels corresponding to the filter elements F1 and F2 are symmetric about the x-axis and are adjacent to each other.

[0049] In some embodiments of the present application, the at least two second type of filter elements include at least one of the following filter elements: a first filter element, a second filter element, a third filter element, a fourth filter element, a fifth filter element, a sixth filter element, a seventh filter element, and an eighth filter element.

[0050] Exemplarily, as Figure 15a shown, in a filter unit 210, the at least two second type of filter elements may include a first filter element F1, a second filter element F2, a third filter element F3, a fourth filter element F4, a fifth filter element F5, a sixth filter element F6, a seventh filter element F7, and an eighth filter element F8. It should be noted that the arrangement of F1, F2, F3, F4, F5, F6, F7, and F8 in the figure is only a schematic arrangement, and F1, F2, F3, F4, F5, F6, F7, and F8 can be arranged arbitrarily in the filter unit 210.

[0051] In this way, in the embodiments of the present application, by providing at least one of the first filter element F1, the second filter element F2, the third filter element F3, the fourth filter element F4, the fifth filter element F5, the sixth filter element F6, the seventh filter element F7, and the eighth filter element F8 in the second type of filter element 212, the image sensor can identify at least one piece of multispectral information, thereby realizing the image adjustment function.

[0052] Moreover, when the second type of filter element 212 includes at least one of the first filter element F1, the second filter element F2, the third filter element F3, the fourth filter element F4, the fifth filter element F5, the sixth filter element F6, the seventh filter element F7, and the eighth filter element F8, there is at least partial overlap in the transmission wavelength ranges of at least two of the above eight filter elements. When performing image adjustment using the spectral feature information obtained based on the second type of filter element 212, the light wavelength range that can be transmitted by at least one of the filter elements in the second type of filter element 212 is used as source data for calculation and processing, and can represent a piece of spectral feature information. Then, the more types of filter elements in the second type of filter element 212, the richer the identified multispectral information, and the better the image adjustment function. ToFigure 15a For example, it includes 8 filter elements, which enables the image sensor to achieve the multi-spectral imaging function by identifying eight spectral information. Moreover, since the eight transmissible light wavelength ranges are relatively wide, the light input amount of the corresponding filter elements is more sufficient, and the amount of collected signals is larger.

[0053] Further, in another exemplary embodiment, the transmissible wavelength ranges of the 8 filter elements can be respectively: The transmissible wavelength range of the first filter element is 420 nm to 460 nm; The transmissible wavelength range of the second filter element is 670 nm to 700 nm; The transmissible wavelength range of the third filter element is 520 nm to 610 nm; The transmissible wavelength range of the fourth filter element is 610 nm to 700 nm; The transmissible wavelength range of the fifth filter element is 430 nm to 510 nm; The transmissible wavelength range of the sixth filter element is 450 nm to 590 nm; The transmissible wavelength range of the seventh filter element is 400 nm to 440 nm; The transmissible wavelength range of the eighth filter element is 560 nm to 700 nm.

[0054] Exemplarily, as Figure 15b shown, it is a schematic diagram of the 8 transmissible wavelength ranges.

[0055] Among them, in the 8 transmissible wavelength ranges, each transmissible wavelength range partially overlaps with at least one other transmissible wavelength range.

[0056] It should be noted that the embodiments of the present application do not limit the types of the transmissible wavelength ranges corresponding to the second filter element. The above 8 transmissible wavelength ranges are only illustrative examples, and other transmissible wavelength ranges may also be included. The specific range of each transmissible wavelength range can also be determined according to the actual situation, and the embodiments of the present application do not make specific limitations.

[0057] In this way, when performing multi-spectral imaging using the spectral feature information obtained based on at least one of the above filter elements, at least two corresponding pixel data can be used as source data for calculation and processing, and can represent a kind of spectral feature information. In the embodiments of the present application, each transmissible wavelength range is relatively wide, and the transmissible wavelength ranges of two or more filter elements partially overlap, so that the light input amount of the corresponding filter elements is more sufficient, and the amount of collected signals is larger.

[0058] In some embodiments of the present application, the at least two second - type filter elements 212 include at least one of the following: cyan filter element, yellow filter element, magenta filter element, purple filter element, orange filter element, and all - pass filter element.

[0059] Among them, the second - type filter elements 212 may include filter elements with different shades. For example, the second - type filter elements 212 may include deep - yellow filter elements, or light - yellow filter elements, etc. Embodiments of the present application do not make specific settings on the shade of the color light that the second - type filter elements 212 can transmit.

[0060] Among them, the second - type filter elements 212 may also include filter elements with different transitional colors. For example, the second - type filter elements 212 may include yellow - green filter elements or cyan - blue filter elements, etc. Embodiments of the present application do not make specific settings on the specific color of the color light that the second - type filter elements 212 can transmit.

[0061] Among them, when performing image adjustment using the spectral feature information obtained based on the second - type filter elements 212, each of the cyan filter element C, yellow filter element Y, magenta filter element M, purple filter element P, orange filter element O, and all - pass filter element W can represent a kind of spectral feature information.

[0062] In this way, embodiments of the present application can enable the image sensor to realize the image adjustment function by identifying at least one piece of multispectral information by setting at least one of the cyan filter element C, yellow filter element Y, magenta filter element M, purple filter element P, orange filter element O, and all - pass filter element W in the second - type filter elements 212. And this at least one piece of multispectral information can be directly used as spectral feature information for image adjustment, simplifying the image adjustment process.

[0063] Further, in another exemplary embodiment, the transmission wavelength range of the cyan filter element is the first transmission wavelength range; the transmission wavelength range of the yellow filter element is the second transmission wavelength range; the transmission wavelength range of the magenta filter element is the third transmission wavelength range; the transmission wavelength range of the purple filter element is the fourth transmission wavelength range; the transmission wavelength range of the orange filter element is the fifth transmission wavelength range. Among them, the first transmission wavelength range, the second transmission wavelength range, the third transmission wavelength range, the fourth transmission wavelength range, and the fifth transmission wavelength range partially overlap or do not overlap. It can be understood that among the five transmission wavelength ranges, there are at least two transmission wavelength ranges that do not overlap.

[0064] Further, in yet another exemplary example, the first transmission wavelength range is 490 nanometers to 515 nanometers; the second transmission wavelength range is 580 nanometers to 595 nanometers; the third transmission wavelength range is 390 nanometers to 700 nanometers; the fourth transmission wavelength range is 380 nanometers to 410 nanometers; the fifth transmission wavelength range is 595 nanometers to 625 nanometers.

[0065] Exemplarily, as Figure 16 shown, the second type of filter element 212 may include a cyan filter element C, a yellow filter element Y, a magenta filter element M, a purple filter element P, an orange filter element O, and an all - pass filter element W.

[0066] Among them, the light wavelength range that the cyan filter element C can transmit is from 490 nanometers to 515 nanometers; Among them, the light wavelength range that the yellow filter element Y can transmit is from 580 nanometers to 595 nanometers; Among them, the light wavelength range that the magenta filter element M can transmit is a target range, and the target range is determined based on the red light wavelength range and the blue light wavelength range; in practical applications, the light wavelength range that the magenta filter element M can transmit can be the red light wavelength range and the blue light wavelength range mixed in a preset ratio. In some embodiments, the magenta filter element M is 390 nanometers to 700 nanometers; Among them, the light wavelength range that the purple filter element P can transmit is in the 380 - nanometer to 410 - nanometer band; the light wavelength range that the orange filter element O can transmit is from 595 nanometers to 625 nanometers; Among them, the light wavelength range that the all - pass filter element W can transmit is the full - range light (white light) band.

[0067] In this way, by providing the cyan filter element C, the yellow filter element Y, the magenta filter element M, the purple filter element P, the orange filter element O, and the all - pass filter element W in the second - type filter element 212 in the embodiments of the present application, the image sensor can identify six spectral information to achieve the image adjustment function. Moreover, the six spectral information can be directly used as six spectral feature information for image adjustment, simplifying the image adjustment process.

[0068] It should be noted that the embodiments of the present application do not limit the types of the transmission wavelength ranges corresponding to the second filter element. The above - mentioned 6 transmission wavelength ranges are only illustrative examples, and other transmission wavelength ranges may also be included. The specific range of each transmission wavelength range can also be determined according to actual situations, and the embodiments of the present application do not make specific limitations.

[0069] In some embodiments of the present application, the second type of filter element may be a broadband filter element. Among them, the broadband filter can allow light in a relatively wide spectral range to pass through, and cover light of multiple wavelengths, such as covering a wide range of the main colors such as red, green, and blue. That is to say, this broadband filter is used to capture color information, and light of the same color will be captured through a relatively wide wavelength range. For example, when the transmission wavelength range of the broadband filter is 620 nm - 750 nm, multiple red, orange, and yellow lights will be allowed to pass through; when the transmission wavelength range of the broadband filter is 495 nm - 570 nm, green light and some yellow light will be allowed to pass through; when the transmission wavelength range of the broadband filter is 450 nm - 495 nm, blue light and violet light will be allowed to pass through. Therefore, when using the broadband filter, each channel will receive a relatively wide spectral range, and the image sensor can calculate the final color based on these different channel signals. Since the captured light information contains a relatively wide range of color information, the color of the generated image is determined by combining based on the light intensity of these wide ranges. That is to say, since the captured color information is relatively broad and contains light of multiple similar colors, the color of the final image will show a "mixing" effect.

[0070] In some embodiments of the present application, the second type of filter element may be a narrowband filter element. Among them, the narrowband filter only allows specific colors or more precise wavelengths to pass through. For example, a red narrowband filter element may only allow light with a wavelength range of 650 nm - 660 nm to pass through, that is, this filter element can only capture red light in a very narrow range; a green narrowband filter element may only allow light with a wavelength range of 500 nm - 510 nm to pass through, that is, this filter element can only capture green light in a very narrow range; a blue narrowband filter element may only allow light with a wavelength range of 450 nm - 460 nm to pass through, that is, this filter element can only capture blue light in a very narrow range. Therefore, when using the narrowband filter element, the light signal received by the corresponding pixel is very precise and basically represents only one specific color. When calculating the color of each pixel subsequently, because the wavelength range of each filter element is very narrow, the signals of each color channel are relatively independent, and the color calculation process will be more precise.

[0071] In some embodiments of the present application, in one of the filter units corresponding to one of the multispectral units, the value range of the proportion of the number of the second type of filter elements is 1% - 100%; where the proportion of the number of the second type of filter elements is the ratio of the number of the second type of filter elements to the total number of filter elements in the filter unit.

[0072] Among them, the percentage range of the number of the second type of filter elements 212 in the total number of all filter elements included in the filter unit 210 can be understood as the multispectral channel density range.

[0073] It is understandable that the pixel array 100 includes at least one multispectral unit 110, and each multispectral unit 110 is composed of at least one multispectral element 120. Each multispectral element 120 includes 64 pixels. In the embodiments of the present application, among the filter units corresponding to each multispectral unit 110, the second type of filter element 212 can be arranged arbitrarily in the filter unit. That is to say, the arrangement of the second type of filter element 212 is not realized based on the multispectral element 120 in the multispectral unit 110, but is realized based on the filter unit 210 corresponding to the multispectral unit 110. In this way, the implementation complexity can be avoided, that is, the second type of pixel 112 is not embedded for each multispectral element 120, the flexibility and diversity of the arrangement of the second type of filter element 212 corresponding to the second type of pixel 112 can be improved, and thus diverse spectral acquisition and image processing can be realized, thereby improving the imaging quality and analysis accuracy.

[0074] In some embodiments of the present application, the multispectral channel density can be 1%.

[0075] Among them, taking the multispectral channel density of 1% as an example, 256×1% = 2.56, and 3 second type of filter elements can be arranged in one filter unit 210.

[0076] In some embodiments of the present application, the multispectral channel density can be 6%.

[0077] Among them, taking the multispectral channel density of 6% as an example, 256×6% = 15.36, and 16 second type of filter elements can be arranged in one filter unit 210.

[0078] In some embodiments of the present application, the multispectral channel density can be 7%.

[0079] Among them, taking the multispectral channel density of 7% as an example, 256×7% = 17.92, and 18 second type of filter elements can be arranged in one filter unit 210.

[0080] In some embodiments of the present application, the multispectral channel density can be 50%.

[0081] Among them, taking the multispectral channel density of 50% as an example, 256×50% = 128, and 128 second type of filter elements can be arranged in one filter unit 210.

[0082] In some embodiments of the present application, the multispectral channel density can be 60%.

[0083] Among them, taking the multispectral channel density of 60% as an example, 256×60% = 153.6, and 154 second-type filter elements can be set in one filter unit 210.

[0084] In some embodiments of the present application, the multispectral channel density can be 100%.

[0085] Among them, taking the multispectral channel density of 100% as an example, 256×100% = 256, and 256 second-type filter elements can be set in one filter unit 210.

[0086] Exemplarily, as Figure 17a shown, in one filter unit 210, the percentage of the number of second-type filter elements 212 in the total number of all filter elements included in the filter unit 210 is 6%. Furthermore, at a 6% multispectral channel density, the image sensor can better balance the image imaging function and the multispectral imaging function. If more multispectral pixel numbers are actually needed, 2 second-type filter elements 212 can be added in the Figure 17a shown filter unit 210, and the arrangement of the second-type filter elements is as Figure 17b shown. Embedding the second-type filter elements 212 at a preset ratio of 7%, the number of second-type filter elements 212 is 256×7% = 18, that is, the corresponding multispectral unit of the filter unit includes 18 second-type pixels. If more multispectral pixel numbers are not actually needed and more RGB pixels are wanted to obtain a better image effect, 2 second-type filter elements 212 can be removed in the Figure 17a shown filter unit 210, and the arrangement of the second-type filter elements is as Figure 17c shown. Embedding the second-type filter elements 212 at a preset ratio of 5.5%, the number of second-type filter elements 212 is 256×5.5% = 14, that is, the corresponding multispectral unit of the filter unit includes 14 second-type pixels.

[0087] It should be noted that the preset ratio can be determined according to the actual situation, and the embodiments of the present application do not make specific limitations. At the same time, Figure 17a 、 Figure 17b 、 Figure 17c The layout manner of the second-type filter elements in

[0088] is only an example, and the embodiments of the present application do not make specific limitations.

[0089] In some embodiments of the present application, with continued reference to Figure 5 , the image sensor further includes an optical array 300, the optical array 300 includes a plurality of optical elements 310, and each optical element 310 covers of the filter elements. Among them, the optical element is used for condensing light, and n is an integer greater than or equal to 0.

[0090] Exemplarily, as Figure 18 shown, in this basic filter unit, one optical element 310 covers two adjacent first-type filter elements 211, one optical element 310 covers four adjacent first-type filter elements 211, one optical element 310 covers two adjacent second-type filter elements 212, and one optical element 310 covers four adjacent second-type filter elements 212. Among them, the optical unit is represented by a dotted circle or ellipse. The optical unit can be an on-chip lens, the on-chip lens can be arranged above the filter element, and the on-chip lens can be a microlens.

[0091] Among them, as Figure 19 shown, the lens array is used to converge the incident light, and make the converged incident light pass through the filter array and focus on the pixel array, reducing the mutual interference between the light scattering and pixels, so that each pixel unit can capture the optical signal more accurately, improving the clarity and fineness of the image.

[0092] In some embodiments of the present application, the shape of the pixels included in each multi-spectral unit is any one of the following: circular, square, triangular, and hexagonal. In this embodiment, by setting the shape of the pixels, the original color and detail information of the image can be better restored from different perspectives.

[0093] Figure 20 is a schematic block diagram of an electronic device provided according to some embodiments of the present application. As Figure 20 shown, the electronic device 2000 in this figure includes an image sensor 2010, and the image sensor 2010 is Figures 1 - 19 any one of the image sensors in the embodiments shown above. The electronic device can implement the various processes of the corresponding embodiments of the above image sensor and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0094] It should be noted that, in this document, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but may also include performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0095] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0096] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them fall within the protection scope of the present application.

Claims

1. An image sensor, characterized in that: include: Pixel array and color filter array; The pixel array includes at least one multi-spectral unit, the multi-spectral unit includes at least two first-type pixels for image imaging and at least two second-type pixels for multi-spectral imaging, each multi-spectral unit is composed of at least one multi-spectral element, the multi-spectral element corresponds to 4 basic color channels, each basic color channel corresponds to 16 pixels distributed in a 4×4 array in the pixel array, and each filter element in the filter unit corresponds to one pixel of the pixel array; The color filter array comprises at least one filter unit arranged opposite to the at least one multi-spectral unit, the filter unit comprises at least two first-type filter elements and at least two second-type filter elements, the first-type filter elements are arranged opposite to the first-type pixels; the second-type filter elements are arranged opposite to the second-type pixels; The transmission wavelength range of the first type of filter element is different from the transmission wavelength range of the second type of filter element, the at least two second type of filter elements are composed of filter elements with at least one transmission wavelength range, and the at least two second type of filter elements included in each transmission wavelength range are distributed in an M×N array, and M and N are both integers greater than 0 and less than or equal to 2.

2. The image sensor according to claim 1, characterized in that M is 1, N is 2; The multi-spectral unit includes The two second type pixels are arranged in an array, and the filter unit includes The two second-type filter elements are distributed in an array, and the two second-type pixels and the two second-type filter elements are arranged in a one-to-one correspondence.

3. The image sensor according to claim 1, characterized in that M is 2, N is 1; The multi-spectral unit includes The two second type pixels are arranged in an array, and the filter unit includes The two second-type filter elements are distributed in an array, and the two second-type pixels and the two second-type filter elements are arranged in a one-to-one correspondence.

4. The image sensor according to claim 1, characterized in that M is 2, N is 2; The multi-spectral unit includes The four second type pixels are arranged in an array, and the filter unit includes The four second-type filter elements are distributed in an array, and the four second-type pixels and the four second-type filter elements are arranged in a one-to-one correspondence.

5. The image sensor according to claim 2 or 3, characterized in that: Each basic color channel corresponds to a 4×4 array; The multi-spectral element comprises two second-type pixels; the two second-type pixels are arranged opposite to two second-type filter elements of a transmission wavelength range; In the case where the two second-type pixels correspond to the same basic color channel, the two second-type pixels are located at any two adjacent positions in the 4×4 array of the same basic color channel.

6. The image sensor according to claim 4, characterized in that Each basic color channel corresponds to a 4×4 array; The multi-spectral element comprises four second-type pixels; the four second-type pixels are arranged opposite to four second-type filter elements of a transmission wavelength range; In the case where the four second-type pixels correspond to the same basic color channel, the four second-type pixels are located at any four adjacent and center-symmetrical positions in a 4×4 array of the same basic color channel.

7. The image sensor according to claim 2 or 4, characterized in that: Each basic color channel corresponds to a 4×4 array; The multi-spectral element comprises at least two second-type pixels; the at least two second-type pixels are arranged opposite to at least two second-type filter elements of a transmission wavelength range; In the case where the at least two second-type pixels correspond to two adjacent basic color channels, the at least two second-type pixels are symmetrically and adjacently distributed along the y-axis; The y-axis is the vertical symmetry axis of the four basic color channels in the multi-spectral element.

8. The image sensor according to claim 3 or 4, characterized in that: Each basic color channel corresponds to a 4×4 array; The multi-spectral element comprises at least two second-type pixels; the at least two second-type pixels are arranged opposite to at least two second-type filter elements of a transmission wavelength range; In a case where the at least two second-type pixels correspond to two upper and lower adjacent basic color channels, the at least two second-type pixels are symmetrically and adjacently distributed along the x-axis; The x-axis is a transverse symmetry axis of the four basic color channels in the multi-spectral element.

9. The image sensor according to claim 1, characterized in that: The at least two second-type filter elements include at least one of the following filter elements: a first filter element, a second filter element, a third filter element, a fourth filter element, a fifth filter element, a sixth filter element, a seventh filter element, and an eighth filter element.

10. The image sensor according to claim 9, characterized in that: The transmission wavelength range of the first filter element is 420 nanometers to 460 nanometers; The transmission wavelength range of the second filter element is 670 nanometers to 700 nanometers; The transmission wavelength range of the third filter element is 520 nanometers to 610 nanometers; The transmission wavelength range of the fourth filter element is 610 nanometers to 700 nanometers; The transmission wavelength range of the fifth filter element is 430 nanometers to 510 nanometers; The transmission wavelength range of the sixth filter element is 450 nanometers to 590 nanometers; The transmission wavelength range of the seventh filter element is 400 nanometers to 440 nanometers; The transmission wavelength range of the eighth filter element is 560 nanometers to 700 nanometers.

11. The image sensor according to claim 1, characterized in that: The at least two second-type filter elements include at least one of the following: a cyan filter element, a yellow filter element, a magenta filter element, a purple filter element, an orange filter element, and an all-pass filter element.

12. The image sensor according to claim 11, characterized in that The transmission wavelength range of the cyan filter element is a first transmission wavelength range; the transmission wavelength range of the yellow filter element is a second transmission wavelength range; the transmission wavelength range of the magenta filter element is a third transmission wavelength range; the transmission wavelength range of the purple filter element is a fourth transmission wavelength range; the transmission wavelength range of the orange filter element is a fifth transmission wavelength range, The first transmission wavelength range, the second transmission wavelength range, the third transmission wavelength range, the fourth transmission wavelength range and the fifth transmission wavelength range partially overlap or do not overlap.

13. The image sensor according to claim 12, characterized in that: The first transmission wavelength range is 490 nanometers to 515 nanometers; The second transmission wavelength range is 580 nanometers to 595 nanometers; The third transmission wavelength range is 390 nanometers to 700 nanometers; The fourth transmission wavelength range is 380 nanometers to 410 nanometers; The fifth transmission wavelength range is 595 nanometers to 625 nanometers.

14. The image sensor according to claim 1, characterized in that: In one of the filter units corresponding to one of the multi-spectral units, the proportion of the second type of filter elements ranges from 1% to 100%; The proportion of the number of the second type of filter elements is the ratio of the number of the second type of filter elements to the total number of filter elements in the filter unit.

15. The image sensor according to claim 14, characterized in that: In one of the filter units corresponding to one of the multi-spectral units, the proportion of the second type of filter elements ranges from 1% to 50%.

16. The image sensor according to claim 14, characterized in that In one of the filter units corresponding to one of the multi-spectral units, the proportion of the second type of filter elements is 6%.

17. The image sensor according to claim 1, characterized in that: The image sensor further comprises an optical array, wherein the optical array comprises a plurality of optical elements, each of which covers The optical element is used for focusing light, and n is an integer greater than or equal to 0.

18. An electronic device, characterized in that: The invention comprises the image sensor according to any one of claims 1 to 17.

Images