Pixel array, image sensor, imaging system and image capturing method

Abstract

Embodiments of the present application disclose a pixel array, an image sensor, an imaging system and an image acquisition method. The pixel array comprises a photoelectric conversion subarray and an optical subarray arranged on the photoelectric conversion subarray. The optical subarray comprises a plurality of symmetric optical elements and a plurality of asymmetric optical elements arranged in a scattered staggered manner. Each symmetric optical element is arranged on a corresponding plurality of photoelectric converters to form a phase detection autofocus pixel arranged symmetrically. Each asymmetric optical element is arranged on a corresponding plurality of photoelectric converters to form an imaging pixel arranged asymmetrically. The asymmetric optical elements and the symmetric optical elements can replace microlenses and filters in existing pixels, so that the incident light can be detected in brightness and color on the basis of realizing the phase detection autofocus function, and various disadvantages caused by the microlenses and the filters in the pixels can be effectively avoided.

Description

Technical Field

[0001] The present application relates to the field of optoelectronic technology, and in particular to a pixel array, an image sensor, an imaging system, and an image acquisition method. Background Art

[0002] In a traditional image sensor, each pixel consists of at least a microlens, a filter, and a photoelectric converter. The microlens focuses light, ensuring it effectively passes through the filter and strikes the photoelectric converter, thereby improving the converter's light collection efficiency. The photoelectric converter then generates an electric charge or current based on the incident light. To produce color images, each pixel is equipped with a red, green, or blue filter. These filters transmit the corresponding color band and block the rest of the visible spectrum.

[0003] However, traditional image sensors have many defects: low white light broadband sensitivity and low spectral resolution; microlenses and filters need to be complexly and precisely shifted relative to the photoelectric converter, resulting in high debugging and setting costs and low efficiency; and microlenses and filters made of organic materials have more process flows and lower yields. Summary of the Invention

[0004] The embodiments of the present application provide a pixel array, an image sensor, an imaging system, an image acquisition method, an electronic device, a computer-readable storage medium, and a computer program product. The microlenses and filters in existing pixels can be replaced by asymmetric optical elements and symmetric optical elements. On the basis of realizing the phase detection autofocus function, the brightness detection and color detection of the incident light can also be realized, which can effectively avoid various disadvantages caused by the microlenses and filters in the pixels, such as low shift debugging efficiency of the microlenses and filters, low production yield, and high cost.

[0005] In a first aspect, an embodiment of the present application provides a pixel array, comprising:

[0006] A photoelectric conversion subarray, comprising a plurality of photoelectric converters arranged in an array;

[0007] an optical subarray, arranged on the photoelectric conversion subarray, the optical subarray comprising a plurality of symmetrical optical elements and a plurality of asymmetrical optical elements that are dispersed and staggered;

[0008] Each of the symmetrical optical elements is disposed on a corresponding plurality of the photoelectric converters to form symmetrically arranged phase detection autofocus pixels;

[0009] Each of the asymmetric optical elements is arranged on a corresponding plurality of the photoelectric converters to form asymmetrically arranged imaging pixels. The asymmetric optical element is used to focus and diffract the incident light to obtain a three-dimensional asymmetric light intensity pattern, which is recorded one-to-one on the corresponding plurality of the photoelectric converters to form a plurality of different spatial signals.

[0010] In a second aspect, an embodiment of the present application provides an image sensor, comprising the above-mentioned pixel array, and configured as follows:

[0011] Detecting the phase difference or focus state of the incident light in multiple directions by the phase detection autofocus pixel, and adjusting the relative position between the incident light and the image sensor based on the detection results in multiple directions to achieve focus; and

[0012] The incident light is collected by the imaging pixels, and within each imaging pixel, the incident light is resolved into multiple spectral components to achieve color detection of the incident light.

[0013] In a third aspect, an embodiment of the present application further provides an imaging system, comprising:

[0014] Imaging lens, used to converge light and obtain incident light;

[0015] An image sensor comprising the above-mentioned pixel array;

[0016] In which, the image sensor is configured to: detect the phase difference or focusing state of the incident light in multiple directions through the phase detection autofocus pixel, and adjust the relative position between the imaging lens and the image sensor based on the detection results in multiple directions to achieve focusing; and collect the incident light through the imaging pixel, and within each of the imaging pixels, resolve the incident light into multiple spectral components to achieve color detection of the incident light.

[0017] In a fourth aspect, an embodiment of the present application further provides an image acquisition method, which is applied to the above-mentioned imaging system, and the method includes:

[0018] The incident light is collected by the phase detection autofocus pixel to obtain a corresponding plurality of phase detection spatial signals, wherein the incident light is formed by focusing light by the imaging lens;

[0019] Detecting the phase difference or focus state of the incident light in multiple directions based on the multiple phase detection spatial signals, and adjusting the relative position between the imaging lens and the image sensor in combination with the detection results in the multiple directions to achieve focusing;

[0020] The incident light is collected by imaging pixels to obtain corresponding multiple spatial signals;

[0021] For each of the imaging pixels, based on the plurality of spatial signals and the plurality of calibration color bands, the incident light is resolved into spectral components corresponding to the plurality of calibration color bands;

[0022] For each phase detection autofocus pixel, interpolation is performed based on spectral components corresponding to a number of imaging pixels adjacent to the phase detection autofocus pixel to determine spectral components corresponding to the phase detection autofocus pixel.

[0023] In a fifth aspect, an embodiment of the present application further provides an electronic device comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any one of the image acquisition methods provided in the embodiments of the present application.

[0024] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, comprising a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any one of the image acquisition methods provided in the embodiments of the present application.

[0025] In the seventh aspect, an embodiment of the present application also provides a computer program product, including a computer program, which is stored in a computer-readable storage medium; when the processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of any one of the image acquisition methods provided in the embodiments of the present application.

[0026] In an embodiment of the present application, the pixel array includes a photoelectric conversion subarray and an optical subarray arranged on the photoelectric conversion subarray. The optical subarray includes a plurality of symmetrical optical elements and a plurality of asymmetric optical elements that are dispersed and staggered. Each symmetrical optical element is arranged on a corresponding plurality of photoelectric converters to form a symmetrically arranged phase detection autofocus pixel. Each asymmetric optical element is arranged on a corresponding plurality of photoelectric converters to form an asymmetric imaging pixel. Asymmetric optical elements and symmetrical optical elements can replace the microlenses and filters in existing pixels. On the basis of realizing the phase detection autofocus function, brightness detection and color detection of the incident light can also be realized, which can effectively avoid various disadvantages caused by the microlenses and filters in the pixels. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 This is a schematic diagram of the pixel structure in a traditional image sensor;

[0029] Figure 2 It is a schematic diagram of the structure of a traditional RGGB image sensor;

[0030] Figure 3 This is a schematic diagram comparing the quantum efficiency (QE) curves of monochrome image sensors and traditional RGGB image sensors in the visible spectrum range;

[0031] Figure 4 It is a structural diagram of an imaging system in the prior art;

[0032] Figure 5 is a graph showing the distribution of the principal ray angle of an imaging lens in an imaging system in the prior art;

[0033] Figure 6 This is a schematic diagram of phase detection autofocus for a traditional image sensor;

[0034] Figure 7 This is a schematic diagram of the process flow of a traditional RGGB image sensor;

[0035] Figure 8 This is a schematic structural diagram of a pixel array in an optional embodiment of the present application;

[0036] Figure 9 is a schematic diagram of phase detection autofocus of a pixel array in an optional embodiment of the present application;

[0037] Figure 10 This is a schematic diagram of a spatial signal collected during phase detection autofocusing of a pixel array in an optional embodiment of the present application;

[0038] Figure 11 is a three-dimensional view of an imaging pixel in an optional embodiment of the present application;

[0039] Figure 12 is a schematic diagram of asymmetric diffraction of different incident light by an asymmetric optical element in an optional embodiment of the present application;

[0040] Figure 13 is a schematic diagram of four calibration ribbons selected in an optional embodiment of the present application;

[0041] Figure 14 1 is a schematic diagram of a calibration process for a pixel array in an optional embodiment of the present application;

[0042] Figure 15 This is a structural diagram of a pixel array in an optional embodiment of the present application;

[0043] Figure 16 is another structural diagram of a pixel array in an optional embodiment of the present application;

[0044] Figure 17 is another structural diagram of a pixel array in an optional embodiment of the present application;

[0045] Figure 18 is another structural diagram of a pixel array in an optional embodiment of the present application;

[0046] Figure 19 is another structural diagram of a pixel array in an optional embodiment of the present application;

[0047] Figure 20 is a flowchart of an image sensor operating method in an optional embodiment of the present application;

[0048] Figure 21 This is a schematic structural diagram of an imaging system in an optional embodiment of the present application;

[0049] Figure 22 This is a graph showing the distribution of the principal ray angles of an imaging lens of an imaging system in an optional embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0051] At the same time, in the description of the embodiments of this application, the terms "first", "second", etc. are used only to distinguish the description and should not be understood as indicating or implying relative importance. Therefore, the features defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of the embodiments of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0052] Research has shown that traditional image sensors have low broadband sensitivity to white light and low spectral resolution. Microlenses and filters require complex and precise shifting relative to photoelectric converters, resulting in high debugging and setup costs and low efficiency. Microlenses and filters made of organic materials have more process flows and lower yields.

[0053] First, if Figure 1 As shown in the figure, in a traditional image sensor, each pixel includes at least a microlens, a filter, and a photoelectric converter (such as a photodiode). The microlens is used to converge light to ensure that the light can effectively pass through the filter and illuminate the photoelectric converter, thereby improving the light collection efficiency of the photoelectric converter. The photoelectric converter then generates an electric charge or current based on the incident light. In order to produce a color image, each pixel is equipped with one of a red filter, a green filter, or a blue filter, which transmits the corresponding color band and blocks the rest of the light in the visible spectrum. These types of filters can be used as follows Figure 2 The Bayer pattern (Red-Green-Green-Blue, RGGB) shown here is repeated in the image sensor. The RGGB pattern refers to the arrangement of pixels that make up the image sensor of a digital camera.

[0054] Since each filter transmits only a narrow spectrum of light, while the rest of the incident light is blocked, at any given photoelectric converter position or pixel position, only one of the color band signals transmitted by the filter can be recorded. In an optional embodiment of the present application, a monochrome image sensor without any color filter is compared with a traditional RGGB image sensor to obtain the quantum efficiency (QE) curves of the two in the visible spectrum range, as shown in FIG. Figure 3 As shown. Figure 3 It can be seen that spectral filtering reduces the white light broadband sensitivity of traditional image sensors by about 3-4 times compared to monochrome image sensors without any color filters. Therefore, the functions required to be implemented in this application include: improving the white light broadband sensitivity of image sensors.

[0055] Furthermore, to infer the block color signal at any given pixel location in an RGGB image sensor, the block color signal at that location must be interpolated from the signals of adjacent color pixels. This interpolation (also known as demosaicing) reduces the spectral resolution of the color image sensor. Therefore, the functionality required by this application also includes improving the spectral resolution of the image sensor.

[0056] In addition, if Figure 4 to Figure 5As shown, in many existing application scenarios, imaging systems are equipped with multi-element imaging lenses that can transmit light to traditional image sensors at extremely high angles (for example, the chief ray angle can reach 35 degrees). In these sensors, the pixels have a stack height of 2-4 microns from the top surface of the microlens to the photosensitive substrate interface. When light is incident on the microlens at a high angle, it is focused by the microlens and experiences a large displacement during its journey through the pixel stack. In order to effectively guide the light to the appropriate photoelectric converter, the microlens and filter must be shifted relative to the photoelectric converter in each pixel.

[0057] Specifically, by Figure 4 and Figure 5 It can be seen that the displacement of the microlens and filter relative to the photoelectric converter is affected by many factors, such as the pixel stack height, the number of stacked layers, the optical properties of the materials, the pixel size, the image sensor's imaging lens's chief ray angle distribution curve, and the imaging lens's focal length f. Therefore, designing microlens and filter offsets for a specific imaging lens and image sensor is typically a complex and expensive process. Any change in the imaging lens design, pixel size, or pixel stack height will result in significant changes to the manufacturing process, requiring verification of the new process's high stability to ensure high-yield parts, which will incur additional costs.

[0058] Therefore, by reducing pixel stacking and, ideally, avoiding the need to shift any part of the pixel stack, it would be very beneficial to significantly reduce the cost of image sensors. During the assembly stage of the imaging system, the multi-element imaging lens needs to be three-dimensionally aligned with the image sensor array to achieve lateral coincidence of the optical centers of the imaging lens and the image sensor. It is also crucial to prevent the imaging lens from tilting relative to the sensor plane so that the image sensor with offset microlenses and filters can sense the correct angle of incidence of light. Therefore, the functions that need to be implemented in this application also include: developing an image sensor that can withstand the problem of misalignment between the imaging lens and the image sensor.

[0059] In addition, image sensors are usually equipped with phase detection autofocus pixels (PDAF pixels), which enable the imaging system to adjust the position between the imaging lens and the image sensor at a very fast speed and obtain the best focused image. Figure 6 As shown, PDAF pixels are generally equipped with micro lenses covering a number of photoelectric converters arranged in a 2×1, 1×2 or 2×2 array, and the corresponding photoelectric converters generally share the same filter.

[0060] In detail, such as Figure 6As shown in the figure, to check the focusing state of the PDAF pixel at any given position, it is necessary to compare the signals between multiple photoelectric converters below the shared microlens. The corresponding signal is based on the light collected by the PDAF pixel at the determined focus state. The relative position between the imaging lens and the image sensor is then adjusted based on the comparison results, so that the imaging light collected by the imaging lens is evenly focused in multiple directions.

[0061] However, image sensors with PDAF pixels not only lose sensitivity due to the absorption filter, but also lose color resolution because PDAF pixels generally do not participate in imaging but are interpolated from adjacent imaging pixels.

[0062] Therefore, conventional image sensors with RGGB imaging pixels and PDAF pixels not only lose sensitivity due to the absorption filter, but also lose color resolution due to the need for interpolation calculations of adjacent imaging pixels.

[0063] Furthermore, the deposition process of microlenses and color filters based on organic polymer materials that are not complementary metal oxide semiconductors (CMOS) requires additional non-CMOS production equipment for placing absorptive filters on the image sensor array. In an alternative embodiment of the present application, the number of process steps, production cycle, and production yield of existing RGGB image sensors are as follows: Figure 7 shown.

[0064] In detail, such as Figure 7 As shown, the addition of non-CMOS process technology lengthens the traditional image sensor production process, and non-CMOS production equipment has a lower yield. Therefore, the inclusion of non-CMOS process technology increases the cost of image sensors. Furthermore, traditional polymer-based absorptive filters are also susceptible to weathering and have a shorter lifespan than inorganic materials produced using CMOS production equipment. Furthermore, for image sensors with PDAF pixels, the deposition process of traditional PDAF pixel microlenses in non-CMOS fabs is significantly limited by pixel size, required focusing characteristics, material viscosity, and other physical and chemical properties. Non-CMOS-based PDAF pixel microlenses typically suffer from significant variations in height and focusing power (radius of curvature), which affects the uniformity of PDAF pixel performance.

[0065] Therefore, the functions that need to be implemented in this application also include: integrating the color imaging function and phase detection autofocus function of the image sensor into the CMOS manufacturing process to reduce the cost of the image sensor and improve the yield of the image sensor.

[0066] In order to solve the above problems, an embodiment of the present application proposes an image sensor based on a pure CMOS process: asymmetric optical elements and symmetric optical elements are designed based on inorganic materials, imaging pixels are designed based on the asymmetric optical elements and the m photoelectric converters thereunder, and PDAF pixels are designed based on the symmetric optical elements and the n photoelectric converters thereunder.

[0067] In the imaging pixel, the asymmetric optical element focuses and diffracts the incident light, producing a three-dimensional asymmetric light intensity pattern, which is recorded on m photoelectric converters sharing the same asymmetric optical element to form different spatial signals. The three-dimensional asymmetric light intensity pattern is formed by the light diffraction and subsequent light self-interference occurring on the asymmetric optical element. The interference properties of light make the m detected spatial signals highly correlated, which causes narrow spectral bands of various colors (hereinafter referred to as color bands) to form m different signal component distributions on the m photoelectric converters. In this way, m representative color bands can be selected within the visible spectrum, and the incident light of unknown color can be split into a superposition combination of m color bands. Based on the spatial response of the asymmetric optical element to the incident light, a system of linear equations with m variables is constructed. , multiply both sides by the inverse matrix of C, and the solution of the m-dimensional linear equation system is In other words, incident light of unknown color can be directly calculated based on the spatial signal vector M (related to m spatial signals) and the inverse matrix of C, parsing the incident light into m spectral components corresponding to m color bands. This allows for simultaneous detection of the incident light brightness and color measurement.

[0068] Where C is an m×m calibration coefficient matrix, X is the color band fraction vector that contributes to the measured spatial signal in the incident light of unknown color, and M is the spatial signal vector obtained by measuring the incident light of unknown color. The calibration coefficient matrix can be directly simplified by substituting the spatial response of the asymmetric optical element to each single color band into a system of m-variable linear equations. m is an integer greater than or equal to 3, and n is an integer greater than or equal to 2. Multiple different representative color bands are selected across the entire visible spectrum, and representative color bands are selected within the small spectral range corresponding to each different color, such as at least three color bands, including red, green, and blue.

[0069] In a PDAF pixel, n photoelectric converters sharing a common symmetrical optical element are arranged in a centrally or axis-symmetrical manner. The symmetrical optical element and the n photoelectric converters form a PDAF pixel that is centrally or axis-symmetrically arranged. The symmetrical optical element focuses and diffracts the incident light. When the focus of the incident light is not ideal, a three-dimensional asymmetric light intensity pattern is generated on the n photoelectric converters. This asymmetric light intensity pattern enables phase difference detection in multiple directions, and the position of the imaging lens is adjusted based on the detected phase difference to achieve focus.

[0070] Based on this, the color of incident light is detected by combining asymmetric optical elements with multi-color band calibration technology, and the focusing state of incident light is detected by combining symmetric optical elements with n symmetrically arranged photoelectric converters, so that the asymmetric optical elements and the symmetric optical elements can replace the microlenses and filters in existing pixels.

[0071] At the same time, because multiple imaging pixels and multiple PDAF pixels are dispersed in the pixel array of the image sensor, they complement each other by interpolating the values ​​of adjacent pixels of different types (focus state or spectrum). For example, the spectral components of the PDAF pixel are interpolated based on the spectral components of several imaging pixels adjacent to the PDAF pixel, thereby realizing the color detection of the incident light at the PDAF pixel. Another example is that the phase difference of the imaging pixel is interpolated based on the phase difference of several PDAF pixels adjacent to the imaging pixel, thereby realizing the focus state detection at the imaging pixel.

[0072] Specifically, the present application provides a pixel array, the pixel array comprising:

[0073] A photoelectric conversion subarray, comprising a plurality of photoelectric converters arranged in an array;

[0074] An optical subarray is arranged on the photoelectric conversion subarray, and the optical subarray includes a plurality of symmetrical optical elements and a plurality of asymmetrical optical elements that are dispersed and staggered;

[0075] Wherein, each symmetrical optical element is arranged on a corresponding plurality of photoelectric converters to form symmetrically arranged phase detection autofocus pixels;

[0076] Each asymmetric optical element is arranged on a corresponding plurality of photoelectric converters to form asymmetrically arranged imaging pixels. The asymmetric optical element is used to focus and diffract the incident light to obtain a three-dimensional asymmetric light intensity pattern, which is recorded one-to-one on the plurality of photoelectric converters corresponding to the asymmetric optical element to form a plurality of different spatial signals.

[0077] For example, Figure 8 As shown, each symmetrical optical element is arranged on the corresponding 2×2 photoelectric converters to form a centrally symmetrical PDAF pixel in the pixel array, and each asymmetrical optical element is arranged on the corresponding 2×2 photoelectric converters to form an asymmetrically arranged imaging pixel in the pixel array. The asymmetric optical element is used to focus and diffract the incident light to obtain a three-dimensional asymmetric light intensity pattern, which is recorded one-to-one on the 2×2 photoelectric converters corresponding to the asymmetric optical element to form four different spatial signals.

[0078] In the embodiments of the present application, asymmetric optical elements and symmetric optical elements can replace the microlenses and filters in existing pixels. On the basis of realizing the phase detection autofocus function, brightness detection and color detection of the incident light can also be realized, which can effectively avoid various disadvantages caused by the microlenses and filters in the pixels.

[0079] Specifically, the symmetrical optical element includes a background structure layer and a symmetrical diffraction structure layer embedded in the background structure layer; wherein the refractive index of the background structure layer is lower than the refractive index of the symmetrical diffraction structure layer.

[0080] For example, Figure 8 As shown, the symmetrical diffraction structure layer is a transparent structure layer and is embedded at the bottom of the background structure layer.

[0081] In some embodiments, the materials of the aforementioned symmetrical optical elements are all inorganic materials.

[0082] The background structure layer may be made of a low refractive index material, such as silicon oxide, which has a refractive index of 1.46; or air, which has a refractive index of 1.

[0083] The symmetrical diffraction structure layer is made of a high refractive index material, such as a material including but not limited to silicon nitride, titanium oxide, tantalum oxide, silicon carbide and silicon, and its refractive index may be in the range of 1.8 to 4.

[0084] In some embodiments, in a symmetrical optical element, the symmetrical diffraction structure layer (i.e., a component of the symmetrical optical element) is coaxially arranged with the background structure layer, and the axis of the symmetrical diffraction structure layer is aligned with the center of the array formed by multiple photoelectric converters corresponding to the symmetrical optical element.

[0085] For example, Figure 8 As shown, the symmetrical diffraction structure layer includes a diffraction cylinder, which is coaxially arranged with the outer cylindrical background structure layer. The four photoelectric converters below the symmetrical optical element are arranged in a 2×2 array, and the axis of the symmetrical diffraction structure layer is aligned with the center of the 2×2 photoelectric conversion sub-array below.

[0086] In some embodiments, the asymmetric optical element includes a background structure layer and an asymmetric diffraction structure layer embedded in the background structure layer; wherein the refractive index of the background structure layer is lower than the refractive index of the asymmetric diffraction structure layer.

[0087] In some embodiments, the materials of the asymmetric optical elements are all inorganic materials.

[0088] The background structure layer may be made of a low refractive index material, such as silicon oxide, which has a refractive index of 1.46; or air, which has a refractive index of 1.

[0089] The asymmetric diffraction structure layer is made of a high refractive index material, such as a material including but not limited to silicon nitride, titanium oxide, tantalum oxide, silicon carbide and silicon, and its refractive index may be in the range of 1.8 to 4.

[0090] In some embodiments, in an asymmetric optical element, the asymmetric diffraction structure layer includes a plurality of diffraction cylinders of different sizes and a plurality of diffraction ring cylinders of different sizes. The plurality of diffraction cylinders and the plurality of diffraction ring cylinders are staggered and asymmetrically arranged inside the background structure layer, so that the diffraction of the incident light by the asymmetric diffraction structure layer is asymmetric diffraction, and the height of the diffraction cylinders and the height of the diffraction ring cylinders are both smaller than the height of the background structure layer.

[0091] For example, Figure 8 As shown, the asymmetric diffraction structure layer includes multiple diffraction cylinders and multiple diffraction ring cylinders. Within the asymmetric diffraction structure layer, multiple diffraction cylinders of different sizes are interlaced with multiple diffraction ring cylinders of different sizes, resulting in asymmetric diffraction of incident light by the asymmetric diffraction structure layer. Within the asymmetric diffraction structure layer, the multiple diffraction cylinders have different diameters, the multiple diffraction ring cylinders have different inner and outer diameters, and the heights of the diffraction cylinders and the diffraction ring cylinders are both less than the height of the background structure layer.

[0092] In some embodiments, the above-mentioned pixel array also includes a substrate, and a unit deep trench isolation structure arranged in the substrate; wherein the photoelectric conversion sub-array is arranged on the substrate; the multiple photoelectric converters corresponding to each optical element are surrounded by a unit deep trench isolation structure to prevent light crosstalk from adjacent optical elements and / or prevent electrical interference from adjacent photoelectric converters, and the optical elements include symmetrical optical elements and asymmetric optical elements.

[0093] The material of the substrate and the material of the unit deep trench isolation structure can be adjusted according to actual conditions, and the embodiments of the present application do not impose any restrictions thereon.

[0094] For example, the substrate is a back-side illuminated silicon (BSI Si) substrate, and the material of the cell deep trench isolation structure is silicon oxide.

[0095] For example, Figure 8As shown, the pixel array also includes a back-illuminated silicon substrate (Back-Side illuminated Si, BSI Si), the photoelectric conversion sub-array is arranged on the substrate, the optical sub-array is arranged on the photoelectric conversion sub-array, and the pixel also includes a cell deep trench isolation structure (Cell Deep Trench Isolation, CDTI) arranged in the back-illuminated silicon substrate, and the cell deep trench isolation structure is arranged around the 2×2 photoelectric converters to physically isolate the 2×2 photoelectric converters under each symmetrical optical element or asymmetric optical element to prevent light crosstalk from adjacent optical elements or electrical interference from adjacent photoelectric converters.

[0096] In some embodiments, phase detection autofocus pixels (i.e., PDAF pixels) are arranged in a centrally symmetrical or axially symmetrical manner. Within the phase detection autofocus pixels, symmetrical optical elements are used to focus and diffract the incident light to obtain a phase detection light intensity pattern, which is recorded one-to-one on corresponding multiple photoelectric converters to form multiple phase detection spatial signals.

[0097] In some embodiments, a phase detection autofocus pixel (ie, PDAF pixel) is configured to detect a phase difference or focus state of incident light in a direction perpendicular to an axis of symmetry based on a plurality of phase detection spatial signals arranged axially symmetrically.

[0098] In detail, such as Figure 8 As shown, in the embodiment of the present application, the values ​​of m and n are both 4, and the multiple photoelectric converters are arranged in a regular array, and the corresponding imaging pixels and PDAF pixels are also arranged in a regular pattern. The entire pixel array can be regarded as a staggered regular arrangement of multiple imaging pixels and multiple PDAF pixels. Each PDAF pixel is surrounded by four adjacent imaging pixels, and each imaging pixel is surrounded by four adjacent PDAF pixels. The number of imaging pixels is approximately equal to the number of PDAF pixels.

[0099] It's understandable that a greater number of PDAF pixels translates to faster phase detection autofocus. However, since PDAF pixels don't directly participate in imaging, even when interpolated from signals from neighboring imaging pixels, the image sensor's color resolution remains relatively low. Therefore, the number of PDAF pixels can be flexibly selected based on actual conditions and is not a specific limitation.

[0100] In detail, such as Figure 8As shown in the embodiment of the present application, in the PDAF pixel, a flat and centrally symmetrically placed symmetrical diffraction structure layer composed of a high refractive index material is used as a light focusing element and is embedded in a background structure layer composed of a low refractive index material to increase the effective refractive index toward the center of the 2×2 photoelectric conversion sub-array, thereby efficiently focusing the incident light onto the 2×2 photoelectric conversion sub-array below. Figure 9 to Figure 10 As shown in Figure 3, the symmetry of the light intensity distribution of the PDAF pixel is similar to that of the light intensity distribution under the traditional non-planar microlens.

[0101] In more detail, at any given height position of the image sensor, the asymmetry of the spatial signal in the 2×2 photoelectric conversion subarray recorded under the symmetrical optical elements in the PDAF pixel encodes information about the focus state of the incident light. Figure 10 As shown on the left side of the image, when the light collected in the lower part of the imaging lens pupil mainly illuminates the lower photoelectric converters of the 2×2 photoelectric conversion sub-array, the spatial signals of the two lower photoelectric converters will be higher than the spatial signals of the two upper photoelectric converters, and the image will be in a defocused state. Figure 10 As shown in the middle, when the light collected in the lower part of the imaging lens pupil mainly illuminates the center of the 2×2 photoelectric conversion sub-array, the spatial signals of the two lower photoelectric converters will be equal to the spatial signals of the two upper photoelectric converters, which is in a clear positive focus state. Figure 10 As shown on the right side of the image, when the light collected in the lower portion of the imaging lens pupil primarily illuminates the upper photoelectric converters of the 2×2 photoelectric conversion subarray, the spatial signals of the two lower photoelectric converters are lower than those of the two upper photoelectric converters, indicating an overfocus state. Therefore, this spatial signal difference across the 2×2 photoelectric conversion subarray can be used to identify the focus state, direction, and distance of the incident light, enabling the image sensor to be moved relative to the imaging lens to achieve optimal focus.

[0102] It can be understood that the spatial signal collected by each photoelectric converter in the PDAF pixel can be called a phase detection spatial signal. In the PDAF pixel, based on multiple phase detection spatial signals arranged in an axially symmetrical manner, the phase difference or focusing state of the incident light in the vertical direction of the symmetry axis can be detected.

[0103] It should be noted that Figure 9 to Figure 10Only the vertical focus adjustment diagram is shown. The four photoelectric converters in the PDAF pixel are arranged in a centrosymmetric 2×2 array. In addition to determining the vertical focus state of incident light based on the signal difference between the upper and lower photoelectric converters, the horizontal focus state of incident light can also be determined by the signal difference between the left and right photoelectric converters, or the diagonal focus state of incident light can be determined by the signal difference between two photoelectric converters on a diagonal line. The analysis of the vertical focus state of incident light can be similarly carried out and will not be repeated here.

[0104] In some embodiments, the imaging pixels are configured to: based on the decomposition of the incident light using multiple calibration color bands, combine the pre-calibration calibration parameters obtained by the pre-calibration calibration of the multiple calibration color bands, and the corresponding multiple spatial signals, perform analytical calibration on the incident light, and parse the incident light into spectral components corresponding to the multiple calibration color bands to perform color detection on the incident light.

[0105] In some embodiments, for each imaging pixel, when performing color detection, multiple calibration color bands are first selected within the visible spectrum to meet the requirements of unfolding and decomposing incident light of unknown color, and the number of selected calibration color bands is the same as the number of photoelectric converters in the imaging pixel.

[0106] In detail, in the embodiments of the present application, Figure 11 As shown, in the imaging pixel, a flat and asymmetrically placed asymmetric diffraction structure layer composed of a high refractive index material is embedded in a background structure layer composed of a low refractive index material as a light focusing element to increase the effective refractive index toward the 2×2 photoelectric conversion sub-array, thereby efficiently focusing the incident light into the 2×2 photoelectric conversion sub-array below, corresponding to the formation of a three-dimensional asymmetric distribution mode of light intensity pattern distribution, which is different from the light intensity distribution under the traditional shape of the non-planar microlens.

[0107] It should be noted that the light intensity pattern distribution of the three-dimensional asymmetric distribution mode can be simply referred to as a three-dimensional asymmetric light intensity pattern. The three-dimensional asymmetric light intensity pattern refers to the incident light after the asymmetric focusing and diffraction of the asymmetric optical element, forming a three-dimensional asymmetric light intensity pattern in the three-dimensional space on the photoelectric converter. The three-dimensional asymmetric light intensity pattern is then projected onto the photosensitive plane of the photoelectric converter, and a two-dimensional asymmetric light intensity distribution is formed on each photoelectric converter at different spatial positions in the photosensitive plane. The light intensity distribution collected on the photoelectric converters at different positions is different, and then based on the photoelectric effect, the corresponding records are made on each photoelectric converter to form multiple spatial signals of different sizes.

[0108] In detail, in the embodiment of the present application, in the imaging pixel, based on Figure 8The asymmetric diffraction of the incident light of four color bands (B=430nm, G2=490nm, G1=550nm and R=610nm) in the visible spectrum of the asymmetric optical element is shown as follows: Figure 12 The three-dimensional asymmetric light intensity pattern is formed by the light diffraction and subsequent light self-interference on the asymmetric optical element. The interference properties of light make the four detected spatial signals highly correlated. Figure 12 It can be seen that the asymmetric distribution patterns of the spatial signals related to incident light of different colors on the four photoelectric converters are different. The asymmetric spatial patterns of the spatial signals recorded on the four photoelectric converters can be regarded as the fingerprint of the incident spectrum / color.

[0109] In this way, four representative color bands (hereinafter referred to as calibration bands) can be selected in the visible spectrum, and the incident light of unknown color can be split into a superposition combination of the four calibration bands. Based on the spatial response of the asymmetric optical element to the incident light, a four-variable linear equation system is constructed. Multiply both sides of the 4-variable linear equation by C The inverse matrix of , the solution of the 4-variable linear equation system is Therefore, the incident light of unknown color can be directly calculated based on the spatial signal vector M (related to the 4 spatial signals) and C The inverse matrix of is calculated to resolve the incident light into four spectral components corresponding to the four calibration color bands.

[0110] That is, the incident light analysis can be decoded into 4 calibration bands by writing a system of linear equations connecting the spatial signal of the photoelectric converter with the spectral components of the measurement signal.

[0111] In order to decode the four measured spatial signals, based on the above four-variable linear equation system, for the imaging pixel whose four photoelectric converters share an asymmetric optical element, four different narrowband bands (i.e., calibration bands) can be used to illuminate the pixel in succession, and the corresponding spatial signal responses can be recorded to form a calibration coefficient matrix C . Then use the calibration coefficient matrix C The inverse matrix of the calibration coefficient matrix is ​​used to calculate the incident light of unknown color and solve the linear equations. C The elements of are used as coefficients for the equations connecting the 4 spatial measurement signals to the 4 calibration strips.

[0112] More specifically, in the embodiment of the present application, based on the selection of four calibration color bands to decompose the incident light of unknown color, the following four-variable linear equation system can be constructed based on the spatial signal response of the incident light after the decomposition of the four calibration color bands for each imaging pixel:

[0113]

[0114] in, Calibration color strips representing incident light bk Component in the photoelectric converter PD ij The spatial signal measured on M ij The calibration coefficient, Calibration color strips representing incident light bk Component to spatial signal M ij The contribution factor, M ij The incident light of unknown color is reflected by the photoelectric converter PD ij The spatial signal measured above; i 、 j is an integer from 1 to 2, k An integer from 1 to 4.

[0115] In detail, in the embodiment of the present application, within the visible spectrum range of 400-640 nm, as Figure 12 to Figure 14 As shown in the figure, blue light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 430 nm is used as the calibration band Band 1 (referred to as b1), green light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 490 nm is used as the calibration band Band 2 (referred to as b2), green light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 550 nm is used as the calibration band Band 3 (referred to as b3), and red light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 610 nm is used as the calibration band Band 4 (referred to as b4).

[0116] It can be understood that the above-mentioned four-variable linear equation system can be written in the following matrix form.

[0117]

[0118] in, C is a 4×4 calibration coefficient matrix, X is the color band fraction vector that contributes to the measured spatial signal in the incident light of unknown color, M is the spatial signal vector obtained by measuring the incident light of unknown color, where

[0119] .

[0120] Multiply both sides of the above matrix equation by C The inverse matrix of , we get the following relationship, which is the solution of the linear equations:

[0121]

[0122] It can be seen from the above matrix equation that after the pixel structure and the selected four calibration color bands are determined, the calibration vector of the pixel structure for each calibration color band is determined accordingly, thereby making the calibration coefficient matrix unique. In this way, the spatial signal response of the pixel structure to the incident light of unknown color can be understood as "encoding the incident light expanded and decomposed according to the four calibration color bands based on the calibration coefficient matrix to obtain four different spatial signals"; at the same time, it can be seen from the above matrix equation that only one of the four calibration color bands can be used at a time to simplify the spatial signal response or simplify the encoding, that is, X One element in is 1 and the other three elements are 0, which simplifies the matrix equation and then the calibration coefficient matrix can be obtained based on the corresponding measured spatial signal. C The calibration coefficient matrix is ​​obtained by combining the calibration coefficient elements or calibration coefficient vectors obtained by solving the spatial signals corresponding to the four calibration bands. C , and the calibration coefficient matrix can be solved C The inverse matrix of C -1 .

[0123] Finally, from the above matrix equation, we can know that the subsequent incident light of unknown color X , in the known calibration coefficient matrix C The inverse matrix of C -1 and incident light X The corresponding spatial signal vector M Based on the calibration coefficient matrix C The inverse matrix of C -1 and the measured spatial signal vector M , solve the incident light, resolve the incident light into the spectral components corresponding to the above four calibration color bands, and realize the color measurement of the incident light while detecting the brightness of the incident light. This process can be understood as "the inverse operation parameters based on the encoding parameters (the inverse matrix C of the calibration coefficient matrix C -1 ) is decoded with the encoded result (4 different spatial signals) to obtain the unfolding decomposition result of the incident light under 4 calibration color bands."

[0124] In detail, in the embodiment of the present application, , only red light with a full width at half maximum (FWHM) of 60nm and a wavelength of 610nm is used as the calibration band Band 4 for pre-calibration of the monochrome band. The corresponding four-variable linear equation system is simplified to:

[0125]

[0126] Meanwhile, according to the actual measurement of the spatial signal vector M , the quantum efficiencies of the corresponding four photoelectric converters are obtained as follows: QE PD 11 = 16.7, QE PD 12 = 12.7, QE PD 21 = 12.7, and QE PD 22 = 14.8.

[0127] In detail, in the embodiment of the present application, let , only the green light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 550 nm is used as the calibration color band Band 3 to perform monochromatic band pre-calibration simplification, and then the corresponding four-element linear equation group is simplified as follows:

[0128]

[0129] Meanwhile, according to the actual measurement of the spatial signal vector M , the quantum efficiencies of the corresponding four photoelectric converters are obtained as follows: QE PD 11 = 20.2, QE PD 12 = 14, QE PD 21 = 16, and QE PD 22 = 17.5.

[0130] In detail, in the embodiment of the present application, let , only the green light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 490 nm is used as the calibration color band Band 2 to perform monochromatic band pre-calibration simplification, and then the corresponding four-element linear equation group is simplified as follows:

[0131]

[0132] Meanwhile, according to the actual measurement of the spatial signal vector M , the quantum efficiencies of the corresponding four photoelectric converters are obtained as follows: QE PD 11 = 19.8, QE PD 12 = 15, QE PD 21 = 15.4, and QE PD 22 = 17.3.

[0133] In detail, in the embodiment of the present application, let , only the blue light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 430 nm is used as the calibration color band Band 1 to perform monochromatic band pre-calibration simplification, and then the corresponding four-element linear equation group is simplified as follows:

[0134]

[0135] At the same time, according to the spatial signal vector M The actual measurement of the four corresponding photoelectric converters yields the following quantum efficiencies: QE PD 11 =18.1, QE PD 12 =16.9, QE PD 21 =14.4, QE PD 22 =19.9.

[0136] In this way, the corresponding calibration coefficient matrix is ​​obtained by combining the pre-calibration simplification of the above four monochrome bands C As shown in the table below, the four calibration coefficients in each column correspond to the pre-calibrated simplified results of a calibration ribbon.

[0137]

[0138] At the same time, for calculation considerations, the four calibration coefficients corresponding to the pre-calibration simplified results of a single calibration ribbon are normalized to obtain the calibration coefficient matrix shown in the following table C .

[0139]

[0140] For the calibration coefficient matrix shown in the table above C Perform inverse calculation to obtain the calibration coefficient matrix C The inverse matrix of C -1 As shown in the following table.

[0141]

[0142] In this way, the corresponding imaging pixels or image sensors are pre-calibrated or pre-calibrated, and then the incident light of unknown color is detected based on the Formula, can be directly based on the calibration coefficient matrix C The inverse matrix of C -1 and the measured spatial signal vector M Calculating incident light X , the incident light X It is resolved into the spectral components corresponding to the four calibration bands, namely ， While detecting the brightness of the incident light, the color of the incident light is measured.

[0143] In addition, in the embodiments of the present application, Figure 14 As shown, based on the above process, the incident light XAfter parsing the spectral components corresponding to the four calibration color bands, each spectral component is multiplied by the total measurement signal in the four photoelectric converters to convert the signal to the least significant bit (LSB) of the measurement value. Finally, the spectral components corresponding to the four calibration color bands are converted to the standard red, green, and blue color space based on the least significant bit for display.

[0144] It should be noted that in the above embodiment, the value of n is 4, and the four photoelectric converters under the symmetrical optical element in the PDAF pixel are distributed in a 2×2 array, but the value of n is not limited to 4; it is understandable that the more the number of photoelectric converters under the symmetrical optical element in the PDAF pixel is (that is, the larger the value of n) and the more they are distributed in a square array, such as Figure 15 The 3×3 array shown, or a larger 4×4 array, 5×5 array, etc., can detect the focusing state of incident light in more directions (diagonal lines at more angles), and can be analyzed similarly, so I will not repeat it here.

[0145] At the same time, when the photoelectric converters under the symmetrical optical elements in the PDAF pixels are centrally symmetrically distributed, they are not necessarily distributed in a square array: they may be non-square matrix distributions, such as 1×2 array, 2×3 array, 2×4 array, etc., which can also perform focus state detection, but the corresponding detection directions are relatively few; they may also be non-square centrally symmetrical distributions, such as Figure 16 As shown, it can also detect the focusing state of incident light in the diagonal direction, horizontal direction and vertical direction, which will not be described in detail here.

[0146] It should be noted that in the above embodiment, m is set to 4, and an imaging pixel is formed based on a 2×2 array of four photoelectric converters and an asymmetric optical element. Four calibration color bands, R, G1, G2, and B, are uniformly selected across the visible spectrum. Pre-calibration is then performed based on the simplified spatial response of a single calibration color band. Subsequently, incident light of unknown color can be parsed into spectral components corresponding to the four calibration color bands. This scenario is applicable to image sensors with a single photoelectric converter arranged in a square configuration and multiple photoelectric converters arranged in a regular array.

[0147] It is understandable that the value of m is not limited to 4, such as Figure 15 As shown, the value of m is 9, and the 9 photoelectric converters are distributed in a 3×3 array. The 9 photoelectric converters distributed in the 3×3 array and an asymmetric optical element thereon constitute an imaging pixel. In addition, with the further expansion of the photoelectric conversion sub-array, the value of m can be further flexibly and freely selected. For example, an asymmetric optical element can be formed correspondingly on 4×4, 5×5, ..., N×N photoelectric converters to constitute an imaging pixel.

[0148] Correspondingly, 3×3, 4×4, 5×5, ..., N×N calibration color bands need to be selected within the visible spectrum, and the calibration color bands must at least cover the three primary colors of red, green, and blue to effectively meet the color decomposition requirements for incident light of various unknown colors. Where N is an integer greater than or equal to 2.

[0149] It should be noted that the more uniform the color distribution of the calibration color strips and the greater their number, the more detailed the color decomposition and analysis of the corresponding incident light, the higher the accuracy of the decomposition and analysis, and the more accurate the decomposition and analysis results, but the corresponding calculation amount is larger and a compromise needs to be considered.

[0150] In this application, at least the optical element in the imaging pixel is required to have an asymmetric structure, so as to form multiple different spatial signal distributions for incident light of different colors based on asymmetric diffraction and self-interference, so as to facilitate the subsequent distinction and calculation of the components of each calibration color band. The photoelectric converters under the asymmetric optical element can be as follows: Figure 8 or Figure 16 The 2×2 matrix distribution shown can also be Figure 15 The 3×3 matrix distribution shown is a centrosymmetric structure. The individual photoelectric converters under the asymmetric optical element can also be distributed in a non-centrosymmetric structure, such as a matrix distribution with photoelectric converters of the same size but unequal rows and columns, or an irregular splicing distribution with photoelectric converters of multiple different sizes.

[0151] It should be noted that when the photoelectric converters under the asymmetric optical element are distributed in a non-centrosymmetric structure, the shape structure of the asymmetric optical element can be adapted and adjusted to ensure the diffraction and focusing effects in all directions as much as possible, and to ensure that light is incident on each photoelectric converter.

[0152] Therefore, in another embodiment of the present application, Figure 17 As shown in the figure, in the imaging pixel shown in the dotted box in the lower left corner, the three photoelectric converters under the asymmetric optical element are regularly arranged in a 3×1 matrix, and each photoelectric converter is set in a rectangular shape; correspondingly, it is necessary to select three different calibration color bands in the visible spectrum range, such as 600nm red light, 525nm green light and 450nm blue light for pre-calibration, and calculate the corresponding calibration coefficient matrix C The inverse matrix of C -1 The specific calibration process can refer to the above embodiment and will not be described in detail here.

[0153] In the above embodiments, Figure 8 、 Figure 15 、 Figure 16 or Figure 17As shown, the size specifications of multiple photoelectric converters in the image sensor are consistent and the multiple photoelectric converters are arranged in a regular array. In some embodiments, there may be at least two photoelectric converters of different sizes in the pixel array, and the corresponding multiple photoelectric converters are arranged in a crisscross pattern, such as Figure 18 and Figure 19 shown.

[0154] In another embodiment of the present application, Figure 18 As shown, in the imaging pixel shown in the dotted box in the lower left corner, the five photoelectric converters under the asymmetric optical element are irregularly distributed in a crisscross pattern. The five photoelectric converters have two different sizes, four of which are rectangular and one is square. Correspondingly, five different calibration color bands need to be selected within the visible spectrum. For example, narrowband light of 610nm, 570nm, 530nm, 490nm and 450nm can be used for pre-calibration to obtain the corresponding calibration coefficient matrix. C The inverse matrix of C -1 The specific calibration process can refer to the above embodiment and will not be described in detail here.

[0155] In another embodiment of the present application, Figure 19 As shown, the pixel array includes imaging pixels of two different structures: in the first imaging pixel shown in the dotted box in the lower left corner, the five photoelectric converters under the asymmetric optical element are irregularly distributed in a criss-cross pattern, and the five photoelectric converters have two different sizes, four of which are rectangular and one is square; in the second imaging pixel shown in the center, the nine photoelectric converters under the asymmetric optical element are distributed in a 3×3 square matrix, and the nine photoelectric converters have only one size, all of which are square.

[0156] Correspondingly, the first imaging pixel needs to select 5 different calibration color bands within the visible spectrum range, and the second imaging pixel needs to select 9 different calibration color bands within the visible spectrum range. For example, the first imaging pixel can be pre-calibrated using narrow-band light of 610nm, 570nm, 530nm, 490nm and 450nm, and the second imaging pixel can be pre-calibrated using narrow-band light of 610nm, 590nm, 570nm, 550nm, 530nm, 510nm, 490nm, 470nm and 450nm.

[0157] The calibration ribbons used by the first imaging pixel and the second imaging pixel can be partially the same or completely different, which is not limited here; since the number of calibration ribbons used for calibration is different, the corresponding calibration coefficient matrix is CThe specific calibration process can refer to the above embodiment and will not be described in detail here.

[0158] In addition, it can be understood that in other optional embodiments of the present application, the pixel array can also include different imaging pixels and PDAF pixels of more specifications and shapes, and the structural types of imaging pixels and PDAF pixels are not limited to one, but there can be multiple different imaging pixels or multiple different PDAF pixels at the same time, which is not limited here.

[0159] Among them, the imaging pixels of multiple different structures refer to at least one of the structures of the asymmetric optical elements and the arrangement structures of each photoelectric converter in the multiple imaging pixels being different, such as the shapes of the asymmetric optical elements being different, the materials of the asymmetric optical elements being different, the number of photoelectric converters being different, the sizes of the photoelectric converters being different, etc.; similarly, the PDAF pixels of multiple different structures refer to at least one of the structures of the symmetric optical elements and the arrangement structures of each photoelectric converter in the multiple PDAF pixels being different.

[0160] In this way, in the pixel array of the embodiment of the present application, the focusing state detection of the incident light can be realized by combining symmetrical optical elements with n symmetrically arranged photoelectric converters, and the color detection of the incident light can be realized by combining asymmetric optical elements with multi-color band calibration technology. It can replace the microlenses and filters in the existing pixels, and on the basis of realizing the phase detection autofocus function, it can also realize the brightness detection and color detection of the incident light.

[0161] Furthermore, the present application also provides an image sensor based on the above pixel array, which includes the above pixel array.

[0162] Specifically, the image sensor is configured as follows:

[0163] Detecting the phase difference or focus state of incident light in multiple directions through phase detection autofocus pixels, and adjusting the relative position between the incident light and the image sensor based on the detection results in multiple directions to achieve focus; and

[0164] The incident light is collected by imaging pixels, and within each imaging pixel, the incident light is resolved into spectral components corresponding to multiple calibration color bands to achieve color detection of the incident light.

[0165] In some embodiments, the image sensor is further configured to: for each phase detection autofocus pixel, interpolate the spectral components corresponding to several imaging pixels adjacent to the phase detection autofocus pixel to determine the spectral components corresponding to the phase detection autofocus pixel, so as to realize color detection of the incident light at the phase detection autofocus pixel.

[0166] In detail, such asFigure 8 As shown, the image sensor includes a plurality of PDAF pixels and imaging pixels that are dispersed and staggered in an array. 2×2 photoelectric converters and a symmetrical optical element thereon constitute a PDAF pixel. Phase difference detection or focus state detection in multiple directions can be performed based on the PDAF pixel to facilitate rapid focus adjustment. 2×2 photoelectric converters and an asymmetric optical element thereon constitute an imaging pixel. The incident light forms an asymmetric light intensity pattern in each imaging pixel based on the asymmetric optical element. Combined with the subsequent calibration of multiple monochromatic bands in the visible spectrum range, it can effectively detect the brightness and color of the incident light in each imaging pixel, and resolve the incident light into spectral components corresponding to the four calibration color bands.

[0167] Similarly, in other optional embodiments of the present application, the number and distribution of photoelectric converters under the PDAF pixels and imaging pixels can have various other alternatives. Furthermore, the corresponding symmetrical and asymmetric optical elements can also have other variations in number, material, and shape, which will not be further described here.

[0168] In more detail, in the embodiment of the present application, the image sensor is as follows Figure 20 The method shown works as follows: first, the focusing state of the incident light in multiple directions is identified based on the PDAF pixel, and then the relative position (mainly the distance) of the imaging lens and the image sensor is adjusted multiple times based on the focusing state information to achieve optimal focus; after achieving optimal focus, the external light source is collected based on the imaging pixel, and the incident light is decomposed into spectral components corresponding to multiple calibration color bands. Then, based on the inverse matrix of the calibration coefficient matrix determined in advance and the real-time spatial signal obtained by the collection, the spectral components of the incident light under each calibration color band are calculated; after analyzing and identifying the spectra of all imaging pixels around the PDAF pixel, the spectrum of the PDAF pixel can be interpolated based on the spectra of the surrounding imaging pixels to make up for the loss of the PDAF pixel.

[0169] Among them, during phase detection autofocus, once data unrelated to the focus state on the image sensor array is retrieved and the position of the image sensor is adjusted relative to the imaging lens, the focus state information on the image sensor can be used to select the pre-calibrated calibration coefficient matrix of each imaging pixel in the image sensor array, and then for each imaging pixel, the corresponding incident light is parsed based on the inverse matrix of the calibration coefficient matrix and the measured spatial signal, and the incident light is parsed into spectral components corresponding to multiple calibration color bands, that is, the incident spectrum of each imaging pixel (also called parsed spectrum, referred to as spectrum) is obtained, and the spectra of multiple adjacent imaging pixels of the PDAF pixel are retrieved, and then the spectrum of the PDAF pixel is interpolated and completed based on the spectra of multiple imaging pixels. The spectrum of the PDAF pixel that ignores the influence of the local focus state is completely determined by the spectral interpolation calculation of the imaging pixels around the PDAF pixel, and depends on the calibration coefficient matrix of the corresponding imaging pixel and the restored incident light color.

[0170] It should be noted that compared to traditional image sensors with color imaging and phase detection autofocus, these sensors no longer use color filters to filter out multiple color components before transferring the remaining single color band components to a photoelectric converter. Instead, they use asymmetric optical elements to transfer all the color band components, resulting in a 2-3 times increase in photosensitivity within the visible spectrum. Furthermore, within the imaging pixel, the brightness and color of the incident light are determined entirely based on pre-calibrated multi-color band components and measured spatial signals, eliminating the need for interpolation calculations based on adjacent imaging pixels and maintaining the image sensor's spectral resolution.

[0171] Furthermore, the optical elements are made entirely of inorganic materials. Compared to traditional image sensors using organic materials, the use of inorganic materials eliminates the need for CMOS process flows like organic materials, allowing the entire image sensor to be manufactured entirely in a CMOS fab, reducing the cost of the image sensor. Furthermore, due to the high process precision and stringent requirements for organic materials, traditional image sensors with microlenses and filters have a low yield of approximately 80%. However, the yield of the all-inorganic CMOS image sensor in this application is very high, approaching 100%. Furthermore, the inorganic materials offer greater compatibility, and by eliminating non-CMOS materials, the possibility of dark current can be further reduced. Consequently, the all-CMOS image sensor offers a longer lifespan and higher reliability.

[0172] It can be understood that in the present application, the PDAF pixels based on symmetric optical elements can effectively realize autofocus, and the imaging pixels based on asymmetric optical elements can effectively realize incident light color analysis detection and incident light brightness detection. On the basis of realizing autofocus, incident light color detection and brightness detection, the pixels in the present application no longer require the microlenses and filters in the existing pixels. Therefore, the pixels in the present application are not limited by the size of the microlens structure and the size of the filter structure, especially the size of the microlens structure, and the corresponding pixel size can be easily reduced.

[0173] At the same time, since adjacent pixels (PDAF pixels or imaging pixels) are optically isolated by the unit deep trench isolation structure, crosstalk is effectively suppressed, and there is almost no color shadow in the entire pixel array or image sensor.

[0174] It should be noted that, in addition to the above-mentioned photoelectric conversion subarray and the above-mentioned optical subarray, the image sensor also includes other structures such as a pixel processing circuit, a controller and an image processor. For details, please refer to the existing technology and will not be repeated here.

[0175] Furthermore, an embodiment of the present application further provides an imaging system, comprising:

[0176] Imaging lens, used to converge light and obtain incident light;

[0177] An image sensor, the image sensor comprising the above-mentioned pixel array;

[0178] Among them, the image sensor is configured to: detect the phase difference or focus state of the incident light in multiple directions through phase detection autofocus pixels, and adjust the relative position between the imaging lens and the image sensor based on the detection results in multiple directions to achieve focusing; and collect the incident light through imaging pixels, and within each imaging pixel, resolve the incident light into multiple spectral components to achieve color detection of the incident light.

[0179] In some embodiments, the image sensor is further configured to: for each phase detection autofocus pixel, interpolate the spectral components corresponding to several imaging pixels adjacent to the phase detection autofocus pixel to determine the spectral components corresponding to the phase detection autofocus pixel, so as to realize color detection of the incident light at the phase detection autofocus pixel.

[0180] For example, Figure 21 As shown, the imaging system includes:

[0181] Imaging lens, which converges the light of the object and forms an image;

[0182] The image sensor realizes automatic focusing based on PDAF pixels, performs asymmetric spatial signal collection on the incident imaging light based on imaging pixels, and combines pre-calibration and calibration of multiple calibration color bands in the visible spectrum range to analyze the incident light into spectral components corresponding to the multiple calibration color bands to detect the brightness and color of the incident light in each imaging pixel.

[0183] In detail, as shown in Figure 21 and Figure 22 In the embodiment of the present application, since the optical element and the subsequent phase detection automatic focusing technology and the multi-color band calibration technology in the visible spectrum range can effectively replace the microlens and filter structure in the existing pixel, the image sensor in the present application does not need any structural offset to adapt to high chief ray angle, and the image sensor in the present application can also adapt to any F number imaging lens. As shown in Figure 22 Experiments prove that two different high chief ray angles CRA and different F number imaging lenses, i.e. Lens 1 and Lens 2, can effectively adapt to the image sensor structure in the present application.

[0184] In this way, the assembly of the imaging system can also be effectively simplified, and different specifications of imaging lenses and different specifications of image sensors can be assembled at any time as needed to obtain an imaging system. On the basis of phase detection automatic focusing based on PDAF pixels, the assembled imaging system can also be calibrated in real time based on pre-calibration and calibration of multiple calibration color bands in the visible spectrum range.

[0185] It should be noted that the pixel and image sensor based on inorganic material optical element are formed first, and then multi-color band calibration is performed based on spatial correlation. Even if the asymmetric optical element in the front-end imaging pixel has printing errors or process errors, causing the structure of the asymmetric optical element to be not ideal, it does not affect the subsequent spectral imaging performance.

[0186] Further, the embodiment of the present application also provides an image acquisition method applied to the above-mentioned imaging system, and the method comprises the following steps:

[0187] S1, collecting the incident light by the phase detection automatic focusing pixel to obtain a plurality of corresponding phase detection spatial signals, wherein the incident light is formed by converging light rays of the imaging lens;

[0188] S2, detecting the phase difference or focusing state of the incident light in multiple directions based on the plurality of phase detection spatial signals, and adjusting the relative position between the imaging lens and the image sensor based on the detection results in the multiple directions to realize focusing;

[0189] S3, collecting the incident light by the imaging pixel to obtain a plurality of corresponding spatial signals;

[0190] S4. For each imaging pixel, based on the multiple spatial signals and the multiple calibration color bands, resolve the incident light into spectral components corresponding to the multiple calibration color bands;

[0191] S5. For each phase detection autofocus pixel, interpolate the spectral components corresponding to several imaging pixels adjacent to the phase detection autofocus pixel to determine the spectral components corresponding to the phase detection autofocus pixel.

[0192] The above-mentioned image acquisition method can also be understood as an image sensor operation method. The image sensor operation method specifically includes performing phase detection autofocus based on PDAF pixels, adjusting the relative position between the imaging lens and the image sensor to achieve target focus; after achieving target focus, performing spatial signal acquisition of incident light based on the imaging pixels, and calculating the incident light based on the acquired spatial signal and the inverse matrix of the pre-calibrated calibration coefficient matrix, decomposing the incident light into spectral components corresponding to m calibration color bands to obtain the spectrum of each imaging pixel; and performing interpolation calculation based on the spectra of the imaging pixels surrounding the PDAF pixel to obtain the spectrum of the PDAF pixel.

[0193] In some embodiments, step S4 of resolving incident light into spectral components corresponding to multiple calibration color bands based on multiple spatial signals and multiple calibration color bands for each imaging pixel includes:

[0194] S41. For each imaging pixel, determine the number m of photoelectric converters in the imaging pixel;

[0195] S42. For each imaging pixel, select m calibration color bands from the visible spectrum to decompose incident light of unknown color;

[0196] S43, performing pre-calibration on each imaging pixel based on m calibration ribbons to obtain pre-calibration parameters;

[0197] S44. For each imaging pixel, analyze and calibrate the incident light in combination with pre-calibrated calibration parameters and m spatial signals corresponding to the incident light to obtain the spectral components of the incident light in each calibration color band; where m is an integer greater than or equal to 3.

[0198] Specifically, in an optional embodiment of the present application, step S4 may further include:

[0199] S401, determining m photoelectric converters that share a target asymmetric optical element;

[0200] S402, selecting m calibration color bands for decomposing incident light of unknown color from a plurality of color bands in the visible spectrum range;

[0201] S403, constructing a linear equation system of m variables based on spatial signal responses of incident light on m photoelectric converters sharing a target asymmetric optical element according to the m calibration color bands;

[0202] S404, simplifying the m-variable linear equation system according to the spatial signal response corresponding to the single calibration color band, and solving the m-variable linear equation system to obtain a calibration coefficient matrix corresponding to the m-variable linear equation system;

[0203] S405. For incident light of unknown color, based on the actual spatial signal responses of the incident light by m photoelectric converters sharing the same asymmetric optical element and the inverse matrix corresponding to the calibration coefficient matrix, the incident light is parsed into spectral components corresponding to the m calibration color bands, thereby obtaining the spectrum of the imaging pixel corresponding to the target asymmetric optical element.

[0204] The target asymmetric optical element refers to an asymmetric optical element corresponding to an imaging pixel used for image acquisition.

[0205] It is understandable that the detailed steps of the above-mentioned image sensor operation method (or the above-mentioned image acquisition method) can be analyzed with reference to the above-mentioned pixel array and the above-mentioned image sensor operation process, and will not be repeated here.

[0206] It should be noted that the various embodiments in the present application are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other.

[0207] Accordingly, an embodiment of the present application further provides an electronic device, which may be a terminal, such as a smartphone, a tablet computer, a laptop computer, a touch screen, a game console, a personal computer (PC), a personal digital assistant (PDA), or the like. Alternatively, the electronic device may be a server.

[0208] The electronic device includes a processor having one or more processing cores, a memory having one or more computer-readable storage media, and a computer program stored in the memory and executable on the processor. The processor is electrically connected to the memory. Those skilled in the art will appreciate that the electronic device structure shown in the figures does not limit the electronic device and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0209] The processor is the control center of an electronic device. It connects the various parts of the entire electronic device using various interfaces and lines. It executes various functions of the electronic device and processes data by running or loading software programs and / or units stored in the memory and calling data stored in the memory. The processor can be a processor CPU, graphics processor GPU, network processor (NP), etc., and can implement or execute the various methods, steps, and logic blocks disclosed in the embodiments of this application.

[0210] In an embodiment of the present application, a processor in an electronic device loads instructions corresponding to one or more application processes into a memory according to the following steps, and the processor runs the application stored in the memory to implement various functions, such as:

[0211] St1, the incident light is collected by the phase detection autofocus pixel to obtain the corresponding multiple phase detection spatial signals, where the incident light is formed by the imaging lens focusing the light;

[0212] St2. Detecting the phase difference or focus state of the incident light in multiple directions based on multiple phase detection spatial signals, and adjusting the relative position between the imaging lens and the image sensor based on the detection results in multiple directions to achieve focusing;

[0213] St3, collecting incident light through imaging pixels to obtain corresponding multiple spatial signals;

[0214] St4. For each imaging pixel, based on the multiple spatial signals and the multiple calibration color bands, resolve the incident light into spectral components corresponding to the multiple calibration color bands;

[0215] St5. For each phase detection autofocus pixel, interpolate the spectral components corresponding to several imaging pixels adjacent to the phase detection autofocus pixel to determine the spectral components corresponding to the phase detection autofocus pixel.

[0216] Furthermore, various functions implemented by running the application stored in the memory can also be described in the aforementioned embodiments and will not be repeated here.

[0217] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0218] Optionally, the electronic device further includes: a touch screen display, a radio frequency circuit, an audio circuit, an input unit, and a power supply. The processor is electrically connected to the touch screen display, the radio frequency circuit, the audio circuit, the input unit, and the power supply, respectively. Those skilled in the art will appreciate that the electronic device structure may include more or fewer components than shown, or may combine certain components or arrange the components differently.

[0219] A touchscreen display can be used to display a graphical user interface (GUI) and receive user commands generated by operating the GUI. A touchscreen display can include a display panel and a touch panel. The display panel can be used to display information input by or provided to the user, as well as various graphical user interfaces (GUIs) of the electronic device. These GUIs can consist of graphics, text, icons, videos, or any combination thereof.

[0220] Optionally, the display panel may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. The touch panel may be configured to collect user touch operations on or near the touch panel (e.g., operations performed by a user using a finger, stylus, or any other suitable object or accessory on or near the touch panel), generate corresponding operation instructions, and execute corresponding programs based on the operation instructions.

[0221] Optionally, the touch panel may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch direction and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor. It can also receive commands from the processor and execute them. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it transmits it to the processor to determine the type of touch event. The processor then provides corresponding visual output on the display panel based on the type of touch event.

[0222] In the embodiments of the present application, the touch panel and display panel can be integrated into a touch display screen to implement input and output functions. However, in some embodiments, the touch panel and display panel can be used as two independent components to implement input and output functions. In other words, the touch display screen can also be used as part of the input unit to implement input functions.

[0223] The radio frequency circuit can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other electronic devices through wireless communication, and to transmit and receive signals between network devices or other electronic devices.

[0224] The audio circuitry can be used for audio interface between the user and the electronic device through the speaker and the microphone. The audio circuitry can convert the received audio data into an electrical signal and transmit the electrical signal to the speaker for conversion into a sound signal and output; on the other hand, the microphone collects the sound signal and converts it into an electrical signal, which is received by the audio circuitry and converted into audio data, and then output to the processor for processing, and then transmitted to another electronic device through the radio frequency circuit, or output to the memory for further processing. The audio circuitry can also include an earphone jack to provide communication between the external earphone and the electronic device.

[0225] The input unit can be used to receive input light and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0226] The power supply is used to supply power to various components of the electronic device. Optionally, the power supply can be connected to the processor logic through the power management system, so that the power management system can realize the functions of managing charging, discharging and power consumption management. The power supply can also include one or more direct current or alternating current power sources, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and any other components.

[0227] The electronic device can also include a camera, a sensor, a wireless fidelity module, a Bluetooth module, and the like, which are not described here.

[0228] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0229] Those of ordinary skill in the art can understand that all or part of the steps of various methods in the above embodiments can be completed by instructions, or by instructions controlling related hardware, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0230] To this end, an embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, when the computer program is executed on an electronic device, the computer program is used to make the electronic device execute any one of the image acquisition methods provided by the embodiments of the present application. For example, the computer program can execute the steps of the following image acquisition method:

[0231] St01, the phase detection autofocus pixel acquires incident light to obtain a plurality of corresponding phase detection space signals, wherein the incident light is formed by converging light rays of an imaging lens;

[0232] St02, detecting phase difference or focusing state of incident light in multiple directions based on multiple phase detection spatial signals, and adjusting relative position between imaging lens and image sensor in combination with detection results in multiple directions to realize focusing;

[0233] St03, collecting incident light by imaging pixels to obtain corresponding multiple spatial signals;

[0234] St04, for each imaging pixel, based on multiple spatial signals and multiple calibration color bands, resolving incident light into spectral components corresponding to multiple calibration color bands;

[0235] St05, for each phase detection auto-focusing pixel, interpolating each spectral component corresponding to several imaging pixels of adjacent phase detection auto-focusing pixels to determine each spectral component corresponding to the phase detection auto-focusing pixel.

[0236] Further, the refinement steps of the above method steps can also refer to the description in the foregoing embodiments, which will not be repeated here.

[0237] The specific implementation of each operation can refer to the foregoing embodiments, which will not be repeated here.

[0238] The computer readable storage medium can include read only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0239] Due to the computer program stored in the computer readable storage medium, any image acquisition method provided by the embodiments of the present application can be executed, and thus the beneficial effects of any image acquisition method provided by the embodiments of the present application can be achieved. Details are described in the foregoing embodiments, which will not be repeated here.

[0240] According to an aspect of the present application, a computer program product is also provided, which includes a computer program stored in a computer readable storage medium; when a processor of an electronic device reads the computer program from the computer readable storage medium, the processor executes the computer program, so that the electronic device executes the method provided in various optional implementation manners in the foregoing embodiments.

[0241] In the computer-readable storage medium, electronic device, computer program product embodiments described above, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can refer to the relevant description of other embodiments. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the computer-readable storage medium, computer program product, electronic device and its corresponding unit described above and the beneficial effects brought by them can refer to the description of the image acquisition method in the above embodiments, and will not be repeated here.

[0242] The pixel array, image sensor, imaging system, image acquisition method, electronic device, computer-readable storage medium and computer program product provided by the embodiments of the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above embodiment description is only used to help understand the method and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the content of the specification should not be understood as a limitation of the present application.

Claims

1. A pixel array, characterized in that: The pixel array comprises: A photoelectric conversion subarray, comprising a plurality of photoelectric converters arranged in an array; an optical subarray, arranged on the photoelectric conversion subarray, the optical subarray comprising a plurality of symmetrical optical elements and a plurality of asymmetrical optical elements that are dispersed and staggered; Each of the symmetrical optical elements is disposed on a corresponding plurality of the photoelectric converters to form symmetrically arranged phase detection autofocus pixels; Each of the asymmetric optical elements is arranged on a corresponding plurality of the photoelectric converters to form asymmetrically arranged imaging pixels. The asymmetric optical element is used to focus and diffract the incident light to obtain a three-dimensional asymmetric light intensity pattern, which is recorded one-to-one on the corresponding plurality of the photoelectric converters to form a plurality of different spatial signals.

2. The pixel array according to claim 1, wherein: The symmetrical optical element includes a background structure layer and a symmetrical diffraction structure layer embedded in the background structure layer; Wherein, the refractive index of the background structure layer is lower than the refractive index of the symmetrical diffraction structure layer.

3. The pixel array according to claim 2, wherein: In the symmetrical optical element, the symmetrical diffraction structure layer and the background structure layer are coaxially arranged, and the axis of the symmetrical diffraction structure layer is aligned with the center of the array formed by the plurality of photoelectric converters corresponding to the symmetrical optical element.

4. The pixel array according to claim 1, wherein: The asymmetric optical element includes a background structure layer and an asymmetric diffraction structure layer embedded in the background structure layer; Wherein, the refractive index of the background structure layer is lower than the refractive index of the asymmetric diffraction structure layer.

5. The pixel array according to claim 4, wherein: In the asymmetric optical element, the asymmetric diffraction structure layer includes a plurality of diffraction cylinders of different sizes and a plurality of diffraction ring cylinders of different sizes. The plurality of diffraction cylinders and the plurality of diffraction ring cylinders are interlaced with each other and asymmetrically arranged inside the background structure layer, so that the diffraction of the incident light by the asymmetric diffraction structure layer is asymmetric diffraction, and the height of the diffraction cylinder and the height of the diffraction ring cylinder are both smaller than the height of the background structure layer.

6. The pixel array according to claim 1, wherein: The pixel array further includes a substrate, and a unit deep trench isolation structure disposed in the substrate; Wherein, the photoelectric conversion sub-array is arranged on the substrate; The multiple photoelectric converters corresponding to each optical element are surrounded by the unit deep trench isolation structure to prevent light crosstalk from adjacent optical elements and / or prevent electrical interference from adjacent photoelectric converters. The optical elements include the symmetrical optical element and the asymmetrical optical element.

7. The pixel array according to claim 1, wherein: The material of the symmetrical optical element and the material of the asymmetrical optical element are both inorganic materials.

8. The pixel array according to claim 1, wherein: The phase detection autofocus pixel is arranged in a centrally symmetrical or axially symmetrical manner. Within the phase detection autofocus pixel, the symmetrical optical element is used to focus and diffract the incident light to obtain a phase detection light intensity pattern, which is recorded one-to-one on the corresponding multiple photoelectric converters to form multiple phase detection spatial signals.

9. The pixel array according to claim 8, wherein: The phase detection autofocus pixel is configured to detect a phase difference or a focus state of the incident light in a direction perpendicular to the axis of symmetry based on a plurality of the phase detection spatial signals arranged in an axisymmetric manner.

10. The pixel array according to claim 1, wherein: The imaging pixel is configured to: on the basis of decomposing the incident light using multiple calibration color bands, combine the pre-calibration calibration parameters obtained by pre-calibration of the multiple calibration color bands, and the corresponding multiple spatial signals, perform analytical calibration on the incident light, and parse the incident light into spectral components corresponding to the multiple calibration color bands to perform color detection on the incident light.

11. The pixel array according to claim 10, wherein: For each imaging pixel, when performing color detection, a plurality of calibration color bands are first selected within the visible spectrum to meet the requirements of unfolding and decomposing the incident light of unknown color, and the number of the selected calibration color bands is the same as the number of the photoelectric converters in the imaging pixel.

12. An image sensor, characterized in that: The image sensor comprises the pixel array according to any one of claims 1 to 11, and is configured as follows: Detecting the phase difference or focus state of the incident light in multiple directions by the phase detection autofocus pixel, and adjusting the relative position between the incident light and the image sensor based on the detection results in multiple directions to achieve focus; and The incident light is collected by the imaging pixels, and within each imaging pixel, the incident light is resolved into multiple spectral components to achieve color detection of the incident light.

13. The image sensor according to claim 12, wherein: The image sensor is further configured to: For each of the phase detection autofocus pixels, interpolation is performed based on the spectral components corresponding to several imaging pixels adjacent to the phase detection autofocus pixel to determine the spectral components corresponding to the phase detection autofocus pixel, so as to achieve color detection of the incident light at the phase detection autofocus pixel.

14. An imaging system, characterized in that: The imaging system comprises: Imaging lens, used to converge light and obtain incident light; An image sensor comprising the pixel array according to any one of claims 1 to 11; In which, the image sensor is configured to: detect the phase difference or focusing state of the incident light in multiple directions through the phase detection autofocus pixel, and adjust the relative position between the imaging lens and the image sensor based on the detection results in multiple directions to achieve focusing; and collect the incident light through the imaging pixel, and within each of the imaging pixels, resolve the incident light into multiple spectral components to achieve color detection of the incident light.

15. The imaging system according to claim 14, wherein: The image sensor is further configured to: for each of the phase detection autofocus pixels, interpolate the spectral components corresponding to several imaging pixels adjacent to the phase detection autofocus pixel to determine the spectral components corresponding to the phase detection autofocus pixel, so as to achieve color detection of the incident light at the phase detection autofocus pixel.

16. An image acquisition method, characterized in that: Applied to the imaging system according to any one of claims 14-15, the method comprising: The incident light is collected by the phase detection autofocus pixel to obtain a corresponding plurality of phase detection spatial signals, wherein the incident light is formed by focusing light by the imaging lens; Detecting the phase difference or focus state of the incident light in multiple directions based on the multiple phase detection spatial signals, and adjusting the relative position between the imaging lens and the image sensor in combination with the detection results in the multiple directions to achieve focusing; The incident light is collected by imaging pixels to obtain corresponding multiple spatial signals; For each of the imaging pixels, based on the plurality of spatial signals and the plurality of calibration color bands, the incident light is resolved into spectral components corresponding to the plurality of calibration color bands; For each phase detection autofocus pixel, interpolation is performed based on spectral components corresponding to a number of imaging pixels adjacent to the phase detection autofocus pixel to determine spectral components corresponding to the phase detection autofocus pixel.

17. The image acquisition method according to claim 16, wherein: The step of resolving, for each of the imaging pixels, the incident light into spectral components corresponding to the plurality of calibration color bands based on the plurality of spatial signals and the plurality of calibration color bands comprises: For each of the imaging pixels, determining the number m of the photoelectric converters in the imaging pixel; For each imaging pixel, m calibration color bands are selected from the visible spectrum to decompose the incident light of unknown color; For each of the imaging pixels, pre-calibration is performed based on the m calibration ribbons to obtain pre-calibration parameters; For each of the imaging pixels, analytically calibrate the incident light in combination with the pre-calibrated calibration parameters and the m spatial signals corresponding to the incident light to obtain spectral components of the incident light in each of the calibration color bands; Here, m is an integer greater than or equal to 3.

Images