Pixel, image sensor, imaging system, calibration method, device and medium

Abstract

The application relates to a pixel, an image sensor, an imaging system, a calibration method, equipment and a medium, the pixel comprising: a photoelectric conversion module comprising n photoelectric converters arranged adjacently in an array form, wherein n is a positive integer greater than or equal to 3; and an asymmetric optical module arranged on the photoelectric conversion module, wherein the asymmetric optical module is used for focusing and diffracting incident light to generate a three-dimensional asymmetric light intensity pattern, and then forming n different spatial signals one by one on the n photoelectric converters in the photoelectric conversion module. The method can analyze the incident light into spectral components corresponding to a plurality of calibration color bands by calibration and demarcation based on associated different spatial signals on a plurality of photoelectric converters and the plurality of calibration color bands in a visible light spectrum range, realize brightness detection and color detection of the incident light, and effectively replace microlenses and optical filters in the pixel, and improve the sensitivity of the image sensor.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optoelectronic technology, and in particular to a pixel, an image sensor, an imaging system, a calibration method, a device and a medium. BACKGROUND

[0002] In an image sensor, in order to generate a color image, each pixel is equipped with one of a red filter, a green filter or a blue filter to transmit a corresponding color band and block the rest of the light in the visible light spectrum.

[0003] However, the image sensor of the related art has a narrow light band transmitted by each filter, and the filter absorbs part of the light, resulting in weakening of the light signal and reducing the sensitivity of the image sensor. SUMMARY

[0004] Embodiments of the present application provide a pixel, an image sensor, an imaging system, a calibration method, a device and a medium to solve the above problems.

[0005] To achieve the above purpose, according to a first aspect of the present application, a pixel is provided, comprising:

[0006] a photoelectric conversion module comprising n photoelectric converters arranged adjacent in an array, wherein n is a positive integer greater than or equal to 3;

[0007] an asymmetric optical module disposed on the photoelectric conversion module, wherein the asymmetric optical module is used to focus and diffract incident light to generate a three-dimensional asymmetric light intensity pattern, and then form n different spatial signals one by one on the n photoelectric converters in the photoelectric conversion module;

[0008] wherein the pixel is configured to, on the basis of decomposing the incident light with n calibration color bands, calibrate the incident light by using pre-calibration parameters obtained by pre-calibration of the n calibration color bands and the n spatial signals, to obtain spectral components of the incident light under each calibration color band.

[0009] Optionally, the photoelectric conversion module includes only one size of the photoelectric converters, and the n photoelectric converters in the photoelectric conversion module are arranged in a regular array.

[0010] Optionally, the photoelectric conversion module includes at least two different sizes of the photoelectric converters, and the n photoelectric converters in the photoelectric conversion module are arranged in an irregular array.

[0011] Optionally, the asymmetric optical module comprises:

[0012] a background structure layer composed of a first material having a first refractive index;

[0013] a diffraction structure layer embedded in the background structure layer, the diffraction structure layer comprising a plurality of components composed of a second material having a second refractive index for focusing diffraction of the incident light, the first refractive index being lower than the second refractive index.

[0014] Optionally, the first material and the second material are inorganic materials.

[0015] Optionally, the components comprise at least two different sizes of diffraction cylinders and at least two different sizes of diffraction circular cylinders, a plurality of the diffraction cylinders and a plurality of the diffraction circular cylinders being staggered and asymmetrically arranged, so that the diffraction of the incident light by the diffraction structure layer is asymmetric diffraction.

[0016] According to a second aspect of the present application, embodiments of the present application further provide an image sensor, the image sensor comprising a plurality of pixels arranged in an array;

[0017] The pixel comprises:

[0018] a photoelectric conversion module comprising n photoelectric converters arranged adjacent in an array, wherein n is a positive integer greater than or equal to 3;

[0019] an asymmetric optical module arranged on the photoelectric conversion module, wherein the asymmetric optical module is used for focusing diffraction of incident light to generate a three-dimensional asymmetric light intensity pattern, thereby forming n different spatial signals one by one on the n photoelectric converters in the photoelectric conversion module;

[0020] The pixel is configured to: on the basis of decomposing the incident light by n calibration color bands, calibrate the incident light by using pre-calibration parameters obtained by pre-calibration of the n calibration color bands and the n spatial signals, to obtain spectral components of the incident light under each calibration color band.

[0021] Optionally, the image sensor comprises only one type of pixel, and each pixel in the image sensor is arranged in a regular array.

[0022] Optionally, the image sensor comprises at least two types of pixels, and each pixel in the image sensor is arranged in a regular array.

[0023] Optionally, in the image sensor, n has only one value, but the array arrangement structures of the n photoelectric converters in at least two photoelectric conversion modules are different.

[0024] Optionally, in the image sensor, n has at least two different values, so that the array arrangement structures of the n photoelectric converters in at least two of the photoelectric conversion modules are different.

[0025] Optionally, in the image sensor, n has only one value, and the array arrangement structures of the n photoelectric converters in each of the photoelectric conversion modules are the same, but the asymmetric optical modules in at least two of the pixels are different.

[0026] Optionally, the pixel further comprises a back-illuminated silicon substrate, and the photoelectric conversion module is arranged on the back-illuminated silicon substrate, and a groove is arranged around the photoelectric conversion module on the back-illuminated silicon substrate.

[0027] According to a third aspect of the present application, the embodiments of the present application further provide an imaging system, the imaging system comprising:

[0028] an imaging lens, configured to converge and image light rays of an object to form incident light;

[0029] an image sensor, configured to diffract the incident light based on a pixel in the image sensor to generate a three-dimensional asymmetric light intensity pattern, and further configured to record n different spatial signals one by one on n photoelectric converters in the pixel; wherein n is a positive integer greater than or equal to 3.

[0030] The pixel is configured to: based on decomposing and expanding the incident light by n calibration color bands, perform analytical calibration on the incident light by using pre-calibration parameters obtained by pre-calibration of the n calibration color bands and the n spatial signals, to obtain spectral components of the incident light under each of the calibration color bands.

[0031] According to a fourth aspect of the present application, the embodiments of the present application further provide a calibration method of an image sensor, applied to an image sensor composed of a plurality of pixels, the method comprising:

[0032] selecting, for each of the pixels, n calibration color bands to decompose and expand incident light;

[0033] performing, for each of the pixels, pre-calibration based on the n calibration color bands to obtain pre-calibration parameters;

[0034] obtaining, for each of the pixels, n different spatial signals obtained by photoelectric collection of the incident light by the pixel;

[0035] performing, for each of the pixels, analytical calibration on the incident light by using the pre-calibration parameters and the n spatial signals, to calculate spectral components of the incident light under each of the calibration color bands.

[0036] wherein the pixel comprises:

[0037] a photoelectric conversion module comprising n photoelectric converters arranged adjacently in an array, wherein n is a positive integer greater than or equal to 3;

[0038] an asymmetric optical module disposed on the photoelectric conversion module, wherein the asymmetric optical module is configured to focus-diffraction the incident light to generate a three-dimensional asymmetric light intensity pattern, and further to record and form n different spatial signals one by one on the n photoelectric converters in the photoelectric conversion module.

[0039] Optionally, the selected n calibration color bands at least include a red spectral color band, a green spectral color band, and a blue spectral color band to meet the unfolding decomposition requirement of the unknown color incident light.

[0040] Optionally, the pre-calibration calibration parameter of each pixel based on the n calibration color bands comprises:

[0041] For each pixel, based on the spatial signal response of the incident light unfolded and decomposed by the n calibration color bands on the n photoelectric converters in the pixel, an n-order linear equation set is constructed, wherein the n-order linear equation set is composed of a calibration coefficient matrix, a color band fraction vector, and a spatial signal vector.

[0042] For each pixel, the n-order linear equation set is simplified and calculated based on the spatial signal response of a single calibration color band to obtain a calibration coefficient vector of the single calibration color band, and the calibration coefficient vectors of the n calibration color bands are integrated to obtain the calibration coefficient matrix.

[0043] For each pixel, the calibration coefficient matrix is subjected to an inverse operation to obtain an inverse matrix of the calibration coefficient matrix, and the inverse matrix of the calibration coefficient matrix is the pre-calibration calibration parameter.

[0044] Optionally, the matrix form of the n-order linear equation set is as follows:

[0045] ,

[0046] wherein, C the n x n calibration coefficient matrix is composed of calibration coefficient vectors corresponding to the n calibration color bands; X the color band fraction vector is a color band fraction vector that contributes to the measured spatial signal in the incident light; MThe n spatial signals obtained by the incident light measurement form a spatial signal vector for the spatial signal.

[0047] Optionally, the spectral components of the incident light under each calibration color band are calculated by analytically calibrating the incident light with the pre-calibration parameters and the n spatial signals for each pixel, including:

[0048] The analytically calibration formula is obtained by deducing based on the n-order linear equation set for each pixel.

[0049]

[0050] wherein, C -1 is an inverse matrix of the calibration coefficient matrix;

[0051] The spatial signal vector is constructed based on the n spatial signals for each pixel, and the inverse matrix of the calibration coefficient matrix and the spatial signal vector are brought into the analytically calibration formula to calculate the color band fraction vector, thereby obtaining the spectral components of the incident light under each calibration color band.

[0052] According to a fifth aspect of the present application, the embodiments of the present application further provide a photoelectric device, including:

[0053] a memory having a computer program stored thereon;

[0054] a processor configured to execute the computer program in the memory to implement the steps of any of the methods provided by the embodiments of the present application.

[0055] According to a sixth aspect of the present application, the embodiments of the present application further provide a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of any of the methods provided by the embodiments of the present application.

[0056] Some embodiments of the present application have at least the following beneficial effects: based on the associated different spatial signals on multiple photoelectric converters and multiple calibration color bands in the visible spectrum range for calibration, the incident light is analytically resolved into spectral components corresponding to the multiple calibration color bands, the brightness detection and color detection of the incident light are realized, the asymmetric optical module and the multi-color band calibration technology can replace the microlens and the filter of the pixel in related technologies, and the problems of reduced white light broadband sensitivity and reduced spectral resolution caused by the microlens and the filter in the pixel can be effectively avoided.

[0057] Other features and advantages of the present application will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF DRAWINGS ​

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0059] In order to more completely understand the present application and its beneficial effects, the following will be described in conjunction with the drawings, wherein the same reference numerals in the following description represent the same parts.

[0060] Figure 1 is a structural schematic diagram of a pixel in a traditional image sensor;

[0061] Figure 2 is a structural schematic diagram of a traditional RGGB image sensor;

[0062] Figure 3 is a comparison schematic diagram of quantum efficiency (QE) curves in the visible spectral range of a monochrome image sensor and a traditional RGGB image sensor;

[0063] Figure 4 is a structural schematic diagram of an imaging system in the related art;

[0064] Figure 5 is an imaging lens chief ray angle distribution curve diagram of an imaging system in the related art;

[0065] Figure 6 is a process flow schematic diagram of a traditional RGGB image sensor;

[0066] Figure 7 is a structural schematic diagram of a pixel in an optional embodiment of the present application;

[0067] Figure 8 is a three-dimensional view of a pixel in an optional embodiment of the present application;

[0068] Figure 9 is a schematic diagram of asymmetric diffraction of different incident lights by an asymmetric optical module in an optional embodiment of the present application;

[0069] Figure 10 is a schematic diagram of four selected calibration color bands in an optional embodiment of the present application;

[0070] Figure 11 is a calibration and marking flow schematic diagram of a pixel in an optional embodiment of the present application;

[0071] Figure 12 is a structural schematic diagram of a pixel in an optional embodiment of the present application;

[0072] Figure 13 is a schematic structural diagram of another pixel in an optional embodiment of the present application;

[0073] Figure 14 is a schematic structural diagram of another pixel in an optional embodiment of the present application;

[0074] Figure 15 This is a schematic structural diagram of an imaging system in an optional embodiment of the present application;

[0075] Figure 16 is a graph showing the distribution of the chief ray angle of the imaging lens of the imaging system in an optional embodiment of the present application;

[0076] Figure 17 This is a schematic structural diagram of an optoelectronic device in an optional embodiment of the present application. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0078] To facilitate understanding of the implementation scheme provided in the embodiments of the present application, the relevant application background of the technical solutions such as pixels, image sensors, imaging systems, image sensor calibration methods, optoelectronic devices and computer-readable storage media provided in the embodiments of the present application is first explained.

[0079] like Figure 1 As shown in the figure, in a traditional image sensor, each pixel includes at least a microlens, a filter and a photoelectric converter (such as a photodiode). The microlens is used to converge light to ensure that the light can effectively pass through the filter and illuminate the photoelectric converter, thereby improving the light collection efficiency of the photoelectric converter. The photoelectric converter generates charge or current based on the incident light. In order to produce a color image, each pixel is equipped with a red filter, a green filter or a blue filter, which transmits the corresponding color band and blocks the rest of the light in the visible spectrum. For example, the filter can be used as follows Figure 2 The RGGB pattern (also called the Bayer pattern) shown is repeatedly arranged in the image sensor.

[0080] Since each filter transmits only a narrow band of spectrum, while the rest of the light is blocked. Therefore, at any given photoelectric converter location or pixel location, only one of the color band signals transmitted by the filter can be recorded. In an optional embodiment of the present application, a monochrome image sensor without any color filter is compared with a conventional RGGB image sensor, and the Quantum Efficiency (QE) curves of both in the visible spectrum range are obtained as shown in Figure 3 It can be seen from Figure 3 that the spectral filtering of the conventional RGGB image sensor reduces the broadband sensitivity of white light by about 3-4 times compared with the monochrome image sensor without any color filter.

[0081] At the same time, in order to infer the block color signal of any given pixel location in the RGGB image sensor, the signals of the adjacent color pixels must be interpolated (also known as demosaicing) to obtain the block color signal of the given location, which reduces the spectral resolution of the color image sensor.

[0082] As shown in Figures 4-5 , in many existing application scenarios, the imaging system is equipped with a multi-element imaging lens, and the imaging lens sends the incident light of the imaging to the image sensor at a very high angle (up to 35 degrees of chief ray angle). The pixel in the related art image sensor has a pixel stack height of 2-4 pm between the top surface of the microlens and the photosensitive substrate interface, and the light incident to the microlens at a high angle is focused by the microlens and experiences a large displacement during the propagation through the pixel stack. Therefore, in each pixel, the microlens and the filter need to be shifted relative to the photoelectric converter to collect light into the appropriate photoelectric converter.

[0083] Specifically, as shown in Figure 4 and Figure 5 , the displacement of the microlens and the filter relative to the photoelectric converter depends on many factors, such as the pixel stack height, the number of pixel stack layers, the optical properties of the materials, the pixel size, the chief ray angle distribution curve of the imaging lens of the image sensor, and the focal length f of the imaging lens. Therefore, the design of the offset of the microlens and the filter in the image sensor for a specific imaging lens is usually a very long and expensive process. Any modification of the imaging lens design, the pixel size, or the pixel stack height also requires additional costs because the manufacturing process has changed substantially and requires very high stability to verify the new process for producing high-yield parts.

[0084] Therefore, it would be highly advantageous to significantly reduce the cost of image sensors by reducing the pixel stack, and by ideally avoiding the necessity of shifting any portion of the pixel stack. During the imaging system assembly phase, the multi-element imaging lens needs to be three-dimensionally aligned with the image sensor array to achieve lateral coincidence of the imaging lens and image sensor optical centers. In addition, it is also critical to prevent the imaging lens from tilting with respect to the sensor plane, as an image sensor with misaligned microlenses and filters can sense light signals at the correct angle of incidence. Therefore, it is desirable to provide an image sensor that can tolerate misalignment of the imaging lens with the image sensor to reduce the process complexity.

[0085] Furthermore, the deposition process of microlenses and color filters based on non-CMOS organic polymer materials requires additional non-CMOS production equipment for setting up the absorptive filters on the image sensor array. In an alternative embodiment of the present application, the number of process fabrication steps, production cycle, and production yield of the existing RGGB image sensor as shown in Figure 6

[0086] As shown in Figure 6 the addition of the non-CMOS process fabrication prolongs the production flow of the image sensor, and the yield of the non-CMOS production equipment is low, therefore, the addition of the non-CMOS process fabrication increases the cost of the image sensor. At the same time, the conventional polymer-based absorptive filters are also susceptible to weathering, and have a shorter lifetime than inorganic materials that use CMOS production equipment. Therefore, it is desirable to incorporate the color imaging function of the image sensor into the CMOS fabrication process to reduce the cost of the image sensor and to improve the yield of the image sensor.

[0087] ​In view of this, the embodiment of the present application proposes a kind of image sensor based on pure CMOS process: design in pixel unit, in each pixel, based on the inorganic material of at least two different refractive indexes design n photoelectric converter shared asymmetric optical module, focusing diffraction is carried out to incident light, generate three-dimensional asymmetric light intensity pattern, and then based on three-dimensional asymmetric light intensity pattern on n photoelectric converter record formation different spatial signal;Three-dimensional asymmetric light intensity pattern is formed by the light diffraction and subsequent self-interference that occur on asymmetric optical module, and the interference property of light makes each spatial signal detected highly correlated, so that the narrow spectral band (hereinafter referred to as color band) of various colors is formed one by one n different signal component distribution on n photoelectric converter, so that n representative calibration color band can be selected in the visible spectral range, and the incident light of unknown color is expanded into the sum of the components of n calibration color band, and then based on the spatial signal response of the incident light expanded and decomposed by n calibration color band on n photoelectric converter, construct n primary linear equation group, wherein the n primary linear equation group is composed of calibration coefficient matrix, color band score vector and spatial signal vector, and calibration coefficient matrix can be calibrated by independent spatial signal of each calibration color band.

[0088] Therefore, based on the n primary linear equation group, the calibration coefficient matrix obtained by pre-calibration and the n spatial signals measured in real time can be used to solve the color band components of the incident light, and the incident light can be analyzed into the spectral components corresponding to the n calibration color bands, so that the brightness detection of the incident light is realized, and the color measurement of the incident light is realized. By combining asymmetric optical module and multi-color band calibration technology, the microlens and filter in the existing pixel are replaced, which helps to reduce the production cost and process complexity of image sensor, and improve the sensitivity and spectral resolution of incident light detection.

[0089] Wherein, n is a positive integer greater than or equal to 3;Select a plurality of different representative calibration color bands in the full range of visible spectral range. For example, representative calibration color bands can be selected in the narrow spectral band corresponding to each different color, and exemplarily, at least three calibration color bands of red, green and blue are covered, i.e. the selected calibration color bands include at least red spectral color band, green spectral color band and blue spectral color band, to meet the expansion and decomposition requirements of the incident light of unknown color.

[0090] Firstly, the embodiment of the present application provides a kind of pixel, pixel includes:

[0091] Photoelectric conversion module, including n photoelectric converters adjacent arrangement in array form, wherein, n is a positive integer greater than or equal to 3;

[0092] an asymmetric optical module, disposed on the photoelectric conversion module, wherein the asymmetric optical module is used to focus and diffract incident light to generate a three-dimensional asymmetric light intensity pattern, and then record the n different spatial signals on the n photoelectric converters in the photoelectric conversion module in a one-to-one correspondence;

[0093] The pixels are configured as follows: on the basis of decomposing the incident light using n calibration color bands, the incident light is analytically calibrated using pre-calibrated calibration parameters and n spatial signals obtained by pre-calibration of the n calibration color bands to obtain the spectral components of the incident light under each calibration color band.

[0094] It should be noted that the three-dimensional asymmetric light intensity pattern refers to the three-dimensional asymmetric light intensity pattern formed in the three-dimensional space on the photoelectric converter after the incident light is asymmetricly focused and diffracted by the asymmetric optical module. The three-dimensional asymmetric light intensity pattern is then projected onto the photosensitive plane of the photoelectric converter, forming a two-dimensional asymmetric light intensity distribution on each photoelectric converter at different spatial positions in the photosensitive plane. The light intensity distribution collected on the photoelectric converters at different positions is different, and then based on the photoelectric effect, the light is recorded one by one on n photoelectric converters to form n spatial signals of different sizes.

[0095] For example, Figures 7-8 As shown, n is 4, and the asymmetric optical module is disposed on a photoelectric conversion module composed of four photoelectric converters. For example, the asymmetric optical module can be disposed on corresponding 2×2 photoelectric converters. The asymmetric optical module focuses and diffracts the incident light to generate a three-dimensional asymmetric light intensity pattern, which is recorded on the 2×2 photoelectric converters within the photoelectric conversion module to form four different spatial signals.

[0096] In some embodiments, the asymmetric optical module comprises:

[0097] A background structure layer, the background structure layer is made of a first material having a first refractive index;

[0098] The diffraction structure layer is embedded in the background structure layer, and the diffraction structure layer includes a plurality of components made of a second material with a second refractive index for focusing and diffracting incident light, and the first refractive index is lower than the second refractive index.

[0099] It is understood that the geometric shape, size, material properties, and arrangement of the components can be flexibly designed according to actual needs to achieve asymmetric focusing and diffraction of the incident light. The incident light can be the light emitted directly by the light source or the light reflected from the object after the light source emits the light, which will not be described in detail here.

[0100] In some embodiments, the component may include a cylindrical, rectangular, V-shaped, annular, or the like.

[0101] In some embodiments, the components can be arranged in a periodic, quasi-periodic, or random manner.

[0102] In some embodiments, the components include at least two different sizes of diffractive cylinders and at least two different sizes of diffractive circular cylinders, and the plurality of diffractive cylinders and the plurality of diffractive circular cylinders are interleaved and asymmetrically arranged, so that the diffractive structure layer diffracts incident light asymmetrically.

[0103] As shown in the example, Figures 7-8 The asymmetric optical module includes:

[0104] a background structure layer having a first refractive index;

[0105] an asymmetric diffractive structure layer having a second refractive index, the asymmetric diffractive structure layer being embedded in the background structure layer, and the second refractive index being different from the first refractive index.

[0106] As shown in the example, Figures 7-8 The asymmetric diffractive structure layer is a transparent structure layer and is embedded at the bottom of the background structure layer, and the asymmetric diffractive structure layer includes a plurality of diffractive cylinders and a plurality of diffractive circular cylinders, the plurality of different sizes of diffractive cylinders and the plurality of different sizes of diffractive circular cylinders are interleaved, so that the asymmetric diffractive structure layer diffracts incident light asymmetrically.

[0107] As shown in the example, Figures 7-8 The diameters of the plurality of diffractive cylinders are different, the inner diameters and the outer diameters of the plurality of diffractive circular cylinders are different, and the heights of the diffractive cylinders and the heights of the diffractive circular cylinders are less than the height of the background structure layer.

[0108] In some embodiments, the first material and the second material are inorganic materials.

[0109] It should be noted that the asymmetric optical module is completely made of inorganic materials, and compared with the traditional image sensor using organic materials, since inorganic materials are used, CMOS process flow such as organic materials is not involved, and the entire image sensor can be manufactured on a CMOS wafer factory, thereby reducing the cost of the image sensor. At the same time, due to the process precision and harsh requirements of organic materials, the yield of the traditional image sensor with microlenses and optical filters is relatively low, about 80%, while the yield of the CMOS image sensor of all inorganic materials in the embodiment is very high, close to 100%, and the compatibility between inorganic materials is better, and the possibility of further reducing the dark current by eliminating non-CMOS materials is further reduced, so that the all-CMOS image sensor provides higher life and higher reliability.

[0110] The background structure layer can be made of a low refractive index material, such as silicon oxide with a refractive index of 1.46, or air with a refractive index of 1; and the asymmetric diffraction structure layer can be made of a high refractive index material, such as silicon nitride, titanium oxide, tantalum oxide, silicon carbide, and silicon, etc., with a refractive index in the range of 1.8-4.

[0111] It can be understood that, in other optional embodiments, the asymmetric optical module can be composed of three or more inorganic materials, without limitation, such as a first background layer with a low refractive index, a second background layer with a medium refractive index, and an asymmetric diffraction structure layer with a high refractive index, which are sequentially embedded one inside another from inside to outside; or a first background layer with a low refractive index, a first asymmetric diffraction structure layer with a medium refractive index embedded in the first background layer, and a second asymmetric diffraction structure layer with a high refractive index embedded in the first background layer, etc.

[0112] In some embodiments, the photoelectric conversion module includes only one size of photoelectric converter, and the n photoelectric converters in the photoelectric conversion module are arranged in a regular array.

[0113] For example, as shown in FIG. 2, the pixel enclosed by the dashed line in FIG. 2 includes four photoelectric converters of the same size, which are arranged in a 2x2 regular array. Figures 7-8 Figure 7 For example, as shown in FIG. 2, the pixel enclosed by the dashed line in FIG. 2 includes four photoelectric converters of the same size, which are arranged in a 2x2 regular array.

[0114] For example, as shown in FIG. 3, the pixel enclosed by the dashed line in FIG. 3 includes three photoelectric converters of the same size, which are arranged in a 1x3 regular array. Figure 12 Figure 12 For example, as shown in FIG. 3, the pixel enclosed by the dashed line in FIG. 3 includes three photoelectric converters of the same size, which are arranged in a 1x3 regular array.

[0115] In some embodiments, the regular array can be linear arrangement, two-dimensional matrix arrangement, etc.

[0116] For example, as shown in FIG. 4, the four photoelectric converters of the same size in the photoelectric conversion module corresponding to the pixel are arranged in a 2x2 two-dimensional matrix. Figures 7-8 For example, as shown in FIG. 4, the four photoelectric converters of the same size in the photoelectric conversion module corresponding to the pixel are arranged in a 2x2 two-dimensional matrix.

[0117] Figure 12 For example, as shown in FIG. 5, the three photoelectric converters of the same size in the photoelectric conversion module corresponding to the pixel are arranged in a 1x3 matrix, which can also be said to be linear arrangement in the horizontal direction.

[0118] In some embodiments, the photoelectric conversion module includes at least two different sizes of photoelectric converters, and the n photoelectric converters in the photoelectric conversion module are arranged in an irregular array. ​​​

[0119] As shown in Figures 13-14 As shown in Figure 13 As shown in the pixel in the upper left corner of the dashed box in FIG. 1B and the pixel in the upper left corner of the dashed box in FIG. 1C, the photoelectric conversion module corresponding to the pixel includes 1 first rectangular photoelectric converter and 2 second rectangular photoelectric converters, and the 3 photoelectric converters are arranged in an irregular array; and the photoelectric conversion module in at least some of the pixels includes only one size of photoelectric converter, as shown in the pixel in the lower right corner of the dashed box in FIG. 1D, the 4 photoelectric converters in the photoelectric conversion module corresponding to the pixel are of the same size, which are square photoelectric converters, and the corresponding photosensitive surface is designed in a square shape, and the 4 photoelectric converters are arranged in a 2x2 regular array. Figure 14 Figure 14 In some embodiments, the specific size or irregular arrangement of each photoelectric converter in the pixel can be customized according to specific needs, for example, larger size photoelectric converters are arranged in certain areas to improve local sensitivity, or smaller size photoelectric converters are used in other areas to achieve higher spatial resolution.

[0120] In some embodiments, the specific size or irregular arrangement of each photoelectric converter in the pixel can be customized according to specific needs, for example, larger size photoelectric converters are arranged in certain areas to improve local sensitivity, or smaller size photoelectric converters are used in other areas to achieve higher spatial resolution.

[0121] Secondly, the embodiments of the present application also provide an image sensor, the image sensor comprising a plurality of pixels arranged in an array;

[0122] The pixel comprises:

[0123] The photoelectric conversion module comprises n photoelectric converters arranged adjacent in an array, wherein n is a positive integer greater than or equal to 3;

[0124] The asymmetric optical module is arranged on the photoelectric conversion module, wherein the asymmetric optical module is used to focus and diffract the incident light to generate a three-dimensional asymmetric light intensity pattern, and then form n different spatial signals one by one on the n photoelectric converters in the photoelectric conversion module;

[0125] The pixel is configured to: on the basis of decomposing and expanding the incident light by the n calibration color bands, calibrate the incident light by the pre-calibration parameters obtained by the pre-calibration and calibration of the n calibration color bands and the n spatial signals to obtain the spectral components of the incident light under each calibration color band.

[0126] As shown in Figures 7-8 The image sensor comprises:

[0127] The photoelectric conversion array comprises a plurality of photoelectric conversion modules arranged in an array;

[0128] ​The asymmetric optical array includes a plurality of asymmetric optical modules arranged on the photoelectric conversion array and arranged in an array, each asymmetric optical module is arranged on a corresponding photoelectric conversion module, n is 4, each photoelectric conversion module includes 4 photoelectric converters arranged in a 2*2 regular array, the asymmetric optical module focuses and diffracts incident light to generate a three-dimensional asymmetric light intensity pattern to record and form 4 different spatial signals on the 2*2 photoelectric converters in the corresponding photoelectric conversion module.

[0129] As shown in the figure, 2*2 photoelectric converters and an asymmetric optical module thereon constitute a pixel (which can also be referred to as a pixel unit), the image sensor includes a plurality of pixels arranged in an array, incident light forms a three-dimensional asymmetric light intensity pattern in each pixel based on the asymmetric optical module, and then forms 4 different spatial signals one by one on the 4 photoelectric converters, and in combination with 4 monochromatic calibration color bands in the subsequent visible light spectrum range, the incident light in each pixel can be effectively detected in brightness and color, and the incident light can be analyzed into spectral components corresponding to the 4 calibration color bands. Figures 7-8

[0130] It should be noted that, compared with the traditional RGGB sensor, in the above image sensor, the single color band component remaining after filtering out multiple color components by the color filter is no longer transmitted to the photoelectric converter, but full-spectrum color band component transmission is performed based on the asymmetric optical module, and the photosensitivity in the visible light spectrum range is increased by 2-3 times. At the same time, the light and dark detection and color detection of the incident light are performed based on the pre-calibration and measured spatial signals of the multi-color band components, and interpolation calculation based on adjacent pixels is not needed, which helps to improve the spectral resolution of the image sensor.

[0131] It can be understood that, since the pixel in the embodiment of the present application can replace the microlens and the filter in the existing pixel by combining the asymmetric optical module and the multi-color band calibration technology, the pixel in the embodiment of the present application is not limited by the structure size of the microlens and the filter, especially not limited by the structure size of the microlens, and the corresponding pixel size can be adaptively reduced.

[0132] It should be noted that, in addition to the above photoelectric conversion array and the above asymmetric optical array, the image sensor also includes other structures such as pixel processing circuit, controller and image processor, and details can be referred to the prior art, which will not be described here.

[0133] In some embodiments, the image sensor only includes a pixel of one structure, and each pixel in the image sensor is arranged in a regular array.

[0134] As shown in the figure, 2*2 photoelectric converters and an asymmetric optical module thereon constitute a pixel (which can also be referred to as a pixel unit), the image sensor includes a plurality of pixels arranged in an array, incident light forms a three-dimensional asymmetric light intensity pattern in each pixel based on the asymmetric optical module, and then forms 4 different spatial signals one by one on the 4 photoelectric converters, and in combination with 4 monochromatic calibration color bands in the subsequent visible light spectrum range, the incident light in each pixel can be effectively detected in brightness and color, and the incident light can be analyzed into spectral components corresponding to the 4 calibration color bands. Figure 7 Figure 12 or Figure 13 ​​As shown, the image sensor includes a plurality of pixels, each pixel has the same structure, i.e. the photoelectric conversion modules in each pixel have the same structure, and the asymmetric optical modules in each pixel have the same structure, and each pixel in the image sensor is arranged in a regular two-dimensional matrix.

[0135] In some embodiments, the image sensor includes at least two different structures of pixels, and each pixel in the image sensor is arranged in a regular array.

[0136] For example, the image sensor includes a plurality of pixels, at least two pixels have different structures, the photoelectric conversion modules in the two pixels have different structures, and / or the asymmetric optical modules have different structures.

[0137] In some embodiments, in the image sensor, n has only one value, but the array arrangement structures of the n photoelectric converters in the at least two photoelectric conversion modules are different.

[0138] For example, the image sensor includes a plurality of pixels, each pixel includes the same number of photoelectric converters, each pixel has the same structure of asymmetric optical module, but the photoelectric conversion modules in some pixels have different structures, i.e. the array arrangement structures of the plurality of photoelectric converters in some pixels are different, such as in the image sensor, the value of n in each pixel is 3, the structure of the photoelectric conversion module in some pixels is as shown in Figure 12 which is formed by 3 rectangular photoelectric converters of the same size arranged in a 1x3 array. Figure 13 which is formed by 3 rectangular photoelectric converters of two different sizes arranged in an irregular array.

[0139] In some embodiments, in the image sensor, n has at least two different values, so that the array arrangement structures of the n photoelectric converters in the at least two photoelectric conversion modules are different.

[0140] For example, the image sensor includes a plurality of pixels, the number of photoelectric converters included in at least two pixels is different, and the array arrangement structures of the plurality of photoelectric converters in the two pixels are different, such as Figure 14 As shown, the value of n in some pixels is 3, and the value of n in some pixels is 4, i.e. even if the structure of the asymmetric optical module in each pixel is the same, but under the premise that the size of the photoelectric conversion module is the same, since the number of photoelectric converters included in some pixels is different, the array arrangement structures of each photoelectric converter in the photoelectric conversion module in the corresponding pixel must be different.

[0141] In some embodiments, in the image sensor, n has only one value, and the array arrangement structure of the n photoelectric converters in each photoelectric conversion module is the same, but the asymmetric optical modules in at least two pixels are different.

[0142] For example, the image sensor includes a plurality of pixels, the number n of photoelectric converters included in each pixel is the same, and the array arrangement structure of the n photoelectric converters in the photoelectric conversion module corresponding to each pixel is the same, but the asymmetric optical modules in at least two pixels are different, such as Figure 13 The asymmetric optical module in some pixels shown is rotated 90° to the right in the vertical projection plane of the incident light. The focus diffraction orientation of the rotated asymmetric optical module is different from that of the unrotated asymmetric optical module. In this way, two different asymmetric optical modules are obtained, or the focus diffraction orientation can be changed directly based on the transformation of the shape and material of the internal components. Figure 13 The structure of the asymmetric optical module in a portion of the pixel is shown.

[0143] In some embodiments, the number n of photoelectric converters in a pixel is 4, and a pixel is formed based on a photoelectric conversion module composed of 2×2 photoelectric converters and an asymmetric optical module. Four calibration color bands, R, G1, G2, and B, are uniformly selected in the visible spectrum range. Pre-calibration is performed based on the spatial signal response corresponding to the single-color calibration color bands, and then the incident light of unknown color can be subsequently parsed into spectral components corresponding to the four calibration color bands. This is suitable for image sensors in which a single photoelectric converter is arranged in a square and multiple photoelectric converters are regularly arranged in an array, such as Figure 7 The 4×6 photoelectric conversion array shown includes 24 photoelectric converters arranged in a 4×6 array, which can be configured as 2×3 pixels. The 2×2 photoelectric converters correspondingly form an asymmetric optical module.

[0144] In addition, if Figure 7 The 4×6 photoelectric conversion array shown can also be configured into 1×2 pixels, with an asymmetric optical module correspondingly formed on 3×3 photoelectric converters to form a pixel. As the photoelectric conversion array expands, an asymmetric optical module can also be formed correspondingly on 4×4, 5×5, ..., N×N photoelectric converters to form a pixel.

[0145] Correspondingly, it is necessary to select 3×3, 4×4, 5×5, ..., N×N calibration color bands within the visible spectrum, and the calibration color bands must cover at least the three primary colors of red, green, and blue to effectively meet the color detection requirements for incident light of various unknown colors. Where N is an integer greater than or equal to 2.

[0146] It can be understood that the more uniform and the more the color distribution of the calibration color band, the more detailed and the higher the accuracy of the color decomposition and analysis of the incident light, but the corresponding calculation is large, which can be selected according to the actual situation.

[0147] Of course, the 4x6 photoelectric conversion array as shown in Figure 7 may also be configured into 2x2 pixels, and a 2x3 photoelectric converter in a pixel is correspondingly formed into an asymmetric optical module to form a pixel, wherein the shape of the asymmetric optical module needs to be correspondingly adjusted, and the diffraction focusing effect is poor in some directions or on some photoelectric converters.

[0148] Therefore, in another embodiment of the present application, as shown in Figure 12 , for an image sensor in which a plurality of photoelectric converters are regularly arranged in an array and each photoelectric converter is rectangular, a 1x3 photoelectric converter as shown in the dashed box can be correspondingly formed into an asymmetric optical module, and three different calibration color bands need to be selected in the visible light spectrum range, such as a 600nm red spectral color band, a 525nm green spectral color band, and a 450nm blue spectral color band for pre-calibration and calibration to obtain the inverse matrix C -1 of the corresponding calibration coefficient matrix C. The specific calibration process can be referred to in the following embodiments, which will not be described here.

[0149] In some embodiments, as shown in Figure 7 or Figure 12 , the size of each photoelectric converter in the image sensor is consistent and the plurality of photoelectric converters are regularly arranged in an array.

[0150] In some embodiments, there can be at least two different sizes of photoelectric converters in the image sensor, and the plurality of photoelectric converters are arranged in a vertical and horizontal staggered manner, as shown in Figure 13 and Figure 14 .

[0151] In another embodiment of the present application, as shown in Figure 13 , for a plurality of photoelectric converters arranged in a vertical and horizontal staggered manner, a 3x3 photoelectric converter as shown in the dashed box can be correspondingly formed into an asymmetric optical module to form a pixel, and three different calibration color bands need to be selected in the visible light spectrum range, such as a 600nm red spectral color band, a 525nm green spectral color band, and a 450nm blue spectral color band for pre-calibration and calibration to obtain the inverse matrix C -1 of the corresponding calibration coefficient matrix C. The specific calibration process can be referred to in the following embodiments, which will not be described here.

[0152] In another embodiment of the present application, as shown in Figure 14As shown, for the plurality of photoelectric converters arranged in part in a longitudinal and transverse staggered manner and in part in an array regular manner, a first asymmetric optical module can be formed on the 3 photoelectric converters shown in the first dashed box, and a second asymmetric optical module can be formed on the 2x2 photoelectric converters shown in the second dashed box. The shape structure of the first asymmetric optical module can be the same as or different from that of the second asymmetric optical module. In this way, the first pixel is formed based on the 3 photoelectric converters and the first asymmetric optical module shared thereon, and the second pixel is formed based on the 2x2 photoelectric converters and the second asymmetric optical module shared thereon.

[0153] The first pixel needs to select 3 different calibration color bands in the visible light spectrum range, and the second pixel needs to select 4 different calibration color bands in the visible light spectrum range. For example, the first pixel can be pre-calibrated and calibrated using a red spectral color band of 600 nm, a green spectral color band of 525 nm, and a blue spectral color band of 450 nm, and the second pixel can be pre-calibrated and calibrated using R, G1, G2, and B, a total of 4 calibration color bands. Alternatively, R, G1, G2, and B, a total of 4 calibration color bands, can be used to pre-calibrate and calibrate the first pixel and the second pixel respectively, and the first pixel can use R, G1, B or R, G2, B, a total of 3 calibration color bands, during pre-calibration and calibration. The specific calibration process can refer to the following embodiments, which will not be described here.

[0154] In some embodiments, the pixel further includes a back-illuminated silicon substrate, and the photoelectric converter is disposed on the back-illuminated silicon substrate. A trench is provided around the periphery of the plurality of photoelectric converters on the back-illuminated silicon substrate.

[0155] For example, as shown in FIG. 1, the pixel includes a back-illuminated silicon substrate (BSI Si), and the photoelectric conversion module is disposed on the back-illuminated silicon substrate. The asymmetric optical module is disposed on the photoelectric conversion module. Figure 8 As shown, the pixel further includes a back-illuminated silicon substrate (Back-Side illuminated Si, BSI Si), and the photoelectric conversion module is disposed on the back-illuminated silicon substrate. The asymmetric optical module is disposed on the photoelectric conversion module, and the pixel further includes a cell deep trench isolation structure (Cell Deep Trench Isolation, CDTI) disposed in the back-illuminated silicon substrate. The cell deep trench isolation structure is disposed around the 2x2 photoelectric converters in the photoelectric conversion module to physically isolate the 2x2 photoelectric converters under each asymmetric optical module and prevent light crosstalk between adjacent asymmetric optical modules.

[0156] The cell deep trench isolation structure can be a material such as silicon oxide, which is not limited here.

[0157] Again, the embodiments of the present application also provide a calibration method of an image sensor, applied to an image sensor composed of a plurality of pixels. The method includes the following steps:

[0158] S1, for each pixel, n calibration color bands are selected to decompose the incident light;

[0159] S2, for each pixel, pre-calibration calibration is performed based on the n calibration color bands to obtain pre-calibration calibration parameters;

[0160] S3, for each pixel, n different spatial signals obtained by photoelectric collection of the incident light by the pixel are obtained;

[0161] S4, for each pixel, the incident light is analyzed and calibrated by the pre-calibration calibration parameters and the n spatial signals to calculate the spectral components of the incident light under each calibration color band;

[0162] The pixel includes:

[0163] The photoelectric conversion module includes n photoelectric converters arranged adjacent in an array, wherein n is a positive integer greater than or equal to 3;

[0164] The asymmetric optical module is arranged on the photoelectric conversion module, wherein the asymmetric optical module is used to focus and diffract the incident light to generate a three-dimensional asymmetric light intensity pattern, and then form n different spatial signals one by one on the n photoelectric converters in the photoelectric conversion module.

[0165] It should be noted that the spatial signal refers to the signal measured on each photoelectric converter by the three-dimensional asymmetric light intensity pattern formed by the incident light of unknown color after asymmetric focusing and diffraction by the asymmetric optical module. Since the light intensity pattern is a three-dimensional asymmetric light intensity pattern, the light intensity pattern on the photosensitive plane of each photoelectric converter is different after projection into the pixel. The spatial signal corresponding to the photoelectric response is not the same, and therefore the spatial signal reflects the distribution characteristics of the incident light in space.

[0166] It can be understood that the spectral component can include the contribution ratio of each calibration color band to the total incident light intensity, such as the contribution ratio of the 600nm red spectral color band being 20%, the contribution ratio of the 525nm green spectral color band being 30%, and the contribution ratio of the 450nm blue spectral color band being 50%.

[0167] In some embodiments, the n calibration color bands selected in step S1 include at least a red spectral color band, a green spectral color band and a blue spectral color band to meet the decomposition requirement of the incident light of unknown color.

[0168] The red spectral color band is used to calibrate and analyze the red component in the incident light, the green spectral color band is used to calibrate and analyze the green component in the incident light, and the blue spectral color band is used to calibrate and analyze the blue component in the incident light.

[0169] The unfolding decomposition refers to a process of decomposing the unknown color incident light into different wavelength components. By using the red spectral color band, the green spectral color band and the blue spectral color band as the calibration color bands, the unknown color light can be decomposed into three basic components of the red spectral color band, the green spectral color band and the blue spectral color band, so that the spectral characteristics thereof can be accurately described.

[0170] In order to ensure that the unknown color incident light can be accurately decomposed and analyzed, the calibration color bands selected in step S1 must be able to cover the spectral range of the unknown color. The red spectral color band, the green spectral color band and the blue spectral color band are commonly used reference color bands in spectral analysis, which can cover most of the visible light range and can effectively decompose and describe the spectral characteristics of the unknown color incident light.

[0171] It should be noted that in step S1, when selecting the calibration color bands, the more the number of different color bands covered by the selected calibration color bands, the more accurate the unfolding decomposition of the multiple calibration color bands to the incident light. Therefore, on the basis of covering the red spectral color band, the green spectral color band and the blue spectral color band, the selected calibration color bands can include more spectral color bands of other colors, such as the yellow spectral color band, the purple spectral color band, etc. Multiple spectral color bands of the same color system with similar colors but different wavelengths can also be selected, such as the green spectral color band with a wavelength of 490 nm and the green spectral color band with a wavelength of 550 nm. However, the corresponding calibration calculation amount is larger and needs to be considered in a balanced manner.

[0172] In step S2, the pre-calibration parameter refers to a parameter obtained through a series of pre-calibration steps for unfolding and decomposing the incident light, which can be obtained by combining the monochromatic band space signal response calculation results of each calibration color band.

[0173] In some embodiments, for each pixel, the step S2 of obtaining the pre-calibration parameter based on the n calibration color bands further includes:

[0174] S21, for each pixel, based on the spatial signal response of the incident light unfolded and decomposed by the n calibration color bands on the n photoelectric converters in the pixel, an n-order linear equation set is constructed, wherein the n-order linear equation set is composed of a calibration coefficient matrix, a color band fraction vector and a spatial signal vector;

[0175] S22, for each pixel, traversing the n calibration color bands, based on the spatial signal response of a single calibration color band, the n-order linear equation set is calculated to obtain a calibration coefficient vector of the single calibration color band, and the calibration coefficient vectors of the n calibration color bands are integrated to obtain the calibration coefficient matrix.

[0176] S23, for each pixel, inverse operation is performed on the calibration coefficient matrix to obtain the inverse matrix of the calibration coefficient matrix, and the inverse matrix of the calibration coefficient matrix is the pre-calibration calibration parameter.

[0177] In step S21, the calibration coefficient matrix represents the calibration coefficient of the spatial signal measured by each calibration color band on each photoelectric converter. Specifically, the plurality of elements of each row of the calibration coefficient matrix represent the calibration coefficient of each calibration color band on the spatial signal of a certain photoelectric converter, and the plurality of elements of each column of the calibration coefficient matrix represent the calibration coefficient of a certain calibration color band on the spatial signal of each photoelectric converter, that is, each element a ij represents the calibration coefficient of the ith spatial signal under the jth calibration color band.

[0178] For example, the calibration method of the image sensor can be further refined to include:

[0179] Step S01, determine the number n of photoelectric converters sharing the same asymmetric optical module;

[0180] Step S02, select n calibration color bands in the visible light spectrum range which can effectively decompose various unknown colors of incident light;

[0181] Step S03, based on the spatial signal response of the n calibration color bands on the n photoelectric converters sharing the same asymmetric optical module, construct an n-order linear equation set;

[0182] Step S04, based on the spatial signal response of a single calibration color band, simplify the n-order linear equation set to solve the calibration coefficient matrix corresponding to the n-order linear equation set;

[0183] Step S05, solve the inverse matrix of the calibration coefficient matrix;

[0184] Step S06, for unknown color incident light, according to the measured spatial signal on the n photoelectric converters sharing the same asymmetric optical module and the inverse matrix of the calibration coefficient matrix, decompose the incident light into the spectral components corresponding to the n calibration color bands.

[0185] The detailed step-by-step process of the calibration method of the image sensor can be referred to in the following embodiments, which will not be described here.

[0186] Within each pixel, the spatial signal response refers to the photoelectric effect response of the asymmetric light intensity spatial distribution formed by the incident calibration color band after passing through the asymmetric focusing diffraction of the asymmetric optical module, forming a corresponding electrical signal. That is, the plurality of spatial signal responses in each pixel can describe the spatial distribution of the incident light on the photoelectric conversion module.

[0187] Exemplarily, within each pixel, the spatial signal response can be obtained by illuminating known calibration color bands (e.g., red spectral color band, green spectral color band, blue spectral color band, etc.) onto the photoelectric conversion module, and recording the electrical signals outputted by each photoelectric converter.

[0188] It can be understood that the spatial signal response within each pixel can be represented as a vector or a matrix, wherein each element corresponds to the electrical signal intensity on each photoelectric converter. For example, for a 2x2 photoelectric conversion array, the spatial signal response can be represented as a 4x1 vector.

[0189] Exemplarily, in the embodiments of the present application, for the asymmetric optical module as shown in Figures 7-8 , the value of n is 4, and when the narrow spectral color bands with wavelengths of 430 nm, 490 nm, 550 nm, and 610 nm in the visible spectral range are selected as the calibration color bands, the asymmetric optical module can effectively asymmetrically focus and diffract the incident light of the above-mentioned four calibration color bands (B=430 nm, G2=490 nm, G1=550 nm, and R=610 nm), as shown in Figure 9 , thus the asymmetric optical module can also effectively asymmetrically diffract various incident light based on the above-mentioned four calibration color bands, and form different spatial signals on the 2x2 photoelectric converters.

[0190] In this way, based on the different spatial signal responses on the four photoelectric converters within the pixel, four representative calibration color bands in the visible spectral range can be selected, a 4x4 linear equation set is constructed, and then the spatial signal response of a single calibration color band is used for calibration to determine the calibration coefficient matrix corresponding to the 4x4 linear equation set; the color band components of the incident light are solved based on the calibration coefficient matrix and the four spatial signals measured in real time, that is, the 2x2 spatial signals are analyzed into 2x2 spectral components of the incident light using the 2x2 calibration color bands.

[0191] In some embodiments, for the general case that the photoelectric conversion module includes n photoelectric converters, the matrix form of the n-linear equation set constructed based on the different spatial signal responses within the pixel can be represented as follows:

[0192] ,

[0193] wherein, C is an n*n calibration coefficient matrix, which is composed of the calibration coefficient vectors corresponding to the n calibration color bands; X is a color band fraction vector, which is the color band fraction vector that contributes to the measured spatial signal in the incident light; M is a spatial signal vector, which is composed of the n spatial signals measured by the incident light.

[0194] Exemplarily, in the embodiments of the present application, for the pixel or image sensor as shown in Figures 7-8 each pixel spatially responds to the incident light decomposed based on the above-mentioned four calibration color bands, forming a four-element linear equation group as follows.

[0195] ;

[0196] ;

[0197] ;

[0198] ;

[0199] wherein, C represents a calibration coefficient matrix of the four calibration color bands, X represents a color band fraction vector of the calibration color bands having contribution to the measured spatial signal in the incident light of unknown color, and M represents a spatial signal vector measured by the incident light of unknown color. represents a calibration coefficient of the spatial signal Mij measured by the calibration color band bk component of the incident light on the photoelectric converter PDij, represents a contribution ratio of the calibration color band bk component of the incident light to the spatial signal Mij, and Mij represents a spatial signal measured by the incident light of unknown color on the photoelectric converter PDij; i and j are integers from 1 to 2, and k is an integer from 1 to 4.

[0200] Exemplarily, in the embodiments of the present application, in the visible light spectrum range of 400-640 nm, as shown in Figures 9-10 a blue light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 430 nm is used as the calibration color band Band 1 (b1), a green light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 490 nm is used as the calibration color band Band 2 (b2), a green light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 550 nm is used as the calibration color band Band 3 (b3), and a red light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 610 nm is used as the calibration color band Band 4 (b4).

[0201] It can be understood that the above-mentioned four-element linear equation group can be written in the following form.

[0202] ;

[0203] wherein, C is a 4x4 calibration coefficient matrix, X is a color band fraction vector of the calibration color bands having contribution to the measured spatial signal in the incident light of unknown color, and M is a spatial signal vector measured by the incident light of unknown color.

[0204] , , .

[0205] The above matrix equation is multiplied by the inverse matrix of C on both sides to obtain the following relationship, which is the solution of the linear equation group:

[0206] ;

[0207] From the above matrix equation, after the pixel structure and the selected four calibration color bands are determined, the calibration vector of the pixel structure for each calibration color band is determined, and then the calibration coefficient matrix is determined uniquely. In this way, the spatial signal response of the pixel structure to the unknown color incident light can be understood as "encoding the incident light decomposed according to the four calibration color bands based on the calibration coefficient matrix to obtain four different spatial signals". At the same time, from the above matrix equation, only one of the four calibration color bands can be used each time to simplify the spatial signal response or the encoding simplification, that is, one element in X is 1 and the other three elements are 0, to simplify the matrix equation, and then the partial elements or vectors in the calibration coefficient matrix C can be solved based on the corresponding measured spatial signal. In combination with the calibration coefficient elements or calibration coefficient vectors solved by the spatial signal response of the four calibration color bands, the complete calibration coefficient matrix C is obtained, and the inverse matrix C -1 of the calibration coefficient matrix C can be solved.

[0208] Finally, from the above matrix equation, for the unknown color incident light X, based on the inverse matrix C -1 of the calibration coefficient matrix C and the spatial signal vector M corresponding to the incident light X, the incident light X can be directly calculated based on the inverse matrix C -1 of the calibration coefficient matrix C and the measured spatial signal vector M. The incident light X is analyzed as the spectral components corresponding to the above four calibration color bands, and the color measurement of the incident light is realized while detecting the brightness of the incident light. This process can be understood as "decoding based on the inverse operation parameter (the inverse matrix C -1 of the calibration coefficient matrix C) and the encoding result (four different spatial signals) to obtain the expansion and decomposition result of the incident light under the four calibration color bands".

[0209] For example, in the embodiment of the present application, let , only the red light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 610 nm is used as the calibration color band Band 4 for single-band pre-calibration simplification, and the corresponding four-element linear equation group is simplified as:

[0210] ;

[0211] ;

[0212] ;

[0213] ;

[0214] At the same time, according to the actual measured value of the spatial signal vector M, the quantum efficiencies of the corresponding four photoelectric converters are respectively: QE PD11 = 16.7, QE PD12 = 12.7, QE PD21 = 12.7, and QE PD22 = 14.8.

[0215] Exemplarily, in the embodiment of the present application, let , only the green light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 550 nm is used as the calibration color band Band 3 to perform monochromatic band pre-calibration simplification, and the corresponding four-element linear equation group is simplified as:

[0216] ;

[0217] ;

[0218] ;

[0219] ;

[0220] At the same time, according to the actual measured value of the spatial signal vector M, the quantum efficiencies of the corresponding four photoelectric converters are respectively: QE PD11 = 20.2, QE PD12 = 14, QE PD21 = 16, and QE PD22 = 17.5.

[0221] Exemplarily, in the embodiment of the present application, let , only the green light with a full width at half maximum (FWHM) of 60 nm and a wavelength of 490 nm is used as the calibration color band Band 2 to perform monochromatic band pre-calibration simplification, and the corresponding four-element linear equation group is simplified as:

[0222] ;

[0223] ;

[0224] ;

[0225] ;

[0226] At the same time, according to the actual measured value of the spatial signal vector M, the quantum efficiencies of the corresponding four photoelectric converters are respectively: QE PD11 = 19.8, QE PD12 = 15, QE PD21 = 15.4, and QE PD22 = 17.3.

[0227] Exemplarily, in the embodiment of the present application, let , only the blue light with FWHM of 60 nm and wavelength of 430 nm as the calibration color band Band 1 is used for monochromatic band pre-calibration simplification, and the corresponding four-element linear equation group is simplified as:

[0228] ;

[0229] ;

[0230] ;

[0231] ;

[0232] At the same time, according to the actual measurement value of the spatial signal vector M, the quantum efficiencies of the corresponding four photoelectric converters are obtained as follows: QE PD11=18.1, QE PD12=16.9, QE PD21=14.4, and QE PD22=19.9.

[0233] In this way, the pre-calibration simplification of the above four monochromatic calibration color bands is integrated to obtain the corresponding calibration coefficient matrix C as shown in Table 1 below, wherein the four calibration coefficients of each column correspond to the pre-calibration simplification result of one calibration color band.

[0234] Table 1

[0235]

[0236] At the same time, for the purpose of calculation, the four calibration coefficients corresponding to the pre-calibration simplification result of a single calibration color band are normalized to obtain the calibration coefficient matrix C as shown in Table 2 below.

[0237] Table 2

[0238]

[0239] In some embodiments, for each pixel, the incident light is analytically calibrated by pre-calibrating the parameters and n spatial signals to calculate the spectral components of the incident light under each calibration color band, including:

[0240] For each pixel, the analytical calibration formula is obtained by deducing based on the n-element linear equation group:

[0241] ,

[0242] wherein, C -1 is the inverse matrix of the calibration coefficient matrix;

[0243] For each pixel, a spatial signal vector is constructed based on the n spatial signals, and the inverse matrix of the calibration coefficient matrix and the spatial signal vector are brought into the analytical calibration formula to calculate the color band score vector and obtain the spectral component of the incident light under each calibration color band.

[0244] For example, the inverse calculation of the calibration coefficient matrix C shown in the above table is performed to obtain the inverse matrix C of the calibration coefficient matrix C -1 As shown in Table 3 below.

[0245] Table 3

[0246]

[0247] In this way, the pre-calibration or pre-calibration of the corresponding pixel or image sensor is realized, and for subsequent unknown color incident light, the spectral component of the incident light is calculated based on the inverse matrix C of the calibration coefficient matrix C The formula can directly calculate the incident light X according to the inverse matrix C of the calibration coefficient matrix C -1 and the measured spatial signal vector M, and the incident light X is analyzed into the spectral components corresponding to the four calibration color bands, that is At the same time of realizing the incident light brightness detection, the color measurement of the incident light is realized.

[0248] In the embodiment of the present application, the spatial signal of the incident light is recorded by the four photoelectric converters Figure 9 It can be known that the asymmetric distribution patterns of the spatial signals of the incident light of different colors on the four photoelectric converters are different, and the asymmetric spatial patterns of the spatial signals recorded on the four photoelectric converters can be regarded as the fingerprints of the incident light spectrum / color, so that a linear equation group connecting the spatial signals of the photoelectric converters and the measured spectral components of the incident light signals can be written to analyze and decode the spectral components corresponding to the four calibration color bands; in order to decode the measured four spatial signals, the pixel sharing the asymmetric optical module of the multiple photoelectric converters can be pre-calibrated by sequentially irradiating the pixel with calibration color bands of different wave bands, and the corresponding spatial signal response is recorded to obtain the calibration coefficient matrix, and the inverse matrix of the calibration coefficient matrix is used to calculate the incident light of unknown color to solve the linear equation group, wherein each element of the calibration coefficient matrix is used as the coefficient of the linear equation group connecting the four spatial measurement signals to the four calibration color bands.

[0249] In the embodiment of the present application, the asymmetric optical module and the multi-color band calibration technology means are combined to replace the microlens and the optical filter in the traditional pixel, and the process of incident light brightness detection and color detection is realized as Figure 11As shown, after the incident light X is resolved into spectral components corresponding to the four calibration color bands based on the above process, each spectral component is multiplied by the total measurement signal of the four photoelectric converters, so that each spectral component is converted into the least significant bit (LSB) of the total measurement signal, and finally the spectral components corresponding to the four calibration color bands are converted into the standard red-green-blue color space based on the least significant bit for display.

[0250] For example, in order to convert the measured spectral components into standardized values, each spectral component needs to be normalized, that is, each spectral component is divided by the total measurement signal of the four photoelectric converters to eliminate the influence of light source intensity variation and ensure the relative proportion of the measurement values.

[0251] Therefore, the embodiment of the present application also provides an imaging system, which comprises:

[0252] An imaging lens is configured to converge and image light rays of an object to form incident light;

[0253] An image sensor is configured to diffract the incident light based on pixels in the image sensor to generate a three-dimensionally asymmetric light intensity pattern, and then record n different spatial signals one by one on n photoelectric converters in the pixels; wherein n is a positive integer greater than or equal to 3.

[0254] The pixel is configured to, based on the decomposition of the incident light with n calibration color bands, resolve and calibrate the incident light by using pre-calibration parameters obtained by pre-calibration of the n calibration color bands and the n spatial signals to obtain spectral components of the incident light under each calibration color band.

[0255] For example, as shown in Figure 15 The imaging system comprises:

[0256] An imaging lens is configured to converge and image light rays of an object to form incident light;

[0257] The image sensor is configured to collect asymmetric spatial signals of the incident light based on the pixels, and combine pre-calibration of four calibration color bands in the visible spectrum to resolve the incident light into spectral components corresponding to the four calibration color bands to detect the brightness and color of the incident light in each pixel.

[0258] For example, as shown in Figure 15 and Figure 16 In the embodiment of the present application, the asymmetric optical module and the subsequent multi-color band calibration technology in the visible spectrum can effectively replace the micro-lens and filter structure in the related pixels, so that the image sensor in the embodiment of the present application does not need any structural offset to adapt to the high chief ray angle, and the image sensor in the embodiment of the present application can also adapt to any F-number F imaging lens. For example,Figure 16 As shown, the tests prove that two different imaging lenses with different chief ray angles CRA and different F numbers, i.e., lens 1 and lens 2, can effectively adapt to the structure of the image sensor in the embodiments of the present application.

[0259] In this way, the assembly of the imaging system can also be simplified, different specifications of imaging lenses + different specifications of image sensors are assembled as needed to obtain the imaging system, and the assembled imaging system is calibrated based on the pre-calibration of the multiple calibration color bands in the visible light spectrum range.

[0260] Finally, as shown in Figure 17 The present application also provides a photoelectric device 1700.

[0261] Specifically, the photoelectric device 1700 includes a processor 1701 having one or more processing cores, a memory 1702 having one or more computer readable storage media, and a computer program stored on the memory 1702 and executable on the processor 1701. The processor 1701 is electrically connected to the memory 1702. Those skilled in the art can understand that the structure of the photoelectric device shown in the figure does not constitute a limitation on the photoelectric device, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0262] The processor 1701 is the control center of the photoelectric device 1700, and connects various parts of the entire photoelectric device 1700 through various interfaces and lines. By running or loading the software programs and / or units stored in the memory 1702, and calling the data stored in the memory 1702, the processor 1701 performs various functions and processes data of the photoelectric device 1700, thereby overall monitoring the photoelectric device 1700. The processor 1701 can be a processor CPU, a graphics processor GPU, a network processor (Network Processor, NP), etc., and can realize or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0263] In the embodiments of the present application, the processor 1701 in the photoelectric device 1700 will load the instructions corresponding to the processes of one or more application programs into the memory 1702, and run the application programs stored in the memory 1702 by the processor 1701, thereby realizing various functions, such as executing the steps of the calibration method of the image sensor described above.

[0264] The specific implementation of each operation can refer to the previous embodiments, which will not be repeated here.

[0265] As shown in Figure 17As shown, the optoelectronic device 1700 further includes an image sensor (not shown in the figure), and the processor 1701 is electrically connected with the image sensor respectively. The image sensor can be the image sensor as described in the above embodiments. Those skilled in the art can understand that Figure 17 The optoelectronic device structure shown in the figure is not a limitation of the optoelectronic device, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0266] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0267] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by instructions controlling related hardware, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0268] Therefore, the embodiments of the present application also provide a computer readable storage medium, which stores a plurality of computer programs. The computer programs can be loaded by a processor to execute any of the image sensor calibration methods provided by the embodiments of the present application. The computer programs can execute the steps of the image sensor calibration method as described above.

[0269] The specific implementation of each operation can refer to the previous embodiments, which will not be repeated here.

[0270] The computer readable storage medium can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0271] Since the computer programs stored in the computer readable storage medium can execute any of the image sensor calibration methods provided by the embodiments of the present application, the beneficial effects of any of the image sensor calibration methods provided by the embodiments of the present application can be achieved. Details are described in the previous embodiments, which will not be repeated here.

[0272] The above describes in detail a pixel, an image sensor, an image sensor calibration method, an optoelectronic device, a computer readable storage medium and an imaging system provided by the embodiments of the present application. The principles and implementation manners are described by specific examples in the embodiments of the present application. The above embodiment descriptions are only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the embodiments of the present application.

[0273] It should be noted that each of the arrangements described in the present application are described with progressive manner, and each arrangement focuses on the differences from other arrangements, and the same or similar parts between arrangements can be referred to each other.

[0274] It should also be noted that the terms such as first and second in the present application are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0275] The above description of the disclosed arrangements enables those skilled in the art to implement or use the present application. Various modifications to the arrangements will be apparent to those skilled in the art, and the general principles defined in the present application can be implemented in other arrangements without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the arrangements shown in the present application, but will conform to the widest scope consistent with the principles and novel features disclosed in the present application.

Claims

1. A pixel unit, characterized in that: The pixel unit includes: A photoelectric conversion module, comprising n photoelectric converters arranged adjacent to each other in an array, wherein n is a positive integer greater than or equal to 3; an asymmetric optical module, disposed on the photoelectric conversion module, wherein the asymmetric optical module is used to focus and diffract incident light to generate a three-dimensional asymmetric light intensity pattern, thereby recording n different spatial signals on the n photoelectric converters in the photoelectric conversion module in a one-to-one correspondence; The pixel unit is configured to: on the basis of decomposing the incident light using n calibration color bands, perform analytical calibration on the incident light using pre-calibrated calibration parameters obtained by pre-calibrating the n calibration color bands and the n spatial signals, to obtain the spectral components of the incident light in each of the calibration color bands; The pre-calibration parameters obtained by pre-calibrating the n calibration ribbons include: For each pixel unit, constructing a system of linear equations of n variables based on spatial signal responses of the incident light decomposed by the n calibration color bands on the n photoelectric converters within the pixel unit, wherein the system of linear equations of n variables is composed of a calibration coefficient matrix, a color band fraction vector, and a spatial signal vector; For each pixel unit, traverse the n calibration color bands, simplify the n-variable linear equation system based on the spatial signal response of the single calibration color band to obtain a calibration coefficient vector for the single calibration color band, and then combine the calibration coefficient vectors of the n calibration color bands to obtain the calibration coefficient matrix; For each pixel unit, an inverse operation is performed on the calibration coefficient matrix to obtain an inverse matrix of the calibration coefficient matrix, and the inverse matrix of the calibration coefficient matrix is ​​the pre-calibrated calibration parameters.

2. The pixel unit according to claim 1, wherein: The photoelectric conversion module includes only the photoelectric converters of one size, and the n photoelectric converters in the photoelectric conversion module are arranged in a regular array.

3. The pixel unit according to claim 1, wherein: The photoelectric conversion module includes at least two photoelectric converters of different sizes, and the n photoelectric converters in the photoelectric conversion module are arranged in an irregular array.

4. The pixel unit according to claim 1, wherein: The asymmetric optical module comprises: A background structure layer, wherein the background structure layer is made of a first material having a first refractive index; A diffraction structure layer is embedded in the background structure layer, wherein the diffraction structure layer includes a plurality of components made of a second material having a second refractive index for focusing and diffracting the incident light, and the first refractive index is lower than the second refractive index.

5. The pixel unit according to claim 4, wherein: The first material and the second material are inorganic materials.

6. The pixel unit according to claim 5, characterized in that: The component includes at least two diffraction cylinders of different sizes and at least two diffraction ring cylinders of different sizes. Multiple diffraction cylinders and multiple diffraction ring cylinders are staggered and asymmetrically arranged so that the diffraction structure layer diffracts the incident light asymmetrically.

7. An image sensor, characterized in that: The image sensor includes a plurality of pixel units arranged in an array; The pixel unit includes: A photoelectric conversion module, comprising n photoelectric converters arranged adjacent to each other in an array, wherein n is a positive integer greater than or equal to 3; an asymmetric optical module, disposed on the photoelectric conversion module, wherein the asymmetric optical module is used to focus and diffract incident light to generate a three-dimensional asymmetric light intensity pattern, thereby recording n different spatial signals on the n photoelectric converters in the photoelectric conversion module in a one-to-one correspondence; The pixel unit is configured to: on the basis of decomposing the incident light using n calibration color bands, perform analytical calibration on the incident light using pre-calibrated calibration parameters obtained by pre-calibrating the n calibration color bands and the n spatial signals, to obtain the spectral components of the incident light in each of the calibration color bands; The pre-calibration parameters obtained by pre-calibrating the n calibration ribbons include: For each pixel unit, constructing a system of linear equations of n variables based on spatial signal responses of the incident light decomposed by the n calibration color bands on the n photoelectric converters within the pixel unit, wherein the system of linear equations of n variables is composed of a calibration coefficient matrix, a color band fraction vector, and a spatial signal vector; For each pixel unit, traverse the n calibration color bands, simplify the n-variable linear equation system based on the spatial signal response of the single calibration color band to obtain a calibration coefficient vector for the single calibration color band, and then combine the calibration coefficient vectors of the n calibration color bands to obtain the calibration coefficient matrix; For each pixel unit, an inverse operation is performed on the calibration coefficient matrix to obtain an inverse matrix of the calibration coefficient matrix, and the inverse matrix of the calibration coefficient matrix is ​​the pre-calibrated calibration parameters.

8. The image sensor according to claim 7, wherein: The image sensor includes only one type of pixel unit structure, and the pixel units in the image sensor are arranged in a regular array.

9. The image sensor according to claim 7, wherein: The image sensor includes at least two pixel units with different structures, and the pixel units in the image sensor are arranged in a regular array.

10. The image sensor according to claim 9, wherein: In the image sensor, n has only one value, but the array arrangement structures of the n photoelectric converters in at least two of the photoelectric conversion modules are different.

11. The image sensor according to claim 9, wherein In the image sensor, n has at least two different values, so that the array arrangement structures of the n photoelectric converters in at least two of the photoelectric conversion modules are different.

12. The image sensor according to claim 9, wherein In the image sensor, n has only one value, and the array arrangement structure of the n photoelectric converters in each photoelectric conversion module is the same, but the asymmetric optical modules in at least two pixel units are different.

13. The image sensor according to claim 8, wherein The pixel unit further includes a back-illuminated silicon substrate, the photoelectric conversion module is arranged on the back-illuminated silicon substrate, and a groove is provided on the back-illuminated silicon substrate surrounding the photoelectric conversion module.

14. An imaging system, characterized in that: The imaging system comprises: Imaging lens, used to converge the light of the object into an image to form incident light; An image sensor configured to focus and diffract incident light based on pixel units in the image sensor to generate a three-dimensional asymmetric light intensity pattern, and then to record n ​​different spatial signals on n photoelectric converters in the pixel units in a one-to-one correspondence; wherein n is a positive integer greater than or equal to 3; The pixel unit is configured to: on the basis of decomposing the incident light using n calibration color bands, perform analytical calibration on the incident light using pre-calibrated calibration parameters obtained by pre-calibrating the n calibration color bands and the n spatial signals, to obtain the spectral components of the incident light in each of the calibration color bands; The pre-calibration parameters obtained by pre-calibrating the n calibration ribbons include: For each pixel unit, constructing a system of linear equations of n variables based on spatial signal responses of the incident light decomposed by the n calibration color bands on the n photoelectric converters within the pixel unit, wherein the system of linear equations of n variables is composed of a calibration coefficient matrix, a color band fraction vector, and a spatial signal vector; For each pixel unit, traverse the n calibration color bands, simplify the n-variable linear equation system based on the spatial signal response of the single calibration color band to obtain a calibration coefficient vector for the single calibration color band, and then combine the calibration coefficient vectors of the n calibration color bands to obtain the calibration coefficient matrix; For each pixel unit, an inverse operation is performed on the calibration coefficient matrix to obtain an inverse matrix of the calibration coefficient matrix, and the inverse matrix of the calibration coefficient matrix is ​​the pre-calibrated calibration parameters.

15. A method for calibrating an image sensor, characterized in that: Applied to an image sensor composed of multiple pixel units, the method includes: For each pixel unit, selecting n calibration color bands to decompose the incident light; For each pixel unit, pre-calibration is performed based on the n calibration ribbons to obtain pre-calibration parameters; For each pixel unit, obtaining n different spatial signals obtained by the pixel unit performing photoelectric collection on the incident light; For each pixel unit, analytically calibrate the incident light using the pre-calibrated calibration parameters and the n spatial signals, and calculate the spectral components of the incident light in each calibration color band; Wherein, the pixel unit includes: A photoelectric conversion module, comprising n photoelectric converters arranged adjacent to each other in an array, wherein n is a positive integer greater than or equal to 3; an asymmetric optical module, disposed on the photoelectric conversion module, wherein the asymmetric optical module is used to focus and diffract the incident light to generate a three-dimensional asymmetric light intensity pattern, and then record the n different spatial signals on the n photoelectric converters in the photoelectric conversion module in a one-to-one correspondence; For each pixel unit, pre-calibration is performed based on the n calibration ribbons to obtain pre-calibration parameters, including: For each pixel unit, constructing a system of linear equations of n variables based on spatial signal responses of the incident light decomposed by the n calibration color bands on the n photoelectric converters within the pixel unit, wherein the system of linear equations of n variables is composed of a calibration coefficient matrix, a color band fraction vector, and a spatial signal vector; For each pixel unit, traverse the n calibration color bands, simplify the n-variable linear equation system based on the spatial signal response of the single calibration color band to obtain a calibration coefficient vector for the single calibration color band, and then combine the calibration coefficient vectors of the n calibration color bands to obtain the calibration coefficient matrix; For each pixel unit, an inverse operation is performed on the calibration coefficient matrix to obtain an inverse matrix of the calibration coefficient matrix, and the inverse matrix of the calibration coefficient matrix is ​​the pre-calibrated calibration parameters.

16. The method according to claim 15, characterized in that The selected n calibration color bands include at least a red spectral color band, a green spectral color band, and a blue spectral color band, so as to meet the requirements of unfolding and decomposing the incident light of unknown color.

17. The method according to claim 15, characterized in that The matrix form of the n-dimensional linear equation system is as follows: ， in, C The calibration coefficient matrix is ​​n×n and is composed of calibration coefficient vectors corresponding to the n calibration color bands; X is the color band fraction vector, which is the color band fraction vector in the incident light that contributes to the measured spatial signal; M is the spatial signal vector, which is composed of n spatial signals obtained by measuring the incident light.

18. The method according to claim 17, characterized in that For each pixel unit, performing analytical calibration on the incident light using the pre-calibrated calibration parameters and the n spatial signals, and calculating the spectral components of the incident light in each calibration color band, includes: For each pixel unit, the analytical calibration formula is obtained by deducing and solving the n-variable linear equation group: ， in, C -1 is the inverse matrix of the calibration coefficient matrix; For each of the pixel units, the spatial signal vector is constructed based on the n spatial signals, and the inverse matrix of the calibration coefficient matrix and the spatial signal vector are substituted into the analytical calibration formula to calculate the color band score vector to obtain the spectral component of the incident light in each of the calibration color bands.

19. A photoelectric device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 15 to 18.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 15 to 18 are implemented.

Images