Pixel structure, image sensor, camera module and color deconstruction method

Abstract

The invention discloses a pixel structure, an image sensor, a camera module and a color deconstruction method, and belongs to the technical field of optical structures.The pixel structure comprises a photodiode assembly which comprises at least three photodiodes; wherein the red light absorption proportion, the green light absorption proportion and the blue light absorption proportion of the at least three photodiodes are different.

Description

Technical Field

[0001] The present application belongs to the field of optical technology, and specifically relates to a pixel structure, an image sensor, a camera module and a color deconstruction method. Background Art

[0002] Camera modules have become standard modules for electronic devices such as smartphones and tablets, and are important tools for obtaining visual information and performing artistic creation. Users' demands for the imaging quality and shooting functions of camera modules on electronic devices are constantly increasing. Among them, image sensors are key components in the imaging chain, and their color reproduction ability is one of the important factors affecting the tone of later images.

[0003] In related technologies, image sensors usually use a Bayer array photosensitive structure, which is a structure that uses red (R), green (G), and blue (B) filters arranged in a mosaic pattern and covered on each photodiode. Taking the red filter as an example, the pixels covered by the filter mainly receive incident light above 560nm. For the components of the incident light with a wavelength lower than 560nm, the absorption rate is low, but it is not zero absorption, that is, there is a low absorption rate for light in this band, and this part is called "spectral crosstalk."

[0004] In the original RAW image obtained by the Bayer array sensor, each pixel can only recognize one color among R, G, and B. If you want to obtain a full-color image, you need to calculate the values ​​of the other two colors corresponding to each pixel position, which is also called demosaicing (DMSC). Specifically, the pixels with missing colors are filled by interpolation to obtain a full-color image.

[0005] However, the above Bayer array and the corresponding DMSC algorithm design will introduce additional errors; based on the green filter in the Bayer array occupying half of the pixels to enhance the brightness resolution, the resolution of the red and blue channels is only 1 / 4 of the image, which results in the overall color resolution being much lower than the brightness resolution, affecting the color reproduction accuracy. This will cause the image to be prone to aliasing; in addition, the filter will absorb part of the light energy, so that the photodiode cannot fully utilize all the incident light, especially in low light conditions, and the signal-to-noise ratio will be significantly reduced. Summary of the invention

[0006] The purpose of the embodiments of the present application is to provide a pixel structure, an image sensor, a camera module and a color deconstruction method, which can improve the imaging quality of the image sensor.

[0007] In a first aspect, some embodiments of the present application provide a pixel structure, the pixel structure comprising:

[0008] A photodiode assembly, wherein the photodiode assembly comprises at least three photodiodes, and the at least three photodiodes are stacked along the incident direction of light;

[0009] Wherein, the red light absorption ratio, green light absorption ratio and blue light absorption ratio of the at least three photodiodes are all different.

[0010] In a second aspect, some embodiments of the present application provide an image sensor, comprising a pixel array, an analog-to-digital conversion (ADC) circuit, and an image signal processor (ISP), wherein the pixel array comprises the pixel structure described in the first aspect;

[0011] A first end of the ADC circuit is electrically connected to a pixel structure in the pixel array, a second end of the ADC circuit is electrically connected to the ISP, and the ADC circuit is used to convert an analog electrical signal of a photodiode component of the pixel structure into a digital signal, and transmit the digital signal to the ISP;

[0012] The color deconstruction function module in the ISP is used to obtain the R pixel signal, the G pixel signal and the B pixel signal of each pixel in the pixel array according to the color deconstruction processing of the digital signal.

[0013] In a third aspect, some embodiments of the present application provide a camera module, which includes the image sensor described in the second aspect.

[0014] In a fourth aspect, some embodiments of the present application provide an electronic device comprising the camera module described in the third aspect.

[0015] In a fifth aspect, some embodiments of the present application provide a color deconstruction method, which is performed by the electronic device described in the fourth aspect, and includes:

[0016] Acquiring analog electrical signals from at least three photodiodes stacked along the incident direction of light;

[0017] Acquiring analog electrical signals of at least three photodiodes through an ADC circuit, wherein the at least three photodiodes are located in a photodiode component of a pixel structure;

[0018] Converting the analog electrical signal into a digital signal through the ADC circuit;

[0019] The color deconstruction function module in the ISP performs color deconstruction processing on the digital signals corresponding to the at least three photodiodes to obtain the R pixel signal, the G pixel signal and the B pixel signal of the pixel structure.

[0020] In the embodiment of the present application, since the photodiode assembly includes at least three photodiodes stacked along the incident direction of the light, the light penetrates the at least three photodiodes in sequence. In the process, each of the at least three photodiodes performs photoelectric conversion on the incident red light, green light and blue light respectively, and generates an analog electrical signal corresponding to the absorbed light intensity; based on the different absorption ratios of the at least three photodiodes stacked along the incident direction of the light for the red light, green light and blue light, the red light, green light and blue light are absorbed by each of the at least three photodiodes in a different ratio, so that the analog electrical signal output by each photodiode is positively correlated to the sum of the light intensity of each light converted by the photodiode. In this way, when the at least three photodiodes output In the process of color deconstruction of the signal, the analog electrical signals output by multiple photodiodes and the absorption ratios of red light, green light and blue light by the multiple photodiodes can be combined to separate the red light illuminance, green light illuminance and blue light illuminance in the light incident on the photodiode component, and based on the fact that the R pixel signal of the pixel structure is proportional to the red light illuminance, the G pixel signal is proportional to the green light illuminance and the B pixel signal is proportional to the blue light illuminance, the R pixel signal, G pixel signal and B pixel signal of the pixel structure can be determined, the spatial resolution of the R pixel signal, G pixel signal and B pixel signal is higher, the signal-to-noise ratio of the image generated based on the R pixel signal, G pixel signal and B pixel signal is higher, and the imaging quality of the image sensor with the pixel structure is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a schematic diagram of a pixel structure provided by some embodiments of the present application;

[0022] Figure 2 is a schematic diagram of a pixel structure provided by some embodiments of the present application;

[0023] Figure 3 is a graph of the optical properties of silicon;

[0024] Figure 4 is a circuit diagram of a pixel structure provided by some embodiments of the present application;

[0025] Figure 5 is a circuit diagram of a pixel structure provided by some embodiments of the present application;

[0026] Figure 6 is a schematic diagram of the structure of an image sensor provided by some embodiments of the present application;

[0027] Figure 7 is a schematic diagram of the structure of an image sensor provided by some embodiments of the present application;

[0028] Figure 8 is a schematic diagram of a pixel array in an image sensor provided by some embodiments of the present application;

[0029] Fig. 9 yes Figure 8 A corresponding relationship diagram between the photodiode in each pixel structure in the pixel array and the analog electrical signal;

[0030] Fig.10 is a flow chart of a color deconstruction method provided by some embodiments of the present application;

[0031] Fig.11 It is a schematic diagram of the structure of an electronic device provided in some embodiments of the present application.

[0032] Description of reference numerals:

[0033] 10-photodiode component, reading module 20, first photodiode 11, second photodiode 12, third photodiode 13, TG module 21, FD node 22, RST switch 23, SF circuit 24, SEL switch 25, first TG switch 211, second TG switch 212, third TG switch 213, fourth TG switch 214. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0035] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0036] See also Figure 1 or Figure 2 , the pixel structure provided in the embodiment of the present application includes:

[0037] A photodiode assembly 10, the photodiode assembly 10 comprises at least three photodiodes, and the at least three photodiodes are stacked along a light incident direction H;

[0038] Among them, the red light absorption ratio, green light absorption ratio and blue light absorption ratio of at least three photodiodes are all different.

[0039] It should be noted that one photodiode assembly 10 may include 3 or more photodiodes.

[0040] For example: Figure 1 As shown, a photodiode assembly 10 includes three photodiodes, which are respectively a first photodiode 11, a second photodiode 12 and a third photodiode 13 along the light incident direction H. If the light propagates from top to bottom, the first photodiode 11, the second photodiode 12 and the third photodiode 13 are stacked in sequence from top to bottom.

[0041] In some embodiments of the present application, a photodiode assembly 10 may include more than three photodiodes. Figure 2 As shown, a photodiode assembly 10 includes four photodiodes, and the four photodiodes along the light incident direction H are respectively a first photodiode 11, a second photodiode 12, a third photodiode 13 and a fourth photodiode 14.

[0042] For the sake of convenience, in the embodiments of the present application, a photodiode assembly 10 including three photodiodes is generally used as an example for illustration, which is not specifically limited herein.

[0043] It is worth pointing out that the red light absorption ratio, green light absorption ratio and blue light absorption ratio of at least three photodiodes are all different. They may be photodiodes located at different depths, and the absorption ratios of red light, green light and blue light are all different. For example, two photodiodes are stacked in sequence along the direction of light propagation. Even if the illuminance and proportion of red light, green light and blue light in the incident light are the same, when the incident light is transmitted to the first photodiode, the absorption ratio of the blue light by the first photodiode is greater than the absorption ratio of red light and green light. In this way, the illuminance of blue light in the light penetrating the first photodiode is lower than the illuminance of red light and green light. When the light penetrating the first photodiode continues to irradiate the second photodiode, due to the decrease in the illuminance of blue light, the absorption ratio of the second photodiode to red light and green light may be greater than the absorption ratio to blue light. In this way, the absorption ratio of the first photodiode to red light, green light and blue light is different from the absorption ratio of the second photodiode to red light, green light and blue light.

[0044] In some embodiments of the present application, the thickness of each photodiode can be designed to achieve different red light absorption ratios, green light absorption ratios, and blue light absorption ratios of at least three photodiodes.

[0045] It should be noted that, based on the different absorption degrees of light by photodiodes of different thicknesses, the absorption ratio of red light, green light and blue light by the photodiode can be changed, and finally the absorption ratios of red light, green light and blue light by at least three photodiodes in the photodiode assembly 10 are all different. For example, for light with a longer wavelength, the thickness of the photodiode required to absorb the light is thicker. Based on this principle, taking the third photodiode 13 as an example, when the thickness of the third photodiode 13 is thin, the light mainly absorbed by the third photodiode 13 may be green light, and when the thickness of the third photodiode 13 is thick, the light mainly absorbed by the third photodiode 13 may be red light.

[0046] By designing the at least three photodiodes to have different thicknesses along the light incidence direction, when the light is transmitted into the at least three photodiodes, the photodiodes located at different depths have different absorption characteristics for the light, which can provide a reliable data basis for subsequent color deconstruction processing based on the light intensity absorbed by the at least three photodiodes.

[0047] In the present embodiment, by designing the thickness of each photodiode, the absorption ratio of red light, green light and blue light by each photodiode in the photodiode assembly 10 can be adjusted, so that the absorption ratio of red light, green light and blue light by each photodiode in the photodiode assembly 10 is flexibly adjustable. In addition, after determining the absorption ratio requirements of red light, green light and blue light by each photodiode in the photodiode assembly 10, the thickness of each photodiode in the photodiode assembly 10 can be designed so that the actual absorption ratio of red light, green light and blue light by each photodiode in the photodiode assembly 10 meets the requirements, thereby realizing flexible customization of the light absorption characteristics of the pixel structure under different spectral conditions.

[0048] In some embodiments of the present application, the specific thickness of the at least three photodiodes is related to the application scenario, pixel size, process technology, etc., and is not specifically limited here. Taking the pixel structure of the embodiment of the present application applied to a mobile phone image sensor as an example, the total thickness of at least three photodiodes in a pixel structure is between 1 and 10 μm. However, depending on the application scenario of the pixel structure, the thickness of at least three photodiodes in the pixel structure can be adjusted accordingly. For example, in the case of an outdoor display device, based on the increase in the thickness of the display screen, the total thickness of at least three photodiodes in a pixel structure is between 100 and 1000 μm. That is, on the basis of ensuring that the process is feasible, the pixels in any image sensor can be set to a structure with at least three layers of photodiodes.

[0049] In other embodiments, the material of the photodiodes can be designed to achieve different red light absorption ratios, green light absorption ratios, and blue light absorption ratios of at least three photodiodes.

[0050] For example, when using silicon material to make photodiodes, the optical properties of silicon can be expressed as follows:

[0051] I(λ,z)=I0(λ,0)e -α(λ)z (1)

[0052] Among them, I0(λ,0) is the magnitude of the incident light of different wavelengths, z is the penetration depth of the light, and α(λ) is the absorption coefficient of silicon for light of different wavelengths (in cm -1 ),like Figure 3 As shown, for the visible light band between the two dotted lines, the absorption coefficient of low-wavelength light, such as blue light, is higher, reaching 1E+5cm -1 Therefore, its photosensitivity process mainly exists in the photodiode close to the incident direction of light, such as Figure 1 The first photodiode 11 in the middle, that is, in the first photodiode 11 close to the incident direction of the light, has the highest absorption ratio for blue light; when the light is transmitted to the second photodiode 12, as the depth of the second photodiode 12 increases, the blue light is almost completely absorbed, and the absorption coefficient of green light is the highest at this time, so the absorption ratio of green light by the second photodiode 12 is the highest; while the absorption coefficient of larger wavelengths, such as red light, is lower, about 1E+3cm -1 , so its photosensitivity process will run through the photodiodes in the entire photodiode assembly 10; if the depth of at least three photodiodes stacked along the incident direction of light is deep enough, the deeper the depth, the higher the proportion of red light absorption, for example, the third photodiode 13 has the highest proportion of red light absorption.

[0053] In some embodiments of the present application, when the photodiode assembly 10 includes three photodiodes, the three photodiodes are composed of an N-type semiconductor, a P-type semiconductor, an N-type semiconductor, and a P-type semiconductor stacked in sequence along the incident direction of light.

[0054] According to the basic theory of semiconductors, doping trivalent element impurities, such as boron, into single crystal silicon can form a P-type semiconductor, and doping pentavalent element impurities, such as phosphorus, can form an N-type semiconductor. The N-type semiconductor and the P-type semiconductor combine to form a PN junction.

[0055] It is worth mentioning that, for the photodiode in the photodiode assembly 10, it works in a reverse bias state by default. After light penetrates into a photodiode, if the photon energy E λ (E λ =hv) is greater than the energy band gap, the electrons will jump from the valence band to the conduction band to form free electrons. These free electrons form a photocurrent with directional movement under the action of the external electric field. From a macroscopic perspective, the process of electrons absorbing light energy and jumping corresponds to the absorption process of light, which is similar to Figure 3 The principle shown is consistent, that is, in the area with shallow incident depth, blue photons have higher frequency, so the energy of single photons is also higher, which makes it easier for electrons to transition to form photocurrent, which is manifested as a higher absorption coefficient of blue light from a macroscopic point of view. In other words, when the incident light enters the photodiode component 10, the light intensity will change with the different irradiation depths, and the absorption speed of different wavelengths will also change.

[0056] In some embodiments of the present application, three PN junctions are formed by using an N-type semiconductor, a P-type semiconductor, an N-type semiconductor and a P-type semiconductor stacked in sequence along the incident direction of light, thereby forming three photodiodes stacked in sequence along the incident direction of light, which can simplify the structure of the photodiode component 10.

[0057] It should be noted that by converting the absorbed light into an electrical signal by each photodiode in the photodiode assembly 10, the electrical signal can only reflect the total illuminance of the light absorbed by each photodiode, but cannot reflect the ratio of red, green and blue light absorbed by the photodiode, thereby making it impossible to determine the specific red pixel signal, green pixel signal and blue pixel signal.

[0058] For example: Figure 1In the pixel structure shown, if the total depth of the photodiode assembly 10 is 10 μm, the thickness of the first photodiode 11 along the incident direction of the light is 3 μm, the thickness of the second photodiode 12 along the incident direction of the light is 3 μm, and the thickness of the third photodiode 13 along the incident direction of the light is 4 μm. In this way, when the incident light is irradiated from the surface of the first photodiode 11 to a depth of 3 μm, since the light absorption coefficient of small wavelengths such as blue light is higher, that is, the absorption is faster, the blue light will excite 1000 photoelectrons in the range of 0-3 μm, that is, the first photodiode 11 excites 1000 photoelectrons due to the absorption of blue light, wherein the process of light energy being absorbed is also the process of photoelectron excitation; and the green light is absorbed slower than the blue light, so it will excite 500 photoelectrons, that is, the first photodiode 11 excites 500 photoelectrons due to the absorption of green light; the red light is absorbed more slowly, so it will excite 200 photoelectrons, that is, the first photodiode 11 excites 200 photoelectrons due to the absorption of red light. In summary, at this time, the first photodiode 11 generates a total of 1700 photoelectrons, and blue light accounts for the largest proportion. However, we cannot distinguish which wavelength of light these 1700 photoelectrons are generated by, that is, the pixel signal has "aliasing".

[0059] In addition, when the incident light further penetrates the photodiode assembly 10 and enters a deeper space, such as the 3-6um space where the second photodiode 12 is located, since most of the energy of the blue light has been lost in the first photodiode 11, only 400 photoelectrons can be generated, that is, the second photodiode 12 excites 400 photoelectrons due to the absorption of blue light; the loss of green light is less than that of blue light, and the absorption is faster than that of red light, and 1000 photoelectrons are generated at this time, that is, the second photodiode 12 excites 1000 photoelectrons due to the absorption of green light; the energy loss of red light is less, but because it is absorbed more slowly, only 600 electrons can be generated, that is, the second photodiode 12 excites 600 photoelectrons due to the absorption of red light. In summary, the second photodiode 12 generates a total of 2000 photoelectrons, and green light accounts for the highest proportion.

[0060] Similarly, in the 6-10um space where the third photodiode 13 is located, since the energy of blue light and green light has been largely absorbed by the first photodiode 11 and the second photodiode 12, the blue light and green light can only produce 100 and 300 electrons at this time, that is, the third photodiode 13 stimulates 100 and 300 photoelectrons respectively due to the absorption of blue and green light. Since red light is absorbed more slowly in the upper two layers, it is retained in deeper layers. Although its absorption is slow, if the depth of the photoelectrons can ensure that the red light is fully absorbed, the third photodiode 13 will produce 1000 photoelectrons, that is, the third photodiode 13 stimulates 1000 photoelectrons due to the absorption of red light. In summary, the third photodiode 13 generates a total of 1400 photoelectrons, and red light accounts for the highest proportion.

[0061] In some embodiments of the present application, since the material of the photodiode is silicon, its absorption of light satisfies Figure 3 According to the above rule, the blue light is absorbed quickly, that is, the blue light accounts for a higher proportion in the output signal of the first photodiode 11, the green light is slightly lower, and the red light is absorbed the slowest. The green light accounts for a higher proportion in the output signal of the second photodiode 12, and the red light accounts for a higher proportion in the output signal of the third photodiode 13. By obtaining the output signals of the three photodiodes stacked along the incident direction of the light, the output signal dominated by blue light, the output signal dominated by green light, and the output signal dominated by red light can be obtained respectively. Based on these three signals and in combination with the appropriate color deconstruction method, the R pixel signal, the G pixel signal, and the B pixel signal can be deciphered therefrom.

[0062] In some embodiments of the present application, the R pixel signal corresponding to the red light absorbed by the photodiode component 10 as a whole, the G pixel signal corresponding to the green light absorbed by the photodiode component 10 as a whole, and the B pixel signal corresponding to the blue light absorbed by the photodiode component 10 as a whole can be used as unknowns, and certain calculations can be performed on parameters such as the electrical signal output by each photodiode in the photodiode component 10 and the absorption ratio of red light, green light and blue light by each photodiode in the photodiode component 10 to obtain the above-mentioned R pixel signal, G pixel signal and B pixel signal, thereby realizing color deconstruction of the pixel structure.

[0063] In other embodiments, the R pixel signal corresponding to the red light absorbed by the photodiode assembly 10 as a whole, the G pixel signal corresponding to the green light absorbed by the photodiode assembly 10 as a whole, and the B pixel signal corresponding to the blue light absorbed by the photodiode assembly 10 as a whole can be used as unknowns, and the electrical signals output by each photodiode in the photodiode assembly 10 in multiple adjacent pixel structures and the absorption ratios of each photodiode in the photodiode assembly 10 to red light, green light and blue light can be calculated to obtain the above-mentioned R pixel signal, G pixel signal and B pixel signal, so as to realize the color deconstruction of the pixel structure. Among them, the light received by multiple adjacent pixel structures is approximately the same, so that the R pixel signal, G pixel signal and B pixel signal of the multiple adjacent pixel structures can be regarded as the same.

[0064] In some embodiments of the present application, the value X of the output signal of the photodiode can be expressed as the following photoelectric conversion formula:

[0065] X=K(m1×R+n1×G+p1×B); (2)

[0066] Among them, K is the conversion coefficient of the photoelectric conversion of the photodiode, and the unit is μV / lux; R, G and B are the illuminance of red, green and blue light incident per unit time, respectively, and the unit is lux*s; usually, light with a wavelength of 390nm to 490nm is defined as blue light, light with a wavelength of 490nm to 590nm is defined as green light, and light with a wavelength of 590nm to 750nm is defined as red light; X is the voltage signal output by a single photodiode, and the unit is μV; m1, n1 and p1 are the absorption ratios of red, green and blue light by the photodiode, respectively.

[0067] For example, if m1=0.08, n1=0.2, and p1=0.65, this means that the proportion of blue light in the X signal is 65%, the proportion of green light is 20%, and the proportion of red light is 8%.

[0068] Based on the fact that the photodiode assembly includes at least three photodiodes stacked along the incident direction of light, at least three photoelectric conversion formulas can be listed to solve the three light intensities of R, G and B.

[0069] In some embodiments of the present application, if a pixel structure includes three photodiodes, the photoelectric conversion formulas of the three photodiodes in the current pixel structure can be combined with the photoelectric conversion formulas of the photodiodes in other adjacent pixel structures to jointly solve the three light intensities of R, G and B.

[0070] For example: Figure 8 As shown, in the pixel array of the image sensor, pixels are periodically arranged in a manner of four adjacent pixel structures as one pixel unit, wherein one pixel unit includes pixel A, pixel B, pixel C, and pixel D;

[0071] The analog electrical signals of the photodiodes located in the first layer among the pixels A, B, C and D are used to determine the B pixel signal; the analog electrical signals of the photodiodes located in the second layer among the pixels A, B, C and D are used to determine the G pixel signal; the analog electrical signals of the photodiodes located in the third layer among the pixels A, B, C and D are used to determine the R pixel signal, and the first layer, the second layer and the third layer are stacked in sequence along the incident direction of light.

[0072] like Fig. 9As shown, if pixel A includes three photodiodes stacked in sequence along the light incident direction: PD1A, PD2A and PD3A; pixel B includes three photodiodes stacked in sequence along the light incident direction: PD1B, PD2B and PD3B; pixel C includes three photodiodes stacked in sequence along the light incident direction: PD1C, PD2C and PD3C; pixel D includes three photodiodes stacked in sequence along the light incident direction: PD1D, PD2D and PD3D; then the photodiodes of the first layer include PD1A, PD1B, PD1C and PD1D, the photodiodes of the second layer include PD2A, PD2, PD2C and PD2D; the photodiodes of the third layer include PD3A, PD3B, PD3C and PD3D.

[0073] Thus, taking the first layer of photodiodes as an example, the following four photoelectric conversion formulas can be listed:

[0074] X1= K(m11*R+m12*G+m13*B); (3)

[0075] X2= K(m21*R+m22*G+m23*B); (4)

[0076] X3= K(m31*R+m32*G+m33*B); (5)

[0077] X4= K(m41*R+m42*G+m43*B); (6)

[0078] Among them, X1, X2, X3 and X4 are the values ​​of the output signals of PD1A, PD1B, PD1C and PD1D respectively; K is the photoelectric conversion coefficient; m11, m12 and m13 are the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of PD1A respectively; m21, m22 and m23 are the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of PD1B respectively; m31, m32 and m33 are the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of PD1C respectively; m41, m42 and m43 are the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of PD1D respectively; R is the R pixel illumination; G is the G pixel illumination; B is the B pixel illumination.

[0079] In this way, based on the above four photoelectric conversion formulas, the illumination of the three pixels R, G, and B can be calculated.

[0080] It is worth mentioning that, since the first-layer photodiode absorbs the highest proportion of blue light, the output signal of the first-layer photodiode has the highest proportion of blue light. For example, X is an electrical signal generated based on 1600 photoelectrons, of which 1000 photoelectrons are photoelectrons excited by absorbing blue light, 400 photoelectrons are photoelectrons excited by absorbing green light, and 200 photoelectrons are photoelectrons excited by absorbing red light. In this way, since the number of photoelectrons excited by absorbing blue light is the largest in the output signal of the first-layer photodiode, the B pixel illumination is calculated based on the output signal of the first-layer photodiode, which can increase the value of the calculated B pixel illumination and reduce the relative error of the B pixel illumination. Therefore, the blue pixel illumination calculated based on the four photoelectric conversion formulas of the first-layer photodiode can be used as the basis for the final calculation of the B pixel signal, so that the reliability of the calculated B pixel signal can be improved.

[0081] In addition, the second layer photodiode can absorb the highest proportion of green light, and the third layer photodiode can absorb the highest proportion of red light. The same principle as above, because the number of photoelectrons excited by absorbing blue light in the output signal of the first layer photodiode is the largest, the B pixel illumination is calculated based on the output signal of the first layer photodiode, and the relative error of the B pixel illumination can be reduced. For the G pixel signal, the green pixel illumination calculated by the four photoelectric conversion formulas of the second layer photodiode can be used as the basis for calculation; for the R pixel signal, the red pixel illumination calculated by the four photoelectric conversion formulas of the third layer photodiode can be used as the basis for calculation. In this way, the relative error of the calculated G pixel illumination and R pixel illumination can be reduced, thereby improving the reliability of the calculated G pixel signal and R pixel signal.

[0082] In other embodiments, if a pixel structure includes greater than or equal to 4 photodiodes, the three pixel illuminations of R, G and B can be solved directly based on at least 4 photoelectric conversion formulas of at least 4 photodiodes.

[0083] For example, if the output signals of the four photodiodes stacked in sequence along the incident direction of the light in the pixel structure are X, Y, W and Z respectively, the following four photoelectric conversion formulas can be obtained:

[0084] X= K(m1*R+n1*G+p1*B); (7)

[0085] Y= K(m2*R+n2*G+p2*B); (8)

[0086] W= K(m3*R+n3*G+p3*B); (9)

[0087] Z= K(m4*R+n4*G+p4*B); (10)

[0088] Among them, X, Y, W and Z are respectively digital signals of four photodiodes stacked in sequence along the incident direction of light in the pixel structure; K is a conversion coefficient; m1, n1 and p1 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X; m2, n2 and p2 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y; m3, n3 and p3 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to W; m4, n4 and p4 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Z; R is the R pixel illumination; G is the G pixel illumination; B is the B pixel illumination.

[0089] In this way, based on the above four photoelectric conversion formulas, the three pixel illuminations of R, G, and B can be calculated, and then based on the three pixel illuminations of R, G, and B, the pixel signals of R, G, and B can be calculated.

[0090] In some implementations of the present application, gain conversion processing may be implemented based on the following formula to calculate pixel signals of R, G, and B:

[0091] R signal =K× R lux ; (11)

[0092] G signal =K× G lux ; (12)

[0093] B signal =K× B lux ; (13)

[0094] Among them, R signal is the R pixel signal, G signal is the G pixel signal, B signal is the B pixel signal; R lux is the R pixel illumination; G lux is the G pixel illumination; B lux is the illumination of pixel B; K is the conversion coefficient of the image sensor.

[0095] It is worth mentioning that the above formula is only an example. In some possible implementations, the number and type of parameters in the above formula can be adjusted, which is not specifically limited here. In addition to formula calculation, other methods can be used to achieve color deconstruction, such as using an artificial intelligence (AI) calculation model to perform color deconstruction processing on the output signal value of the photodiode and the absorption ratio of the photodiode to red, green and blue light to obtain R pixel signals, G pixel signals and B pixel signals.

[0096] In some embodiments of the present application, the absorption ratio of each photodiode in the photodiode assembly 10 to red light, green light and blue light can be obtained through simulation measurement, or can be calculated based on the structural parameters of each photodiode in the photodiode assembly 10. In this way, the absorption ratio coefficient of each photodiode in the photodiode assembly 10 to red light, green light and blue light can be determined in advance, so that in the color deconstruction process, the absorption ratio coefficient can be used as a calculation basis, which can simplify the color deconstruction process and save the computing power of the color deconstruction process.

[0097] In some embodiments of the present application, Figure 4 As shown, the pixel structure also includes:

[0098] The reading module 20 is electrically connected to the photodiode assembly 10 and is used to read the analog electrical signals of the at least three photodiodes.

[0099] In some embodiments of the present application, the reading module 20 can transmit the analog electrical signals of the at least three photodiodes to an external circuit for signal processing to obtain the R pixel signal, the G pixel signal, and the B pixel signal of the pixel structure. For example, the analog electrical signals of the at least three photodiodes are transmitted to an ADC circuit, and after being converted into digital signals by the ADC circuit, the color deconstruction function module in the ISP performs color deconstruction processing on the digital signals to obtain the R pixel signal, the G pixel signal, and the B pixel signal of the pixel structure.

[0100] In some embodiments of the present application, the reading module 20 reads the analog electrical signals of the at least three photodiodes, indicating that the reading module 20 reads the voltage signal of each of the at least three photodiodes, wherein the voltage signal can reflect the illuminance of the light absorbed by the corresponding photodiode, and the illuminance is proportional to the pixel signal, so that after decomposing the illuminance of red, green and blue light in the light absorbed by the pixel structure, the illuminance of red, green and blue light can be gain converted to obtain R pixel signal, G pixel signal and B pixel signal.

[0101] In some embodiments of the present application, a reading module 20 is used to read the analog electrical signal of each photodiode in the photodiode assembly 10, so that the analog electrical signals of at least three photodiodes read by the reading module 20 can be used as a basis for color deconstruction processing, providing a data basis for determining the R pixel signal, G pixel signal and B pixel signal of the pixel structure.

[0102] In this embodiment, a reading module 20 is provided in the photodiode component 10, so that the analog electrical signals of at least three photodiodes in the photodiode component 10 can be read. There is no need to provide an independent reading circuit for each photodiode in the photodiode component 10, which can simplify the structure of the image sensor.

[0103] As an optional implementation, Figure 4 As shown, the read module 20 includes a transfer gate (TG) module 21, a floating diffusion (FD) node 22, a reset (RST) switch 23, a source follower (SF) circuit 24 and a select (SEL) switch 25;

[0104] A first end of the TG module 21 is electrically connected to the anode of each photodiode in the photodiode assembly 10, and a second end of the TG module 21 is electrically connected to a first end of the FD node 22 and a first end of the RST switch 23, respectively;

[0105] The second end of the RST switch 23 is connected to the power supply;

[0106] A second end of the FD node 22 is electrically connected to a second end of the SF circuit 24, and a third end of the FD node 22 is grounded;

[0107] The second end of the SF circuit 24 is connected to the power supply, the third end of the SF circuit 24 is electrically connected to the first end of the SEL switch 25, and the second end of the SEL switch 25 is the output end of the reading module 20;

[0108] The TG module 21 is used to control the reading module 20 to read the analog electrical signal of each photodiode of the at least three photodiodes respectively.

[0109] In some embodiments of the present application, the TG module 21 can adjust the photodiode connected to the FD node 22 so that each of the at least three photodiodes is connected to the FD node 22 in a certain order, so that the reading module 20 can read the analog electrical signal of each of the at least three photodiodes respectively in this order.

[0110] For example: Figure 5 As shown, if the at least three photodiodes include a first photodiode 11, a second photodiode 12 and a third photodiode 13;

[0111] The TG module 21 includes a first TG switch 211, a second TG switch 212, a third TG switch 213 and a fourth TG switch 214; wherein, the first end of the first TG switch 211 is electrically connected to the anode of the first photodiode 11, the first end of the second TG switch 212 is electrically connected to the anode of the second photodiode 12, the first end of the third TG switch 213 is electrically connected to the anode of the third photodiode 13, the second end of the first TG switch 211, the second end of the second TG switch 212 and the second end of the third TG switch 213 are respectively electrically connected to the first end of the fourth TG switch 214, and the second end of the fourth TG switch 214 is respectively electrically connected to the first end of the FD node 22 and the first end of the RST switch 23, that is, the second end of the fourth TG switch 214 serves as the second end of the TG module 21.

[0112] When the first TG switch 211 and the fourth TG switch 214 are closed, and the second TG switch 212 and the third TG switch 213 are disconnected, the TG module 21 is used to control the reading module 20 to read the analog electrical signal output by the first photodiode 11; when the second TG switch 212 and the fourth TG switch 214 are closed, and the first TG switch 211 and the third TG switch 213 are disconnected, the TG module 21 is used to control the reading module 20 to read the analog electrical signal output by the second photodiode 12; when the third TG switch 213 and the fourth TG switch 214 are closed, and the first TG switch 211 and the second TG switch 212 are disconnected, the TG module 21 is used to control the reading module 20 to read the analog electrical signal output by the third photodiode 13.

[0113] During operation, by switching the switching state of each TG switch in the TG module 21, the photodiode corresponding to the analog electrical signal read by the reading module 20 can be changed. In this way, the reading module 20 can adopt a time-sharing reading method to read the analog electrical signals of the at least three photodiodes in sequence according to a certain order.

[0114] For example, the TG module 21 can select one of the first photodiode 11, the second photodiode 12, and the third photodiode 13 to be connected to the FD node 22. For example, if the first photodiode 11 is connected to the FD node 22 through the TG module 21, the first photodiode 11 converts the optical signal into photoelectrons, and the charge is converted into voltage through the FD node 22 and read out through the SF circuit 24 and the SEL switch 25. In this way, the analog electrical signal output by the first photodiode 11 can be read out. In a similar manner, when the second photodiode 12 is connected to the FD node 22 through the TG module 21, the second photodiode 12 is connected to the FD node 23. 2, after the second photodiode 12 converts the optical signal into photoelectrons, the charges are converted into voltages through the FD node 22, and are read out through the SF circuit 24 and the SEL switch 25, so that the analog electrical signal output by the second photodiode 12 can be read out; when the third photodiode 13 is connected to the FD node 22 through the TG module 21, after the third photodiode 13 converts the optical signal into photoelectrons, the charges are converted into voltages through the FD node 22, and are read out through the SF circuit 24 and the SEL switch 25, so that the analog electrical signal output by the third photodiode 13 can be read out.

[0115] In the above-mentioned embodiments of the present application, on the one hand, by setting a corresponding TG switch for each photodiode, when the output signal of a certain photodiode is read through the reading module 20, the TG switch corresponding to the photodiode is controlled to be turned on, and the TG switches corresponding to other photodiodes are turned off, so that crosstalk between the output signals of different photodiodes can be effectively avoided; on the other hand, through the four TG switches, during the signal reading process of the current frame, the exposure of the next frame can be started by disconnecting the fourth TG switch 214, which can shorten the time interval between two adjacent frames of images, thereby improving the frame rate of the image sensor; and, through the added fourth TG switch 214, the Correlated Double Sampling (CDS) function can also be realized. For example, by turning on the fourth TG switch 214, the reset signal and the exposure signal are read respectively in the reset stage and the signal reading stage, wherein the reset signal is the noise when there is no light, and the reset noise can be suppressed by subtracting the reset signal and the exposure signal read respectively in the reset stage and the signal reading stage.

[0116] It should be noted that the SEL switch 25 is used for pixel selection. For example, for an image sensor with a large number of pixels, the signal read by the ISP can be selected to be the signal of the pixel where the SEL switch 25 is in the on state by controlling the switching state of the SEL switch 25 in each pixel.

[0117] In some embodiments of the present application, the FD node 22 and the SF circuit 24 can be reset by the RST switch 23. For example, after reading the analog electrical signal output by the first photodiode 11, the charge generated by the first photodiode 11 remains on the FD node 22 and the SF circuit 24. At this time, by controlling the RST switch 23 to be turned on, the potential on the FD node 22 and the SF circuit 24 can be reset to a default initial value, in preparation for reading the analog electrical signal output by the second photodiode 12 or the third photodiode 13.

[0118] In some embodiments of the present application, the reading module 20 may further include other devices or circuit structures, for example, Figure 5 As shown, the reading module 20 also includes a bias current source 26, and the second end of the SEL switch 25 is also electrically connected to the bias current source 26, so that the output voltage of the reading module 20 can be grounded according to the bias current of the bias current source 26 through the SF circuit 24 and the SEL switch 25.

[0119] In some embodiments of the present application, a reading circuit capable of reading the output signal of each photodiode in the pixel structure is designed in the pixel structure, so that a pixel structure has at least three output signals, and then by transmitting the at least three output signals to an external circuit, such as an analog-to-digital converter (ADC) circuit and an image signal processor (ISP) module of an image sensor, the external circuit can perform color deconstruction processing on the at least three output signals based on the absorption ratio of red light, green light and blue light corresponding to each output signal, so as to separate the illuminance of red light, green light and blue light absorbed by the pixel structure, and then perform gain conversion processing on the illuminance of red light, green light and blue light to obtain R pixel signal, G pixel signal and B pixel signal. In this embodiment, a pixel structure has at least three output signals, and based on the at least three output signals, three pixel values ​​of R, G and B can be deconstructed, and in the process of color deconstructing the signals output by at least three photodiodes, the analog electrical signals output by multiple photodiodes and the absorption ratios of red light, green light and blue light respectively absorbed by the multiple photodiodes are combined. The absorption ratio of green light and blue light is used to separate the red light illuminance, green light illuminance and blue light illuminance from the light incident on the photodiode component. Based on the fact that the R pixel signal of the pixel structure is proportional to the red light illuminance, the G pixel signal is proportional to the green light illuminance and the B pixel signal is proportional to the blue light illuminance, the R pixel signal, G pixel signal and B pixel signal of the pixel structure can be determined. Since the scheme can obtain RGB color information in the full pixel area and there is no loss caused by the absorption of light energy by the additional filter, the spatial resolution of the R pixel signal, G pixel signal and B pixel signal determined by color deconstruction is higher. At the same time, the signal-to-noise ratio of the image generated based on the R pixel signal, G pixel signal and B pixel signal is higher, thereby improving the imaging quality of the image sensor. Among them, the signal-to-noise ratio can represent the ratio of the pixel signal to the noise, for example, the signal-to-noise ratio is equal to S / N, S is the value of the pixel signal, and N is the noise value; the spatial resolution can represent the number of pixels per unit area, for example: the spatial resolution is equal to the number of R, G and B three-color pixels per unit area.

[0120] Some embodiments of the present application also provide an image sensor, such as Figure 6 As shown, the image sensor includes a pixel array 101, an ADC circuit 102 and an ISP 103, and the pixel array 101 includes any one of the pixel structures provided in the foregoing embodiments of the present application;

[0121] A first end of the ADC circuit 102 is electrically connected to a pixel structure in the pixel array 101, and a second end of the ADC circuit 102 is electrically connected to a module in the ISP 103. The ADC circuit 102 is used to convert an analog electrical signal of the photodiode component 10 of the pixel structure into a digital signal, and transmit the digital signal to the ISP 103;

[0122] The color deconstruction function module in the ISP 103 is used to obtain an R pixel signal, a G pixel signal, and a B pixel signal of each pixel in the pixel array 101 by performing color deconstruction processing on the digital signal.

[0123] It should be noted that the image sensor in the embodiment of the present application can be any type of image sensor that uses a photodiode for light sensing, including but not limited to any type of image sensor such as back-illuminated, front-illuminated or stacked.

[0124] In some embodiments of the present application, when the pixel structure includes a reading module 20, the first end of the ADC circuit 102 is electrically connected to the output end of the reading module 20, and is used to obtain the analog electrical signal, such as voltage, output by each photodiode of each pixel structure in the pixel array 101, and convert these analog electrical signals into corresponding digital signals, such as voltage values.

[0125] When the pixel structure does not include the reading module 20, the ADC circuit 102 can also obtain the analog electrical signal output by each photodiode of each pixel structure in the pixel array 101 in other ways, such as obtaining the current signal output by a photodiode of a pixel structure each time in a preset order, and outputting a corresponding digital signal according to the current size, etc., which is not specifically limited here.

[0126] In some embodiments of the present application, the color deconstruction function module in ISP 103 can perform color deconstruction processing on the digital signals of at least three photodiodes at a certain pixel position according to the red light absorption ratio, green light absorption ratio and blue light absorption ratio of at least three photodiodes at the pixel position to obtain the R pixel signal, G pixel signal and B pixel signal of the pixel position; and then based on the same method, obtain the R pixel signal, G pixel signal and B pixel signal of each pixel in the pixel array 101.

[0127] For example, the color deconstruction function module in ISP 103 can adopt a color deconstruction method that matches the pixel structure in the pixel array, wherein different pixel structures can correspond to different color deconstruction methods. Since the pixel structure in the image sensor is determined, the color deconstruction method adopted by the color deconstruction function module in ISP 103 corresponds to the pixel structure. For example, in the case where a single pixel structure includes 3 photodiodes, the color deconstruction function module in ISP 103 can perform color deconstruction based on the photoelectric conversion formula of each photodiode in the 2×2 pixel array, that is, formulas (15) to (17) in the present application; for another example, in the case where a single pixel structure includes 4 photodiodes, the color deconstruction function module in ISP 103 can perform color deconstruction based on the photoelectric conversion formula of the 4 photodiodes in the single pixel structure, that is, formulas (7) to (10) in the present application. According to the color deconstruction method, the color deconstruction function module in ISP 103 can deconstruct one by one to obtain the R pixel signal, G pixel signal and B pixel signal of each pixel in the pixel array 101.

[0128] In some embodiments of the present application, the image sensor may further include other structures or devices, such as Figure 8 As shown, the image sensor may further include an amplifier circuit 104, which is electrically connected between the pixel array 101 and the ADC circuit 102, and is used to amplify the analog electrical signals of the pixels in the pixel array 101, so that the ADC circuit 102 performs analog-to-digital conversion on the amplified analog electrical signals, thereby improving the accuracy of the analog-to-digital conversion while reducing the power consumption of the pixel array 101.

[0129] Among related technologies, the most widely used image sensor is the complementary metal-oxide-semiconductor (CMOS) image sensor (CIS). Compared with charge coupled devices (CCD), CMOS has lower cost and power consumption, and its readout method is more flexible and efficient, and its market share is increasing year by year.

[0130] The working mechanism of CIS is as follows: the pixels distributed in an array contain photodiodes, which convert the incident light signal into an electrical signal, and pass it through the amplification circuit and the analog-to-digital conversion (ADC) circuit to form a digital signal matrix, i.e., an image. It is then packaged and transmitted to the image signal processing (ISP) module for data processing according to the mobile industry processor interface (MIPI) protocol.

[0131] Based on the above devices, the workflow of the Camera Compact Module (CCM) is as follows: First, the Voice Coil Motor (VCM) drives the lens to focus, focusing the light carrying visual information through the lens, and filtering out the infrared band invisible to the human eye through the infrared cutoff filter, and imaging it to the image sensor. The photodiode converts the light signal into an analog electrical signal, and forms a digital signal matrix through amplification and digital-to-analog conversion, that is, the original (RAW) domain image. Then, after being processed by the color deconstruction function module in the ISP, it is compressed and stored in the memory or displayed on the display.

[0132] In the process of CIS photosensitivity, quantum efficiency (QE) is an important indicator to characterize the utilization rate of light energy: the higher the QE, the higher the conversion efficiency of optical signals to electrical signals, that is, the better the photosensitivity of the image sensor.

[0133] The definition of quantum efficiency is as follows:

[0134]

[0135] Among them, N e N is the number of electrons generated by a single photodiode after being exposed to light; ph is the number of photons incident on a unit pixel.

[0136] Image sensors are key components in the imaging chain, and their color reproduction capability is one of the important factors that affect the tone of later images. In related technologies, image sensors based on Bayer arrays are still the mainstream in the industry, that is, by placing three filters, R, G, and B, above each pixel, each filter acts as a bandpass filter for light of different wavelengths, that is, it can only selectively transmit light of certain wavelengths, thereby achieving color recording.

[0137] In the RAW image obtained by the Bayer array sensor, each pixel can only identify one color among R, G, and B. If you want to obtain a full-color image, you need to calculate the values ​​of the other two colors corresponding to each pixel position, which is called "demosaicing" (DMSC). At present, there are a series of related algorithms in the industry, which usually fill in the pixels with certain colors through interpolation to obtain a full-color image.

[0138] However, the above-mentioned Bayer array and the corresponding DMSC algorithm design have the following major defects:

[0139] 1) Information loss caused by interpolation algorithm:

[0140] The Bayer array structure means that each pixel can only obtain information of one wavelength among R, G, and B, such as the light intensity information of red, green, or blue. The information of the other two color channels needs to be obtained through interpolation calculation of neighboring pixels. This interpolation algorithm will introduce additional errors and loss of details, especially at the edges or high-frequency areas, which is prone to artifacts or distortion.

[0141] 2) Decreased spatial resolution:

[0142] Since the green filter in the Bayer array occupies half of the pixels to enhance the brightness resolution, the resolution of the red and blue channels is only 1 / 4 of the image, which causes the overall color resolution to be much lower than the brightness resolution, affecting the accuracy of color reproduction. This will cause the image to be prone to aliasing, also known as moiré.

[0143] 3) Low efficiency of light energy utilization:

[0144] The filters in the Bayer array absorb part of the light energy, so that the photodiode cannot fully utilize all the incident light. Especially in low-light environments, the signal-to-noise ratio will be significantly reduced.

[0145] In some embodiments of the present application, it is not necessary to set red, green and blue filters on the image sensor. On the one hand, the structural complexity of the camera module can be simplified, and light can be directly irradiated onto the pixel array 101 on the image sensor without penetrating the filter. In this way, the light energy utilization efficiency and spatial resolution of the image sensor can be improved. Moreover, in the process of color deconstruction of the signals output by at least three photodiodes, the analog electrical signals output by multiple photodiodes and the absorption ratios of the multiple photodiodes to red light, green light and blue light can be combined to separate the photodiode components incident on the pixel structure. The red light illuminance, green light illuminance and blue light illuminance in the light, and based on the pixel structure, the R pixel signal is proportional to the red light illuminance, the G pixel signal is proportional to the green light illuminance and the B pixel signal is proportional to the blue light illuminance, the R pixel signal, G pixel signal and B pixel signal of the pixel structure can be determined, and the spatial resolution of the R pixel signal, G pixel signal and B pixel signal is higher, and the signal-to-noise ratio of the image generated based on the R pixel signal, G pixel signal and B pixel signal is higher. Therefore, in some embodiments of the present application, the color deconstruction method adopted by the color deconstruction function module in the ISP has a higher spatial resolution.

[0146] In some embodiments of the present application, Fig. 9 As shown, at least four adjacent pixel structures in the pixel array 101 are distributed in a rectangular array;

[0147] The B pixel signal is obtained by performing color deconstruction processing on the digital signals corresponding to the photodiodes located in the first layer in the at least four adjacent pixel structures;

[0148] The G pixel signal is obtained by performing color deconstruction processing on digital signals corresponding to the photodiodes located in the second layer in the at least four adjacent pixel structures;

[0149] The R pixel signal is obtained by performing color deconstruction processing on the digital signals corresponding to the photodiodes located in the third layer in the at least four adjacent pixel structures;

[0150] The second layer is sandwiched between the first layer and the third layer, and the first layer is located on a side of the second layer facing the incident light.

[0151] It should be noted that if Figure 8 As shown, the four adjacent pixel structures are distributed in a 2×2 array.

[0152] In some embodiments of the present application, four adjacent pixel structures distributed in a rectangular array can be regarded as a pixel unit, which has unified R, G and B pixel signals, that is, the analog electrical signals of the four pixel structures in one pixel unit and the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of each photodiode in the four pixel structures, which are used to determine a set of R, G and B pixel signals.

[0153] It should be noted that the photodiodes located in the first layer of the at least four adjacent pixel structures, due to their shallow depth, mainly absorb blue light with a shorter wavelength, that is, the analog electrical signal output by the photodiodes of the first layer is a photoelectric conversion signal mainly absorbing blue light; the photodiodes located in the second layer of the at least four adjacent pixel structures, due to their slightly deeper depth than the photodiodes of the first layer, mainly absorb green light with a slightly longer wavelength than blue light, that is, the analog electrical signal output by the photodiodes of the second layer is a photoelectric conversion signal mainly absorbing green light; the photodiodes located in the third layer of the at least four adjacent pixel structures, due to their deeper depth than the photodiodes of the first and second layers, mainly absorb red light with a longer wavelength than blue and green light, that is, the analog electrical signal output by the photodiodes of the third layer is a photoelectric conversion signal mainly absorbing red light.

[0154] In the above-mentioned embodiments of the present application, the B pixel signal can be determined based on the first-layer photodiode analog electrical signal dominated by blue light, the G pixel signal can be determined based on the second-layer photodiode analog electrical signal dominated by green light, and the R pixel signal can be determined based on the third-layer photodiode analog electrical signal dominated by red light. This can reduce the relative errors of the calculated R pixel illuminance, G pixel illuminance and R pixel illuminance, and improve the reliability of the calculated R pixel signal, G pixel signal and R pixel signal.

[0155] In some embodiments of the present application, the red light absorption ratio coefficient, the green light absorption ratio coefficient and the blue light absorption ratio coefficient of the photodiode in different pixel structures in a pixel unit may be different, for example, Figure 8 and Fig. 9As shown, if a pixel unit includes four pixel structures distributed in a 2×2 array, namely pixel A, pixel B, pixel C, and pixel D, wherein pixel A includes three photodiodes stacked in sequence along the incident direction of light: PD1A, PD2A, and PD3A; pixel B includes three photodiodes stacked in sequence along the incident direction of light: PD1B, PD2B, and PD3B; pixel C includes three photodiodes stacked in sequence along the incident direction of light: PD1C, PD2C, and PD3C; pixel D includes three photodiodes stacked in sequence along the incident direction of light: PD1D, PD2D, and PD3D; wherein, for the photodiodes PD1A, PD1B, PD1C, and PD1D also located in the first layer, the red light absorption ratio coefficient, green light absorption ratio coefficient, and blue light absorption ratio coefficient of these four photodiodes may be different, such as the thicknesses of PD1A, PD1B, PD1C, and PD1D are different.

[0156] In this way, the photodiodes in different pixel structures can have different photoelectric conversion formulas to facilitate the calculation of the R pixel illumination, the G pixel illumination and the B pixel illumination.

[0157] In some embodiments of the present application, Figure 8 As shown, taking a 2×2 pixel structure as an example, the pixel array 101 includes a large number of pixel units, which may be periodically arranged in a 2×2 pixel structure.

[0158] In some embodiments of the present application, Figure 7 As shown, the ADC circuit 102 includes a first column ADC circuit 1021 and a second column ADC circuit 1022;

[0159] The first column ADC circuit 1021 is used to convert the analog electrical signals of the photodiode components in the pixel structures located in odd rows in the pixel array 101 into digital signals;

[0160] The second column ADC circuit 1022 is used to convert analog electrical signals of the photodiode components in the pixel structures located in even-numbered rows in the pixel array 101 into digital signals.

[0161] In this embodiment, since the image sensor needs to deconstruct a group of R, G, and B pixel signals through at least 4 pixel structures, and each pixel needs to read out the information of at least 3 photodiodes, the overall data volume is large. By adopting a dual-column ADC readout circuit design, the simultaneous readout of two rows of signals is achieved, which can shorten the readout time by half and help improve the image sensor's frame rate.

[0162] Some embodiments of the present application also provide a camera module, which includes any image sensor provided by the aforementioned embodiments of the present application.

[0163] The camera module (Camera Compact Module, CCM) is an important tool for acquiring visual information. In some embodiments of the present application, the camera module is generally composed of a lens (Lens), an infrared cutoff filter (IR Cut Filter), an image sensor (Image Sensor) and a voice coil motor (VCM), which are not specifically limited here.

[0164] The image sensor in the camera module provided in some embodiments of the present application does not need to set a corresponding filter for each pixel, and can allow light to directly irradiate the pixel array on the image sensor without passing through the filter. In this way, the light energy utilization efficiency and spatial resolution of the image sensor can be improved. Moreover, the color deconstruction method used in some embodiments of the present application can obtain all color information at each pixel point, which can improve the color restoration accuracy, which makes the color representation of the image more accurate, especially in the edges and detail areas, avoiding color aliasing.

[0165] An embodiment of the present application also provides an electronic device, which includes any camera module provided in the aforementioned embodiments of the present application.

[0166] Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. The non-mobile electronic device may also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which has a camera module, and the embodiments of the present application are not specifically limited thereto.

[0167] The electronic device provided in the embodiment of the present application can utilize the camera module provided in the aforementioned embodiment of the present application to improve the signal-to-noise ratio of images such as photos or videos taken using the camera module. To avoid repetition, it will not be described here.

[0168] Some embodiments of the present application also provide a color deconstruction method, which can be executed by the electronic device provided by the aforementioned embodiments of the present application.

[0169] like Fig.10 As shown, the color deconstruction method includes the following steps:

[0170] Step 1001: Obtain analog electrical signals of at least three photodiodes through an ADC circuit, wherein the at least three photodiodes are located in a photodiode component of a pixel structure.

[0171] In some embodiments of the present application, when the pixel structure includes a reading module, analog electrical signals of at least three photodiodes can be read by the reading module.

[0172] Step 1002: Convert the analog electrical signal into a digital signal through the ADC circuit.

[0173] In some embodiments of the present application, before converting the analog electrical signal into a digital signal, the analog electrical signal may be processed in a certain manner, such as amplified, and the amplified analog electrical signal may be converted into a digital signal.

[0174] Step 1003: Perform color deconstruction processing on the digital signals corresponding to the at least three photodiodes through the color deconstruction function module in the ISP to obtain the R pixel signal, the G pixel signal and the B pixel signal of the pixel structure.

[0175] In some embodiments of the present application, the red light absorption ratio, green light absorption ratio and blue light absorption ratio of each of at least three photodiodes can be pre-stored. At this time, the color deconstruction function module in the ISP can perform color deconstruction processing on the digital signals corresponding to the at least three photodiodes based on the red light absorption ratio, green light absorption ratio and blue light absorption ratio of each of the at least three photodiodes to obtain the R pixel signal, G pixel signal and B pixel signal of the pixel structure.

[0176] In some embodiments of the present application, an image sensor of an electronic device includes a pixel array having a large number of pixels. In this case, each pixel may adopt the same or different light absorption ratio.

[0177] For example, if the photodiodes in the same layer of the pixel array use the same set of absorption coefficients, where the set of absorption coefficients includes red light absorption coefficients, green light absorption coefficients, and blue light absorption coefficients, then the electronic device only needs to pre-store at least three sets of absorption coefficients, that is, a set of absorption coefficients corresponding to at least three layers of photodiodes.

[0178] As another example: if a pixel structure includes three photodiodes, and the photodiodes in at least four adjacent pixel structures in the pixel array, even if they are located in the same layer, use a different set of absorption coefficients, then the image sensor or camera module can pre-store at least 3×4 sets of absorption coefficients, that is, the absorption coefficient of each photodiode in each pixel structure in the at least four pixel structures.

[0179] In some embodiments of the present application, the photoelectric conversion formula described in the aforementioned embodiments of the present application can be used to calculate the R pixel illumination, G pixel illumination and B pixel illumination of the pixel structure, and the R pixel signal, G pixel signal and B pixel signal of the pixel structure can be calculated based on the R pixel illumination, G pixel illumination and B pixel illumination.

[0180] In other embodiments, an AI computing model can also be used to deconstruct the R pixel signal, G pixel signal, and B pixel signal of the pixel structure, such as using the absorption ratio of each of at least three photodiodes to red light, green light, and blue light, and the digital signals corresponding to the at least three photodiodes as input signals of the AI ​​computing model, so that the AI ​​computing model outputs the R pixel signal, G pixel signal, and B pixel signal of the pixel structure.

[0181] The color deconstruction method of the present application can realize color deconstruction of the pixel structure provided by the aforementioned embodiments of the present application to obtain pixel signals of the three colors R, G and B in the light absorbed by the pixel structure. Based on the color deconstruction method of the present application, all color information is obtained at each pixel point, so that the R, G, B pixel information obtained by the color structure is obtained based on the photoelectric conversion calculation of the real absorbed light, without the need for guessing color, which can improve the color restoration accuracy.

[0182] As an optional implementation, a pixel structure includes four photodiodes, and the color deconstruction function module in the ISP performs color deconstruction processing on the digital signals corresponding to the at least three photodiodes to obtain the R pixel signal, the G pixel signal and the B pixel signal of the pixel structure, including:

[0183] Calculate the R pixel illumination, the G pixel illumination and the B pixel illumination of the pixel structure according to the first set of formulas;

[0184] Performing gain conversion processing on the R pixel illumination to obtain an R pixel signal;

[0185] Performing gain conversion processing on the G pixel illumination to obtain a G pixel signal;

[0186] Performing gain conversion processing on the B pixel illumination to obtain a B pixel signal;

[0187] The first set of formulas is as follows:

[0188] X= K(m1*R+n1*G+p1*B); (7)

[0189] Y= K(m2*R+n2*G+p2*B); (8)

[0190] W= K(m3*R+n3*G+p3*B); (9)

[0191] Z= K(m4*R+n4*G+p4*B); (10)

[0192] Among them, X, Y, W and Z are respectively digital signals of four photodiodes stacked in sequence along the incident direction of light in the pixel structure; K is a conversion coefficient; m1, n1 and p1 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X; m2, n2 and p2 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y; m3, n3 and p3 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to W; m4, n4 and p4 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Z; R is the R pixel illumination; G is the G pixel illumination; B is the B pixel illumination.

[0193] In this embodiment, when a pixel structure includes four photodiodes stacked along the propagation direction of light, the R pixel illumination, G pixel illumination and B pixel illumination can be calculated based on the photoelectric conversion formulas of the four photodiodes respectively, and the R pixel signal, G pixel signal and B pixel signal can be determined based on the R pixel illumination, G pixel illumination and B pixel illumination respectively. In this way, the three types of pixel information R, G and B can be deconstructed in a single pixel structure, so that all color information can be obtained at each pixel structure, which can improve the color reproduction accuracy.

[0194] It is worth mentioning that when a pixel structure includes more than 4 photodiodes, one or more groups of R pixel illumination, G pixel illumination and B pixel illumination can be calculated based on the photoelectric conversion formulas of more than 4 photodiodes, and gain conversion processing can be performed on at least one group of R pixel illumination, G pixel illumination and B pixel illumination to obtain R pixel signals, G pixel signals and B pixel signals.

[0195] For example: when multiple groups of R pixel illumination, G pixel illumination and B pixel illumination are calculated, the corresponding average R pixel illumination, average G pixel illumination and average B pixel illumination of the multiple groups of R pixel illumination, G pixel illumination and B pixel illumination can be taken, and gain conversion processing can be performed on the average R pixel illumination, average G pixel illumination and average B pixel illumination respectively to obtain R pixel signal, G pixel signal and B pixel signal.

[0196] As an optional implementation, a pixel structure includes three photodiodes, and four adjacent pixel structures are distributed in a rectangular array, and the color deconstruction function module in the ISP performs color deconstruction processing on the digital signals corresponding to the at least three photodiodes to obtain the R pixel signal, the G pixel signal and the B pixel signal of the pixel structure, including:

[0197] According to the second group of formulas, the B pixel illumination of the four adjacent pixel structures is determined, and the B pixel illumination is subjected to gain conversion processing to obtain a B pixel signal; wherein the second group of formulas is a photoelectric conversion formula of a first photodiode array in the four adjacent pixel structures, and the first photodiode array includes four photodiodes located in a first layer in the four adjacent pixel structures;

[0198] According to a third group of formulas, the G pixel illumination of the four adjacent pixel structures is determined, and the G pixel illumination is subjected to gain conversion processing to obtain a G pixel signal; wherein the third group of formulas is a photoelectric conversion formula of a second photodiode array in the four adjacent pixel structures, and the second photodiode array includes four photodiodes located in a second layer in the four adjacent pixel structures;

[0199] According to the fourth group of formulas, the R pixel illumination of the four adjacent pixel structures is determined, and the R pixel illumination is gain converted to obtain an R pixel signal; wherein the fourth group of formulas are photoelectric conversion formulas of the third photodiode array in the four adjacent pixel structures, and the third photodiode array includes four photodiodes located in the third layer of the four adjacent pixel structures.

[0200] Similar to the aforementioned pixel structure embodiment, the three photodiodes of a pixel structure mainly absorb blue light with a shorter wavelength, green light with a medium wavelength, and red light with a longer wavelength, respectively. In this way, the output signals of the three photodiodes stacked along the incident direction of the light are respectively an output signal dominated by blue light, an output signal dominated by green light, and an output signal dominated by red light.

[0201] In some embodiments of the present application, the B pixel illumination of the four adjacent pixel structures can be determined according to the second group of formulas, and then the B pixel illumination can be gain-converted to obtain the B pixel signal; similarly, the G pixel illumination of the four adjacent pixel structures can be determined according to the third group of formulas, and then the G pixel illumination can be gain-converted to obtain the G pixel signal; in addition, the R pixel illumination of the four adjacent pixel structures can be determined according to the fourth group of formulas, and then the R pixel illumination can be gain-converted to obtain the R pixel signal.

[0202] In this embodiment, the B pixel illumination of the four adjacent pixel structures can be determined according to the output signal dominated by blue light; the G pixel illumination of the four adjacent pixel structures can be determined according to the output signal dominated by green light; and the R pixel illumination of the four adjacent pixel structures can be determined according to the output signal dominated by red light, thereby improving the reliability of the calculated G pixel signal and R pixel signal.

[0203] In some embodiments of the present application, the second set of formulas is as follows:

[0204]

[0205] Among them, X1, X2, X3 and X4 are respectively digital signals of the four photodiodes in the first photodiode array; K is the conversion coefficient; m11, m12 and m13 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X1; m21, m22 and m23 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X2; m31, m32 and m33 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X3; m41, m42 and m43 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X4; R1 is the R pixel illuminance of the first photodiode array; G1 is the G pixel illuminance of the first photodiode array; B1 is the B pixel illuminance of the first photodiode array.

[0206] In some embodiments of the present application, the third set of formulas is as follows:

[0207]

[0208] Among them, Y1, Y2, Y3 and Y4 are respectively the digital signals of the four photodiodes in the second photodiode array; K is the conversion coefficient; h11, h12 and h13 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y1; h21, h22 and h23 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y2; h31, h32 and h33 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y3; h41, h42 and h43 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y4; R2 is the R pixel illuminance of the second photodiode array; G2 is the G pixel illuminance of the second photodiode array; B2 is the B pixel illuminance of the second photodiode array.

[0209] In some embodiments of the present application, the fourth set of formulas is as follows:

[0210]

[0211] Among them, W1, W2, W3 and W4 are respectively digital signals of the four photodiodes in the third photodiode array; K is the conversion coefficient; l11, l12 and l13 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W1; l21, l22 and l23 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W2; l31, l32 and l33 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W3; l41, l42 and l43 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W4; R3 is the R pixel illuminance of the third photodiode array; G3 is the G pixel illuminance of the third photodiode array; B3 is the B pixel illuminance of the third photodiode array.

[0212] It is worth mentioning that when there are less than 4 photodiodes in a pixel structure, it may be impossible to calculate the R pixel illumination, G pixel illumination and B pixel illumination of the pixel structure based on the photoelectric conversion formula of the photodiode in the same pixel structure. For example, when three photoelectric conversion formulas are used to solve the three unknowns of R pixel illumination, G pixel illumination and B pixel illumination, the calculation results of R pixel illumination, G pixel illumination and B pixel illumination cannot be solved.

[0213] In this embodiment, the photoelectric conversion formula of the photodiodes in the adjacent four pixel structures can be combined to solve the R pixel illumination, the G pixel illumination and the B pixel illumination. Moreover, the B pixel signal is determined based on the B pixel illumination obtained by solving the photoelectric conversion formula of the first layer of photodiodes that mainly absorb blue light, the G pixel signal is determined based on the G pixel illumination obtained by solving the photoelectric conversion formula of the second layer of photodiodes that mainly absorb green light, and the R pixel signal is determined based on the R pixel illumination obtained by solving the photoelectric conversion formula of the third layer of photodiodes that mainly absorb red light. This can improve the reliability of the B pixel signal, G pixel signal and R pixel signal finally calculated.

[0214] It is worth mentioning that the above formulas (15) to (17) in the embodiments of the present application are only examples. In some possible implementations, the above formulas (15) to (17) can be adaptively adjusted, such as adjusting the type and number of parameters in the formulas according to the number of photodiodes in the pixel array. The first group of formulas, the second group of formulas and the third group of formulas are not limited to the above formulas (15) to (17).

[0215] For ease of understanding, the color deconstruction method of this application is illustrated by taking the following scenario as an example:

[0216] like Fig. 9 As shown, in the pixel array of the image sensor, pixels are periodically arranged in a manner that four adjacent pixel structures are used as a pixel unit, wherein one pixel unit includes pixel A, pixel B, pixel C, and pixel D; wherein each pixel structure includes Figure 3 The three photodiodes shown are arranged in the direction of light propagation. For example, pixel A includes three photodiodes stacked in sequence along the light incident direction: PD1A, PD2A and PD3A; pixel B includes three photodiodes stacked in sequence along the light incident direction: PD1B, PD2B and PD3B; pixel C includes three photodiodes stacked in sequence along the light incident direction: PD1C, PD2C and PD3C; pixel D includes three photodiodes stacked in sequence along the light incident direction: PD1D, PD2D and PD3D;

[0217] The following photoelectric conversion formulas can be obtained for the four photodiodes located in the first layer in the four adjacent pixel structures: PD1A, PD1B, PD1C and PD1D:

[0218]

[0219] Taking the first layer of photodiodes as an example, if the thickness Z11 of PD1A is as small as 1μm, PD1A absorbs less blue light, m13=0.3; the thickness Z21 of PD1B is relatively larger, 2μm, and it can absorb more blue light, m23=0.5; the thickness Z31 of PD1C continues to increase to 3μm, m33=0.7; the thickness Z41 of PD1D is 3.5μm, m43=0.8.

[0220] It can be seen that the photoelectric coefficient of the above equations can be changed by changing the thickness of the photodiode. In the design stage of the image sensor, the thickness of different photosensitive layers can be designed as needed, and then the absorption ratio coefficient in the above formula can be simulated or calculated according to the light absorption characteristics.

[0221] If the conversion coefficient K of the image sensor is 0.005μV / lux, by designing the thickness of each layer of photodiodes in the four pixels, taking the first layer of photodiodes as an example, its matrix coefficient is:

[0222]

[0223] At this time, the output signals of the first layer photodiodes PD1A, PD1B, PD1C and PD1D are: X1 = 74μV; X2 = 125μV; X3 = 191μV; X4 = 224μV. Then, the R, G, and B pixel illuminations at the pixel position can be calculated as follows: the red pixel illumination R lux The green pixel illuminance is 100 lux. lux is 200 lux, blue pixel illuminance B lux is 150 lux, and the corresponding R pixel signal, G pixel signal, and B pixel signal can be calculated based on the following:

[0224] R signal =K× R lux ; (11)

[0225] G signal =K× G lux ; (12)

[0226] B signal =K× B lux ; (13)

[0227] Among them, R signal is the R pixel signal, G signal is the G pixel signal, B signal is the B pixel signal; R lux is the R pixel illumination; G lux is the G pixel illumination; B luxis the illumination of pixel B; K is the conversion coefficient of the image sensor. signal It is equal to 0.005μV / lux multiplied by 100lux, which is 0.5μV. Similarly, we can solve for G signal 1μV, B signal is 0.75μV.

[0228] Although the above method only uses X1 to X4 signals to solve the R pixel signal, G pixel signal and B pixel signal, considering that X1 to X4 are the signals of the photodiodes of the first layer, the absorption of green light and red light is relatively low at this time, the R signal and G signal More susceptible to noise, in this case, the B can be calculated based on the X1 to X4 signals. signal The value of G is calculated based on the Y1-Y4 signal signal The value of R is calculated based on the W1-W4 signal. signal The value of .

[0229] Among them, the following photoelectric conversion formula is based on the four photodiodes located in the second layer in the above four adjacent pixel structures: PD2A, PD2, PD2C and PD2D:

[0230]

[0231] Based on the following photoelectric conversion formula of the four photodiodes located in the third layer in the above four adjacent pixel structures: PD3A, PD3B, PD3C and PD3D:

[0232]

[0233] The definitions of the above formulas refer to the explanations in the aforementioned pixel structure of this application and will not be repeated here.

[0234] Among them, G is calculated based on the Y1-Y4 signal signal The value of R is calculated based on the W1-W4 signal. signal The value of B is calculated based on the X1 to X4 signals. signal The principle and process of setting the value of are the same and will not be repeated here.

[0235] Fig.11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0236] The electronic device 1100 includes but is not limited to: a radio frequency unit 1101, a network module 1102, an audio output unit 1103, an input unit 1104, a sensor 1105, a display unit 1106, a user input unit 1107, an interface unit 1108, a memory 1109, a processor 1110 and a camera module 1111 and other components.

[0237] Those skilled in the art will appreciate that the electronic device 1100 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.11 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0238] The camera module 1111 includes any image sensor provided in the aforementioned embodiments of the present application, and the camera module 1111 is used for:

[0239] Acquiring analog electrical signals of at least three photodiodes through an ADC circuit, wherein the at least three photodiodes are located in a photodiode assembly of a pixel structure;

[0240] Converting the analog electrical signal into a digital signal through the ADC circuit;

[0241] The color deconstruction function module in the ISP performs color deconstruction processing on the digital signals corresponding to the at least three photodiodes to obtain the R pixel signal, the G pixel signal and the B pixel signal of the pixel structure.

[0242] In some embodiments of the present application, a pixel structure includes 4 photodiodes;

[0243] The color deconstruction function module in the ISP performs color deconstruction processing on the digital signals corresponding to the at least three photodiodes to obtain the R pixel signal, the G pixel signal and the B pixel signal of the pixel structure, including:

[0244] According to the first set of formulas, calculating the R pixel illumination, the G pixel illumination and the B pixel illumination of the pixel structure;

[0245] Performing gain conversion processing on the R pixel illumination to obtain an R pixel signal;

[0246] Performing gain conversion processing on the G pixel illumination to obtain a G pixel signal;

[0247] Performing gain conversion processing on the B pixel illumination to obtain a B pixel signal;

[0248] The first set of formulas is as follows:

[0249] X= K(m1*R+n1*G+p1*B); (7)

[0250] Y= K(m2*R+n2*G+p2*B); (8)

[0251] W= K(m3*R+n3*G+p3*B); (9)

[0252] Z= K(m4*R+n4*G+p4*B); (10)

[0253] Among them, X, Y, W and Z are respectively digital signals of four photodiodes stacked in sequence along the incident direction of light in the pixel structure; K is a conversion coefficient; m1, n1 and p1 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X; m2, n2 and p2 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y; m3, n3 and p3 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to W; m4, n4 and p4 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Z; R is the R pixel illumination; G is the G pixel illumination; B is the B pixel illumination.

[0254] In some embodiments of the present application, a pixel structure includes three photodiodes, and four adjacent pixel structures are distributed in a rectangular array;

[0255] The color deconstruction function module in the ISP performs color deconstruction processing on the digital signals corresponding to the at least three photodiodes to obtain the R pixel signal, the G pixel signal and the B pixel signal of the pixel structure, including:

[0256] According to the second group of formulas, the B pixel illumination of the four adjacent pixel structures is determined, and the B pixel illumination is subjected to gain conversion processing to obtain a B pixel signal; wherein the second group of formulas is a photoelectric conversion formula of a first photodiode array in the four adjacent pixel structures, and the first photodiode array includes four photodiodes located in a first layer in the four adjacent pixel structures;

[0257] According to a third group of formulas, the G pixel illumination of the four adjacent pixel structures is determined, and the G pixel illumination is subjected to gain conversion processing to obtain a G pixel signal; wherein the third group of formulas is a photoelectric conversion formula of a second photodiode array in the four adjacent pixel structures, and the second photodiode array includes four photodiodes located in a second layer in the four adjacent pixel structures;

[0258] According to the fourth group of formulas, the R pixel illumination of the four adjacent pixel structures is determined, and the R pixel illumination is gain converted to obtain an R pixel signal; wherein the fourth group of formulas are photoelectric conversion formulas of the third photodiode array in the four adjacent pixel structures, and the third photodiode array includes four photodiodes located in the third layer of the four adjacent pixel structures.

[0259] In some embodiments of the present application, the second set of formulas is as follows:

[0260]

[0261] Among them, X1, X2, X3 and X4 are respectively digital signals of the four photodiodes in the first photodiode array; K is the conversion coefficient; m11, m12 and m13 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X1; m21, m22 and m23 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X2; m31, m32 and m33 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X3; m41, m42 and m43 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X4; R1 is the R pixel illuminance of the first photodiode array; G1 is the G pixel illuminance of the first photodiode array; B1 is the B pixel illuminance of the first photodiode array.

[0262] In some embodiments of the present application, the third set of formulas is as follows:

[0263]

[0264] Among them, Y1, Y2, Y3 and Y4 are respectively the digital signals of the four photodiodes in the second photodiode array; K is the conversion coefficient; h11, h12 and h13 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y1; h21, h22 and h23 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y2; h31, h32 and h33 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y3; h41, h42 and h43 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y4; R2 is the R pixel illuminance of the second photodiode array; G2 is the G pixel illuminance of the second photodiode array; B2 is the B pixel illuminance of the second photodiode array.

[0265] In some embodiments of the present application, the fourth set of formulas is as follows:

[0266]

[0267] Among them, W1, W2, W3 and W4 are respectively digital signals of the four photodiodes in the third photodiode array; K is the conversion coefficient; l11, l12 and l13 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W1; l21, l22 and l23 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W2; l31, l32 and l33 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W3; l41, l42 and l43 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W4; R3 is the R pixel illuminance of the third photodiode array; G3 is the G pixel illuminance of the third photodiode array; B3 is the B pixel illuminance of the third photodiode array.

[0268] It can be understood that the electronic device of the embodiment of the present application can achieve Fig.10 Each process of the color deconstruction method shown can obtain Fig.10 The color deconstruction method shown has similar beneficial effects, and will not be described again here to avoid repetition.

[0269] It should be understood that in the embodiment of the present application, the input unit 1104 may include a graphics processor (Graphics Processing Unit, GPU) 11041 and a microphone 11042, and the graphics processor 11041 processes the image data of the static picture or video obtained by the camera module 1111 in the video capture mode or the image capture mode. The display unit 1106 may include a display panel 11061, and the display panel 11061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1107 includes a touch panel 11071 and at least one of other input devices 11072. The touch panel 11071 is also called a touch screen. The touch panel 11071 may include two parts: a touch detection device and a touch controller. Other input devices 11072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0270] The memory 1109 can be used to store software programs and various data. The memory 1109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instruction required for at least one function, such as a sound playback function, an image playback function, and the like. In addition, the memory 1109 may include a volatile memory or a non-volatile memory, or the memory 1109 may include both a volatile and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0271] The processor 1110 may include one or more processing units; optionally, the processor 1110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1110.

[0272] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be noted that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0273] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium, such as ROM / RAM, a disk, or an optical disk, and includes a number of instructions for a terminal to execute the methods described in each embodiment of the present application, wherein the terminal can be a mobile phone, a computer, a server, or a network device, etc.

[0274] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A pixel structure, characterized in that: include: A photodiode assembly, wherein the photodiode assembly comprises at least three photodiodes, and the at least three photodiodes are stacked along the incident direction of light; Wherein, the red light absorption ratio, green light absorption ratio and blue light absorption ratio of the at least three photodiodes are all different.

2. The pixel structure according to claim 1, characterized in that: In the case where the photodiode assembly includes three photodiodes, the three photodiodes are composed of an N-type semiconductor, a P-type semiconductor, an N-type semiconductor and a P-type semiconductor which are sequentially stacked along the incident direction of light.

3. The pixel structure according to claim 1, characterized in that: Also includes: A reading module, the reading module is electrically connected to the photodiode assembly, and the reading module is used to read the analog electrical signals of the at least three photodiodes.

4. The pixel structure according to claim 3, characterized in that: The read module includes a transmission gate TG module, a floating diffusion FD node, a reset RST switch, a source follower SF circuit and a selection SEL switch; A first end of the TG module is electrically connected to an anode of each photodiode in the photodiode assembly, and a second end of the TG module is electrically connected to a first end of the FD node and a first end of the RST switch, respectively; The second end of the RST switch is connected to a power source; The second end of the FD node is electrically connected to the second end of the SF circuit, and the third end of the FD node is grounded; The second end of the SF circuit is connected to a power source, the third end of the SF circuit is electrically connected to the first end of the SEL switch, and the second end of the SEL switch is the output end of the reading module; The TG module is used to control the reading module to read the analog electrical signal of each photodiode of the at least three photodiodes respectively.

5. The pixel structure according to claim 4, characterized in that: The at least three photodiodes include a first photodiode, a second photodiode and a third photodiode; The TG module includes a first TG switch, a second TG switch, a third TG switch and a fourth TG switch; A first end of the first TG switch is electrically connected to the anode of the first photodiode, a first end of the second TG switch is electrically connected to the anode of the second photodiode, a first end of the third TG switch is electrically connected to the anode of the third photodiode, a second end of the first TG switch, a second end of the second TG switch, and a second end of the third TG switch are all electrically connected to a first end of the fourth TG switch, and a second end of the fourth TG switch is electrically connected to a first end of the FD node and a first end of the RST switch, respectively; When the first TG switch and the fourth TG switch are closed, and the second TG switch and the third TG switch are opened, the TG module is used to control the reading module to read the analog electrical signal of the first photodiode; When the second TG switch and the fourth TG switch are closed, and the first TG switch and the third TG switch are opened, the TG module is used to control the reading module to read the analog electrical signal of the second photodiode; When the third TG switch and the fourth TG switch are closed and the first TG switch and the second TG switch are opened, the TG module is used to control the reading module to read the analog electrical signal of the third photodiode.

6. An image sensor, characterized in that: The image sensor comprises a pixel array, an analog-to-digital conversion ADC circuit and an image signal processor ISP, wherein the pixel array comprises a pixel structure as described in any one of claims 1 to 5; A first end of the ADC circuit is electrically connected to a pixel structure in the pixel array, a second end of the ADC circuit is electrically connected to the ISP, and the ADC circuit is used to convert an analog electrical signal of a photodiode component of the pixel structure into a digital signal, and transmit the digital signal to the ISP; The color deconstruction function module in the ISP is used to perform color deconstruction processing on the digital signal to obtain an R pixel signal, a G pixel signal and a B pixel signal of each pixel in the pixel array.

7. The image sensor according to claim 6, characterized in that: At least four adjacent pixel structures in the pixel array are distributed in a rectangular array; The B pixel signal is obtained by performing color deconstruction processing on the digital signals corresponding to the photodiodes located in the first layer in the at least four adjacent pixel structures; The G pixel signal is obtained by performing color deconstruction processing on digital signals corresponding to the photodiodes located in the second layer in the at least four adjacent pixel structures; The R pixel signal is obtained by performing color deconstruction processing on the digital signals corresponding to the photodiodes located in the third layer in the at least four adjacent pixel structures; The first layer is sandwiched between the first layer and the third layer, and the first layer is located on a side of the second layer facing the incident light.

8. The image sensor according to claim 6 or 7, characterized in that: The ADC circuit includes a first column ADC circuit and a second column ADC circuit; The first column ADC circuit is used to convert analog electrical signals of the photodiode components in the pixel structures located in odd rows in the pixel array into digital signals; The second column ADC circuit is used to convert analog electrical signals of the photodiode components in the pixel structures located in even-numbered rows in the pixel array into digital signals.

9. A camera module, characterized in that: An image sensor comprising any one of claims 6 to 8.

10. An electronic device, characterized in that: Including the camera module described in claim 9.

11. A color deconstruction method, executed by the electronic device according to claim 10, characterized in that: The method comprises: Acquiring analog electrical signals of at least three photodiodes through an ADC circuit, wherein the at least three photodiodes are located in a photodiode assembly of the pixel structure; Converting the analog electrical signal into a digital signal through the ADC circuit; The color deconstruction function module in the ISP performs color deconstruction processing on the digital signals corresponding to the at least three photodiodes to obtain the R pixel signal, the G pixel signal and the B pixel signal of the pixel structure.

12. The color deconstruction method according to claim 11, characterized in that: A pixel structure includes 4 photodiodes; The color deconstruction function module in the ISP performs color deconstruction processing on the digital signals corresponding to the at least three photodiodes to obtain the R pixel signal, the G pixel signal and the B pixel signal of the pixel structure, including: According to the first set of formulas, calculating the R pixel illumination, the G pixel illumination and the B pixel illumination of the pixel structure; Performing gain conversion processing on the R pixel illumination to obtain an R pixel signal; Performing gain conversion processing on the G pixel illumination to obtain a G pixel signal; Performing gain conversion processing on the B pixel illumination to obtain a B pixel signal; The first set of formulas is as follows: X=K(m1*R+n1*G+p1*B); Y=K(m2*R+n2*G+p2*B); W = K (m3*R+n3*G+p3*B); Z=K(m4*R+n4*G+p4*B); Among them, X, Y, W and Z are respectively digital signals of four photodiodes stacked in sequence along the incident direction of light in the pixel structure; K is a conversion coefficient; m1, n1 and p1 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X; m2, n2 and p2 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y; m3, n3 and p3 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to W; m4, n4 and p4 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Z; R is the R pixel illumination; G is the G pixel illumination; B is the B pixel illumination.

13. The color deconstruction method according to claim 11, characterized in that: One pixel structure includes three photodiodes, and four adjacent pixel structures are arranged in a rectangular array; The color deconstruction function module in the ISP performs color deconstruction processing on the digital signals corresponding to the at least three photodiodes to obtain the R pixel signal, the G pixel signal and the B pixel signal of the pixel structure, including: According to the second group of formulas, the B pixel illumination of the four adjacent pixel structures is determined, and the B pixel illumination is subjected to gain conversion processing to obtain a B pixel signal; wherein the second group of formulas is a photoelectric conversion formula of a first photodiode array in the four adjacent pixel structures, and the first photodiode array includes four photodiodes located in a first layer in the four adjacent pixel structures; According to a third group of formulas, the G pixel illumination of the four adjacent pixel structures is determined, and the G pixel illumination is subjected to gain conversion processing to obtain a G pixel signal; wherein the third group of formulas is a photoelectric conversion formula of a second photodiode array in the four adjacent pixel structures, and the second photodiode array includes four photodiodes located in a second layer in the four adjacent pixel structures; According to the fourth group of formulas, the R pixel illumination of the four adjacent pixel structures is determined, and the R pixel illumination is gain converted to obtain an R pixel signal; wherein the fourth group of formulas are photoelectric conversion formulas of the third photodiode array in the four adjacent pixel structures, and the third photodiode array includes four photodiodes located in the third layer of the four adjacent pixel structures.

14. The color deconstruction method according to claim 13, characterized in that: The second set of formulas is as follows: Among them, X1, X2, X3 and X4 are respectively digital signals of the four photodiodes in the first photodiode array; K is the conversion coefficient; m11, m12 and m13 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X1; m21, m22 and m23 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X2; m31, m32 and m33 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X3; m41, m42 and m43 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to X4; R1 is the R pixel illuminance of the first photodiode array; G1 is the G pixel illuminance of the first photodiode array; B1 is the B pixel illuminance of the first photodiode array.

15. The color deconstruction method according to claim 13, characterized in that: The third set of formulas is as follows: Among them, Y1, Y2, Y3 and Y4 are respectively the digital signals of the four photodiodes in the second photodiode array; K is the conversion coefficient; h11, h12 and h13 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y1; h21, h22 and h23 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y2; h31, h32 and h33 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y3; h41, h42 and h43 are respectively the red light absorption ratio coefficient, green light absorption ratio coefficient and blue light absorption ratio coefficient of the photodiode corresponding to Y4; R2 is the R pixel illuminance of the second photodiode array; G2 is the G pixel illuminance of the second photodiode array; B2 is the B pixel illuminance of the second photodiode array.

16. The color deconstruction method according to claim 13, characterized in that: The fourth set of formulas is as follows: Among them, W1, W2, W3 and W4 are respectively digital signals of the four photodiodes in the third photodiode array; K is the conversion coefficient; l11, l12 and l13 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W1; l21, l22 and l23 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W2; l31, l32 and l33 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W3; l41, l42 and l43 are respectively the red light absorption coefficient, green light absorption coefficient and blue light absorption coefficient of the photodiode corresponding to W4; R3 is the R pixel illuminance of the third photodiode array; G3 is the G pixel illuminance of the third photodiode array; B3 is the B pixel illuminance of the third photodiode array.

Images