Image sensor, camera assembly, and mobile terminal

Abstract

An image sensor (10), a camera module (40), and a mobile terminal (90). The image sensor (10) includes panchromatic pixels and color pixels. The color pixels have a narrower spectral response than the panchromatic pixels, and the panchromatic pixels have a larger full well capacity than the color pixels.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of imaging technology, and in particular, to an image sensor, a camera assembly, and a mobile terminal. BACKGROUND

[0002] Mobile terminals such as mobile phones are often equipped with a camera to realize a photographing function. The camera is provided with an image sensor. In order to realize the collection of a color image, a color pixel is usually arranged in the image sensor, and the color pixel is arranged in a Bayer array form. In order to improve the imaging quality of the image sensor in a dark environment, a white pixel with higher sensitivity than the color pixel is added to the image sensor in the related art. SUMMARY

[0003] The present application provides an image sensor, a camera assembly, and a mobile terminal.

[0004] One aspect of the present application provides an image sensor. The image sensor includes a panchromatic pixel and a color pixel. The color pixel has a narrower spectral response than the panchromatic pixel, and the panchromatic pixel has a larger full well capacity than the color pixel.

[0005] In another aspect, the present application also provides a camera assembly. The camera assembly includes an image sensor. The image sensor includes a panchromatic pixel and a color pixel. The color pixel has a narrower spectral response than the panchromatic pixel, and the panchromatic pixel has a larger full well capacity than the color pixel.

[0006] In yet another aspect, the present application also provides a mobile terminal. The mobile terminal includes a housing and an image sensor mounted in the housing. The image sensor includes a panchromatic pixel and a color pixel. The color pixel has a narrower spectral response than the panchromatic pixel, and the panchromatic pixel has a larger full well capacity than the color pixel.

[0007] Additional aspects and advantages of the present application will be apparent from the following description of the application, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0008] The above and / or additional aspects and advantages of the present application can become apparent and be easily understood from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

[0009] Figure 1 is a schematic diagram of different color channel exposure saturation times;

[0010] Figure 2 is a schematic diagram of an image sensor in an embodiment of the present application;

[0011] Figure 3 is a schematic diagram of a pixel circuit in an embodiment of the present application;

[0012] Figure 4A is a partial cross-sectional schematic diagram of a pixel array in an embodiment of the present application;

[0013] Figure 4B is a schematic diagram of the arrangement of photoelectric conversion elements (or color filters) in the pixel array of Figure 4A

[0014] Figure 5A is a partial cross-sectional schematic diagram of a pixel array in an embodiment of the present application;

[0015] Figure 5B is a schematic diagram of the arrangement of photoelectric conversion elements (or color filters) in the pixel array of Figure 5A

[0016] Figure 5C is a schematic diagram of the arrangement of photoelectric conversion elements (or color filters) in the pixel array of Figure 5A

[0017] is a partial cross-sectional schematic diagram of a pixel array in an embodiment of the present application; Figure 6A

[0018] Figure 6B is a schematic diagram of the arrangement of color filters in the pixel array of Figure 6A

[0019] Figure 6C is a schematic diagram of the arrangement of photoelectric conversion elements in the pixel array of Figure 6A

[0020] Figure 7A is a partial cross-sectional schematic diagram of a pixel array in an embodiment of the present application;

[0021] Figure 7B is a schematic diagram of the arrangement of color filters in the pixel array of Figure 7A

[0022] Figure 7C is a schematic diagram of the arrangement of photoelectric conversion elements in the pixel array of Figure 7A

[0023] Figure 8A is a partial cross-sectional schematic diagram of a pixel array in an embodiment of the present application;

[0024] Figure 8B is a schematic diagram of the arrangement of color filters in the pixel array of Figure 8A

[0025] Figure 8C is a schematic diagram of the arrangement of photoelectric conversion elements in the pixel array of Figure 8A ​​​​​​​​A schematic diagram of the arrangement of photoelectric conversion elements in a pixel array;

[0026] Figure 9A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0027] Figure 9B yes Figure 9A A schematic diagram of the arrangement of photoelectric conversion elements (or filters) in a pixel array;

[0028] Figure 10A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0029] Figure 10B yes Figure 10A A schematic diagram of the arrangement of photoelectric conversion elements (or filters) in a pixel array;

[0030] Figure 10C yes Figure 10A Another schematic diagram of the arrangement of photoelectric conversion elements (or filters) in a pixel array;

[0031] Figure 11A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0032] Figure 11B yes Figure 11A A schematic diagram of the filter arrangement in the pixel array;

[0033] Figure 11C yes Figure 11A A schematic diagram of the arrangement of photoelectric conversion elements in a pixel array;

[0034] Figure 12A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0035] Figure 12B yes Figure 12A A schematic diagram of the filter arrangement in the pixel array;

[0036] Figure 12C yes Figure 12A A schematic diagram of the arrangement of photoelectric conversion elements in a pixel array;

[0037] Figure 13A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0038] Figure 13B yes Figure 13A A schematic diagram of the filter arrangement in the pixel array;

[0039] Figure 13C yes Figure 13A A schematic diagram of the arrangement of photoelectric conversion elements in a pixel array;

[0040] Figure 14 is a partial cross-sectional schematic view of yet another pixel array in an embodiment of the application;

[0041] Figure 15 is a partial cross-sectional schematic view of yet another pixel array in an embodiment of the application;

[0042] Figure 16 is a schematic view of a pixel array and exposure control line connection in an embodiment of the application;

[0043] Figure 17 is a schematic view of a minimum repeating unit pixel arrangement in an embodiment of the application;

[0044] Figure 18 is a schematic view of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0045] Figure 19 is a schematic view of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0046] Figure 20 is a schematic view of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0047] Figure 21 is a schematic view of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0048] Figure 22 is a schematic view of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0049] Figure 23 is a schematic view of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0050] Figure 24 is a schematic view of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0051] Figure 25 is a schematic view of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0052] Figure 26 is a schematic view of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0053] Figure 27 is a schematic view of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0054] Figure 28 is a schematic view of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0055] Figure 29 is a schematic diagram of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0056] Figure 30 is a schematic diagram of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0057] Figure 31 is a schematic diagram of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0058] Figure 32 is a schematic diagram of yet another minimum repeating unit pixel arrangement in an embodiment of the application;

[0059] Figure 33 is a schematic diagram of a camera assembly in an embodiment of the application;

[0060] Figure 34 is a schematic diagram of a flow of an image acquisition method in certain embodiments of the application;

[0061] Figure 35 is a schematic diagram of a principle of an image acquisition method in the related art;

[0062] Figure 36 is a schematic diagram of a principle of a light image acquisition method in an embodiment of the application;

[0063] Figure 37 is a schematic diagram of another principle of a light image acquisition method in an embodiment of the application;

[0064] Figures 38 to 41 is a schematic diagram of a flow of an image acquisition method in certain embodiments of the application;

[0065] Figure 42 is a schematic diagram of yet another principle of a light image acquisition method in an embodiment of the application;

[0066] Figure 43 is a schematic diagram of still another principle of a light image acquisition method in an embodiment of the application;

[0067] Figure 44 is a schematic diagram of still another principle of a light image acquisition method in an embodiment of the application;

[0068] Figure 45 is a schematic diagram of still another principle of a light image acquisition method in an embodiment of the application;

[0069] Figure 46 is a schematic diagram of still another principle of a light image acquisition method in an embodiment of the application;

[0070] Figure 47is a schematic diagram of a mobile terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0071] Embodiments of the present application are described below in detail, examples of which are shown in the accompanying drawings, in which the same or similar reference numerals are used throughout to designate the same or similar elements or elements having the same or similar functions. The embodiments described below by reference to the drawings are exemplary only, and are for the purpose of explanation only and are not to be taken as limiting of the present application.

[0072] Referring to Figure 4A The present application provides an image sensor 10 including panchromatic pixels and color pixels. The color pixels have a narrower spectral response than the panchromatic pixels, and the panchromatic pixels have a larger full well capacity than the color pixels.

[0073] Referring to Figure 33 The present application also provides a camera module 40. The camera module 40 includes an image sensor 10 and a processing chip 20. The image sensor 10 includes panchromatic pixels and color pixels. The color pixels have a narrower spectral response than the panchromatic pixels, and the panchromatic pixels have a larger full well capacity than the color pixels. The color pixels and the panchromatic pixels form a two-dimensional pixel array. The two-dimensional pixel array includes a minimum repeating unit, each minimum repeating unit contains a plurality of sub-units, each sub-unit includes a plurality of single-color pixels and a plurality of panchromatic pixels. The image sensor 10 is configured to be exposed to acquire a panchromatic raw image and a color raw image. The processing chip 20 is configured to: process the color raw image to treat all pixels in each sub-unit as a single-color large pixel corresponding to a single color in the sub-unit, and output pixel values of the single-color large pixels to obtain a color intermediate image; process the panchromatic raw image to obtain a panchromatic intermediate image; and process the color intermediate image and / or the panchromatic intermediate image to obtain a target image.

[0074] Referring to Figure 47 The present application also provides a mobile terminal 90. The mobile terminal includes an image sensor 50 and a processor 60. The image sensor 50 includes panchromatic pixels and color pixels. The color pixels have a narrower spectral response than the panchromatic pixels, and the panchromatic pixels have a larger full well capacity than the color pixels. The color pixels and the panchromatic pixels form a two-dimensional pixel array. The two-dimensional pixel array includes a minimum repeating unit, each minimum repeating unit contains a plurality of sub-units, each sub-unit includes a plurality of single-color pixels and a plurality of panchromatic pixels. The image sensor 50 is configured to be exposed to acquire a panchromatic raw image and a color raw image. The processor 60 is configured to: process the color raw image to treat all pixels in each sub-unit as a single-color large pixel corresponding to a single color in the sub-unit, and output pixel values of the single-color large pixels to obtain a color intermediate image; process the panchromatic raw image to obtain a panchromatic intermediate image; and process the color intermediate image and / or the panchromatic intermediate image to obtain a target image.

[0075] The embodiments of the present application are further illustrated below with reference to the drawings.

[0076] In a color image sensor, different color pixels receive different exposure amount in a unit time, and after saturation of some colors, some colors are not exposed to the ideal state. For example, exposure to 60%-90% of the saturation exposure amount can have better signal-to-noise ratio and accuracy, but the embodiments of the present application are not limited thereto.

[0077] Figure 1 Take RGBW (red, green, blue, and full color) four pixels as an example for illustration. Referring to Figure 1 , Figure 1 In the figure, the horizontal axis is the exposure time, the vertical axis is the exposure amount, Q is the saturation exposure amount, LW is the exposure curve of the full color pixel W, LG is the exposure curve of the green pixel G, LR is the exposure curve of the red pixel R, and LB is the exposure curve of the blue pixel.

[0078] As can be seen from Figure 1 , the slope of the exposure curve LW of the full color pixel W is the largest, that is, the full color pixel W can obtain more exposure amount in a unit time, and it reaches saturation at t1. The slope of the exposure curve LG of the green pixel G is the second, and the green pixel saturates at t2. The slope of the exposure curve LR of the red pixel R is the third, and the red pixel saturates at t3. The slope of the exposure curve LB of the blue pixel B is the smallest, and the blue pixel saturates at t4. At t1, the full color pixel W has already saturated, while the exposure of the R, G, and B pixels has not reached the ideal state.

[0079] In the related art, the exposure time of the RGBW four pixels is commonly controlled. For example, the exposure time of each row of pixels is the same, connected to the same exposure control line, and controlled by the same exposure control signal. For example, still referring to Figure 1 In the 0-t1 time period, the RGBW four pixel units can work normally, but in this interval, RGB has a short exposure time and a small exposure amount, which will cause low brightness, low signal-to-noise ratio, and even not bright enough color in image display. In the t1-t4 time period, the W pixel is saturated and cannot work, and the exposure amount data cannot truly reflect the target.

[0080] Based on the above reasons, one aspect of the present application provides an image sensor, which increases the full well capacity of the full color pixel, so that the full well capacity of the full color pixel can be greater than that of the color pixel, thereby avoiding the problem of premature saturation of the full color pixel, balancing the exposure of the full color pixel W and the color pixel (including but not limited to RGB), and thereby improving the image shooting quality.

[0081] It should be noted that, Figure 1 The exposure curve in FIG. 10 is only an example, and the slope and relative relationship of the curve vary depending on the difference in the pixel response wavelength band. The present application is not limited to the case shown in FIG. 10. For example, when the red pixel R response wavelength band is narrow, the red pixel R exposure curve slope can be lower than the blue pixel B exposure curve slope. Figure 1

[0082] Next, the basic structure of the image sensor 10 will be described first. Referring to FIG. 11, Figure 2 Figure 2 is a schematic diagram of the image sensor 10 in the present embodiment. The image sensor 10 includes a pixel array 11, a vertical drive unit 12, a control unit 13, a column processing unit 14, and a horizontal drive unit 15.

[0083] For example, the image sensor 10 can employ a complementary metal oxide semiconductor (CMOS) photosensitive element or a charge-coupled device (CCD) photosensitive element.

[0084] For example, the pixel array 11 includes a plurality of pixels (not shown in the drawing) arranged in a two-dimensional array, and each pixel includes a photoelectric conversion element 117 (shown in FIG. 12). Each pixel converts light into an electric charge according to the intensity of light incident thereon. Figure 2 Figure 3 For example, the vertical drive unit 12 includes a shift register and an address decoder. The vertical drive unit 12 includes a readout scan and a reset scan function. The readout scan refers to sequentially scanning unit pixels row by row and reading signals from the unit pixels row by row. For example, the signals output from each pixel in the pixel row selected and scanned are transferred to the column processing unit 14. The reset scan is used to reset the electric charge, and the photoelectric charge of the photoelectric conversion element 117 is discarded, so that the accumulation of new photoelectric charge can be started.

[0085] For example, the signal processing performed by the column processing unit 14 is a correlated double sampling (CDS) process. In the CDS process, the reset level and the signal level output from each pixel in the selected pixel row are taken out, and the level difference is calculated. Thus, the signals of the pixels in the row are obtained. The column processing unit 14 can have an analog-to-digital (A / D) conversion function for converting the analog pixel signals into a digital format.

[0086] For example, the signal processing performed by the column processing unit 14 is a correlated double sampling (CDS) process. In the CDS process, the reset level and the signal level output from each pixel in the selected pixel row are taken out, and the level difference is calculated. Thus, the signals of the pixels in the row are obtained. The column processing unit 14 can have an analog-to-digital (A / D) conversion function for converting the analog pixel signals into a digital format.

[0087] ​​​For example, the horizontal drive unit 15 includes a shift register and an address decoder. The horizontal drive unit 15 sequentially scans the pixel array 11 column by column. Each pixel column is sequentially processed by the column processing unit 14 and sequentially output by a selection scan operation performed by the horizontal drive unit 15.

[0088] For example, the control unit 13 configures the timing signals according to the operation mode, and controls the vertical drive unit 13, the column processing unit 14 and the horizontal drive unit 15 to work cooperatively using a plurality of timing signals.

[0089] Figure 3 is a schematic diagram of a pixel circuit 110 in an embodiment of the present application. Figure 3 The pixel circuit 110 is applied in Figure 2 each pixel of the pixel array 11. The working principle of the pixel circuit 110 will be described below in combination with Figure 2 and Figure 3 .

[0090] As shown in Figure 3 , the pixel circuit 110 includes a photoelectric conversion element 117 (e.g., a photodiode PD), an exposure control circuit 116 (e.g., a transfer transistor 112), a reset circuit (e.g., a reset transistor 113), an amplification circuit (e.g., an amplification transistor 114) and a selection circuit (e.g., a selection transistor 115). In an embodiment of the present application, the transfer transistor 112, the reset transistor 113, the amplification transistor 114 and the selection transistor 115 are MOS transistors, but are not limited thereto.

[0091] For example, referring to Figure 2 and Figure 3 , the gate TG of the transfer transistor 112 is connected to the vertical drive unit 12 through an exposure control line (not shown in the figure); the gate RG of the reset transistor 113 is connected to the vertical drive unit 12 through a reset control line (not shown in the figure); and the gate SEL of the selection transistor 114 is connected to the vertical drive unit 12 through a selection line (not shown in the figure). The exposure control circuit 116 (e.g., the transfer transistor 112) in each pixel circuit 110 is electrically connected to the photoelectric conversion element 117, and is used to transfer the potential accumulated by the photoelectric conversion element 117 after being illuminated by light. For example, the photoelectric conversion element 117 includes a photodiode PD, and the anode of the photodiode PD is connected to the ground, for example. The photodiode PD converts the received light into electric charges. The cathode of the photodiode PD is connected to a floating diffusion unit FD via the exposure control circuit 116 (e.g., the transfer transistor 112). The floating diffusion unit FD is connected to the gate of the amplification transistor 114 and the source of the reset transistor 113.

[0092] For example, the exposure control circuit 116 is a transfer transistor 112, and the control terminal TG of the exposure control circuit 116 is the gate of the transfer transistor 112. When a pulse of an effective level (for example, a VPIX level) is transmitted to the gate of the transfer transistor 112 through an exposure control line (not shown in the figure), the transfer transistor 112 is turned on. The transfer transistor 112 transmits the photoelectrically converted charge of the photodiode PD to the floating diffusion unit FD.

[0093] For example, the drain of the reset transistor 113 is connected to the pixel power supply VPIX. The source of the reset transistor 113 is connected to the floating diffusion unit FD. Before the charge is transferred from the photodiode PD to the floating diffusion unit FD, a pulse of an effective reset level is transmitted to the gate of the reset transistor 113 through a reset line, and the reset transistor 113 is turned on. The reset transistor 113 resets the floating diffusion unit FD to the pixel power supply VPIX.

[0094] For example, the gate of the amplification transistor 114 is connected to the floating diffusion unit FD. The drain of the amplification transistor 114 is connected to the pixel power supply VPIX. After the floating diffusion unit FD is reset by the reset transistor 113, the amplification transistor 114 outputs a reset level through the output terminal OUT via the selection transistor 115. After the charge of the photodiode PD is transferred by the transfer transistor 112, the amplification transistor 114 outputs a signal level through the output terminal OUT via the selection transistor 115.

[0095] For example, the drain of the selection transistor 115 is connected to the source of the amplification transistor 114. The source of the selection transistor 115 is connected to the column processing unit 14 through the output terminal OUT. Figure 2 When a pulse of an effective level is transmitted to the gate of the selection transistor 115 through a selection line, the selection transistor 115 is turned on. The signal output by the amplification transistor 114 is transmitted to the column processing unit 14 through the selection transistor 115.

[0096] It should be noted that the pixel structure of the pixel circuit 110 in the embodiments of the present application is not limited to the structure shown in Figure 3 For example, the pixel circuit 110 can have a three-transistor pixel structure, in which the functions of the amplification transistor 114 and the selection transistor 115 are completed by one transistor. For example, the exposure control circuit 116 is also not limited to the single transfer transistor 112, and other electronic devices or structures having a control terminal to control the on-off function can also be used as the exposure control circuit in the embodiments of the present application. The single transfer transistor 112 is simple in implementation, low in cost, and easy to control.

[0097] Figures 4A to 13C It is shown that Figure 2Fig. 1 shows a schematic diagram of a cross section of a pixel array 11 of an image sensor 10 according to an embodiment of the present application, and a schematic diagram of an arrangement of photoelectric conversion elements 117 (or filters 1182) in the pixel array 11. In the embodiment shown, the full-color pixels and the color pixels are arranged alternately, and the color pixels have a narrower spectral response than the full-color pixels. Each full-color pixel and each color pixel includes a microlens 1181, a filter 1182, an isolation layer 1183, and a photoelectric conversion element 117. The microlens 1181, the filter 1182, the isolation layer 1183, and the photoelectric conversion element 117 are arranged in sequence along the light-receiving direction of the image sensor 10. The photoelectric conversion element 117 can convert the received light into electric charges. Specifically, the photoelectric conversion element 117 includes a substrate 1171 and an n-well layer 1172 formed inside the substrate 1171, and the n-well layer 1172 can convert light into electric charges. The isolation layer 1183 is arranged on one surface of the photoelectric conversion element 117 (specifically, one surface of the substrate 1171). Since the substrate 1171 is not completely flat, it is difficult to arrange the filter 1182 directly on the surface of the substrate 1171. The isolation layer 1183 is arranged on one surface of the substrate 1171, and the flatness of the surface of the isolation layer 1183 that is away from the substrate 1171 is higher, so that the filter 1182 can be arranged on the surface. The filter 1182 is arranged on the surface of the isolation layer 1183 that is away from the substrate 1171, and the filter 1182 can allow light of a specific wavelength band to pass through. The microlens 1181 is arranged on the side of the filter 1182 that is away from the isolation layer 1183, and the microlens 1181 can converge light and direct more incident light to the photoelectric conversion element 117. The full-well capacity of the photoelectric conversion element 117 is related to the volume of the n-well layer 1172 of the photoelectric conversion element 117. The larger the volume of the n-well layer 1172, the larger the full-well capacity. Figures 4A to 13C In any of the embodiments shown, the volume of the n-well layer 1172 of the full-color pixel is larger than the volume of the n-well layer 1172 of the color pixel, so that the full-well capacity of the full-color pixel is larger than the full-well capacity of the color pixel, the exposure Q at which the full-color pixel saturates is increased, and the time period during which the full-color pixel reaches saturation is increased. Thus, the problem of premature saturation of the full-color pixel is avoided, the exposure of the full-color pixel and the color pixel can be balanced, and the image capturing quality is improved.

[0098] For example, Figure 4A and Figure 4B are a schematic diagram of a cross section of a pixel array 11 of an image sensor 10 according to an embodiment of the present application, and a schematic diagram of an arrangement of photoelectric conversion elements 117 (or filters 1182), respectively. As shown in Figure 4AAs shown, the size of the multiple cross sections of the isolation layer 1183 of each pixel (the same pixel) along the light receiving direction is equal; the size of the multiple cross sections of the n-well layer 1172 of each pixel (the same pixel) along the light receiving direction is equal; the size of the cross section of the n-well layer 1172 of the full-color pixel is equal to the size of the cross section of the n-well layer 1172 of the color pixel; the depth H1 of the n-well layer 1172 of the full-color pixel is greater than the depth H2 of the n-well layer 1172 of the color pixel, so that the volume of the n-well layer 1172 of the full-color pixel is greater than the volume of the n-well layer 1172 of the color pixel, and the full-color pixel has a greater full-well capacity than the color pixel.

[0099] It should be noted that the cross section of the isolation layer 1183 is a cross section of the isolation layer taken along a direction perpendicular to the light receiving direction, and the cross section of the n-well layer 1172 is a cross section of the n-well layer 1172 taken along a direction perpendicular to the light receiving direction. The shape and size of the cross section of the isolation layer 1183 of each pixel correspond to the shape and size of the cross section of the n-well layer 1172 of the pixel. The cross section can be a rectangle, a square, a parallelogram, a rhombus, a pentagon, a hexagon, etc., which is not limited herein.

[0100] Along the light receiving direction, the size of the multiple cross sections of the n-well layer 1172 (or the isolation layer 1183) of the same pixel is equal, which means that the multiple cross sections have the same area, and the corresponding side lengths of the multiple cross sections are equal. The size of the cross section of the n-well layer 1172 of the full-color pixel is equal to the size of the cross section of the n-well layer 1172 of the color pixel, which means that the area of the cross section of the n-well layer 1172 of the full-color pixel is equal to the area of the cross section of the n-well layer 1172 of the color pixel. The side length of the shape formed by the cross section of the n-well layer 1172 of the full-color pixel can be equal to or different from the side length of the shape formed by the cross section of the n-well layer 1172 of the color pixel. For example, Figure 4B The cross sections of the n-well layer 1172 of the full-color pixel and the color pixel shown are rectangles, which include a length and a width. The area of the cross section of the n-well layer 1172 of the full-color pixel is equal to the area of the cross section of the n-well layer 1172 of the color pixel. The length L 全 of the cross section of the n-well layer 1172 of the full-color pixel is equal to the length L 彩 of the cross section of the n-well layer 1172 of the color pixel. The width W 全 of the cross section of the n-well layer 1172 of the full-color pixel is equal to the width W 彩 of the cross section of the n-well layer 1172 of the color pixel. In other examples, L 全 may not be equal to L 彩 , and W 全 may not be equal to W 彩The area of the cross section of the n-well layer 1172 of the full-color pixel can be equal to the area of the cross section of the n-well layer 1172 of the color pixel. The explanation of the cross section of the n-well layer 1172 (or the isolation layer 1183), the size of each cross section of the n-well layer 1172 (or the isolation layer 1183) of each pixel, and the size of the cross section of the n-well layer 1172 of the full-color pixel being equal to the size of the cross section of the n-well layer 1172 of the color pixel is the same as the explanation herein.

[0101] For example, Figure 5A is a schematic view of a cross section of the pixel array 11 of another embodiment of the present application taken along the light-receiving direction, Figure 5B and Figure 5C is Figure 5A two schematic views of the arrangement of the plurality of photoelectric conversion elements 117 (or the plurality of filters 1182) in the pixel array 11 of Figure 5A As shown, the size of each cross section of the isolation layer 1183 of each pixel (the same pixel) is equal along the light-receiving direction; the size of each cross section of the n-well layer 1172 of each pixel (the same pixel) is equal along the light-receiving direction; the size of the cross section of the n-well layer 1172 of the full-color pixel is greater than the size of the cross section of the n-well layer 1172 of the color pixel; and the depth H1 of the n-well layer 1172 of the full-color pixel is equal to the depth H2 of the n-well layer 1172 of the color pixel. In this way, the volume of the n-well layer 1172 of the full-color pixel is greater than the volume of the n-well layer 1172 of the color pixel, and the full-color pixel has a greater full-well capacity than the color pixel. Of course, in other embodiments, Figure 5A the depth H1 of the n-well layer 1172 of the full-color pixel can also be greater than the depth H2 of the n-well layer 1172 of the color pixel.

[0102] It should be noted that the size of the cross section of the n-well layer 1172 of the full-color pixel being greater than the size of the cross section of the n-well layer 1172 of the color pixel means that the area of the cross section of the n-well layer 1172 of the full-color pixel is greater than the area of the cross section of the n-well layer 1172 of the color pixel, and the length of the side of the shape formed by the cross section of the n-well layer 1172 of the full-color pixel can be partially or entirely greater than the length of the side of the shape formed by the cross section of the n-well layer 1172 of the color pixel. For example, as shown in Figure 5B the length L 全 of the cross section of the n-well layer 1172 of the full-color pixel is greater than the length L 彩 of the cross section of the n-well layer 1172 of the color pixel; the width W 全 of the cross section of the n-well layer 1172 of the full-color pixel is equal to the width W 彩 of the cross section of the n-well layer 1172 of the color pixel; and as shown in Figure 5C the length L全 the length L of the cross section of the n-well layer 1172 of the color pixel 彩 the width W of the cross section of the n-well layer 1172 of the full-color pixel 全 greater than the width W of the cross section of the n-well layer 1172 of the color pixel 彩 The explanation below for the size of the cross section of the n-well layer of the full-color pixel being greater than the size of the cross section of the n-well layer 1172 of the color pixel is the same as the explanation here.

[0103] For example, Figures 6A to 6C are, respectively, a cross-sectional view of the pixel array 11 taken along the light-receiving direction, a schematic view of the arrangement of the plurality of filters 1182, and a schematic view of the arrangement of the plurality of photoelectric conversion elements 117 of yet another embodiment of the present application. As shown in Figure 6A the size of the cross section of the isolation layer 1183 of each pixel (the same pixel) is equal along the light-receiving direction; the size of the cross section of the n-well layer 1172 of each full-color pixel (the same full-color pixel) gradually increases along the light-receiving direction, the size of the cross section of the n-well layer 1172 of each color pixel (the same color pixel) gradually decreases, and the smallest cross section of the n-well layer 1172 of the full-color pixel is equal in size to the largest cross section of the n-well layer 1172 of the color pixel; the depth H1 of the n-well layer 1172 of the full-color pixel is equal to the depth H2 of the n-well layer 1172 of the color pixel. As shown in Figure 6B the size of the cross section of the filter 1182 of the full-color pixel is equal to the size of the cross section of the filter 1182 of the color pixel (both the area and the corresponding side length are equal), but as shown in Figure 6C in fact, the size of the cross section of the n-well layer 1172 in the photoelectric conversion element 117 of the full-color pixel (except for the smallest cross section) is greater than the size of the cross section of the n-well layer 1172 in the photoelectric conversion element 117 of the color pixel. This makes the volume of the n-well layer 1172 of the full-color pixel greater than the volume of the n-well layer 1172 of the color pixel, and the full-color pixel has a greater full-well capacity than the color pixel.

[0104] In other embodiments, Figure 6A the smallest cross section of the n-well layer 1172 of the full-color pixel can also be greater in size than the largest cross section of the n-well layer of the color pixel, and the depth H1 of the n-well layer 1172 of the full-color pixel can also be greater than the depth H2 of the n-well layer 1172 of the color pixel.

[0105] For example, Figures 7A to 7C are, respectively, a cross-sectional view of the pixel array 11 taken along the light-receiving direction, a schematic view of the arrangement of the plurality of filters 1182, and a schematic view of the arrangement of the plurality of photoelectric conversion elements 117 of yet another embodiment of the present application. As shown in Figure 7A As shown, the size of the multiple cross sections of the isolation layer 1183 of each panchromatic pixel (the same panchromatic pixel) gradually increases along the light receiving direction, the size of the multiple cross sections of the isolation layer 1183 of each color pixel (the same color pixel) gradually decreases, the size of the cross section of the n-well layer 1172 of each panchromatic pixel gradually increases along the light receiving direction, the size of the cross section of the n-well layer 1172 of each color pixel gradually decreases, and the size of the smallest cross section of the n-well layer 1172 of the panchromatic pixel is equal to the size of the largest cross section of the n-well layer 1172 of the color pixel; the depth H1 of the n-well layer 1172 of the panchromatic pixel is equal to the depth H2 of the n-well layer 1172 of the color pixel. As shown, Figure 7B As shown, although the size of the cross section of the filter 1182 of the panchromatic pixel is equal to the size of the cross section of the filter 1182 of the color pixel (both the area and the corresponding side length are equal), as shown, Figure 7C As shown, the size of the cross section of the n-well layer 1172 in the panchromatic photoelectric conversion element 117 is actually greater than the size of the cross section of the n-well layer 1172 in the color photoelectric conversion element 117 (except for the smallest cross section). In this way, the volume of the n-well layer 1172 of the panchromatic pixel is greater than the volume of the n-well layer 1172 of the color pixel, and the panchromatic pixel has a greater full-well capacity than the color pixel.

[0106] In other embodiments, Figure 7A In the above embodiment, the size of the smallest cross section of the n-well layer 1172 of the panchromatic pixel can also be greater than the size of the largest cross section of the n-well layer of the color pixel, and the depth H1 of the n-well layer 1172 of the panchromatic pixel can also be greater than the depth H2 of the n-well layer 1172 of the color pixel.

[0107] For example, Figures 8A to 8C respectively, a cross-sectional view of the pixel array 11 along the light receiving direction, a schematic view of the arrangement of the multiple filters 1182, and a schematic view of the arrangement of the multiple photoelectric conversion elements 117 of another embodiment of the present application. As shown, Figure 8A As shown, the size of the multiple cross sections of the isolation layer 1183 of each panchromatic pixel (the same panchromatic pixel) gradually increases along the light receiving direction, the size of the multiple cross sections of the isolation layer 1183 of each color pixel (the same color pixel) gradually decreases, the size of the cross section of the n-well layer 1172 of each pixel is equal along the light receiving direction, the size of the cross section of the n-well layer 1172 of the panchromatic pixel is greater than the size of the cross section of the n-well layer 1172 of the color pixel, and the depth H1 of the n-well layer 1172 of the panchromatic pixel is equal to the depth H2 of the n-well layer 1172 of the color pixel. Figure 8B As shown, although the size of the cross section of the filter 1182 of the panchromatic pixel is equal to the size of the cross section of the filter 1182 of the color pixel (both the area and the corresponding side length are equal), as shown in FIG. 11A, the size of the cross section (except the cross section of the minimum size) of the n-well layer 1172 in the photoelectric conversion element 117 of the panchromatic pixel is actually greater than the size of the cross section of the n-well layer 1172 in the photoelectric conversion element 117 of the color pixel. This makes the volume of the n-well layer 1172 of the panchromatic pixel greater than the volume of the n-well layer 1172 of the color pixel, and the panchromatic pixel has a greater full-well capacity than the color pixel. Figure 8C As shown, although the size of the cross section of the filter 1182 of the panchromatic pixel is equal to the size of the cross section of the filter 1182 of the color pixel (both the area and the corresponding side length are equal), as shown in FIG. 11A, the size of the cross section (except the cross section of the minimum size) of the n-well layer 1172 in the photoelectric conversion element 117 of the panchromatic pixel is actually greater than the size of the cross section of the n-well layer 1172 in the photoelectric conversion element 117 of the color pixel. This makes the volume of the n-well layer 1172 of the panchromatic pixel greater than the volume of the n-well layer 1172 of the color pixel, and the panchromatic pixel has a greater full-well capacity than the color pixel.

[0108] In other embodiments, Figure 8A In some embodiments, the depth H1 of the n-well layer 1172 of the panchromatic pixel can also be greater than the depth H2 of the n-well layer 1172 of the color pixel. Figure 8A In some embodiments, the size of the smallest cross section of the isolation layer 1183 of the panchromatic pixel can also be greater than the size of the largest cross section of the isolation layer 1183 of the color pixel.

[0109] Figures 4A to 8C In the pixel array 11 in any one of the embodiments shown, the depth H3 of the photoelectric conversion element 117 of the panchromatic pixel is equal to the depth H4 of the photoelectric conversion element 117 of the color pixel, specifically, the depth H3 of the substrate 1171 of the panchromatic pixel is equal to the depth H4 of the substrate 1171 of the color pixel. When H3 and H4 are equal, the surface of the substrate 1171 of the panchromatic pixel away from the filter 1182 is in the same horizontal plane as the surface of the substrate 1171 of the color pixel away from the filter 1182, which can reduce the complexity of the design and manufacture of the readout circuit.

[0110] Figures 4A to 8CEach pixel in any of the embodiments shown also includes a light guide layer 1184. The light guide layer 1184 is formed within the isolation layer 1183, and the refractive index of the light guide layer 1184 is greater than the refractive index of the isolation layer 1183. Along a direction perpendicular to the light-receiving direction, the isolation layer 1183, the light guide layer 1184, and the isolation layer 1183 of each pixel are sequentially arranged. For example, along a direction perpendicular to the light-receiving direction, the isolation layer 1183 of the panchromatic pixel W, the light guide layer 1184 of the panchromatic pixel W, and the isolation layer 1183 of the panchromatic pixel W are sequentially arranged; the isolation layer 1183 of the color pixel A, the light guide layer 1184 of the color pixel A, and the isolation layer 1183 of the color pixel A are sequentially arranged; the isolation layer 1183 of the color pixel B, the light guide layer 1184 of the color pixel B, and the isolation layer 1183 of the color pixel B are sequentially arranged, and so on. The purpose of setting the light guide layer 1184 in the isolation layer 1183 is to make the light passing through the filter 1182 undergo total internal reflection in the structure composed of the isolation layer 1183 and the light guide layer 1184, thereby converging the light and allowing more light to enter the corresponding photoelectric conversion element 117. The n-potential well layer 1172 in the photoelectric conversion element 117 can receive the light passing through the light guide layer 1184 and convert the light into electric charge.

[0111] In one example, the refractive index of the light guide layer 1184 is equal at all locations. This design simplifies the design of the light guide layer and reduces the manufacturing difficulty of the pixel array 11. In another example, the refractive index of the light guide layer 1184 gradually increases along the light-receiving direction of the image sensor 10. This design enhances the light-gathering ability of the light guide layer 1184, allowing more light to enter the photoelectric conversion element 117.

[0112] like Figures 4A to 6C As shown, along the light-receiving direction, the dimensions of multiple cross-sections of the isolation layer 1183 for each pixel are all equal, and the dimensions of multiple cross-sections of the light guide layer 1184 for each pixel are also equal. This design simplifies the manufacturing process of the light guide layer 1184. Of course, in other embodiments, when the dimensions of multiple cross-sections of the isolation layer 1183 for each pixel are equal along the light-receiving direction, the structure of the light guide layer 1184 can also be such that the dimensions of multiple cross-sections of the light guide layer 1184 for each pixel gradually decrease along the light-receiving direction. This design enhances the light-gathering ability of the light guide layer 1184, allowing more light to enter the photoelectric conversion element 117.

[0113] like Figure 7A and Figure 8AAs shown, along the light receiving direction, the size of the multiple cross sections of the isolation layer 1183 of each panchromatic pixel gradually increases, the size of the multiple cross sections of the isolation layer 1183 of each color pixel gradually decreases, and the size of the cross section of the light guide layer 1184 of each panchromatic pixel and the light guide layer 1184 of each color pixel gradually decreases. Such a design can enhance the light collecting capability of the light guide layer 1184, so that more light can enter the photoelectric conversion element 117. Of course, in other embodiments, when along the light receiving direction, the size of the multiple cross sections of the isolation layer 1183 of each panchromatic pixel gradually increases, and the size of the multiple cross sections of the isolation layer 1183 of each color pixel gradually decreases, the structure of the light guide layer 1184 can also be that: along the light receiving direction, the size of the multiple cross sections of the isolation layer 1183 of each pixel is equal. Such a design can simplify the manufacturing process of the light guide layer 1184.

[0114] The depth of the light guide layer 1184 is equal to the depth of the isolation layer 1183, so as to enhance the light collecting capability of the light guide layer 1184. Compared with the thickness of the isolation layer in the existing image sensor, the thickness of the isolation layer 1183 of the present application is greater, for example, greater than a predetermined thickness, so as to form a longer light path and improve the light collecting effect of the structure composed of the light guide layer 1184 and the isolation layer 1183.

[0115] Figures 4A to 8C Each pixel in any of the embodiments shown further comprises an optical isolation interlayer 1185. The optical isolation interlayer 1185 is arranged between the isolation layers 1183 of two adjacent pixels. For example, one optical isolation interlayer 1185 is arranged between the isolation layer 1183 of the panchromatic pixel W and the isolation layer 1183 of the color pixel A, another optical isolation interlayer 1185 is arranged between the isolation layer 1183 of the panchromatic pixel W and the isolation layer 1183 of the color pixel B, and so on. The optical isolation interlayer 1185 can be made of at least one of tungsten, titanium, aluminum and copper. The optical isolation interlayer 1185 can prevent the light incident on a certain pixel from entering another pixel adjacent to the certain pixel, so as to avoid bringing noise to other pixels.

[0116] Figures 4A to 8C The light guide layer 1184 in each pixel in any of the embodiments shown can be replaced by a light collecting lens 1186. Specifically, as shown in FIG. 1A, the image sensor 10 in the embodiment shown in FIG. 1A is the same as the image sensor 10 in the embodiment shown in FIG. 1A except that the light guide layer 1184 is replaced by the light collecting lens 1186. Figures 9A to 13C As shown in FIG. 1B, the image sensor 10 in the embodiment shown in FIG. 1B is the same as the image sensor 10 in the embodiment shown in FIG. 1B except that the light guide layer 1184 is replaced by the light collecting lens 1186. Figure 9A As shown in FIG. 1C, the image sensor 10 in the embodiment shown in FIG. 1C is the same as the image sensor 10 in the embodiment shown in FIG. 1C except that the light guide layer 1184 is replaced by the light collecting lens 1186. Figure 4A As shown in FIG. 1D, the image sensor 10 in the embodiment shown in FIG. 1D is the same as the image sensor 10 in the embodiment shown in FIG. 1D except that the light guide layer 1184 is replaced by the light collecting lens 1186. Figure 10A As shown in FIG. 1E, the image sensor 10 in the embodiment shown in FIG. 1E is the same as the image sensor 10 in the embodiment shown in FIG. 1E except that the light guide layer 1184 is replaced by the light collecting lens 1186. Figure 5A As shown in FIG. 1F, the image sensor 10 in the embodiment shown in FIG. 1F is the same as the image sensor 10 in the embodiment shown in FIG. 1F except that the light guide layer 1184 is replaced by the light collecting lens 1186. Figure 11A As shown in FIG. 1G, the image sensor 10 in the embodiment shown in FIG. 1G is the same as the image sensor 10 in the embodiment shown in FIG. 1G except that the light guide layer 1184 is replaced by the light collecting lens 1186. Figure 6A same, Figure 12A The structure of the image sensor 10, except for the condenser lens 1186, is the same as... Figure 7A The image sensor 10 in it is the same. Figure 13A The structure of the image sensor 10, except for the condenser lens 1186, is the same as... Figure 8A The image sensor 10 in this paper is the same, and will not be described again here. Figures 9A to 13C The microlens 1181, filter 1182, isolation layer 1183, optical isolation interlayer 1185, and photoelectric conversion element 117 (substrate 1171 and n-potential well layer 1172) are described.

[0117] like Figures 9A to 13C As shown, each panchromatic pixel and each color pixel include a condenser lens 1186, which is disposed within the isolation layer 1183 of the corresponding pixel. The condenser lens 1186 can concentrate light, allowing more light passing through the filter 1182 to enter the photoelectric conversion element 117. When each pixel is provided with a condenser lens 1186, condenser lenses 1186 with different radii of curvature can be designed according to the needs of different pixels. For example, the radius of curvature of the condenser lens 1186 of the color pixel is greater than that of the condenser lens 1186 of the panchromatic pixel, thereby making the light-gathering ability of the condenser lens 1186 of the color pixel higher than that of the condenser lens 1186 of the panchromatic pixel.

[0118] In other embodiments, only some pixels may include the condenser lens 1186. For example, the condenser lens 1186 may not be provided in the pan-color pixels, but it may be provided in the color pixels. For example, as Figure 11A and Figure 12A In the illustrated embodiment, along the light-receiving direction, the cross-section of the n-potential well layer 1172 of the pan-color pixel gradually increases, while the cross-section of the n-potential well layer of the color pixel gradually decreases. Most of the light passing through the filter 1182 of the pan-color pixel can enter the photoelectric conversion element 117 of the pan-color pixel, while only a smaller portion of the light passing through the filter 1182 of the color pixel can enter the photoelectric conversion element 117 of the color pixel. Therefore, a condensing lens 1186 can be placed only in the isolation layer 1183 of the color pixel, thereby utilizing the light-concentrating effect of the condensing lens 1186 to allow more light to enter the photoelectric conversion element 117 of the color pixel. Placing the condensing lens 1186 only in some pixels can reduce the manufacturing cost of the image sensor 10.

[0119] When a condenser lens 1186 is provided in a pixel, an anti-reflective film can be provided on the side of each condenser lens 1186 opposite to the photoelectric conversion element 117. The anti-reflective film can be used to reduce light interference and avoid the influence of light interference on the imaging effect of the image sensor 10.

[0120] Referring to Figure 14 and Figure 15 , the image sensor 10 further includes a barrier layer 1187. The barrier layer 1187 can be disposed between the photoelectric conversion elements 117 of two adjacent pixels. For example, one barrier layer 1187 is disposed between the photoelectric conversion element 117 of the panchromatic pixel W and the photoelectric conversion element 117 of the color pixel A, another barrier layer 1187 is disposed between the photoelectric conversion element 117 of the panchromatic pixel W and the photoelectric conversion element 117 of the color pixel B, and so on. Exemplarily, the barrier layer 1187 can be a Deep Trench Isolation (DTI). The barrier layer 1187 can prevent the light entering the photoelectric conversion element 117 of a certain pixel from entering the photoelectric conversion element 117 of another pixel adjacent to the certain pixel, avoiding bringing noise to the photoelectric conversion element 117 of the another pixel.

[0121] In addition to the above-mentioned setting the full well capacity of the panchromatic pixel to be greater than the full well capacity of the color pixel, in the embodiments of the present application, different full well capacities can be set for color pixels of different colors. Specifically, the full well capacity corresponding to the sensitivity of the color pixel can be set according to the sensitivity of the color pixel (the shorter the time length of the exposure amount reaching saturation, the higher the sensitivity of the pixel). For example, as shown in Figure 1 , the sensitivity of the green pixel > the sensitivity of the red pixel > the sensitivity of the blue pixel, the full well capacity of the color pixel can be set as: the full well capacity of the green pixel > the full well capacity of the red pixel > the full well capacity of the blue pixel. Wherein, the way to increase the full well capacity of the color pixel is similar to the way to increase the full well capacity of the panchromatic pixel, for example, one way can be: when the cross-sectional areas of the n-well layers 1172 of the pixels are all the same, i.e. S W =S G =S R =S B , the relationship of the depths of the n-well layers 1172 of the pixels can be H W >H G >H R >H B ; and for another example, when the depths of the n-well layers 1172 of the pixels are all the same, i.e. H W =H G =H R =H B , the relationship of the cross-sectional areas of the n-well layers 1172 of the pixels can be S W >S G >S R >S B, other cases are not described here. In this way, different full well capacities can be set according to different sensitivities, so that the exposure of pixels of each color can be balanced, and the image shooting quality can be improved.

[0122] On the basis of setting the full well capacity of the panchromatic pixel to be greater than the full well capacity of the color pixel, the exposure of the panchromatic pixel and the color pixel can be further balanced by independently controlling the exposure time of the panchromatic pixel and the exposure time of the color pixel.

[0123] Figure 16 is a schematic diagram of a pixel array 11 and a connection mode of an exposure control line according to an embodiment of the present application. The pixel array 11 is a two-dimensional pixel array. The two-dimensional pixel array includes a plurality of panchromatic pixels and a plurality of color pixels, wherein the color pixels have a narrower spectral response than the panchromatic pixels. The pixels in the pixel array 11 are arranged in the following manner:

[0124] W A W B

[0125] A W B W

[0126] W B W C

[0127] B W C W

[0128] It should be noted that, for the convenience of illustration, Figure 16 only part of the pixels in the pixel array 11 are shown in FIG. 11, and other pixels and connection lines around them are replaced by ellipses “…”.

[0129] As shown in FIG. 11, Figure 16 pixels 1101, 1103, 1106, 1108, 1111, 1113, 1116, and 1118 are panchromatic pixels W, pixels 1102 and 1105 are first color pixels A (for example, red pixels R), pixels 1104, 1107, 1112, and 1115 are second color pixels B (for example, green pixels G), and pixels 1114 and 1117 are third color pixels C (for example, blue pixels Bu). From Figure 16As can be seen, the control end TG of the exposure control circuit in the panchromatic pixels W (pixels 1101, 1103, 1106 and 1108) is connected to a first exposure control line TX1, and the control end TG of the exposure control circuit in the panchromatic pixels W (1111, 1113, 1116 and 1118) is connected to another first exposure control line TX1; the control end TG of the exposure control circuit in the first color pixels A (pixels 1102 and 1105), the control end TG of the exposure control circuit in the second color pixels B (pixels 1104 and 1107) are connected to a second exposure control line TX2, and the control end TG of the exposure control circuit in the second color pixels B (pixels 1112 and 1115), the control end TG of the exposure control circuit in the third color pixels C (pixels 1114 and 1117) are connected to another second exposure control line TX2. Each first exposure control line TX1 can control the exposure time of the panchromatic pixels through a first exposure control signal; each second exposure control line TX2 can control the exposure time of the color pixels (for example, the first color pixels A and the second color pixels B, the second color pixels B and the third color pixels C) through a second exposure control signal. Thus, the exposure time of the panchromatic pixels and the color pixels can be independently controlled. For example, the color pixels can continue to be exposed after the exposure of the panchromatic pixels is completed, so as to achieve a desired imaging effect.

[0130] Please refer to Figure 2 and Figure 16 , the first exposure control line TX1 and the second exposure control line TX2 are connected to the vertical driving unit 12 in Figure 2 , and transmit the corresponding exposure control signals in the vertical driving unit 12 to the control end TG of the exposure control circuit of the pixels in the pixel array 11.

[0131] It can be understood that since there are multiple pixel row groups in the pixel array 11, the vertical driving unit 12 is connected to multiple first exposure control lines TX1 and multiple second exposure control lines TX2. The multiple first exposure control lines TX1 and the multiple second exposure control lines TX2 correspond to the corresponding pixel row groups.

[0132] For example, the first first exposure control line TX1 corresponds to the panchromatic pixels in the first and second rows; the second first exposure control line TX1 corresponds to the panchromatic pixels in the third and fourth rows, and so on. The third first exposure control line TX1 corresponds to the panchromatic pixels in the fifth and sixth rows; the fourth first exposure control line TX1 corresponds to the panchromatic pixels in the seventh and eighth rows, and the correspondence between the first exposure control lines TX1 below and the panchromatic pixels in the rows below will not be described again. The signal timing of the signals transmitted by different first exposure control lines TX1 will also be different, which is configured by the vertical driving unit 12.

[0133] For example, the first second exposure control line TX2 corresponds to the color pixels in the first and second rows; the second second exposure control line TX2 corresponds to the color pixels in the third and fourth rows, and so on. The third second exposure control line TX2 corresponds to the color pixels in the fifth and sixth rows; the fourth second exposure control line TX2 corresponds to the color pixels in the seventh and eighth rows, and the correspondence between the subsequent second exposure control lines TX2 and the color pixels in the rows below is not elaborated further. The signal timing transmitted by different second exposure control lines TX2 will also be different, and this signal timing is also configured by the vertical drive unit 12.

[0134] Figures 17 to 32 Various image sensors 10 are shown. Figure 2 An example of pixel arrangement shown. See also Figure 2 ,and Figures 17 to 32 The image sensor 10 includes a two-dimensional pixel array (i.e., a plurality of color pixels (e.g., a plurality of first color pixels A, a plurality of second color pixels B, and a plurality of third color pixels C) and a plurality of panchromatic pixels W) composed of multiple color pixels (e.g., a plurality of first color pixels A, a plurality of second color pixels B, and a plurality of third color pixels C) and a plurality of panchromatic pixels W). Figure 16 The pixel array 11 shown is an example. The color pixels have a narrower spectral response than the panchromatic pixels. The response spectrum of a color pixel is, for example, a portion of the response spectrum of a panchromatic pixel W. The two-dimensional pixel array includes minimal repeating units (…). Figures 17 to 32 Examples of pixel minimum repeating units in various image sensors 10 are shown. A two-dimensional pixel array consists of multiple minimum repeating units, which are replicated and arranged in rows and columns. In each minimum repeating unit, a panchromatic pixel W is positioned along a first diagonal direction D1, and a colored pixel is positioned along a second diagonal direction D2, where the first diagonal direction D1 differs from the second diagonal direction D2. The first exposure time of at least two adjacent panchromatic pixels along the first diagonal direction D1 is controlled by a first exposure signal, and the second exposure time of at least two adjacent colored pixels along the second diagonal direction D2 is controlled by a second exposure signal, thereby achieving independent control of the exposure time of the panchromatic pixels and the exposure time of the colored pixels. Each minimum repeating unit includes multiple sub-units, each sub-unit including multiple single-color pixels (e.g., multiple first-color pixels A, multiple second-color pixels B, or multiple third-color pixels C) and multiple panchromatic pixels W. For example, please combine... Figure 3 and Figure 16Pixels 1101-1108 and 1111-1118 form a minimum repeating unit. Among them, pixels 1101, 1103, 1106, 1108, 1111, 1113, 1116, and 1118 are full-color pixels, while pixels 1102, 1104, 1105, 1107, 1112, 1114, 1115, and 1117 are color pixels. Pixels 1101, 1102, 1105, and 1106 form a sub-unit, where pixels 1101 and 1106 are full-color pixels, and pixels 1102 and 1105 are single-color pixels (e.g., first-color pixel A); pixels 1103, 1104, 1107, and 1108 form a sub-unit, where pixels 1103 and 1108 are full-color pixels, and pixels 1104 and 1107 are single-color pixels (e.g., second-color pixel B). Pixels 1111, 1112, 1115, and 1116 form a subunit, where pixels 1111 and 1116 are full-color pixels, and pixels 1112 and 1115 are single-color pixels (e.g., second-color pixels B); pixels 1113, 1114, 1117, and 1118 form a subunit, where pixels 1113 and 1118 are full-color pixels, and pixels 1114 and 1117 are single-color pixels (e.g., third-color pixels C).

[0135] For example, the smallest repeating unit has the same number of pixels in both rows and columns. Examples of such smallest repeating units include, but are not limited to, 4x4, 6x6, 8x8, and 10x10 units. Similarly, the sub-units within the smallest repeating unit have the same number of pixels in both rows and columns. Examples of such sub-units include, but are not limited to, 2x2, 3x3, 4x4, and 5x5 units. This configuration helps to balance the resolution and color performance of the image in both row and column directions, improving display quality.

[0136] For example, Figure 17 This is a schematic diagram of a minimum repeating unit of 1181 pixels in an embodiment of this application; the minimum repeating unit is 4 rows and 4 columns of 16 pixels, and the sub-unit is 2 rows and 2 columns of 4 pixels, arranged as follows:

[0137] WAWB

[0138] AWBW

[0139] WBWC

[0140] BWCW

[0141] W represents a full-color pixel; A represents the first color pixel among multiple color pixels; B represents the second color pixel among multiple color pixels; and C represents the third color pixel among multiple color pixels.

[0142] For example, such as Figure 17 As shown, the panchromatic pixels W are arranged in a first diagonal direction D1 (i.e. Figure 17 the direction connecting the upper left corner and the lower right corner), and the color pixels are arranged in a second diagonal direction D2 (e.g. Figure 17 the direction connecting the lower left corner and the upper right corner), and the first diagonal direction D1 is different from the second diagonal direction D2. For example, the first diagonal and the second diagonal are perpendicular. The first exposure time of two panchromatic pixels W adjacent in the first diagonal direction D1 (e.g. the two panchromatic pixels in the first row and the second column from the upper left) is controlled by a first exposure signal, and the second exposure time of at least two color pixels adjacent in the second diagonal direction D2 (e.g. the two color pixels in the fourth row and the first column and the third row and the second column from the upper left) is controlled by a second exposure signal.

[0143] It should be noted that the first diagonal direction D1 and the second diagonal direction D2 are not limited to diagonal lines, but also include directions parallel to the diagonal lines, for example Figure 16 In this case, the panchromatic pixels 1101, 1106, 1113, and 1118 are arranged in the first diagonal direction D1, and the panchromatic pixels 1103 and 1108 are also arranged in the first diagonal direction D1, and the panchromatic pixels 1111 and 1116 are also arranged in the first diagonal direction D1; the second color pixels 1104, 1107, 1112, and 1115 are arranged in the second diagonal direction D2, and the first color pixels 1102 and 1105 are also arranged in the second diagonal direction D2, and the third color pixels 1114 and 1117 are also arranged in the second diagonal direction D2, and the following Figures 18 to 32 In this case, the first diagonal direction D1 and the second diagonal direction D2 are explained in the same way as here. The "direction" here is not a single direction, and can be understood as a concept of "straight line" indicating arrangement, and can have a bidirectional direction at both ends of the straight line.

[0144] It should be understood that the terms "upper", "lower", "left", "right", and the like in this case and in the following indicate the positional or locational relationship based on the positional or locational relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0145] For example, as Figure 17As shown, the panchromatic pixels of the first and second rows are connected together by the first exposure control line TX1 in a "W" shape to achieve separate control of the panchromatic pixel exposure time. The color pixels (A and B) of the first and second rows are connected together by the second exposure control line TX2 in a "W" shape to achieve separate control of the color pixel exposure time. The panchromatic pixels of the third and fourth rows are connected together by the first exposure control line TX1 in a "W" shape to achieve separate control of the panchromatic pixel exposure time. The color pixels (B and C) of the third and fourth rows are connected together by the second exposure control line TX2 in a "W" shape to achieve separate control of the color pixel exposure time. For example, the first exposure signal is transmitted via the first exposure control line TX1, and the second exposure signal is transmitted via the second exposure control line TX2. For example, the first exposure control line TX1 is in a "W" shape and is electrically connected to the control ends of the exposure control circuits in the panchromatic pixels of the adjacent two rows; the second exposure control line TX2 is in a "W" shape and is electrically connected to the control ends of the exposure control circuits in the color pixels of the adjacent two rows. The specific connection manner can be referred to the foregoing description of the connection and pixel circuit. Figure 3 and Figure 16 the relevant part of the description.

[0146] It should be noted that the first exposure control line TX1 and the second exposure control line TX2 in a "W" shape do not mean that the physical wiring must be strictly arranged in a "W" shape, but only the connection manner corresponds to the arrangement of the panchromatic pixels and the color pixels. For example, the "W" shaped exposure control line is arranged to correspond to the "W" shaped pixel arrangement manner, which has simple wiring, good resolution and color of the pixel arrangement, and realizes independent control of the panchromatic pixel exposure time and the color pixel exposure time at low cost.

[0147] For example, Figure 18 is a schematic diagram of another pixel arrangement of the minimum repeating unit 1182 in the embodiment of the present application. The minimum repeating unit is 4 rows and 4 columns of 16 pixels, the subunit is 2 rows and 2 columns of 4 pixels, and the arrangement manner is:

[0148] A W B W

[0149] W A W B

[0150] B W C W

[0151] W B W C

[0152] W represents a panchromatic pixel; A represents a first color pixel in a plurality of color pixels; B represents a second color pixel in the plurality of color pixels; and C represents a third color pixel in the plurality of color pixels.

[0153] For example, as Figure 18 shown, the panchromatic pixels W are arranged in the first diagonal direction D1 (i.e. Figure 18 The color pixels are arranged in the second diagonal direction D2 (for example, the direction connecting the upper right corner and the lower left corner) corresponding to the arrangement of the first color pixel A, the second color pixel B and the third color pixel C. Figure 18 The color pixels are arranged in the second diagonal direction D2 (for example, the direction connecting the upper right corner and the lower left corner) corresponding to the arrangement of the first color pixel A, the second color pixel B and the third color pixel C.

[0154] For example, as shown in FIG. 11, the first row and the second row of the full-color pixels are connected together by the first exposure control line TX1 in the shape of “W” to realize the separate control of the exposure time of the full-color pixels. The first row and the second row of the color pixels (A and B) are connected together by the second exposure control line TX2 in the shape of “W” to realize the separate control of the exposure time of the color pixels. The third row and the fourth row of the full-color pixels are connected together by the first exposure control line TX1 in the shape of “W” to realize the separate control of the exposure time of the full-color pixels. The third row and the fourth row of the color pixels (B and C) are connected together by the second exposure control line TX2 in the shape of “W” to realize the separate control of the exposure time of the color pixels. Figure 18

[0155] For example, as shown in FIG. 11, the first row and the second row of the full-color pixels are connected together by the first exposure control line TX1 in the shape of “W” to realize the separate control of the exposure time of the full-color pixels. The first row and the second row of the color pixels (A and B) are connected together by the second exposure control line TX2 in the shape of “W” to realize the separate control of the exposure time of the color pixels. The third row and the fourth row of the full-color pixels are connected together by the first exposure control line TX1 in the shape of “W” to realize the separate control of the exposure time of the full-color pixels. The third row and the fourth row of the color pixels (B and C) are connected together by the second exposure control line TX2 in the shape of “W” to realize the separate control of the exposure time of the color pixels. Figure 19 is a schematic diagram of another arrangement of the minimum repeating unit 1183 in an embodiment of the present application. Figure 20 is a schematic diagram of another arrangement of the minimum repeating unit 1184 in an embodiment of the present application. In the embodiment of FIG. 12, the first color pixel A is a red pixel R; the second color pixel B is a green pixel G; and the third color pixel C is a blue pixel Bu. Figure 19 Figure 20 In the embodiments of FIGS. 11 and 12, the arrangement of the first color pixel A, the second color pixel B and the third color pixel C is as follows. Figure 17 Figure 18 In the embodiments of FIGS. 11 and 12, the arrangement of the first color pixel A, the second color pixel B and the third color pixel C is as follows.

[0156] It should be noted that in some embodiments, the response wavelength band of the full-color pixel W is the visible light wavelength band (for example, 400nm-760nm). For example, an infrared filter is arranged on the full-color pixel W to filter out the infrared light. In some embodiments, the response wavelength band of the full-color pixel W is the visible light wavelength band and the near-infrared wavelength band (for example, 400nm-1000nm), which is matched with the response wavelength band of the photoelectric conversion element 117 (for example, the photodiode PD) in the image sensor 10. For example, the full-color pixel W can not be provided with a filter, and the response wavelength band of the full-color pixel W is determined by the response wavelength band of the photodiode, i.e., the two are matched. Embodiments of the present application include but are not limited to the above wavelength band range.

[0157] ​​​For example, Figure 21 is a schematic view of another pixel arrangement of a minimum repeating unit 1185 in an embodiment of the present application. Figure 22 is a schematic view of another pixel arrangement of a minimum repeating unit 1186 in an embodiment of the present application. In Figure 21 and Figure 22 , the first color pixel A is a red pixel R; the second color pixel B is a yellow pixel Y; and the third color pixel C is a blue pixel Bu, respectively. Figure 17 and Figure 18 , the first color pixel A is a magenta pixel M; the second color pixel B is a cyan pixel Cy; and the third color pixel C is a yellow pixel Y, respectively.

[0158] For example, Figure 23 is a schematic view of another pixel arrangement of a minimum repeating unit 1187 in an embodiment of the present application. Figure 24 is a schematic view of another pixel arrangement of a minimum repeating unit 1188 in an embodiment of the present application. In Figure 23 and Figure 24 , the first color pixel A is a magenta pixel M; the second color pixel B is a cyan pixel Cy; and the third color pixel C is a yellow pixel Y, respectively. Figure 17 and Figure 18 , the first color pixel A is a magenta pixel M; the second color pixel B is a cyan pixel Cy; and the third color pixel C is a yellow pixel Y, respectively.

[0159] For example, Figure 25 is a schematic view of another pixel arrangement of a minimum repeating unit 1191 in an embodiment of the present application. The minimum repeating unit is 6 rows and 6 columns of 36 pixels, the subunit is 3 rows and 3 columns of 9 pixels, and the arrangement is as follows:

[0160] W A W B W B

[0161] A W A W B W

[0162] W A W B W B

[0163] B W B W C W

[0164] W B W C W C

[0165] B W B W C W

[0166] W represents a panchromatic pixel; A represents a first color pixel in a plurality of color pixels; B represents a second color pixel in the plurality of color pixels; and C represents a third color pixel in the plurality of color pixels.

[0167] For example, as Figure 25As shown, the panchromatic pixels of the first and second rows are connected together by a first exposure control line TX1 in a "W" pattern to enable individual control of the panchromatic pixel exposure time. The color pixels (A and B) of the first and second rows are connected together by a second exposure control line TX2 in a "W" pattern to enable individual control of the color pixel exposure time. The panchromatic pixels of the third and fourth rows are connected together by a first exposure control line TX1 in a "W" pattern to enable individual control of the panchromatic pixel exposure time. The color pixels (A, B and C) of the third and fourth rows are connected together by a second exposure control line TX2 in a "W" pattern to enable individual control of the color pixel exposure time. The panchromatic pixels of the fifth and sixth rows are connected together by a first exposure control line TX1 in a "W" pattern to enable individual control of the panchromatic pixel exposure time. The color pixels (B and C) of the fifth and sixth rows are connected together by a second exposure control line TX2 in a "W" pattern to enable individual control of the color pixel exposure time.

[0168] For example, Figure 26 is a schematic diagram of yet another arrangement of the pixels of the minimum repeating unit 1192 in an embodiment of the application. The minimum repeating unit is 6 rows by 6 columns of 36 pixels, the subunit is 3 rows by 3 columns of 9 pixels, and the arrangement is:

[0169] A W A W B W

[0170] W A W B W B

[0171] A W A W B W

[0172] W B W C W C

[0173] B W B W C W

[0174] W B W C W C

[0175] W represents a panchromatic pixel; A represents a first color pixel of a plurality of color pixels; B represents a second color pixel of the plurality of color pixels; and C represents a third color pixel of the plurality of color pixels.

[0176] For example, as Figure 26As shown, the panchromatic pixels of the first and second rows are connected together by the first exposure control line TX1 in a "W" shape to achieve individual control of the panchromatic pixel exposure time. The color pixels (A and B) of the first and second rows are connected together by the second exposure control line TX2 in a "W" shape to achieve individual control of the color pixel exposure time. The panchromatic pixels of the third and fourth rows are connected together by the first exposure control line TX1 in a "W" shape to achieve individual control of the panchromatic pixel exposure time. The color pixels (A, B and C) of the third and fourth rows are connected together by the second exposure control line TX2 in a "W" shape to achieve individual control of the color pixel exposure time. The panchromatic pixels of the fifth and sixth rows are connected together by the first exposure control line TX1 in a "W" shape to achieve individual control of the panchromatic pixel exposure time. The color pixels (B and C) of the fifth and sixth rows are connected together by the second exposure control line TX2 in a "W" shape to achieve individual control of the color pixel exposure time.

[0177] For example, Figure 27 is a schematic diagram of another pixel arrangement of a minimum repeating unit 1193 in an embodiment of the present application. Figure 28 is a schematic diagram of another pixel arrangement of a minimum repeating unit 1194 in an embodiment of the present application. In Figure 27 and Figure 28 In the embodiments of Figure 25 and Figure 26 , the first color pixel A is a red pixel R; the second color pixel B is a green pixel G; and the third color pixel C is a blue pixel Bu.

[0178] For example, in other embodiments, the first color pixel A is a red pixel R; the second color pixel B is a yellow pixel Y; and the third color pixel C is a blue pixel Bu. For example, in other embodiments, the first color pixel A is a magenta pixel M; the second color pixel B is a cyan pixel Cy; and the third color pixel C is a yellow pixel Y. Embodiments of the present application include but are not limited to this. The circuit specific connection mode is described above and will not be described here.

[0179] For example, Figure 29 is a schematic diagram of another pixel arrangement of a minimum repeating unit 1195 in an embodiment of the present application. The minimum repeating unit is 8 rows and 8 columns of 64 pixels, the subunit is 4 rows and 4 columns of 16 pixels, and the arrangement mode is:

[0180] W A W A W B W B

[0181] A W A W B W B W

[0182] W A W A W B W B

[0183] AWAWBWBW

[0184] WBWBWCWC

[0185] BWBWCWCW

[0186] WBWBWCWC

[0187] BWBWCWCW

[0188] W represents a full-color pixel; A represents the first color pixel among multiple color pixels; B represents the second color pixel among multiple color pixels; and C represents the third color pixel among multiple color pixels.

[0189] For example, such as Figure 29 As shown, the panchromatic pixels in the first and second rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the first and second rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the third and fourth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the third and fourth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the fifth and sixth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (B and C) in the fifth and sixth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the seventh and eighth rows are connected together by a first exposure control line TX1 in a "W" shape to achieve individual control of the exposure time of the panchromatic pixels. The color pixels (B and C) in the seventh and eighth rows are connected together by a second exposure control line TX2 in a "W" shape to achieve individual control of the exposure time of the color pixels.

[0190] For example, Figure 30 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1196 pixels in an embodiment of this application. The smallest repeating unit is 8 rows and 8 columns, consisting of 64 pixels, and the sub-unit is 4 rows and 4 columns, consisting of 16 pixels. The arrangement is as follows:

[0191] AWAWBWBW

[0192] WAWAWBWB

[0193] AWAWBWBW

[0194] WAWAWBWB

[0195] BWBWCWCW

[0196] WBWBWCWC

[0197] BWBWCWCW

[0198] WBWBWCWC

[0199] W represents a full-color pixel; A represents the first color pixel among multiple color pixels; B represents the second color pixel among multiple color pixels; and C represents the third color pixel among multiple color pixels.

[0200] For example, such as Figure 30 As shown, the panchromatic pixels in the first and second rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the first and second rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the third and fourth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the third and fourth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the fifth and sixth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (B and C) in the fifth and sixth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the seventh and eighth rows are connected together by a first exposure control line TX1 in a "W" shape to achieve individual control of the exposure time of the panchromatic pixels. The color pixels (B and C) in the seventh and eighth rows are connected together by a second exposure control line TX2 in a "W" shape to achieve individual control of the exposure time of the color pixels.

[0201] For example, Figure 31 This is a schematic diagram of another arrangement of the smallest repeating unit 1197 pixels in an embodiment of this application. Figure 32 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1198 pixels in an embodiment of this application. Figure 31 and Figure 32 In the embodiments, respectively corresponding to Figure 29 and Figure 30 The arrangement is as follows: the first color pixel A is the red pixel R; the second color pixel B is the green pixel G; and the third color pixel C is the blue pixel Bu.

[0202] For example, in other embodiments, the first color pixel A is a red pixel R; the second color pixel B is a yellow pixel Y; and the third color pixel C is a blue pixel Bu. For example, the first color pixel A is a magenta pixel M; the second color pixel B is a cyan pixel Cy; and the third color pixel C is a yellow pixel Y. Embodiments of this application include, but are not limited to, these. Specific circuit connections are described above and will not be repeated here.

[0203] As can be seen from the above embodiments, such as Figures 17 to 32 As shown, image sensor 10 ( Figure 2 (As shown) includes multiple colored pixels and multiple panchromatic pixels W arranged in a matrix, with the colored pixels and panchromatic pixels spaced apart in both row and column directions.

[0204] For example, alternating between full-color pixels, color pixels, full-color pixels, color pixels, etc. along the row direction.

[0205] For example, alternating between solid-color pixels, colored pixels, solid-color pixels, colored pixels, etc. along the column direction.

[0206] Please combine Figure 16 The first exposure control line TX1 is electrically connected to the control terminal TG (e.g., the gate of the transfer transistor 112) of the exposure control circuit 116 in the (2n-1)th and (2n)th rows of full-color pixels W; the second exposure control line TX2 is electrically connected to the control terminal TG (e.g., the gate of the transfer transistor 112) of the exposure control circuit 116 in the (2n-1)th and (2n)th rows of color pixels; n is a natural number greater than or equal to 1.

[0207] For example, when n=1, the first exposure control line TX1 is electrically connected to the control terminal TG of the exposure control circuit 116 in the panchromatic pixels W of the first and second rows; the second exposure control line TX2 is electrically connected to the control terminal TG of the exposure control circuit 116 in the color pixels of the first and second rows. When n=2, the first exposure control line TX1 is electrically connected to the control terminal TG of the exposure control circuit 116 in the panchromatic pixels W of the third and fourth rows; the second exposure control line TX2 is electrically connected to the control terminal TG of the exposure control circuit 116 in the color pixels of the third and fourth rows. And so on, which will not be elaborated further here.

[0208] In some embodiments, the first exposure time is shorter than the second exposure time. The first exposure time is determined based on the n-potential well layer 1172 of the panchromatic pixels. Figure 4A The second exposure time can be determined based on the n-potential well layer 1172 of the color pixel (as shown). Figure 4A As shown in the figure, this is used to determine the determination.

[0209] Please see Figure 33The application provides a camera assembly 40. The camera assembly 40 comprises the image sensor 10, the processing chip 20 and the lens 30 according to any one of the above embodiments. The image sensor 10 is electrically connected to the processing chip 20. The lens 30 is arranged on the light path of the image sensor 10. The processing chip 20 can be packaged in the same housing of the camera assembly 40 together with the image sensor 10 and the lens 30; or the image sensor 10 and the lens 30 are packaged in the housing, and the processing chip 20 is arranged outside the housing.

[0210] The application also provides an image acquisition method which can be applied to the camera assembly 40. Figure 33 As shown in the figure, the image acquisition method comprises: Figure 34

[0211] 01: controlling exposure of a two-dimensional pixel array to obtain a panchromatic raw image and a color raw image;

[0212] 02: processing the color raw image to regard all pixels of each subunit as a monochrome large pixel corresponding to a single color in the subunit, and outputting the pixel value of the monochrome large pixel to obtain a color intermediate image;

[0213] 03: processing the panchromatic raw image to obtain a panchromatic intermediate image; and

[0214] 04: processing the color intermediate image and / or the panchromatic intermediate image to obtain a target image.

[0215] Please refer to Figure 2 and Figure 33 The image acquisition method of the application can be implemented by the camera assembly 40. Among them, step 01 can be implemented by the image sensor 10. Steps 02, 03 and 04 can be implemented by the processing chip 20. That is to say, the image sensor 10 can be exposed to obtain a panchromatic raw image and a color raw image. The processing chip 20 can be used to process the color raw image to regard all pixels of each subunit as a monochrome large pixel corresponding to a single color in the subunit, and output the pixel value of the monochrome large pixel to obtain a color intermediate image. The processing chip 20 can also be used to process the panchromatic raw image to obtain a panchromatic intermediate image, and process the color intermediate image and / or the panchromatic intermediate image to obtain a target image.

[0216] Please combine Figure 35 ​In the related art, if the pixel array of the image sensor includes both the panchromatic pixels and the color pixels, the image sensor will fit the pixel value of each panchromatic pixel in the pixel array to the pixel value of other color pixels when the image sensor is working, so as to output an original image including only the color pixels. Specifically, taking a pixel A as a red pixel R, a pixel B as a green pixel G, and a pixel C as a blue pixel Bu as an example, after the column processing unit in the image sensor reads out the pixel values of the plurality of red pixels R, the pixel values of the plurality of green pixels G, the pixel values of the plurality of blue pixels Bu, and the pixel values of the plurality of panchromatic pixels W, the image sensor will first fit the pixel value of each panchromatic pixel W to the red pixel R, the green pixel G, and the blue pixel Bu adjacent to the panchromatic pixel, and then convert the image in the non-Bayer array arrangement into an original image in the Bayer array arrangement for output, so as to enable the processing chip to perform subsequent processing on the original image, such as performing interpolation processing on the original image to obtain a full-color image (in which the pixel value of each pixel is composed of three components of red, green, and blue) and the like. In this processing manner, the image sensor needs to perform a relatively complex algorithm, and the amount of calculation is relatively large. In addition, since the high-throughput platform does not support the processing of the image in the non-Bayer array arrangement, it is necessary to increase additional hardware (for example, an additional processing chip) in the image sensor to perform the processing of converting the image in the non-Bayer array arrangement into the original image in the Bayer array arrangement.

[0217] The image acquisition method and the camera assembly 40 of the present application can reduce the amount of calculation of the image sensor and avoid increasing additional hardware in the image sensor.

[0218] Specifically, please refer to Figure 2 and Figure 36 When the user requests to take a photo, the vertical driving unit 12 in the image sensor 10 controls the plurality of panchromatic pixels and the plurality of color pixels in the two-dimensional pixel array to be exposed, and the column processing unit 14 reads out the pixel value of each panchromatic pixel and the pixel value of each color pixel. The image sensor 10 does not perform the operation of fitting the pixel value of the panchromatic pixel to the pixel value of the color pixel, but directly outputs a panchromatic original image according to the pixel values of the plurality of panchromatic pixels, and directly outputs a color original image according to the pixel values of the plurality of color pixels.

[0219] As Figure 36As shown, the full-color raw image includes a plurality of full-color pixels W and a plurality of null pixels N (NULL), where the null pixels are neither full-color pixels nor color pixels, and the positions of the null pixels N in the full-color raw image can be regarded as positions without pixels, or the pixel values of the null pixels can be regarded as zero. Comparing the two-dimensional pixel array with the full-color raw image, it can be seen that for each sub-unit in the two-dimensional pixel array, the sub-unit includes two full-color pixels W and two color pixels (color pixel A, color pixel B, or color pixel C). The full-color raw image also has a sub-unit corresponding to each sub-unit in the two-dimensional pixel array, and the sub-unit of the full-color raw image includes two full-color pixels W and two null pixels N, and the positions of the two null pixels N correspond to the positions of the two color pixels in the sub-unit of the two-dimensional pixel array.

[0220] Similarly, the color raw image includes a plurality of color pixels and a plurality of null pixels N, where the null pixels are neither full-color pixels nor color pixels, and the positions of the null pixels N in the color raw image can be regarded as positions without pixels, or the pixel values of the null pixels can be regarded as zero. Comparing the two-dimensional pixel array with the color raw image, it can be seen that for each sub-unit in the two-dimensional pixel array, the sub-unit includes two full-color pixels W and two color pixels. The color raw image also has a sub-unit corresponding to each sub-unit in the two-dimensional pixel array, and the sub-unit of the color raw image includes two color pixels and two null pixels N, and the positions of the two null pixels N correspond to the positions of the two full-color pixels W in the sub-unit of the two-dimensional pixel array.

[0221] After the processing chip 20 receives the full-color raw image and the color raw image output by the image sensor 10, the full-color raw image can be further processed to obtain a full-color intermediate image, and the color raw image can be further processed to obtain a color intermediate image. For example, the color raw image can be converted into a color intermediate image in the manner shown. As shown in FIG. 3, the color raw image is converted into a color intermediate image by performing a color interpolation on the color raw image. Figure 37 Figure 37 ​As shown, the color original image includes a plurality of subunits, each of which includes a plurality of null pixels N and a plurality of single-color pixels (also referred to as single-color pixels). Specifically, some subunits include two null pixels N and two single-color pixels A, some subunits include two null pixels N and two single-color pixels B, and some subunits include two null pixels N and two single-color pixels C. The processing chip 20 can regard all the pixels in a subunit including a null pixel N and a single-color pixel A as a single-color large pixel A corresponding to the single color A in the subunit, regard all the pixels in a subunit including a null pixel N and a single-color pixel B as a single-color large pixel B corresponding to the single color B in the subunit, and regard all the pixels in a subunit including a null pixel N and a single-color pixel C as a single-color large pixel C corresponding to the single color C in the subunit. In this way, the processing chip 20 can form a color intermediate image according to a plurality of single-color large pixels A, a plurality of single-color large pixels B, and a plurality of single-color large pixels C. If the color original image including a plurality of null pixels N is regarded as an image having a second resolution, then the color intermediate image obtained in the manner shown is an image having a first resolution, where the first resolution is less than the second resolution. After obtaining the panchromatic intermediate image and the color intermediate image, the processing chip 20 can further process the panchromatic intermediate image and / or the color intermediate image to obtain a target image. Specifically, the processing chip 20 can process only the panchromatic intermediate image to obtain the target image, the processing chip 20 can process only the color intermediate image to obtain the target image, and the processing chip 20 can process both the panchromatic intermediate image and the color intermediate image to obtain the target image. The processing chip 20 can determine the processing manner of the two intermediate images according to actual requirements. Figure 37 The color intermediate image obtained in the manner shown is an image having a first resolution, where the first resolution is less than the second resolution. After obtaining the panchromatic intermediate image and the color intermediate image, the processing chip 20 can further process the panchromatic intermediate image and / or the color intermediate image to obtain a target image. Specifically, the processing chip 20 can process only the panchromatic intermediate image to obtain the target image, the processing chip 20 can process only the color intermediate image to obtain the target image, and the processing chip 20 can process both the panchromatic intermediate image and the color intermediate image to obtain the target image. The processing chip 20 can determine the processing manner of the two intermediate images according to actual requirements.

[0222] In the image acquisition method of the embodiments of the present application, the image sensor 10 can directly output the panchromatic original image and the color original image, and the subsequent processing of the panchromatic original image and the color original image is performed by the processing chip 20, so that the image sensor 10 does not need to perform the operation of fitting the pixel value of the panchromatic pixel W into the pixel value of the color pixel, the operation amount of the image sensor 10 is reduced, and the design of the image sensor 10 can be simplified without the need to add new hardware in the image sensor 10 to support the image sensor 10 to perform image processing.

[0223] In some embodiments, the step 01 of controlling exposure of the two-dimensional pixel array to obtain the panchromatic original image and the color original image can be implemented in various manners.

[0224] Please refer to Figure 38 In one example, the step 01 includes:

[0225] 011: controlling all the panchromatic pixels and all the color pixels in the two-dimensional pixel array to be exposed at the same time;

[0226] 012: outputting pixel values of all the panchromatic pixels to obtain a panchromatic raw image; and

[0227] 013: outputting pixel values of all the color pixels to obtain a color raw image.

[0228] Referring to Figure 33 , steps 011, 012 and 013 can be implemented by the image sensor 10. That is, all the panchromatic pixels and all the color pixels in the image sensor 10 are exposed at the same time. The image sensor 10 can output pixel values of all the panchromatic pixels to obtain a panchromatic raw image, and can also output pixel values of all the color pixels to obtain a color raw image.

[0229] Referring to Figure 2 and Figure 16 , the panchromatic pixels and the color pixels can be exposed at the same time, wherein an exposure time of the panchromatic pixels can be less than or equal to an exposure time of the color pixels. Specifically, when the first exposure time of the panchromatic pixels is equal to the second exposure time of the color pixels, the exposure start time and the exposure end time of the panchromatic pixels are the same as the exposure start time and the exposure end time of the color pixels, respectively. When the first exposure time is less than the second exposure time, the exposure start time of the panchromatic pixels is later than or equal to the exposure start time of the color pixels, and the exposure end time of the panchromatic pixels is earlier than the exposure end time of the color pixels; or, when the first exposure time is less than the second exposure time, the exposure start time of the panchromatic pixels is later than the exposure start time of the color pixels, and the exposure end time of the panchromatic pixels is earlier than or equal to the exposure end time of the color pixels. After the panchromatic pixels and the color pixels are exposed, the image sensor 10 outputs pixel values of all the panchromatic pixels to obtain a panchromatic raw image, and outputs pixel values of all the color pixels to obtain a color raw image. The panchromatic raw image can be output before the color raw image, or the color raw image can be output before the panchromatic raw image, or the panchromatic raw image and the color raw image can be output at the same time. The output order of the two is not limited herein. The panchromatic pixels and the color pixels exposed at the same time can reduce the acquisition time of the panchromatic raw image and the color raw image, and accelerate the acquisition process of the panchromatic raw image and the color raw image. The panchromatic pixels and the color pixels exposed at the same time have great advantages in modes with high requirements for image output speed, such as quick shooting and continuous shooting.

[0230] Referring to Figure 39 , in another example, step 01 includes:

[0231] 014: controlling all the panchromatic pixels and all the color pixels in the two-dimensional pixel array to be exposed at different times;

[0232] 015: outputting pixel values of all the panchromatic pixels to obtain a panchromatic raw image; and

[0233] 016: output the pixel values of all the color pixels to obtain a color raw image.

[0234] Referring to Figure 33 , the step 014, the step 015 and the step 016 can be implemented by the image sensor 10. That is to say, all the panchromatic pixels and all the color pixels in the image sensor 10 are exposed in time division. The image sensor 10 can output the pixel values of all the panchromatic pixels to obtain a panchromatic raw image, and can also output the pixel values of all the color pixels to obtain a color raw image.

[0235] Specifically, the panchromatic pixels and the color pixels can be exposed in time division, wherein the exposure time of the panchromatic pixels can be less than or equal to the exposure time of the color pixels. Specifically, regardless of whether the first exposure time is equal to the second exposure time, the manner in which all the panchromatic pixels and all the color pixels are exposed in time division can be: (1) all the panchromatic pixels first perform exposure for the first exposure time, and after all the panchromatic pixels are exposed, all the color pixels perform exposure for the second exposure time; (2) all the color pixels first perform exposure for the second exposure time, and after all the color pixels are exposed, all the panchromatic pixels perform exposure for the first exposure time. After the panchromatic pixels and the color pixels are exposed, the image sensor 10 outputs the pixel values of all the panchromatic pixels to obtain a panchromatic raw image, and outputs the pixel values of all the color pixels to obtain a color raw image. The output manner of the panchromatic raw image and the color raw image can be: (1) when the panchromatic pixels are exposed first, the image sensor 10 can output the panchromatic raw image during the exposure of the color pixels, or can output the panchromatic raw image and the color raw image in sequence after the exposure of the color pixels is completed; (2) when the color pixels are exposed first, the image sensor 10 can output the color raw image during the exposure of the panchromatic pixels, or can output the color raw image and the panchromatic raw image in sequence after the exposure of the panchromatic pixels is completed; (3) regardless of which one of the panchromatic pixels and the color pixels is exposed first, the image sensor 10 can output the panchromatic raw image and the color raw image simultaneously after all the pixels are exposed. The control logic of the manner in which the panchromatic pixels and the color pixels are exposed in time division in the present example is relatively simple.

[0236] The image sensor 10 can simultaneously have the functions of controlling the panchromatic pixels and the color pixels to be exposed simultaneously and controlling the panchromatic pixels and the color pixels to be exposed in time division as shown in Figure 38 and Figure 39 . Which one of the two exposure manners is adopted by the image sensor 10 in the process of collecting an image can be selected autonomously according to actual requirements. For example, the simultaneous exposure manner can be adopted in the fast shooting, continuous shooting and other modes to meet the requirement of rapid image output; the time-division exposure manner can be adopted in the ordinary shooting mode to simplify the control logic, etc.

[0237] Figure 38 and Figure 39 In the two examples shown in FIGS. 10 and 11, the exposure order of the panchromatic pixels and the color pixels can be controlled by the control unit 13 in the image sensor 10.

[0238] Figure 38 and Figure 39 In the two examples shown in FIGS. 10 and 11, the exposure time of the panchromatic pixels can be controlled by the first exposure signal, and the exposure time of the color pixels can be controlled by the second exposure signal.

[0239] Specifically, please refer to Figure 16 As an example, the image sensor 10 can control at least two panchromatic pixels in the first diagonal direction with the first exposure signal for a first exposure time, and control at least two color pixels in the second diagonal direction with the second exposure signal for a second exposure time, where the first exposure time can be less than or equal to the second exposure time. Specifically, the vertical driving unit 12 in the image sensor 10 transmits the first exposure signal through the first exposure control line TX1 to control at least two panchromatic pixels in the first diagonal direction for a first exposure time, and transmits the second exposure signal through the second exposure control line TX2 to control at least two panchromatic pixels in the second diagonal direction for a second exposure time. After all panchromatic pixels and all color pixels are exposed, the image sensor 10 directly outputs a panchromatic raw image and a color raw image without performing the process of fitting the pixel values of the plurality of panchromatic pixels to the pixel values of the color pixels, as shown in FIG. 12. Figure 36

[0240] Please refer to Figure 2 and Figure 17 As another example, the image sensor 10 can control the panchromatic pixels in the 2n-1th row and the 2n th row with the first exposure signal for a first exposure time, and control the color pixels in the 2n-1th row and the 2n th row with the second exposure signal for a second exposure time, where the first exposure time can be less than or equal to the second exposure time. Specifically, the first exposure control line TX1 in the image sensor 10 is connected to the control end TG of all panchromatic pixels in the 2n-1th row and the 2n th row, and the second exposure control line TX2 is connected to the control end TG of all color pixels in the 2n-1th row and the 2n th row. The vertical driving unit 12 transmits the first exposure signal through the first exposure control line TX1 to control the panchromatic pixels in the 2n-1th row and the 2n th row for a first exposure time, and transmits the second exposure signal through the second exposure control line TX2 to control the color pixels in the 2n-1th row and the 2n th row for a second exposure time. After all panchromatic pixels and all color pixels are exposed, the image sensor 10 directly outputs a panchromatic raw image and a color raw image without performing the process of fitting the pixel values of the plurality of panchromatic pixels to the pixel values of the color pixels, as shown in FIG. 13. Figure 36 ​As shown, the image sensor 10 does not perform the process of fitting the pixel values of the plurality of panchromatic pixels into the pixel values of the color pixels, but directly outputs a panchromatic raw image and a color raw image.

[0241] In some embodiments, the processing chip 20 can determine the relative relationship between the first exposure time and the second exposure time according to the ambient brightness. For example, the image sensor 10 can first control the panchromatic pixels to be exposed and output a panchromatic raw image, and the processing chip 20 analyzes the pixel values of the plurality of panchromatic pixels in the panchromatic raw image to determine the ambient brightness. When the ambient brightness is less than or equal to a brightness threshold, the image sensor 10 controls the panchromatic pixels to be exposed with a first exposure time equal to the second exposure time; when the ambient brightness is greater than the brightness threshold, the image sensor 10 controls the panchromatic pixels to be exposed with a first exposure time less than the second exposure time. When the ambient brightness is greater than the brightness threshold, the relative relationship between the first exposure time and the second exposure time can be determined according to the brightness difference between the ambient brightness and the brightness threshold, for example, the greater the brightness difference, the smaller the ratio of the first exposure time to the second exposure time. For example, when the brightness difference is within a first range [a, b), the ratio of the first exposure time to the second exposure time is V1:V2; when the brightness difference is within a second range [b, c), the ratio of the first exposure time to the second exposure time is V1:V3; when the brightness difference is greater than or equal to c, the ratio of the first exposure time to the second exposure time is V1:V4, where V1

[0242] Referring to Figure 40 In some embodiments, step 02 comprises:

[0243] 021: merging the pixel values of all the pixels in each subunit to obtain the pixel value of a monochromatic large pixel; and

[0244] 022: forming a color intermediate image according to the pixel values of the plurality of monochromatic large pixels, the color intermediate image having a first resolution.

[0245] Referring to Figure 33 In some embodiments, both step 021 and step 022 can be implemented by the processing chip 20. That is, the processing chip 20 can be used to merge the pixel values of all the pixels in each subunit to obtain the pixel value of a monochromatic large pixel, and form a color intermediate image according to the pixel values of the plurality of monochromatic large pixels, the color intermediate image having a first resolution. Wherein, the color intermediate image has a first resolution.

[0246] Specifically, as Figure 37As shown, for the single-color large pixel A, the processing chip 20 can add the pixel values of all the pixels in the sub-unit including the empty pixel N and the single-color pixel A, and take the added result as the pixel value of the single-color large pixel A corresponding to the sub-unit, wherein the pixel value of the empty pixel N can be regarded as zero, and the same below; the processing chip 20 can add the pixel values of all the pixels in the sub-unit including the empty pixel N and the single-color pixel B, and take the added result as the pixel value of the single-color large pixel B corresponding to the sub-unit; the processing chip 20 can add the pixel values of all the pixels in the sub-unit including the empty pixel N and the single-color pixel C, and take the added result as the pixel value of the single-color large pixel C corresponding to the sub-unit. In this way, the processing chip 20 can obtain the pixel values of the plurality of single large pixels A, the pixel values of the plurality of single-color large pixels B, and the pixel values of the plurality of single-color large pixels C. The processing chip 20 can further form a color intermediate image according to the pixel values of the plurality of single large pixels A, the pixel values of the plurality of single-color large pixels B, and the pixel values of the plurality of single-color large pixels C. As shown in the following figure, when the single color A is red R, the single color B is green G, and the single color C is blue Bu, the color intermediate image is an image arranged in a Bayer array. Of course, the manner in which the processing chip 20 obtains the color intermediate image is not limited to this. Figure 37

[0247] In some embodiments, please refer to Figure 33 and Figure 41 When the camera assembly 40 is in different modes, different modes correspond to different target images. The processing chip 20 can first determine which mode the camera assembly 40 is in, and then process the color intermediate image and / or the full-color intermediate image according to the mode of the camera assembly 40 to obtain the target image corresponding to the mode. The target image at least includes four types of target images: a first target image, a second target image, a third target image, and a fourth target image. The mode of the camera assembly 40 at least includes: (1) the mode is a preview mode, and the target image in the preview mode can be the first target image or the second target image; (2) the mode is an imaging mode, and the target image in the imaging mode can be the second target image, the third target image, or the fourth target image; (3) the mode is both the preview mode and the low-power mode, and the target image at this time is the first target image; (4) the mode is both the preview mode and the non-low-power mode, and the target image at this time is the second target image; (5) the mode is both the imaging mode and the low-power mode, and the target image at this time is the second target image or the third target image; (6) the mode is both the imaging mode and the non-low-power mode, and the target image at this time is the fourth target image.

[0248] Please refer to Figure 41 In one example, when the target image is the first target image, step 04 includes:

[0249] ​040: Interpolate each large monochrome pixel in the intermediate color image to obtain pixel values ​​for the other two colors besides that single color, and output them to obtain a first target image with a first resolution.

[0250] Please see Figure 33 Step 040 can be implemented by the processing chip 20. That is, the processing chip 20 can be used to interpolate each monochrome large pixel in the color intermediate image to obtain pixel values ​​of two other colors besides that monochrome color and output them to obtain a first target image with a first resolution.

[0251] Specifically, please combine Figure 42 Assuming that a large monochrome pixel A is a red pixel R, a large monochrome pixel B is a green pixel G, and a large monochrome pixel C is a blue pixel Bu, then the intermediate color image is an image arranged in a Bayer array. The processing chip 20 needs to perform demosaicing (i.e., interpolation) on the intermediate color image so that the pixel value of each large monochrome pixel simultaneously has the three components R, G, and B. For example, linear interpolation can be used to calculate the pixel values ​​of the two colors other than the single color of the large monochrome pixel. Taking the large monochrome pixel C... 2，2 (“C 2，2 For example, C represents the pixel in the second row and second column from the top left. The large monochrome pixel C... 2，2 Pixel value P(C) that has only the component of color C 2，2 Furthermore, it is also necessary to calculate the pixel value P(A) of color A at position C of the monochrome large pixel. 2，2 ) and the pixel value P(B) of color B 2，2 ), then P(A) 2，2 )=α1·P(A 3，1 )+α2·P(A 3，3 )+α3·P(A 1，3 )+α4·P(A 1，1 ), P(B 2，2 )=β1·P(B 1，2 )+β2·P(B 2，1 )+β3·P(B 2，3 )+β4·P(B 3，2 ), where α1~α4 and β1~β4 are interpolation coefficients, and α1+α2+α3+α4=1, β1+β2+β3+β4=1. The above P(A 2，2 ) and P(B 2，2 The calculation method for P(A) is only an example, P(A) 2，2 ) and P(B 2，2 It can also be calculated using other interpolation methods besides linear interpolation, and no restrictions are imposed here.

[0252] After the processing chip 20 calculates the pixel values of the three components of each monochromatic large pixel, the processing chip 20 can calculate the final pixel value of the monochromatic large pixel according to the three pixel values, i.e., A+B+C. It should be noted that A+B+C does not mean that the three pixels are directly added to obtain the final pixel value of the monochromatic large pixel, but only means that the monochromatic large pixel includes the three color components A, B, and C. The processing chip 20 can form a first target image according to the final pixel values of a plurality of monochromatic large pixels. Since the color intermediate image has the first resolution, the first target image is obtained by interpolation processing of the color intermediate image. The processing chip 20 does not perform interpolation processing on the color intermediate image, and thus the resolution of the first target image is also the first resolution. The processing algorithm of the processing chip 20 for processing the color intermediate image to obtain the first target image is relatively simple, and the processing speed is relatively fast. The camera assembly 40 uses the first target image as a preview image when the mode is the preview mode or the low-power-consumption mode, which can meet the demand of the preview mode on the output speed and save the power consumption of the camera assembly 40.

[0253] Please refer to Figure 41 In another example, when the target image is the second target image, step 03 includes:

[0254] 031: Process the panchromatic raw image, take all the pixels of each subunit as a panchromatic large pixel, and output the pixel value of the panchromatic large pixel to obtain a panchromatic intermediate image having the first resolution;

[0255] Step 04 includes:

[0256] 041: Separate the colors and luminance of the color intermediate image to obtain a color-luminance separation image having the first resolution;

[0257] 042: Fuse the luminance of the panchromatic intermediate image and the luminance of the color-luminance separation image to obtain a luminance-corrected color image having the first resolution; and

[0258] 043: Perform interpolation processing on each monochromatic large pixel in the luminance-corrected color image to obtain the pixel values of the other two colors except the monochromatic color and output to obtain a second target image having the first resolution.

[0259] Please refer to Figure 33Steps 031, 041, 042, and 043 can all be implemented by the processing chip 20. That is, the processing chip 20 can process the panchromatic original image, taking all pixels of each sub-unit as panchromatic large pixels and outputting the pixel values ​​of the panchromatic large pixels to obtain a panchromatic intermediate image, which has a first resolution. The processing chip 20 can also be used to separate the color and brightness of the color intermediate image to obtain a color-brightness separation image with a first resolution, fuse the brightness of the panchromatic intermediate image and the brightness of the color-brightness separation image to obtain a brightness-corrected color image with a first resolution, and perform interpolation processing on each monochrome large pixel in the brightness-corrected color image to obtain pixel values ​​of the other two colors besides that single color and output them to obtain a second target image with a first resolution.

[0260] Specifically, the panchromatic original image can be obtained through... Figure 43 The method shown transforms it into a panchromatic intermediate image. For example... Figure 43 As shown, the panchromatic original image comprises multiple sub-units, each sub-unit including two empty pixels N and two panchromatic pixels W. The processing chip 20 can use all pixels in each sub-unit including empty pixels N and panchromatic pixels W as the corresponding panchromatic large pixel W. Thus, the processing chip 20 can form a panchromatic intermediate image based on the multiple panchromatic large pixels W. If the panchromatic original image including multiple empty pixels N is considered an image with a second resolution, then according to... Figure 43 The panchromatic intermediate image obtained in the manner shown is an image with a first resolution, where the first resolution is smaller than the second resolution.

[0261] As an example, the processing chip 20 can use all pixels of each sub-unit in the panchromatic original image as the panchromatic large pixel W corresponding to that sub-unit in the following manner: The processing chip 20 first merges the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of the panchromatic large pixel W, and then forms a panchromatic intermediate image based on the pixel values ​​of multiple panchromatic large pixels W. Specifically, for each panchromatic large pixel, the processing chip 20 can add all pixel values ​​of the sub-unit including the empty pixel N and the panchromatic pixel W, and use the result of the addition as the pixel value of the panchromatic large pixel W corresponding to that sub-unit, where the pixel value of the empty pixel N can be regarded as zero. In this way, the processing chip 20 can obtain the pixel values ​​of multiple panchromatic large pixels W.

[0262] After the processing chip 20 obtains the pancolor intermediate image and the color intermediate image, it can perform a fusion process on the pancolor intermediate image and the color intermediate image to obtain the second target image.

[0263] For example, such as Figure 43 As shown, the processing chip 20 first separates the color and brightness of the intermediate color image to obtain a color-brightness separated image. Figure 43L in the color and brightness separation image represents luminance, and CLR represents color. Specifically, assuming that the single-color pixel A is a red pixel R, the single-color pixel B is a green pixel G, and the single-color pixel C is a blue pixel Bu, then: (1) the processing chip 20 can convert the color intermediate image in the RGB space into a color and brightness separation image in the YCrCb space, at which time Y in YCrCb is the luminance L in the color and brightness separation image, and Cr and Cb in YCrCb are the color CLR in the color and brightness separation image; (2) the processing chip 20 can also convert the color intermediate image in RGB into a color and brightness separation image in Lab, at which time L in Lab is the luminance L in the color and brightness separation image, and a and b in Lab are the color CLR in the color and brightness separation image. It should be noted that, Figure 43 L+CLR in the color and brightness separation image shown does not mean that the pixel value of each pixel is added by L and CLR, but only means that the pixel value of each pixel is composed of L and CLR.

[0264] Subsequently, the processing chip 20 fuses the luminance of the color and brightness separation image and the luminance of the full-color intermediate image. For example, the pixel value of each full-color pixel W in the full-color intermediate image is the luminance value of each full-color pixel, and the processing chip 20 can add L of each pixel in the color and brightness separation image and W of the full-color pixel at the corresponding position in the full-color intermediate image, so as to obtain the pixel value after luminance correction. The processing chip 20 forms a color and brightness separation image after luminance correction according to a plurality of pixel values after luminance correction, and converts the color and brightness separation image after luminance correction into a luminance-corrected color image by using color space conversion.

[0265] When the single-color large pixel A is a red pixel R, the single-color large pixel B is a green pixel G, and the single-color large pixel C is a blue pixel Bu, the luminance-corrected color image is a Bayer array arranged image, and the processing chip 20 needs to perform interpolation processing on the luminance-corrected color image, so that each single-color large pixel after luminance correction has R, G, and B three components. The processing chip 20 can perform interpolation processing on the luminance-corrected color image to obtain a second target image. For example, a linear interpolation method can be used to obtain the second target image, and the linear interpolation process is similar to the interpolation process in the foregoing step 040, which will not be described here.

[0266] Since the luminance-corrected color image has the first resolution, the second target image is obtained by interpolation processing on the luminance-corrected color image, and the processing chip 20 does not perform interpolation processing on the luminance-corrected color image, the resolution of the second target image is also the first resolution. Since the second target image is obtained by fusing the luminance of the color intermediate image and the luminance of the panchromatic intermediate image, the second target image has a better imaging effect. When the mode is the preview mode and the non-low-power mode, the second target image is used as the preview image, which can improve the preview effect of the preview image. When the mode is the imaging mode and the low-power mode, the second target image is used as the image provided to the user. Since the second target image is obtained without interpolation processing, the power consumption of the camera assembly 40 can be reduced to some extent, which can meet the use requirement in the low-power mode. Meanwhile, the luminance of the second target image is relatively high, which can meet the luminance requirement of the user on the target image.

[0267] Please refer to Figure 41 In yet another example, when the target image is the third target image, step 04 includes:

[0268] 044: performing interpolation processing on the color intermediate image to obtain a color interpolated image having a second resolution, the corresponding sub-units in the color interpolated image being arranged in a Bayer array, the second resolution being greater than the first resolution; and

[0269] 045: performing interpolation processing on all single-color pixels in the color interpolated image to obtain pixel values of two other colors except the single color and outputting to obtain the third target image having the second resolution.

[0270] Please refer to Figure 33 Step 044 and step 045 can be implemented by the processing chip 20. That is, the processing chip 20 can be used to perform interpolation processing on the color intermediate image to obtain a color interpolated image having a second resolution, the corresponding sub-units in the color interpolated image being arranged in a Bayer array, the second resolution being greater than the first resolution. The processing chip 20 can also be used to perform interpolation processing on all single-color pixels in the color interpolated image to obtain pixel values of two other colors except the single color and outputting to obtain the third target image having the second resolution.

[0271] Specifically, please refer to Figure 44, the processing chip 20 splits each monochrome large pixel in the color intermediate image into four color pixels, and the four color pixels form a subunit in the color interpolated image, each subunit includes three color pixels, one color pixel A, two color pixel B, and one color pixel C. When the color pixel A is a red pixel R, the color pixel B is a green pixel G, and the color pixel C is a blue pixel Bu, the multiple color pixels in each subunit are arranged in a Bayer array. Thus, the color interpolated image including multiple subunits is a Bayer array image. The processing chip 20 can perform interpolation on the color interpolated image to obtain a third target image. For example, a linear interpolation method can be used to obtain the second target image, and the linear interpolation process is similar to the interpolation process in step 040, which will not be described here. The third target image is an image obtained by interpolation, and the resolution (i.e., the second resolution) of the third target image is larger than the resolution (i.e., the first resolution) of the color intermediate image. When the mode is the preview mode and the non-low-power mode, the third target image is used as a preview image, and a clearer preview image can be obtained. When the mode is the imaging mode and the low-power mode, the third target image is used as an image provided to the user. Since the third target formation process does not need to perform luminance fusion with the panchromatic intermediate image, the power consumption of the camera assembly 40 can be reduced to a certain extent, while the clarity of the captured image can be met.

[0272] Please refer to Figure 41 In yet another example, when the target image is a fourth target image, step 03 includes:

[0273] 032: performing interpolation on the panchromatic raw image to obtain pixel values of all pixels in each subunit to obtain a panchromatic intermediate image having a second resolution;

[0274] Step 04 includes:

[0275] 046: performing interpolation on the color intermediate image to obtain a color interpolated image having a second resolution, and corresponding subunits in the color interpolated image are arranged in a Bayer array, and the second resolution is greater than the first resolution;

[0276] 047: separating the color and luminance of the color interpolated image to obtain a color-luminance-separated image having a second resolution;

[0277] 048: fusing the luminance of the panchromatic interpolated image and the luminance of the color-luminance-separated image to obtain a luminance-corrected color image having a second resolution; and

[0278] 049: performing interpolation on all single-color pixels in the luminance-corrected color image to obtain pixel values of two other colors except the single color and output to obtain a fourth target image having a second resolution.

[0279] Please see Figure 33 Steps 032, 046, 047, 048, and 049 can all be implemented by the processing chip 20. That is, the processing chip 20 can be used to interpolate the panchromatic original image, obtaining the pixel values ​​of all pixels in each sub-unit to obtain a panchromatic intermediate image with a second resolution. The processing chip 20 can also be used to interpolate the color intermediate image to obtain a color interpolated image with a second resolution, where the corresponding sub-units in the color interpolated image are arranged in a Bayer array, and the second resolution is greater than the first resolution. The processing chip 20 can also be used to separate the color and brightness of the color interpolated image to obtain a color-brightness separated image with a second resolution, fuse the brightness of the panchromatic interpolated image and the brightness of the color-brightness separated image to obtain a brightness-corrected color image with a second resolution, and perform interpolation processing on all single-color pixels in the brightness-corrected color image to obtain pixel values ​​of the other two colors besides the single color and output them to obtain a fourth target image with a second resolution.

[0280] Specifically, the processing chip 20 first performs interpolation processing on the first-resolution panchromatic original image to obtain the second-resolution panchromatic intermediate image. Please refer to... Figure 46 The original panchromatic image comprises multiple sub-units, each including two empty pixels N and two panchromatic pixels W. The processing chip 20 needs to replace each empty pixel N in each sub-unit with a panchromatic pixel W and calculate the pixel value of each panchromatic pixel W located at the position of the empty pixel N after replacement. For each empty pixel N, the processing chip 20 replaces the empty pixel N with a panchromatic pixel W and determines the pixel value of the replaced panchromatic pixel W based on the pixel values ​​of the other panchromatic pixels W adjacent to the replaced panchromatic pixel W. Figure 46 The empty pixel N in the panchromatic original image shown 1，8 ("empty pixel N") 1，8 Taking the empty pixel N (counting from the top left, first row, eighth column, the same below) as an example, empty pixel N 1，8 Replace with full-color pixel W 1，8 , with full-color pixels W 1，8 The adjacent pixels are the panchromatic pixels W in the original panchromatic image. 1，7 And full-color pixel W 2，8 As an example, the full-color pixel W can be... 1，7 Pixel values ​​and full-color pixels W 2，8 The average pixel value is used as the panchromatic pixel W. 1，8 The pixel value. Figure 46 The empty pixel N in the panchromatic original image shown 2，3 For example, empty pixel N 2，3 Replace with full-color pixel W 2，3 , with full-color pixels W 2，3The adjacent panchromatic pixel is a panchromatic pixel W in the panchromatic original image 1，3 , the panchromatic pixel W 2，2 , the panchromatic pixel W 2，4 , and the panchromatic pixel W 3，3 As an example, the processing chip 20 takes the mean value of the pixel value of the panchromatic pixel W 1，3 , the pixel value of the panchromatic pixel W 2，2 , the pixel value of the panchromatic pixel W 2，4 , and the pixel value of the panchromatic pixel W 3，3 as the pixel value of the replaced panchromatic pixel W 2，3 .

[0281] After the processing chip 20 obtains the panchromatic intermediate image and the color intermediate image, the processing chip 20 can perform fusion processing on the panchromatic intermediate image and the color intermediate image to obtain a fourth target image.

[0282] First, the processing chip 20 can perform interpolation processing on the color intermediate image of the first resolution to obtain a color interpolation image of the second resolution, as shown in Figure 45 The specific interpolation manner is similar to that in step 045, and is not described herein.

[0283] Subsequently, as shown in Figure 45 , the processing chip 20 can separate the color and luminance of the color interpolation image to obtain a color-luminance separation image, Figure 45 wherein L in the color-luminance separation image represents luminance, and CLR represents color. Specifically, assuming that the single-color pixel A is a red pixel R, the single-color pixel B is a green pixel G, and the single-color pixel C is a blue pixel Bu, (1) the processing chip 20 can convert the color interpolation image in the RGB space into a color-luminance separation image in the YCrCb space, at which time Y in YCrCb is the luminance L in the color-luminance separation image, and Cr and Cb in YCrCb are the color CLR in the color-luminance separation image; (2) the processing chip 20 can also convert the color interpolation image in RGB into a color-luminance separation image in Lab, at which time L in Lab is the luminance L in the color-luminance separation image, and a and b in Lab are the color CLR in the color-luminance separation image. It should be noted that Figure 45 L+CLR in the color-luminance separation image shown in

[0284] Subsequently, as shown in Figure 46As shown, the processing chip 20 can fuse the luminance of the color and brightness separation image and the luminance of the full-color intermediate image. For example, the pixel value of each full-color pixel W in the full-color intermediate image is the luminance value of each full-color pixel, and the processing chip 20 can add the L of each pixel in the color and brightness separation image to the W of the full-color pixel at the corresponding position in the full-color intermediate image, so as to obtain the pixel value after luminance correction. The processing chip 20 forms a color and brightness separation image after luminance correction according to a plurality of pixel values after luminance correction, and converts the color and brightness separation image after luminance correction into a luminance-corrected color image, which has a second resolution.

[0285] When the color pixel A is a red pixel R, the color pixel B is a green pixel G, and the color pixel C is a blue pixel Bu, the luminance-corrected color image is a Bayer array-arranged image, and the processing chip 20 needs to perform interpolation processing on the luminance-corrected color image so that each color pixel after luminance correction has R, G, and B components. The processing chip 20 can perform interpolation processing on the luminance-corrected color image to obtain a fourth target image. For example, a linear interpolation method can be used to obtain the fourth target image, and the linear interpolation process is similar to the interpolation process in the foregoing step 40, which will not be described here.

[0286] Since the fourth target image is obtained by fusing the luminance of the color intermediate image and the luminance of the full-color intermediate image, and the fourth target image has a large resolution, the fourth target image has better luminance and clarity. When the mode is both the imaging mode and the non-low-power mode, the fourth target image can be used as the image provided to the user, which can meet the user's requirements for the quality of the captured image.

[0287] In some embodiments, the image acquisition method can further include obtaining the ambient brightness. This step can be implemented by the processing chip 20, and the specific implementation manner is as described above, which will not be described here. When the ambient brightness is greater than the brightness threshold, the first target image or the third target image can be used as the target image; when the ambient brightness is less than or equal to the brightness threshold, the second target image or the fourth target image can be used as the target image. It can be understood that when the ambient brightness is high, the luminance of the first target image and the second target image obtained from the color intermediate image is sufficient to meet the user's requirements for the luminance of the target image, and thus it is not necessary to fuse the luminance of the full-color intermediate image to improve the luminance of the target image, which can not only reduce the calculation amount of the processing chip 20, but also reduce the power consumption of the camera assembly 40. When the ambient brightness is low, the luminance of the first target image and the second target image obtained from the color intermediate image can not meet the user's requirements for the luminance of the target image, and thus the second target image or the fourth target image obtained by fusing the luminance of the full-color intermediate image can be used as the target image to improve the luminance of the target image.

[0288] Please refer to Figure 47 This application also provides a mobile terminal 90. The mobile terminal 90 may be a mobile phone, tablet computer, laptop computer, smart wearable device (such as smartwatch, smart bracelet, smart glasses, smart helmet, etc.), head-mounted display device, virtual reality device, etc., and is not limited thereto.

[0289] The mobile terminal 90 includes an image sensor 50, a processor 60, a memory 70, and a housing 80, all of which are installed within the housing 80. The image sensor 50 is connected to the processor 60, and the image sensor 50 can be the image sensor 10 described in any of the above embodiments. Figure 33 (As shown). The processor 60 can perform operations with the camera assembly 40 ( Figure 33 The processor 60 has the same function as the processing chip 20 shown in the diagram; in other words, the processor 60 can perform the functions of the processing chip 20 described in any of the above embodiments. The memory 70 is connected to the processor 60 and can store data processed by the processor 60, such as target images. The processor 60 can be mounted on the same substrate as the image sensor 50, in which case the image sensor 50 and the processor 60 can be considered as a single camera assembly 40. Alternatively, the processor 60 can be mounted on a different substrate than the image sensor 50.

[0290] The mobile terminal 90 of this application has an image sensor 50 that can directly output pancolor original images and color original images. The subsequent processing of the pancolor original images and color original images is performed by the processor 60. The image sensor 50 does not need to perform the operation of fitting the pixel values ​​of the pancolor pixels W to the pixel values ​​of the color pixels. The computational load of the image sensor 50 is reduced, and no new hardware needs to be added to the image sensor 50 to support the image sensor 50 to perform image processing, which simplifies the design of the image sensor 50.

[0291] In the description of this specification, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0292] Any processes or methods described in the flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions (or steps) of the process, and / or that the various processes described herein can be understood as representing executable instructions, code segments, or portions of code which include one or more steps for implementing the functions (or steps) of the processes, and that the various processes described herein can be implemented with or without the use of hardware, software, firmware, or any combination thereof.

[0293] Although the embodiments of the present application have been shown and described above, it should be understood by those skilled in the art that the above-described embodiments are exemplary only, and should not be understood as limiting the present application, and that changes, modifications, substitutions, and variations of the above-described embodiments can be made by those skilled in the art within the scope of the present application.

Claims

1. An image sensor, characterized in that, include: Full-color pixels; and A color pixel having a narrower spectral response than the panchromatic pixel having a larger full-well capacity than the color pixel; Each pixel includes a photoelectric conversion element, and each photoelectric conversion element includes a substrate and an n-potential well layer formed in the substrate. Along the light-receiving direction of the image sensor, the cross-sectional size of the n-potential well layer of the pancolor pixel gradually increases, and the cross-sectional size of the n-potential well layer of the color pixel gradually decreases. The smallest cross-sectional size of the n-potential well layer of the pancolor pixel is larger than the largest cross-sectional size of the n-potential well layer of the color pixel.

2. The image sensor according to claim 1, characterized in that, The depth of the photoelectric conversion element of the full-color pixel is equal to the depth of the photoelectric conversion element of the color pixel.

3. The image sensor according to claim 1, characterized in that, Each pixel includes a microlens, a filter, and an isolation layer, and the microlens, the filter, the isolation layer, and the photoelectric conversion element are arranged sequentially along the light-receiving direction of the image sensor.

4. The image sensor according to claim 3, characterized in that, Along the light-receiving direction, the cross-sectional size of the isolation layer of the panchromatic pixel gradually increases, while the cross-sectional size of the isolation layer of the color pixel gradually decreases.

5. The image sensor according to claim 1, characterized in that, The panchromatic pixels and the colored pixels form a two-dimensional pixel array. The two-dimensional pixel array includes a minimum repeating unit. In the minimum repeating unit, the panchromatic pixels are arranged in a first diagonal direction, and the colored pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. The first exposure time of at least two adjacent panchromatic pixels in the first diagonal direction is controlled by a first exposure signal, and the second exposure time of at least two adjacent color pixels in the second diagonal direction is controlled by a second exposure signal.

6. The image sensor according to claim 5, characterized in that, The first exposure time is shorter than the second exposure time.

7. The image sensor according to claim 5, characterized in that, The image sensor also includes: The first exposure control line is electrically connected to the control terminals of the exposure control circuits in at least two of the panchromatic pixels adjacent to the first diagonal direction; and The second exposure control line is electrically connected to the control terminals of the exposure control circuits in at least two color pixels adjacent to the second diagonal direction; The first exposure signal is transmitted via the first exposure control line, and the second exposure signal is transmitted via the second exposure control line.

8. The image sensor according to claim 7, characterized in that, The first exposure control line is "W" shaped and is electrically connected to the control terminal of the exposure control circuit in the two adjacent rows of full-color pixels; The second exposure control line is "W" shaped and is electrically connected to the control terminal of the exposure control circuit in the adjacent two rows of color pixels.

9. The image sensor according to claim 7 or 8, characterized in that, The exposure control circuit is electrically connected to the photoelectric conversion element and is used to transfer the potential accumulated in the photoelectric conversion element after being exposed to light.

10. The image sensor according to claim 7, characterized in that, The exposure control circuit is a transfer transistor, and the control terminal of the exposure control circuit is the gate of the transfer transistor.

11. The image sensor according to claim 5, characterized in that, The smallest repeating unit is 4 rows and 4 columns, consisting of 16 pixels, arranged as follows: WAWB AWBW WBWC BWCW Wherein, W represents the full-color pixel; A represents the first color pixel among the plurality of color pixels; B represents the second color pixel among the plurality of color pixels; C represents the third color pixel among the plurality of color pixels.

12. The image sensor according to claim 5, characterized in that, The smallest repeating unit is 4 rows and 4 columns, consisting of 16 pixels, arranged as follows: AWBW WAWB BWCW WBWC Wherein, W represents the full-color pixel; A represents the first color pixel among the plurality of color pixels; B represents the second color pixel among the plurality of color pixels; C represents the third color pixel among the plurality of color pixels.

13. The image sensor according to claim 11 or 12, characterized in that, The first color pixel A is the red pixel R; The second color pixel B is the green pixel G; The third color pixel C is the blue pixel Bu.

14. The image sensor according to claim 11 or 12, characterized in that, The first color pixel A is the red pixel R; The second color pixel B is the yellow pixel Y; The third color pixel C is the blue pixel Bu.

15. The image sensor according to claim 11 or 12, characterized in that, The first color pixel A is the magenta pixel M; The second color pixel B is the cyan pixel Cy; The third color pixel C is the yellow pixel Y.

16. The image sensor according to any one of claims 5, 11, and 12, characterized in that, The response band of the full-color pixel is the visible light band.

17. The image sensor according to any one of claims 5, 11, and 12, characterized in that, The response bands of the full-color pixels are the visible light band and the near-infrared band, which match the response bands of the photoelectric conversion elements in the image sensor.

18. A camera assembly, characterized in that, include: The image sensor according to any one of claims 1-4.

19. A camera assembly, characterized in that, include: The image sensor according to any one of claims 5-17.

20. The camera assembly according to claim 19, characterized in that, Each of the minimum repeating units comprises multiple sub-units, each sub-unit including multiple single-color pixels and multiple pan-color pixels; the image sensor is used for exposure to acquire pan-color raw images and color raw images; the camera assembly further includes a processing chip, the processing chip being used for: The original color image is processed to treat all pixels of each sub-unit as a single-color large pixel corresponding to a single color in that sub-unit, and the pixel value of the single-color large pixel is output to obtain a color intermediate image. Process the panchromatic original image to obtain a panchromatic intermediate image; and Process the color intermediate image and / or the panchromatic intermediate image to obtain the target image.

21. The camera assembly according to claim 20, characterized in that, All the panchromatic pixels and all the color pixels in the image sensor are exposed simultaneously; The image sensor outputs the pixel values ​​of all the panchromatic pixels to obtain the panchromatic original image, and outputs the pixel values ​​of all the color pixels to obtain the color original image.

22. The camera assembly according to claim 20, characterized in that, All the panchromatic pixels and all the color pixels in the image sensor are exposed in a time-division manner; The image sensor outputs the pixel values ​​of all the panchromatic pixels to obtain the panchromatic original image, and outputs the pixel values ​​of all the color pixels to obtain the color original image.

23. The camera assembly according to claim 20, characterized in that, The processing chip is also used to acquire ambient brightness; When the ambient brightness is greater than the brightness threshold, the first exposure time is less than the second exposure time.

24. The camera assembly according to claim 20, characterized in that, The processing chip is also used for: Merge the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of the monochrome large pixel; and The intermediate color image is formed based on the pixel values ​​of multiple monochrome large pixels, and the intermediate color image has a first resolution.

25. The camera assembly according to claim 24, characterized in that, The processing chip is also used for: Interpolation is performed on each of the monochrome large pixels in the color intermediate image to obtain pixel values ​​for the other two colors besides the monochrome color, and the results are output to obtain a first target image with the first resolution.

26. The camera assembly according to claim 24, characterized in that, The processing chip is also used for: The panchromatic original image is processed by taking all pixels of each sub-unit as a panchromatic large pixel and outputting the pixel value of the panchromatic large pixel to obtain a panchromatic intermediate image, wherein the panchromatic intermediate image has the first resolution; The color and brightness of the intermediate color image are separated to obtain a color-brightness separated image with the first resolution; The brightness of the panchromatic intermediate image and the brightness of the color-brightness separated image are fused to obtain a brightness-corrected color image with the first resolution; and Interpolation is performed on each of the large monochrome pixels in the brightness-corrected color image to obtain pixel values ​​for the other two colors besides the single color, and the results are output to obtain a second target image with the first resolution.

27. The camera assembly according to claim 26, characterized in that, The processing chip is also used for: Merge the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of the full-color large pixel; and The panchromatic intermediate image is formed based on the pixel values ​​of multiple panchromatic large pixels.

28. The camera assembly according to claim 24, characterized in that, The processing chip is also used for: The intermediate color image is interpolated to obtain an interpolated color image with a second resolution, wherein the corresponding sub-units in the interpolated color image are arranged in a Bayer array, and the second resolution is greater than the first resolution; and Interpolation is performed on all single-color pixels in the color interpolated image to obtain pixel values ​​for two other colors besides the single color, and the results are output to obtain a third target image with the second resolution.

29. The camera assembly according to claim 24, characterized in that, The processing chip is also used for: The panchromatic original image is interpolated to obtain the pixel values ​​of all pixels in each sub-unit to obtain a panchromatic intermediate image with a second resolution; The intermediate color image is interpolated to obtain an interpolated color image with a second resolution, wherein the corresponding sub-units in the interpolated color image are arranged in a Bayer array, and the second resolution is greater than the first resolution; The color and brightness of the color interpolated image are separated to obtain a color-brightness separated image with the second resolution; The brightness of the full-color interpolated image and the brightness of the color-brightness separated image are fused to obtain a brightness-corrected color image with the second resolution; and Interpolation is performed on all single-color pixels in the brightness-corrected color image to obtain pixel values ​​for the other two colors besides the single color, and the results are output to obtain a fourth target image with the second resolution.

30. The camera assembly according to any one of claims 25-29, characterized in that, The image sensor is applied to the camera assembly, and different modes correspond to different target images when the camera assembly is in different modes.

31. The camera assembly according to claim 30, characterized in that, When the mode is preview mode, the target image is either a first target image or a second target image; When the mode is imaging mode, the target image is a second target image, a third target image, or a fourth target image.

32. The camera assembly according to claim 30, characterized in that, When the mode is both preview mode and low power mode, the target image is the first target image; When the mode is both the preview mode and the non-low power mode, the target image is the second target image; When the mode is both the imaging mode and the low-power mode, the target image is either the second target image or the third target image; When the mode is both the imaging mode and the non-low power mode, the target image is the fourth target image.

33. The camera assembly according to any one of claims 25-29, characterized in that, The processing chip is also used to acquire ambient brightness; When the ambient brightness is greater than a brightness threshold, the target image is either a first target image or a third target image; When the ambient brightness is less than the brightness threshold, the target image is either a second target image or a fourth target image.

34. A mobile terminal, characterized in that, include: chassis; and The image sensor according to any one of claims 1-4, wherein the image sensor is mounted inside the housing.

35. A mobile terminal, characterized in that, include: chassis; and The image sensor according to any one of claims 5-17, wherein the image sensor is mounted inside the housing.

36. The mobile terminal according to claim 35, characterized in that, Each of the minimum repeating units comprises multiple sub-units, each sub-unit including multiple single-color pixels and multiple pan-color pixels; the image sensor is used for exposure to acquire pan-color raw images and color raw images; the mobile terminal further includes a processor, the processor being used for: The original color image is processed to treat all pixels of each sub-unit as a single-color large pixel corresponding to a single color in that sub-unit, and the pixel value of the single-color large pixel is output to obtain a color intermediate image. Process the panchromatic original image to obtain a panchromatic intermediate image; and Process the color intermediate image and / or the panchromatic intermediate image to obtain the target image.

37. The mobile terminal according to claim 36, characterized in that, All the panchromatic pixels and all the color pixels in the image sensor are exposed simultaneously; The image sensor outputs the pixel values ​​of all the panchromatic pixels to obtain the panchromatic original image, and outputs the pixel values ​​of all the color pixels to obtain the color original image.

38. The mobile terminal according to claim 36, characterized in that, All the panchromatic pixels and all the color pixels in the image sensor are exposed in a time-division manner; The image sensor outputs the pixel values ​​of all the panchromatic pixels to obtain the panchromatic original image, and outputs the pixel values ​​of all the color pixels to obtain the color original image.

39. The mobile terminal according to claim 36, characterized in that, The processor is also used to acquire ambient brightness; When the ambient brightness is greater than the brightness threshold, the first exposure time is less than the second exposure time.

40. The mobile terminal according to claim 36, characterized in that, The processor is also used for: Merge the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of the monochrome large pixel; and The intermediate color image is formed based on the pixel values ​​of multiple monochrome large pixels, and the intermediate color image has a first resolution.

41. The mobile terminal according to claim 40, characterized in that, The processor is also used for: Interpolation is performed on each of the monochrome large pixels in the color intermediate image to obtain pixel values ​​for the other two colors besides the monochrome color, and the results are output to obtain a first target image with the first resolution.

42. The mobile terminal according to claim 40, characterized in that, The processor is also used for: The panchromatic original image is processed by taking all pixels of each sub-unit as a panchromatic large pixel and outputting the pixel value of the panchromatic large pixel to obtain a panchromatic intermediate image, wherein the panchromatic intermediate image has the first resolution; The color and brightness of the intermediate color image are separated to obtain a color-brightness separated image with the first resolution; The brightness of the panchromatic intermediate image and the brightness of the color-brightness separated image are fused to obtain a brightness-corrected color image with the first resolution; and Interpolation is performed on each of the large monochrome pixels in the brightness-corrected color image to obtain pixel values ​​for the other two colors besides the single color, and the results are output to obtain a second target image with the first resolution.

43. The mobile terminal according to claim 42, characterized in that, The processor is also used for: Merge the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of the full-color large pixel; and The panchromatic intermediate image is formed based on the pixel values ​​of multiple panchromatic large pixels.

44. The mobile terminal according to claim 40, characterized in that, The processor is also used for: The intermediate color image is interpolated to obtain an interpolated color image with a second resolution, wherein the corresponding sub-units in the interpolated color image are arranged in a Bayer array, and the second resolution is greater than the first resolution; and Interpolation is performed on all single-color pixels in the color interpolated image to obtain pixel values ​​for two other colors besides the single color, and the results are output to obtain a third target image with the second resolution.

45. The mobile terminal according to claim 40, characterized in that, The processor is also used for: The panchromatic original image is interpolated to obtain the pixel values ​​of all pixels in each sub-unit to obtain a panchromatic intermediate image with a second resolution; The intermediate color image is interpolated to obtain an interpolated color image with a second resolution, wherein the corresponding sub-units in the interpolated color image are arranged in a Bayer array, and the second resolution is greater than the first resolution; The color and brightness of the color interpolated image are separated to obtain a color-brightness separated image with the second resolution; The brightness of the full-color interpolated image and the brightness of the color-brightness separated image are fused to obtain a brightness-corrected color image with the second resolution; and Interpolation is performed on all single-color pixels in the brightness-corrected color image to obtain pixel values ​​for the other two colors besides the single color, and the results are output to obtain a fourth target image with the second resolution.

46. ​​The mobile terminal according to any one of claims 41-45, characterized in that, The image sensor is applied to the mobile terminal, and different target images correspond to different modes when the mobile terminal is in different modes.

47. The mobile terminal according to claim 46, characterized in that, When the mode is preview mode, the target image is either a first target image or a second target image; When the mode is imaging mode, the target image is a second target image, a third target image, or a fourth target image.

48. The mobile terminal according to claim 46, characterized in that, When the mode is both preview mode and low power mode, the target image is the first target image; When the mode is both the preview mode and the non-low power mode, the target image is the second target image; When the mode is both the imaging mode and the low-power mode, the target image is either the second target image or the third target image; When the mode is both the imaging mode and the non-low power mode, the target image is the fourth target image.

49. The mobile terminal according to any one of claims 41-45, characterized in that, The processor is also used to acquire ambient brightness; When the ambient brightness is greater than a brightness threshold, the target image is either a first target image or a third target image; When the ambient brightness is less than the brightness threshold, the target image is either a second target image or a fourth target image.

Images