Image sensor, camera assembly, and mobile terminal

Abstract

An image sensor (10), a camera assembly (40), and a mobile terminal (90) are disclosed. The image sensor (10) includes a plurality of pixels. At least a portion of the plurality of pixels includes an isolation layer (1183), a condenser lens (1186), and a photoelectric conversion element (117). The condenser lens (1186) is disposed within the isolation layer (1183). The photoelectric conversion element (117) is used to receive light passing through the condenser lens (1186).

Description

Technical Field

[0001] This application relates to the field of imaging technology, and in particular to an image sensor, camera assembly, and mobile terminal. Background Technology

[0002] Mobile devices such as smartphones are often equipped with cameras to enable photo taking. Cameras contain image sensors. To capture color images, image sensors typically include multiple pixels arranged in a two-dimensional array. When an image sensor is operating, optical crosstalk can occur between adjacent pixels. Summary of the Invention

[0003] This application provides an image sensor, a camera assembly, and a mobile terminal.

[0004] One aspect of this application provides an image sensor. The image sensor includes a plurality of pixels. At least a portion of the plurality of pixels includes an isolation layer, a condenser lens, and a photoelectric conversion element. The condenser lens is disposed within the isolation layer. The photoelectric conversion element is used to receive light passing through the condenser lens.

[0005] In another aspect, this application also provides a camera assembly. The camera assembly includes an image sensor. The image sensor includes a plurality of pixels. At least a portion of the plurality of pixels includes an insulating layer, a condenser lens, and a photoelectric conversion element. The condenser lens is disposed within the insulating layer. The photoelectric conversion element is used to receive light passing through the condenser lens.

[0006] In another aspect, this application also provides a mobile terminal. The mobile terminal includes a housing and an image sensor, the image sensor being mounted within the housing. The image sensor includes a plurality of pixels. At least a portion of the plurality of pixels includes an insulating layer, a condenser lens, and a photoelectric conversion element. The condenser lens is disposed within the insulating layer. The photoelectric conversion element is used to receive light passing through the condenser lens.

[0007] Additional aspects and advantages of the embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0008] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0009] Figure 1 This is a schematic diagram of the image sensor in the embodiments of this application;

[0010] Figure 2 This is a schematic diagram of a pixel circuit according to an embodiment of this application;

[0011] Figure 3 This is a schematic diagram of the exposure saturation time for different color channels;

[0012] Figure 4A This is a partial cross-sectional schematic diagram of a pixel array according to an embodiment of this application;

[0013] Figure 4B yes Figure 4A A schematic diagram of the arrangement of photoelectric conversion elements (or filters) in a pixel array;

[0014] Figure 5A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0015] Figure 5B yes Figure 5A A schematic diagram of the arrangement of photoelectric conversion elements (or filters) in a pixel array;

[0016] Figure 5C yes Figure 5A Another schematic diagram of the arrangement of photoelectric conversion elements (or filters) in a pixel array;

[0017] Figure 6A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0018] Figure 6B yes Figure 6A A schematic diagram of the filter arrangement in the pixel array;

[0019] Figure 6C yes Figure 6A A schematic diagram of the arrangement of photoelectric conversion elements in a pixel array;

[0020] Figure 7A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0021] Figure 7B yes Figure 7A A schematic diagram of the filter arrangement in the pixel array;

[0022] Figure 7C yes Figure 7A A schematic diagram of the arrangement of photoelectric conversion elements in a pixel array;

[0023] Figure 8A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0024] Figure 8B yes Figure 8A A schematic diagram of the filter arrangement in the pixel array;

[0025] Figure 8Cyes Figure 8A A schematic diagram of the arrangement of photoelectric conversion elements in a pixel array;

[0026] Figure 9 This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0027] Figure 10A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0028] Figure 10B yes Figure 10A A schematic diagram of the arrangement of photoelectric conversion elements (or filters) in a pixel array;

[0029] Figure 11A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0030] Figure 11B yes Figure 11A A schematic diagram of the arrangement of photoelectric conversion elements (or filters) in a pixel array;

[0031] Figure 11C yes Figure 11A Another schematic diagram of the arrangement of photoelectric conversion elements (or filters) in a pixel array;

[0032] Figure 12A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0033] Figure 12B yes Figure 12A A schematic diagram of the filter arrangement in the pixel array;

[0034] Figure 12C yes Figure 12A A schematic diagram of the arrangement of photoelectric conversion elements in a pixel array;

[0035] Figure 13A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0036] Figure 13B yes Figure 13A A schematic diagram of the filter arrangement in the pixel array;

[0037] Figure 13C yes Figure 13A A schematic diagram of the arrangement of photoelectric conversion elements in a pixel array;

[0038] Figure 14A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0039] Figure 14B yes Figure 14AA schematic diagram of the filter arrangement in the pixel array;

[0040] Figure 14C yes Figure 14A A schematic diagram of the arrangement of photoelectric conversion elements in a pixel array;

[0041] Figure 15A This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0042] Figure 15B This is a partial cross-sectional schematic diagram of another pixel array in the embodiments of this application;

[0043] Figure 16 This is a schematic diagram of the connection method between the pixel array and the exposure control line in the embodiment of this application;

[0044] Figure 17 This is a schematic diagram of a minimum repeating unit pixel arrangement in an embodiment of this application;

[0045] Figure 18 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0046] Figure 19 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0047] Figure 20 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0048] Figure 21 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0049] Figure 22 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0050] Figure 23 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0051] Figure 24 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0052] Figure 25 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0053] Figure 26 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0054] Figure 27This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0055] Figure 28 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0056] Figure 29 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0057] Figure 30 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0058] Figure 31 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0059] Figure 32 This is a schematic diagram of another smallest repeating unit pixel arrangement in the embodiments of this application;

[0060] Figure 33 This is a schematic diagram of a camera assembly according to an embodiment of this application;

[0061] Figure 34 This is a flowchart illustrating the image acquisition method according to certain embodiments of this application;

[0062] Figure 35 This is a schematic diagram illustrating the principle of image acquisition methods in related technologies;

[0063] Figure 36 This is a schematic diagram illustrating the principle of the optical image acquisition method in the embodiments of this application;

[0064] Figure 37 This is another schematic diagram illustrating the principle of the optical image acquisition method in the embodiments of this application;

[0065] Figures 38 to 41 This is a flowchart illustrating the image acquisition method according to certain embodiments of this application;

[0066] Figure 42 This is another schematic diagram illustrating the principle of the optical image acquisition method in the embodiments of this application;

[0067] Figure 43 This is another schematic diagram illustrating the principle of the optical image acquisition method in the embodiments of this application;

[0068] Figure 44 This is another schematic diagram illustrating the principle of the optical image acquisition method in the embodiments of this application;

[0069] Figure 45 This is another schematic diagram illustrating the principle of the optical image acquisition method in the embodiments of this application;

[0070] Figure 46 This is another schematic diagram illustrating the principle of the optical image acquisition method in the embodiments of this application;

[0071] Figure 47 This is a schematic diagram of a mobile terminal according to an embodiment of this application. Detailed Implementation

[0072] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0073] Please see Figure 4A This application provides an image sensor 10, which includes a plurality of pixels. At least some of the pixels include an isolation layer 1183, a condenser lens 1186, and a photoelectric conversion element 117. The condenser lens 1186 is disposed within the isolation layer 1183. The photoelectric conversion element 117 is used to receive light passing through the condenser lens 1186.

[0074] Please see Figure 4A and Figure 33 This application also provides a camera assembly 40. The camera assembly 40 includes an image sensor 10. The image sensor 10 includes a plurality of pixels. At least a portion of the plurality of pixels includes an isolation layer 1183, a condenser lens 1186, and a photoelectric conversion element 117. The condenser lens 1186 is disposed within the isolation layer 1183. The photoelectric conversion element 117 is used to receive light passing through the condenser lens 1186.

[0075] Please see Figure 4A and Figure 47 This application also provides a mobile terminal 90. The mobile terminal includes an image sensor 50 and a housing 80. The image sensor 50 is mounted on the housing 80. The image sensor 50 includes a plurality of pixels. At least a portion of the plurality of pixels includes an isolation layer 1183, a condenser lens 1186, and a photoelectric conversion element 117. The condenser lens 1186 is disposed within the isolation layer 1183. The photoelectric conversion element 117 is used to receive light passing through the condenser lens 1186. Embodiments of this application will be further described below with reference to the accompanying drawings.

[0076] In image sensors containing multiple pixels arranged in a multi-dimensional pixel array, when non-perpendicular light passes through the microlens and filter of a pixel, some of the light may incident on the photoelectric conversion elements of adjacent pixels, resulting in optical crosstalk. For image sensors containing pixels of multiple colors, optical crosstalk between adjacent pixels can cause color mixing problems, thus affecting image quality.

[0077] For the reasons mentioned above, such as Figure 4A As shown, this application provides an image sensor 10. By additionally adding an isolation layer 1183 and a condenser lens 1186 disposed within the isolation layer 1183 in each pixel, the light passing through the microlens 1181 and filter 1182 of each pixel can be focused by the condenser lens 1186 and incident on the photoelectric conversion element 117 of that pixel, thus avoiding the problem of light crosstalk between adjacent pixels.

[0078] Next, we will introduce the basic structure of the image sensor 10. Please refer to [link / reference needed]. Figure 1 , Figure 1 This is a schematic diagram of the image sensor 10 in an embodiment of this application. The image sensor 10 includes a pixel array 11, a vertical driving unit 12, a control unit 13, a column processing unit 14, and a horizontal driving unit 15.

[0079] For example, the image sensor 10 may employ a complementary metal-oxide-semiconductor (CMOS) photosensitive element or a charge-coupled device (CCD) photosensitive element.

[0080] For example, pixel array 11 includes a plurality of pixels arranged in a two-dimensional array. Figure 1 (not shown in the image), each pixel includes a photoelectric conversion element 117 ( Figure 2 (As shown). Each pixel converts light into electrical charge based on the intensity of the light incident on it.

[0081] For example, the vertical drive unit 12 includes a shift register and an address decoder. The vertical drive unit 12 includes readout scan and reset scan functions. Readout scan refers to sequentially scanning unit pixels row by row, reading signals from these unit pixels one row at a time. For example, the signal output from each pixel in the selected and scanned pixel row is transmitted to the column processing unit 14. Reset scan is used to reset the charge; the photocharge of the photoelectric conversion element 117 is discarded, allowing the accumulation of new photocharge to begin.

[0082] For example, the signal processing performed by column processing unit 14 is correlated double sampling (CDS) processing. In CDS processing, the reset level and signal level output from each pixel in the selected pixel row are extracted, and the level difference is calculated. Thus, the signal of the pixels in a row is obtained. Column processing unit 14 may have an analog-to-digital (A / D) conversion function for converting analog pixel signals into digital format.

[0083] For example, the horizontal drive unit 15 includes a shift register and an address decoder. The horizontal drive unit 15 sequentially scans the pixel array 11 column by column. Through the selection scan operation performed by the horizontal drive unit 15, each pixel column is processed sequentially by the column processing unit 14 and output sequentially.

[0084] For example, the control unit 13 configures timing signals according to the operating mode and uses a variety of timing signals to control the vertical drive unit 13, column processing unit 14 and horizontal drive unit 15 to work together.

[0085] Figure 2 This is a schematic diagram of a pixel circuit 110 in an embodiment of this application. Figure 2 The 110 mid-pixel circuit is used in Figure 1 In each pixel. (The following is combined with...) Figure 1 and Figure 2 The working principle of pixel circuit 110 is explained.

[0086] like Figure 2 As shown, the pixel circuit 110 includes a photoelectric conversion element 117 (e.g., a photodiode PD), an exposure control circuit 116 (e.g., a transfer transistor 112), a reset circuit (e.g., a reset transistor 113), an amplification circuit (e.g., an amplification transistor 114), and a selection circuit (e.g., a selection transistor 115). In embodiments of this application, the transfer transistor 112, reset transistor 113, amplification transistor 114, and selection transistor 115 are, for example, MOSFETs, but are not limited thereto.

[0087] For example, see Figure 1 and Figure 2The gate TG of transfer transistor 112 is connected to vertical drive unit 12 via an exposure control line (not shown); the gate RG of reset transistor 113 is connected to vertical drive unit 12 via a reset control line (not shown); and the gate SEL of select transistor 114 is connected to vertical drive unit 12 via a select line (not shown). An exposure control circuit 116 (e.g., transfer transistor 112) in each pixel circuit 110 is electrically connected to photoelectric conversion element 117 to transfer the potential accumulated in photoelectric conversion element 117 after illumination. For example, photoelectric conversion element 117 includes a photodiode PD, the anode of which is connected to ground, for example. The photodiode PD converts received light into electrical charge. The cathode of the photodiode PD is connected to floating diffusion unit FD via exposure control circuit 116 (e.g., transfer transistor 112). Floating diffusion unit FD is connected to the gate of amplification transistor 114 and the source of reset transistor 113.

[0088] For example, the exposure control circuit 116 is a transfer transistor 112, and the control terminal TG of the exposure control circuit 116 is the gate of the transfer transistor 112. When an effective level pulse (e.g., VPIX level) is transmitted to the gate of the transfer transistor 112 through the exposure control line (not shown in the figure), the transfer transistor 112 is turned on. The transfer transistor 112 transfers the charge converted by the photodiode PD to the floating diffusion unit FD.

[0089] For example, the drain of reset transistor 113 is connected to the pixel power supply VPIX. The source of reset transistor 113 is connected to the floating diffusion cell FD. Before charge is transferred from photodiode PD to floating diffusion cell FD, a pulse of effective reset level is transmitted to the gate of reset transistor 113 via the reset line, turning on reset transistor 113. Reset transistor 113 resets floating diffusion cell FD to pixel power supply VPIX.

[0090] For example, the gate of amplifying transistor 114 is connected to the floating diffusion unit FD. The drain of amplifying transistor 114 is connected to the pixel power supply VPIX. After the floating diffusion unit FD is reset by reset transistor 113, amplifying transistor 114 outputs a reset level via select transistor 115 through output terminal OUT. After the charge of photodiode PD is transferred by transfer transistor 112, amplifying transistor 114 outputs a signal level via select transistor 115 through output terminal OUT.

[0091] For example, the drain of the selector transistor 115 is connected to the source of the amplifying transistor 114. The source of the selector transistor 115 is connected to the output terminal OUT. Figure 1The column processing unit 14 is used. When an active level pulse is transmitted to the gate of the selection transistor 115 via the selection line, the selection transistor 115 is turned on. The signal output by the amplification transistor 114 is transmitted to the column processing unit 14 via the selection transistor 115.

[0092] It should be noted that the pixel structure of the pixel circuit 110 in this embodiment is not limited to... Figure 2 The structure is shown. For example, pixel circuit 110 can have a three-transistor pixel structure, in which the functions of amplification transistor 114 and selection transistor 115 are performed by one transistor. For example, exposure control circuit 116 is not limited to a single transfer transistor 112; other electronic devices or structures with control terminal control functions can be used as exposure control circuits in the embodiments of this application. The implementation of a single transfer transistor 112 is simple, low-cost, and easy to control.

[0093] The condenser lens 1186 can be applied to image sensors containing only color pixels (including but not limited to RGB), or to image sensors containing both panchromatic and color pixels, to improve the image quality of the image sensor. However, besides optical crosstalk affecting the image quality of the image sensor, pixel sensitivity (i.e., the amount of exposure received per unit time) also affects the image quality. For example, in an image sensor containing both panchromatic and color pixels, pixels of different colors receive different amounts of exposure per unit time; after some colors are saturated, some colors are not yet exposed to the ideal state. For example, exposure to 60%-90% of the saturation exposure can have a better signal-to-noise ratio and accuracy, but the embodiments of this application are not limited to this.

[0094] Figure 3 This example uses four pixel types: RGBW (red, green, blue, and full color). See also... Figure 3 , Figure 3 The horizontal axis represents exposure time, the vertical axis represents exposure amount, Q represents saturated exposure amount, LW represents the exposure curve of the full-color pixel W, LG represents the exposure curve of the green pixel G, LR represents the exposure curve of the red pixel R, and LB represents the exposure curve of the blue pixel.

[0095] from Figure 3 As can be seen, the exposure curve LW of the panchromatic pixel W has the steepest slope, meaning that the panchromatic pixel W can obtain more exposure per unit time, reaching saturation at time t1. The exposure curve LG of the green pixel G has the next steepest slope, and the green pixel saturates at time t2. The exposure curve LR of the red pixel R has the next steepest slope, and the red pixel saturates at time t3. The exposure curve LB of the blue pixel B has the smallest slope, and the blue pixel saturates at time t4. At time t1, the panchromatic pixel W is already saturated, while the exposures of the R, G, and B pixels have not yet reached their ideal state.

[0096] In related technologies, the exposure time of the four pixel types (RGBW) is controlled jointly. For example, the exposure time of each row of pixels is the same, they are connected to the same exposure control line, and are controlled by the same exposure control signal. See also... Figure 3 During the 0-t1 time period, all four pixel types (RGB, W, and RGB) can function normally. However, in this range, RGB pixels experience shorter exposure times and lower exposure amounts, resulting in lower brightness, lower signal-to-noise ratio, and even less vibrant colors in the displayed image. During the t1-t4 time period, W pixels become overexposed due to saturation and cannot function properly; the exposure data no longer accurately reflects the target.

[0097] To improve the image quality of the image sensor 10, in addition to solving the optical crosstalk problem by adding a condenser lens 1186, the full-well capacity of the pancolor pixels can be increased so that the full-well capacity of the pancolor pixels is greater than that of the color pixels, thereby avoiding the problem of premature saturation of the pancolor pixels and improving the image capture quality.

[0098] It should be noted that, Figure 3 The exposure curve shown is merely an example; the slope and relative relationships of the curve will vary depending on the pixel response band, and this application is not limited to this example. Figure 3 This is illustrated in the example. For instance, when the band response of the red pixel R is relatively narrow, the slope of the exposure curve for the red pixel R may be lower than the slope of the exposure curve for the blue pixel B.

[0099] Figures 4A to 8C It shows Figure 1The diagram shows various cross-sectional views of a portion of the pixels in the pixel array 11 along the light-receiving direction of the image sensor 10, as well as a schematic diagram of the arrangement of the photoelectric conversion element 117 (or filter 1182) in the pixel array 11. The panchromatic pixels and color pixels are spaced apart, with the color pixels having a narrower spectral response than the panchromatic pixels. Each panchromatic pixel and each color pixel includes a microlens 1181, a filter 1182, a condenser lens 1186, and a photoelectric conversion element 117. Along the light-receiving direction of the image sensor 10, the microlens 1181, filter 1182, isolation layer 1183, and photoelectric conversion element 117 are arranged sequentially. The photoelectric conversion element 117 converts the received light into electrical charge. Specifically, the photoelectric conversion element 117 includes a substrate 1171 and an n-well layer 1172 formed inside the substrate 1171, which enables the conversion of light into electrical charge. An isolation layer 1183 is disposed on one surface of the photoelectric conversion element 117 (specifically, one surface of the substrate 1171). Since the substrate 1171 is not perfectly flat, the filter 1182 is difficult to directly mount on the surface of the substrate 1171. The isolation layer 1183 is disposed on one surface of the substrate 1171, with a higher flatness on the side of the isolation layer 1183 away from the substrate 1171, facilitating the mounting of the filter 1182 on this surface. The filter 1182 is disposed on the surface of the isolation layer 1183 away from the substrate 1171, allowing light of a specific wavelength to pass through. A microlens 1181 is disposed on the side of the filter 1182 away from the isolation layer 1183. The microlens 1181 is used to converge light, guiding more incident light to the photoelectric conversion element 117. A condenser lens 1186 is disposed within the isolation layer 1183. The condenser lens 1186 can be used to converge the light passing through the microlens 1181 and the filter 1182, thereby allowing more light to enter the corresponding photoelectric conversion element 117 and avoiding optical crosstalk between adjacent pixels. The full-well capacity of the photoelectric conversion element 117 is related to the volume of the n-potential well layer of the photoelectric conversion element 117; the larger the volume of the n-potential well layer 1172, the larger the full-well capacity. Figures 4A to 8C In any of the embodiments shown, the volume of the n-well layer 1172 of the pan-color pixel is larger than the volume of the n-well layer 1172 of the color pixel, thereby making the full-well capacity of the pan-color pixel greater than that of the color pixel, increasing the exposure Q of the pan-color pixel saturation, and lengthening the time for the pan-color pixel to reach saturation. This avoids the problem of premature saturation of the pan-color pixel and can balance the exposure of the pan-color pixel and the color pixel. Thus, the image quality of the image sensor 10 is improved by designing the condenser lens 1186 and the design that the full-well capacity of the pan-color pixel is greater than that of the color pixel.

[0100] For example, Figure 4A and Figure 4BThese are, respectively, a cross-sectional schematic diagram of a pixel array 11 along the light-receiving direction according to an embodiment of this application, and a schematic diagram of the arrangement of multiple photoelectric conversion elements 117 (or multiple filters 1182). Figure 4A As shown, along the light-receiving direction, the dimensions of multiple cross-sections of the isolation layer 1183 for each pixel (within the same pixel) are all equal; the condenser lens 1186 is disposed within the isolation layer 1183; along the light-receiving direction, the dimensions of multiple cross-sections of the n-well layer 1172 for each pixel (within the same pixel) are all equal; the cross-sectional dimension of the n-well layer 1172 of the pan-color pixel is equal to the cross-sectional dimension of the n-well layer 1172 of the color pixel; the depth H1 of the n-well layer 1172 of the pan-color pixel is greater than the depth H2 of the n-well layer 1172 of the color pixel. This results in the volume of the n-well layer 1172 of the pan-color pixel being larger than the volume of the n-well layer 1172 of the color pixel, giving the pan-color pixel a larger full-well capacity than the color pixel. Furthermore, Figure 4A In the image sensor 10 shown, the condenser lens 1186 focuses the light so that more light enters the corresponding photoelectric conversion element 117, thus avoiding optical crosstalk problems.

[0101] It should be noted that the cross-section of the isolation layer 1183 is a cross-section of the isolation layer taken along a direction perpendicular to the light-receiving direction, and the cross-section of the n-potential well layer 1172 is a cross-section of the n-potential well layer 1172 taken along a direction perpendicular to the light-receiving direction. The shape and size of the cross-section of the isolation layer 1183 of each pixel correspond to the cross-section of the n-potential well layer 1172 of that pixel. The cross-section can be a rectangle, square, parallelogram, rhombus, pentagon, hexagon, or other polygons, and there are no restrictions here.

[0102] Along the light-receiving direction, the equal dimensions of multiple cross-sections of the n-well layer 1172 (or isolation layer 1183) of the same pixel mean that the multiple cross-sections have the same area, and the corresponding side lengths of the multiple cross-sections are equal. The equal dimensions of the cross-section of the n-well layer 1172 of the pan-color pixel and the n-well layer 1172 of the color pixel mean that the area of ​​the cross-section of the n-well layer 1172 of the pan-color pixel is equal to the area of ​​the cross-section of the n-well layer 1172 of the color pixel. The side length of the shape formed by the cross-section of the n-well layer 1172 of the pan-color pixel and the side length of the shape formed by the cross-section of the corresponding n-well layer 1172 of the color pixel may be equal or unequal. For example, Figure 4B The cross-sections of the n-potential well layers 1172 for both the pan-color pixel and the color pixel shown are rectangular, including both length and width. The area of ​​the cross-section of the n-potential well layer 1172 for the pan-color pixel is equal to the area of ​​the cross-section of the n-potential well layer 1172 for the color pixel, and the length L of the cross-section of the n-potential well layer 1172 for the pan-color pixel is... 全 The length L of the cross-section of the n-potential well layer 1172 equal to that of the color pixel 彩The width W of the cross-section of the n-potential well layer 1172 of the full-color pixel 全 The width W of the cross-section of the n-potential well layer 1172 equal to that of the color pixel 彩 In other examples, L 全 It may not be equal to L 彩 W 全 It may not be equal to W 彩 This is achieved as long as the area of ​​the cross-section of the n-well layer 1172 of the full-color pixel is equal to the area of ​​the cross-section of the n-well layer 1172 of the color pixel. The explanations below regarding the cross-section of the n-well layer 1172 (or isolation layer 1183), the equal dimensions of multiple cross-sections of the n-well layer 1172 (or isolation layer 1183) of each pixel, and the equality of the cross-section dimensions of the n-well layer 1172 of the full-color pixel and the n-well layer 1172 of the color pixel are the same as those here.

[0103] For example, Figure 5A This is a schematic cross-sectional view of the pixel array 11 along the light-receiving direction according to another embodiment of this application. Figure 5B and Figure 5C yes Figure 5A A schematic diagram showing the arrangement of multiple photoelectric conversion elements 117 (or multiple filters 1182) in the pixel array 11. Figure 5A As shown, along the light-receiving direction, the dimensions of multiple cross-sections of the isolation layer 1183 for each pixel (within the same pixel) are all equal; the condenser lens 1186 is disposed within the isolation layer 1183; along the light-receiving direction, the dimensions of multiple cross-sections of the n-well layer 1172 for each pixel (within the same pixel) are all equal; the cross-sectional dimension of the n-well layer 1172 of the pan-color pixel is larger than that of the n-well layer 1172 of the color pixel; the depth H1 of the n-well layer 1172 of the pan-color pixel is equal to the depth H2 of the n-well layer 1172 of the color pixel. This results in the volume of the n-well layer 1172 of the pan-color pixel being larger than that of the n-well layer 1172 of the color pixel, giving the pan-color pixel a larger full-well capacity than the color pixel. Furthermore, Figure 5A In the image sensor 10 shown, the condenser lens 1186 focuses the light so that more light enters the corresponding photoelectric conversion element 117, thus avoiding optical crosstalk problems.

[0104] Of course, in other embodiments... Figure 5A The depth H1 of the n-well layer 1172 of the medium-color pixel can also be greater than the depth H2 of the n-well layer 1172 of the color pixel.

[0105] It should be noted that the cross-sectional size of the n-well layer 1172 of the full-color pixel being larger than that of the n-well layer 1172 of the color pixel means that the area of ​​the cross-section of the n-well layer 1172 of the full-color pixel is larger than the area of ​​the cross-section of the n-well layer 1172 of the color pixel. The side length of the shape formed by the cross-section of the n-well layer of the full-color pixel can be partially or entirely larger than the side length of the shape formed by the cross-section of the corresponding n-well layer 1172 of the color pixel. For example, such as... Figure 5B As shown, the length L of the cross-section of the n-potential well layer 1172 of the full-color pixel is... 全 The length L of the cross-section of the n-potential well layer 1172, which is larger than the color pixel, is... 彩 The width W of the cross-section of the n-potential well layer 1172 of the full-color pixel 全 The width W of the cross-section of the n-potential well layer 1172 equal to that of the color pixel 彩 ;like Figure 5C As shown, the length L of the cross-section of the n-potential well layer 1172 of the full-color pixel is... 全 The length L of the cross-section of the n-potential well layer 1172 equal to that of the color pixel 彩 The width W of the cross-section of the n-potential well layer 1172 of the full-color pixel 全 The width W of the cross-section of the n-potential well layer 1172, which is larger than that of the color pixel, is... 彩 The explanation below regarding the fact that the cross-sectional size of the n-well layer 1172 for the full-color pixel is larger than that for the n-well layer 1172 for the color pixel is the same as the explanation here.

[0106] For example, Figures 6A to 6C These are, respectively, a cross-sectional schematic diagram of the pixel array 11 along the light-receiving direction according to another embodiment of this application, a schematic diagram of the arrangement of multiple filters 1182, and a schematic diagram of the arrangement of multiple photoelectric conversion elements 117. Figure 6A As shown, along the light-receiving direction, the dimensions of multiple cross-sections of the isolation layer 1183 for each pixel (within the same pixel) are all equal; the condenser lens 1186 is disposed within the isolation layer 1183; along the light-receiving direction, the cross-sectional dimensions of the n-well layer 1172 for each pancolor pixel (within the same pancolor pixel) gradually increase, and the cross-sectional dimensions of the n-well layer 1172 for each color pixel (within the same color pixel) gradually decrease, and the smallest cross-sectional dimension of the n-well layer 1172 of the pancolor pixel is equal to the largest cross-sectional dimension of the n-well layer 1172 of the color pixel; the depth H1 of the n-well layer 1172 of the pancolor pixel is equal to the depth H2 of the n-well layer 1172 of the color pixel. Figure 6B As shown, although the cross-sectional dimensions of the filter 1182 for the full-color pixel are equal to those of the filter 1182 for the color pixel (the area and corresponding side lengths are equal), as... Figure 6CAs shown, the cross-sectional area (excluding the smallest cross-section) of the n-well layer 1172 in the full-color pixel photoelectric conversion element 117 is actually larger than that of the n-well layer 1172 in the color pixel photoelectric conversion element 117. This results in the n-well layer 1172 of the full-color pixel having a larger volume than the n-well layer 1172 of the color pixel, giving the full-color pixel a larger full-well capacity than the color pixel. Furthermore, Figure 6A In the image sensor 10 shown, the condenser lens 1186 focuses the light so that more light enters the corresponding photoelectric conversion element 117, thus avoiding optical crosstalk problems.

[0107] In other embodiments, Figure 6A The minimum cross-sectional size of the n-well layer 1172 of the pancolor pixel can also be larger than the maximum cross-sectional size of the n-well layer of the color pixel, and the depth H1 of the n-well layer 1172 of the pancolor pixel can also be larger than the depth H2 of the n-well layer 1172 of the color pixel.

[0108] For example, Figures 7A to 7C These are, respectively, a cross-sectional view of the pixel array 11 along the light-receiving direction, a schematic diagram of the arrangement of multiple filters 1182, and a schematic diagram of the arrangement of multiple photoelectric conversion elements 117, according to another embodiment of this application. Figure 7A As shown, along the light-receiving direction, the dimensions of multiple cross-sections of the isolation layer 1183 for each panchromatic pixel (within the same panchromatic pixel) gradually increase, while the dimensions of multiple cross-sections of the isolation layer 1183 for each color pixel (within the same color pixel) gradually decrease; a condenser lens 1186 is disposed within the isolation layer 1183; along the light-receiving direction, the dimensions of the cross-section of the n-potential well layer 1172 for each panchromatic pixel gradually increase, while the dimensions of the cross-section of the n-potential well layer 1172 for each color pixel gradually decrease, and the smallest cross-section of the n-potential well layer 1172 of the panchromatic pixel is equal to the largest cross-section of the n-potential well layer 1172 of the color pixel; the depth H1 of the n-potential well layer 1172 of the panchromatic pixel is equal to the depth H2 of the n-potential well layer 1172 of the color pixel. Figure 7B As shown, although the cross-sectional dimensions of the filter 1182 for the full-color pixel are equal to those of the filter 1182 for the color pixel (the area and corresponding side lengths are equal), as... Figure 7C As shown, the cross-sectional area (excluding the smallest cross-section) of the n-well layer 1172 in the full-color pixel photoelectric conversion element 117 is actually larger than that of the n-well layer 1172 in the color pixel photoelectric conversion element 117. This results in the n-well layer 1172 of the full-color pixel having a larger volume than the n-well layer 1172 of the color pixel, giving the full-color pixel a larger full-well capacity than the color pixel. Furthermore, Figure 7AIn the image sensor 10 shown, the condenser lens 1186 focuses the light so that more light enters the corresponding photoelectric conversion element 117, thus avoiding optical crosstalk problems.

[0109] In other embodiments, Figure 7A The minimum cross-sectional size of the n-well layer 1172 of the pancolor pixel can also be larger than the maximum cross-sectional size of the n-well layer of the color pixel, and the depth H1 of the n-well layer 1172 of the pancolor pixel can also be larger than the depth H2 of the n-well layer 1172 of the color pixel.

[0110] For example, Figures 8A to 8C This is a cross-sectional schematic diagram of the pixel array 11 along the light-receiving direction, a schematic diagram of the arrangement of multiple filters 1182, and a schematic diagram of the arrangement of multiple photoelectric conversion elements 117, according to another embodiment of this application. Figure 8A As shown, along the light-receiving direction, the dimensions of multiple cross-sections of the isolation layer 1183 for each panchromatic pixel (within the same panchromatic pixel) gradually increase, while the dimensions of multiple cross-sections of the isolation layer 1183 for each color pixel (within the same color pixel) gradually decrease. Furthermore, the smallest cross-section of the isolation layer 1183 for the panchromatic pixel is equal to the largest cross-section of the isolation layer 1183 for the color pixel. A condenser lens 1186 is disposed within the isolation layer 1183. Along the light-receiving direction, the dimensions of multiple cross-sections of the n-potential well layer 1172 for each pixel are all equal. The cross-sectional dimension of the n-potential well layer 1172 for the panchromatic pixel is larger than that for the color pixel. The depth H1 of the n-potential well layer 1172 for the panchromatic pixel is equal to the depth H2 of the n-potential well layer 1172 for the color pixel. Figure 8B As shown, although the cross-sectional dimensions of the filter 1182 for the full-color pixel are equal to those of the filter 1182 for the color pixel (the area and corresponding side lengths are equal), as... Figure 8C As shown, the cross-sectional area (excluding the smallest cross-section) of the n-well layer 1172 in the full-color pixel photoelectric conversion element 117 is actually larger than that of the n-well layer 1172 in the color pixel photoelectric conversion element 117. This results in the n-well layer 1172 of the full-color pixel having a larger volume than the n-well layer 1172 of the color pixel, giving the full-color pixel a larger full-well capacity than the color pixel. Furthermore, Figure 8A In the image sensor 10 shown, the condenser lens 1186 focuses the light so that more light enters the corresponding photoelectric conversion element 117, thus avoiding optical crosstalk problems.

[0111] In other embodiments, Figure 8A The depth H1 of the n-well layer 1172 of the medium-color pixel can also be greater than the depth H2 of the n-well layer 1172 of the color pixel. Figure 8AThe smallest cross-sectional size of the isolation layer 1183 for the medium-color pixels can also be larger than the largest cross-sectional size of the isolation layer 1183 for the color pixels.

[0112] Figures 4A to 8C In any embodiment of the image sensor 10, each pixel is provided with a condenser lens 1186. When each pixel is provided with a condenser lens 1186, condenser lenses 1186 with different radii of curvature can be designed according to the needs of different pixels. For example, the radius of curvature of the condenser lens 1186 of the color pixel is greater than the radius of curvature of the condenser lens 1186 of the pancolor pixel, thereby making the light-gathering ability of the condenser lens 1186 of the color pixel higher than that of the condenser lens 1186 of the pancolor pixel, etc.

[0113] In other embodiments, only some pixels may include the condenser lens 1186, while the condenser lens 1186 may not be provided in the full-color pixels, and the condenser lens 1186 may be provided in the color pixels. For example, please refer to... Figure 9 In the illustrated embodiment, along the light-receiving direction, the cross-section of the n-potential well layer 1172 of the pan-color pixel gradually increases, while the cross-section of the n-potential well layer of the color pixel gradually decreases. Most of the light passing through the filter 1182 of the pan-color pixel can enter the photoelectric conversion element 117 of the pan-color pixel, while only a smaller portion of the light passing through the filter 1182 of the color pixel can enter the photoelectric conversion element 117 of the color pixel. Therefore, a condensing lens 1186 can be placed only in the isolation layer 1183 of the color pixel, thereby utilizing the light-concentrating effect of the condensing lens 1186 to allow more light to enter the photoelectric conversion element 117 of the color pixel. Placing the condensing lens 1186 only in some pixels can reduce the manufacturing cost of the image sensor 10.

[0114] When a condenser lens 1186 is provided in a pixel, an anti-reflective film can be provided on the side of each condenser lens 1186 opposite to the photoelectric conversion element 117. The anti-reflective film can be used to reduce light interference and avoid the influence of light interference on the imaging effect of the image sensor 10.

[0115] Figures 4A to 8C In any of the illustrated embodiments of the pixel array 11, the depth H3 of the photoelectric conversion element 117 of the full-color pixel is equal to the depth H4 of the photoelectric conversion element 117 of the color pixel. Specifically, the depth H3 of the substrate 1171 of the full-color pixel is equal to the depth H4 of the substrate 1171 of the color pixel. When H3 and H4 are equal, the surface of the substrate 1171 of the full-color pixel that is away from the filter 1182 is in the same horizontal plane as the surface of the substrate 1171 of the color pixel that is away from the filter 1182, which can reduce the complexity of the readout circuit design and manufacturing.

[0116] Figures 4A to 8CEach pixel in any of the embodiments shown also includes an optical isolation layer 1185. The optical isolation layer 1185 is disposed between the isolation layers 1183 of two adjacent pixels. For example, one optical isolation layer 1185 is disposed between the isolation layer 1183 of the panchromatic pixel W and the isolation layer 1183 of the color pixel A, and another optical isolation layer 1185 is disposed between the isolation layer 1183 of the panchromatic pixel W and the isolation layer 1183 of the color pixel B, etc. The optical isolation layer 1185 may be made of at least one material selected from tungsten, titanium, aluminum, and copper. The optical isolation layer 1185 prevents light incident on a pixel from entering another pixel adjacent to that pixel, avoiding noise to other pixels, i.e., avoiding optical crosstalk.

[0117] Figures 4A to 8C In any of the embodiments shown, the light-concentrating lens 1186 within each pixel can be replaced with the light guide layer 1184. Specifically, as Figures 10A to 14C As shown, Figure 10A The structure of the image sensor 10, except for the light guide layer 1184, is consistent with... Figure 4A The image sensor 10 in it is the same. Figure 11A The structure of the image sensor 10, except for the light guide layer 1184, is consistent with... Figure 5A The image sensor 10 in it is the same. Figure 12A The structure of the image sensor 10, except for the light guide layer 1184, is consistent with... Figure 6A same, Figure 13A The structure of the image sensor 10, except for the light guide layer 1184, is consistent with... Figure 7A The image sensor 10 in it is the same. Figure 14A The structure of the image sensor 10, except for the light guide layer 1184, is consistent with... Figure 8A The image sensor 10 in this paper is the same, and will not be described again here. Figures 10A to 14C The microlens 1181, filter 1182, isolation layer 1183, optical isolation interlayer 1185, and photoelectric conversion element 117 (substrate 1171 and n-potential well layer 1172) are described.

[0118] like Figures 10A to 14CAs shown, the light guide layer 1184 is formed within the isolation layer 1183, and the refractive index of the light guide layer 1184 is greater than that of the isolation layer 1183. Along a direction perpendicular to the light-receiving direction, the isolation layer 1183, the light guide layer 1184, and the isolation layer 1183 of each pixel are sequentially arranged. For example, along a direction perpendicular to the light-receiving direction, the isolation layer 1183 of the panchromatic pixel W, the light guide layer 1184 of the panchromatic pixel W, and the isolation layer 1183 of the panchromatic pixel W are sequentially arranged; the isolation layer 1183 of the color pixel A, the light guide layer 1184 of the color pixel A, and the isolation layer 1183 of the color pixel A are sequentially arranged; the isolation layer 1183 of the color pixel B, the light guide layer 1184 of the color pixel B, and the isolation layer 1183 of the color pixel B are sequentially arranged, and so on. The purpose of setting the light guide layer 1184 in the isolation layer 1183 is to make the light passing through the filter 1182 undergo total internal reflection in the structure composed of the isolation layer 1183 and the light guide layer 1184, thereby converging the light and allowing more light to enter the corresponding photoelectric conversion element 117, which can avoid the problem of light crosstalk between adjacent pixels. The n-potential well layer 1172 in the photoelectric conversion element 117 can receive the light passing through the light guide layer 1184 and convert the light into electric charge.

[0119] In one example, the refractive index of the light guide layer 1184 is equal at all locations. This design simplifies the design of the light guide layer and reduces the manufacturing difficulty of the pixel array 11. In another example, the refractive index of the light guide layer 1184 gradually increases along the light-receiving direction of the image sensor 10. This design enhances the light-gathering ability of the light guide layer 1184, allowing more light to enter the photoelectric conversion element 117.

[0120] like Figures 10A to 12C As shown, along the light-receiving direction, the dimensions of multiple cross-sections of the isolation layer 1183 for each pixel are all equal, and the dimensions of multiple cross-sections of the light guide layer 1184 for each pixel are also equal. This design simplifies the manufacturing process of the light guide layer 1184. Of course, in other embodiments, when the dimensions of multiple cross-sections of the isolation layer 1183 for each pixel are equal along the light-receiving direction, the structure of the light guide layer 1184 can also be such that the dimensions of multiple cross-sections of the light guide layer 1184 for each pixel gradually decrease along the light-receiving direction. This design enhances the light-gathering ability of the light guide layer 1184, allowing more light to enter the photoelectric conversion element 117.

[0121] like Figure 13A and Figure 14AAs shown, along the light-receiving direction, the dimensions of multiple cross-sections of the isolation layer 1183 for each panchromatic pixel gradually increase, while the dimensions of multiple cross-sections of the isolation layer 1183 for each color pixel gradually decrease. The dimensions of the cross-sections of both the light guide layer 1184 for each panchromatic pixel and the light guide layer 1184 for each color pixel also gradually decrease. This design enhances the light-gathering capability of the light guide layer 1184, allowing more light to enter the photoelectric conversion element 117. Of course, in other embodiments, when the dimensions of multiple cross-sections of the isolation layer 1183 for each panchromatic pixel gradually increase and the dimensions of multiple cross-sections of the isolation layer 1183 for each color pixel gradually decrease along the light-receiving direction, the structure of the light guide layer 1184 can also be such that the dimensions of multiple cross-sections of the isolation layer 1183 for each pixel are all equal along the light-receiving direction. This design simplifies the manufacturing process of the light guide layer 1184.

[0122] The depth of the light guide layer 1184 is equal to the depth of the isolation layer 1183, thereby enhancing the light-gathering capability of the light guide layer 1184. Compared with the thickness of the isolation layer in existing image sensors, the thickness of the isolation layer 1183 in this application is larger, for example, greater than a predetermined thickness, thereby forming a longer optical path and improving the light-gathering effect of the structure composed of the light guide layer 1184 and the isolation layer 1183.

[0123] Please see Figure 15A and Figure 15B The image sensor 10 also includes a barrier layer 1187. The barrier layer 1187 can be disposed between the photoelectric conversion elements 117 of two adjacent pixels. For example, one barrier layer 1187 can be disposed between the photoelectric conversion element 117 of a panchromatic pixel W and the photoelectric conversion element 117 of a color pixel A, and another barrier layer 1187 can be disposed between the photoelectric conversion element 117 of a panchromatic pixel W and the photoelectric conversion element 117 of a color pixel B, etc. For example, the barrier layer 1187 can be a deep trench isolation (DTI). The barrier layer 1187 can prevent light entering the photoelectric conversion element 117 of a certain pixel from entering the photoelectric conversion elements 117 of other pixels adjacent to that pixel, thus avoiding noise to the photoelectric conversion elements 117 of other pixels.

[0124] In addition to setting the full-well capacity of the full-color pixels to be greater than that of the color pixels as described above, in this embodiment, different full-well capacities can also be set for color pixels of different colors. Specifically, the full-well capacity corresponding to the sensitivity of the color pixel can be set according to its sensitivity (the shorter the exposure time to reach saturation, the higher the sensitivity of the pixel). For example, as... Figure 1As shown, the sensitivity of the green pixel > the sensitivity of the red pixel > the sensitivity of the blue pixel. Therefore, the full-well capacity of the color pixels can be set as follows: full-well capacity of green pixels > full-well capacity of red pixels > full-well capacity of blue pixels. The method for increasing the full-well capacity of the color pixels is similar to the method for increasing the full-well capacity of the pan-color pixels. For example, one method is to ensure that the cross-sectional area of ​​the n-potential well layer 1172 of each pixel is the same, i.e., S... W =S G =S R =S B Then, the relationship between the depths of the n-potential well layer 1172 of each pixel can be expressed as H. W >H G >H R >H B For example, when the depth of the n-well layer 1172 of each pixel is the same, i.e., H W =H G =H R =H B Then, the relationship between the cross-sectional areas of the n-potential well layer 1172 of each pixel can be expressed as S W >S G >S R >S B Other cases will not be elaborated here. Thus, different full-well capacities can be set according to different sensitivities, thereby balancing the exposure of pixels of various colors and improving image quality.

[0125] Based on setting the full well capacity of the pancolor pixels to be greater than that of the color pixels, the exposure of the pancolor pixels and the color pixels can be further balanced by independently controlling the exposure time of the pancolor pixels and the exposure time of the color pixels.

[0126] Figure 16 This is a schematic diagram of a pixel array 11 and the connection method of exposure control lines according to an embodiment of this application. The pixel array 11 is a two-dimensional pixel array. The two-dimensional pixel array includes multiple panchromatic pixels and multiple color pixels, wherein the color pixels have a narrower spectral response than the panchromatic pixels. The pixel arrangement in the pixel array 11 is as follows:

[0127]

[0128] It should be noted that, for the sake of illustration, Figure 16 Only a portion of the pixels in pixel array 11 are shown; other surrounding pixels and connecting lines are replaced with ellipses “…”.

[0129] like Figure 16As shown, pixels 1101, 1103, 1106, 1108, 1111, 1113, 1116, and 1118 are full-color pixels W; pixels 1102 and 1105 are first-color pixels A (e.g., red pixels R); pixels 1104, 1107, 1112, and 1115 are second-color pixels B (e.g., green pixels G); and pixels 1114 and 1117 are third-color pixels C (e.g., blue pixels Bu). From Figure 16 As can be seen, the control terminal TG of the exposure control circuit in the panchromatic pixel W (pixels 1101, 1103, 1106, and 1108) is connected to a first exposure control line TX1, and the control terminal TG of the exposure control circuit in the panchromatic pixel W (pixels 1111, 1113, 1116, and 1118) is connected to another first exposure control line TX1; the control terminal TG of the exposure control circuit in the first color pixel A (pixels 1102 and 1105), the control terminal TG of the exposure control circuit in the second color pixel B (pixels 1104 and 1107) are connected to a second exposure control line TX2, the control terminal TG of the exposure control circuit in the second color pixel B (pixels 1112 and 1115), and the control terminal TG of the exposure control circuit in the third color pixel C (pixels 1114 and 1117) are connected to another second exposure control line TX2. Each first exposure control line TX1 can control the exposure duration of the panchromatic pixels via a first exposure control signal; each second exposure control line TX2 can control the exposure duration of the color pixels (e.g., first color pixel A, second color pixel B, second color pixel C, and third color pixel C) via a second exposure control signal. This allows for independent control of the exposure duration of the panchromatic pixels and the color pixels. For example, it is possible to continue exposing the color pixels after the panchromatic pixel exposure has ended, achieving the desired imaging effect.

[0130] Please refer to Figure 2 and Figure 16 The first exposure control line TX1 and the second exposure control line TX2 are... Figure 2 The vertical drive unit 12 is connected to the pixel array 11, and the corresponding exposure control signal in the vertical drive unit 12 is transmitted to the control terminal TG of the exposure control circuit of the pixel in the pixel array 11.

[0131] Understandably, since the pixel array 11 contains multiple pixel row groups, the vertical drive unit 12 connects to multiple first exposure control lines TX1 and multiple second exposure control lines TX2. The multiple first exposure control lines TX1 and multiple second exposure control lines TX2 correspond to the corresponding pixel row groups.

[0132] For example, the first exposure control line TX1 corresponds to the full-color pixels in the first and second rows; the second exposure control line TX1 corresponds to the full-color pixels in the third and fourth rows, and so on. The third exposure control line TX1 corresponds to the full-color pixels in the fifth and sixth rows; the fourth exposure control line TX1 corresponds to the full-color pixels in the seventh and eighth rows, and the correspondence between the subsequent exposure control lines TX1 and the full-color pixels below them is not elaborated further. The signal timing transmitted by different exposure control lines TX1 will also be different, and this signal timing is configured by the vertical drive unit 12.

[0133] For example, the first second exposure control line TX2 corresponds to the color pixels in the first and second rows; the second second exposure control line TX2 corresponds to the color pixels in the third and fourth rows, and so on. The third second exposure control line TX2 corresponds to the color pixels in the fifth and sixth rows; the fourth second exposure control line TX2 corresponds to the color pixels in the seventh and eighth rows, and the correspondence between the subsequent second exposure control lines TX2 and the color pixels in the rows below is not elaborated further. The signal timing transmitted by different second exposure control lines TX2 will also be different, and this signal timing is also configured by the vertical drive unit 12.

[0134] Figures 17 to 32 Various image sensors 10 are shown. Figure 2 Example of pixel arrangement shown. See also Figure 2 ,and Figures 17 to 32 The image sensor 10 includes a two-dimensional pixel array (i.e., a plurality of color pixels (e.g., a plurality of first color pixels A, a plurality of second color pixels B, and a plurality of third color pixels C) and a plurality of panchromatic pixels W) composed of multiple color pixels (e.g., a plurality of first color pixels A, a plurality of second color pixels B, and a plurality of third color pixels C) and a plurality of panchromatic pixels W). Figure 16 The pixel array 11 shown is an example. The color pixels have a narrower spectral response than the panchromatic pixels. The response spectrum of a color pixel is, for example, a portion of the response spectrum of a panchromatic pixel W. The two-dimensional pixel array includes minimal repeating units (…). Figures 17 to 32Examples of pixel minimum repeating units in various image sensors 10 are shown. A two-dimensional pixel array consists of multiple minimum repeating units, which are replicated and arranged in rows and columns. In each minimum repeating unit, a panchromatic pixel W is positioned along a first diagonal direction D1, and a colored pixel is positioned along a second diagonal direction D2, where the first diagonal direction D1 differs from the second diagonal direction D2. The first exposure time of at least two adjacent panchromatic pixels along the first diagonal direction D1 is controlled by a first exposure signal, and the second exposure time of at least two adjacent colored pixels along the second diagonal direction D2 is controlled by a second exposure signal, thereby achieving independent control of the exposure time of the panchromatic pixels and the exposure time of the colored pixels. Each minimum repeating unit includes multiple sub-units, each sub-unit including multiple single-color pixels (e.g., multiple first-color pixels A, multiple second-color pixels B, or multiple third-color pixels C) and multiple panchromatic pixels W. For example, please combine... Figure 2 and Figure 16 Pixels 1101-1108 and 1111-1118 form a minimum repeating unit. Among them, pixels 1101, 1103, 1106, 1108, 1111, 1113, 1116, and 1118 are full-color pixels, while pixels 1102, 1104, 1105, 1107, 1112, 1114, 1115, and 1117 are color pixels. Pixels 1101, 1102, 1105, and 1106 form a sub-unit, where pixels 1101 and 1106 are full-color pixels, and pixels 1102 and 1105 are single-color pixels (e.g., first-color pixel A); pixels 1103, 1104, 1107, and 1108 form a sub-unit, where pixels 1103 and 1108 are full-color pixels, and pixels 1104 and 1107 are single-color pixels (e.g., second-color pixel B). Pixels 1111, 1112, 1115, and 1116 form a subunit, where pixels 1111 and 1116 are full-color pixels, and pixels 1112 and 1115 are single-color pixels (e.g., second-color pixels B); pixels 1113, 1114, 1117, and 1118 form a subunit, where pixels 1113 and 1118 are full-color pixels, and pixels 1114 and 1117 are single-color pixels (e.g., third-color pixels C).

[0135] For example, the smallest repeating unit has the same number of pixels in both rows and columns. Examples of such smallest repeating units include, but are not limited to, 4x4, 6x6, 8x8, and 10x10 units. Similarly, the sub-units within the smallest repeating unit have the same number of pixels in both rows and columns. Examples of such sub-units include, but are not limited to, 2x2, 3x3, 4x4, and 5x5 units. This configuration helps to balance the resolution and color performance of the image in both row and column directions, improving display quality.

[0136] For example, Figure 17 This is a schematic diagram of a minimum repeating unit of 1181 pixels in an embodiment of this application; the minimum repeating unit is 4 rows and 4 columns of 16 pixels, and the sub-unit is 2 rows and 2 columns of 4 pixels, arranged as follows:

[0137]

[0138] W represents a full-color pixel; A represents the first color pixel among multiple color pixels; B represents the second color pixel among multiple color pixels; and C represents the third color pixel among multiple color pixels.

[0139] For example, such as Figure 17 As shown, the full-color pixel W is set in the first diagonal direction D1 (i.e. Figure 17 The direction connecting the top left and bottom right corners), the color pixels are set in the second diagonal direction D2 (e.g. Figure 17 The direction connecting the lower left and upper right corners), the first diagonal direction D1 is different from the second diagonal direction D2. For example, the first and second diagonals are perpendicular. The first exposure time of two panchromatic pixels W adjacent to each other in the first diagonal direction D1 (e.g., two panchromatic pixels in the first row and first column and the second row and second column from the upper left) is controlled by the first exposure signal, and the second exposure time of at least two colored pixels adjacent to each other in the second diagonal direction D2 (e.g., two colored pixels B in the fourth row and first column and the third row and second column from the upper left) is controlled by the second exposure signal.

[0140] It should be noted that the first diagonal direction D1 and the second diagonal direction D2 are not limited to the diagonal itself, but also include directions parallel to the diagonal, for example... Figure 16 In the diagram, panchromatic pixels 1101, 1106, 1113, and 1118 are located in the first diagonal direction D1; panchromatic pixels 1103 and 1108 are also located in the first diagonal direction D1; panchromatic pixels 1111 and 1116 are also located in the first diagonal direction D1; second color pixels 1104, 1107, 1112, and 1115 are located in the second diagonal direction D2; first color pixels 1102 and 1105 are also located in the second diagonal direction D2; and third color pixels 1114 and 1117 are also located in the second diagonal direction D2. (The rest of the text is omitted as it is incomplete.) Figures 18 to 32 The explanation of the first diagonal direction D1 and the second diagonal direction D2 is the same as here. The "direction" here is not a single point of view, but can be understood as the concept of a "straight line" indicating the arrangement, which can have two directions at both ends of the line.

[0141] It should be understood that the terms “upper,” “lower,” “left,” “right,” etc., used herein and below indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0142] For example, such as Figure 17 As shown, the panchromatic pixels in the first and second rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the first and second rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the third and fourth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (B and C) in the third and fourth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. For example, the first exposure signal is transmitted via the first exposure control line TX1, and the second exposure signal is transmitted via the second exposure control line TX2. For example, the first exposure control line TX1 is "W"-shaped and electrically connected to the control terminals of the exposure control circuits in the adjacent two rows of panchromatic pixels; the second exposure control line TX2 is "W"-shaped and electrically connected to the control terminals of the exposure control circuits in the adjacent two rows of colored pixels. For specific connection methods, please refer to the above. Figure 2 and Figure 16 The relevant section describes the connections and pixel circuitry.

[0143] It should be noted that the "W" shape of the first exposure control line TX1 and the second exposure control line TX2 does not mean that the physical routing must strictly follow the "W" shape. It only requires that the connection method corresponds to the arrangement of the panchromatic and color pixels. For example, setting the "W"-shaped exposure control lines corresponds to a "W"-shaped pixel arrangement. This setting is simple to route, and the pixel arrangement has good resolution and color performance, achieving independent control of the exposure time of the panchromatic and color pixels at a low cost.

[0144] For example, Figure 18 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1182 pixels in an embodiment of this application. The smallest repeating unit is 4 rows and 4 columns, consisting of 16 pixels, and the sub-unit is 2 rows and 2 columns, consisting of 4 pixels. The arrangement is as follows:

[0145]

[0146] W represents a full-color pixel; A represents the first color pixel among multiple color pixels; B represents the second color pixel among multiple color pixels; and C represents the third color pixel among multiple color pixels.

[0147] For example, such as Figure 18 As shown, the full-color pixel W is set in the first diagonal direction D1 (i.e. Figure 18 The direction connecting the top right and bottom left corners), the color pixels are set in the second diagonal direction D2 (e.g. Figure 18 (The direction connecting the top left and bottom right corners). For example, the first diagonal and the second diagonal are perpendicular. The first exposure time of two adjacent panchromatic pixels W in the first diagonal direction D1 (e.g., two panchromatic pixels in the first row and second column from the top left and the first column from the top left) is controlled by the first exposure signal, and the second exposure time of at least two adjacent colored pixels in the second diagonal direction (e.g., two colored pixels A in the first row and first column from the top left and the second column from the top left) is controlled by the second exposure signal.

[0148] For example, such as Figure 18 As shown, the panchromatic pixels in the first and second rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the first and second rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the third and fourth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (B and C) in the third and fourth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time.

[0149] For example, Figure 19 This is a schematic diagram of another arrangement of the smallest repeating unit 1183 pixels in the embodiments of this application. Figure 20 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1184 pixels in an embodiment of this application. Figure 19 and Figure 20 In the embodiments, respectively corresponding to Figure 17 and Figure 18 The arrangement is as follows: the first color pixel A is the red pixel R; the second color pixel B is the green pixel G; and the third color pixel C is the blue pixel Bu.

[0150] It should be noted that in some embodiments, the response band of the panchromatic pixel W is the visible light band (e.g., 400nm-760nm). For example, an infrared filter is provided on the panchromatic pixel W to filter out infrared light. In some embodiments, the response band of the panchromatic pixel W is both the visible light band and the near-infrared band (e.g., 400nm-1000nm), which matches the response band of the photoelectric conversion element 117 (e.g., photodiode PD) in the image sensor 10. For example, the panchromatic pixel W may not have a filter, and the response band of the panchromatic pixel W is determined by the response band of the photodiode, i.e., the two match. The embodiments of this application include, but are not limited to, the above-mentioned band ranges.

[0151] For example, Figure 21 This is a schematic diagram of another arrangement of the smallest repeating unit 1185 pixels in the embodiments of this application. Figure 22 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1186 pixels in an embodiment of this application. Figure 21 and Figure 22 In the embodiments, respectively corresponding to Figure 17 and Figure 18 The arrangement is as follows: the first color pixel A is the red pixel R; the second color pixel B is the yellow pixel Y; and the third color pixel C is the blue pixel Bu.

[0152] For example, Figure 23 This is a schematic diagram of another arrangement of the smallest repeating unit 1187 pixels in the embodiments of this application. Figure 24 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1188 pixels in an embodiment of this application. Figure 23 and Figure 24 In the embodiments, respectively corresponding to Figure 17 and Figure 18 The arrangement is as follows: the first color pixel A is magenta pixel M; the second color pixel B is cyan pixel Cy; and the third color pixel C is yellow pixel Y.

[0153] For example, Figure 25 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1191 pixels in an embodiment of this application. The smallest repeating unit is 6 rows and 6 columns, consisting of 36 pixels, and the sub-unit is 3 rows and 3 columns, consisting of 9 pixels. The arrangement is as follows:

[0154]

[0155]

[0156] W represents a full-color pixel; A represents the first color pixel among multiple color pixels; B represents the second color pixel among multiple color pixels; and C represents the third color pixel among multiple color pixels.

[0157] For example, such as Figure 25 As shown, the panchromatic pixels in the first and second rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the first and second rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the third and fourth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A, B, and C) in the third and fourth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the fifth and sixth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (B and C) in the fifth and sixth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time.

[0158] For example, Figure 26 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1192 pixels in an embodiment of this application. The smallest repeating unit is 6 rows and 6 columns, consisting of 36 pixels, and the sub-unit is 3 rows and 3 columns, consisting of 9 pixels. The arrangement is as follows:

[0159]

[0160] W represents a full-color pixel; A represents the first color pixel among multiple color pixels; B represents the second color pixel among multiple color pixels; and C represents the third color pixel among multiple color pixels.

[0161] For example, such as Figure 26 As shown, the panchromatic pixels in the first and second rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the first and second rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the third and fourth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A, B, and C) in the third and fourth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the fifth and sixth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (B and C) in the fifth and sixth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time.

[0162] For example, Figure 27This is a schematic diagram of another arrangement of the smallest repeating unit 1193 pixels in the embodiments of this application. Figure 28 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1194 pixels in an embodiment of this application. Figure 27 and Figure 28 In the embodiments, respectively corresponding to Figure 25 and Figure 26 The arrangement is as follows: the first color pixel A is the red pixel R; the second color pixel B is the green pixel G; and the third color pixel C is the blue pixel Bu.

[0163] For example, in other embodiments, the first color pixel A is a red pixel R; the second color pixel B is a yellow pixel Y; and the third color pixel C is a blue pixel Bu. For example, in other embodiments, the first color pixel A is a magenta pixel M; the second color pixel B is a cyan pixel Cy; and the third color pixel C is a yellow pixel Y. Embodiments of this application include, but are not limited to, these. Specific circuit connection methods are described above and will not be repeated here.

[0164] For example, Figure 29 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1195 pixels in an embodiment of this application. The smallest repeating unit is 8 rows and 8 columns, consisting of 64 pixels, and the sub-unit is 4 rows and 4 columns, consisting of 16 pixels. The arrangement is as follows:

[0165]

[0166] W represents a full-color pixel; A represents the first color pixel among multiple color pixels; B represents the second color pixel among multiple color pixels; and C represents the third color pixel among multiple color pixels.

[0167] For example, such as Figure 29As shown, the panchromatic pixels in the first and second rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the first and second rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the third and fourth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the third and fourth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the fifth and sixth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (B and C) in the fifth and sixth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the seventh and eighth rows are connected together by a first exposure control line TX1 in a "W" shape to achieve individual control of the exposure time of the panchromatic pixels. The color pixels (B and C) in the seventh and eighth rows are connected together by a second exposure control line TX2 in a "W" shape to achieve individual control of the exposure time of the color pixels.

[0168] For example, Figure 30 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1196 pixels in an embodiment of this application. The smallest repeating unit is 8 rows and 8 columns, consisting of 64 pixels, and the sub-unit is 4 rows and 4 columns, consisting of 16 pixels. The arrangement is as follows:

[0169]

[0170] W represents a full-color pixel; A represents the first color pixel among multiple color pixels; B represents the second color pixel among multiple color pixels; and C represents the third color pixel among multiple color pixels.

[0171] For example, such as Figure 30As shown, the panchromatic pixels in the first and second rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the first and second rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the third and fourth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (A and B) in the third and fourth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the fifth and sixth rows are connected together by a "W"-shaped first exposure control line TX1 to achieve individual control of the panchromatic pixel exposure time. The colored pixels (B and C) in the fifth and sixth rows are connected together by a "W"-shaped second exposure control line TX2 to achieve individual control of the colored pixel exposure time. The panchromatic pixels in the seventh and eighth rows are connected together by a first exposure control line TX1 in a "W" shape to achieve individual control of the exposure time of the panchromatic pixels. The color pixels (B and C) in the seventh and eighth rows are connected together by a second exposure control line TX2 in a "W" shape to achieve individual control of the exposure time of the color pixels.

[0172] For example, Figure 31 This is a schematic diagram of another arrangement of the smallest repeating unit 1197 pixels in an embodiment of this application. Figure 32 This is a schematic diagram illustrating another arrangement of the smallest repeating unit of 1198 pixels in an embodiment of this application. Figure 31 and Figure 32 In the embodiments, respectively corresponding to Figure 29 and Figure 30 The arrangement is as follows: the first color pixel A is the red pixel R; the second color pixel B is the green pixel G; and the third color pixel C is the blue pixel Bu.

[0173] For example, in other embodiments, the first color pixel A is a red pixel R; the second color pixel B is a yellow pixel Y; and the third color pixel C is a blue pixel Bu. For example, the first color pixel A is a magenta pixel M; the second color pixel B is a cyan pixel Cy; and the third color pixel C is a yellow pixel Y. Embodiments of this application include, but are not limited to, these. Specific circuit connections are described above and will not be repeated here.

[0174] As can be seen from the above embodiments, such as Figures 17 to 32 As shown, image sensor 10 ( Figure 2 (As shown) includes multiple colored pixels and multiple panchromatic pixels W arranged in a matrix, with the colored pixels and panchromatic pixels spaced apart in both row and column directions.

[0175] For example, alternating between full-color pixels, color pixels, full-color pixels, color pixels, etc. along the row direction.

[0176] For example, alternating between solid-color pixels, colored pixels, solid-color pixels, colored pixels, etc. along the column direction.

[0177] Please combine Figure 16 The first exposure control line TX1 is electrically connected to the control terminal TG (e.g., the gate of the transfer transistor 112) of the exposure control circuit 116 in the (2n-1)th and (2n)th rows of full-color pixels W; the second exposure control line TX2 is electrically connected to the control terminal TG (e.g., the gate of the transfer transistor 112) of the exposure control circuit 116 in the (2n-1)th and (2n)th rows of color pixels; n is a natural number greater than or equal to 1.

[0178] For example, when n=1, the first exposure control line TX1 is electrically connected to the control terminal TG of the exposure control circuit 116 in the panchromatic pixels W of the first and second rows; the second exposure control line TX2 is electrically connected to the control terminal TG of the exposure control circuit 116 in the color pixels of the first and second rows. When n=2, the first exposure control line TX1 is electrically connected to the control terminal TG of the exposure control circuit 116 in the panchromatic pixels W of the third and fourth rows; the second exposure control line TX2 is electrically connected to the control terminal TG of the exposure control circuit 116 in the color pixels of the third and fourth rows. And so on, which will not be elaborated further here.

[0179] In some embodiments, the first exposure time is shorter than the second exposure time. The first exposure time is determined based on the n-potential well layer 1172 of the panchromatic pixels. Figure 4A The second exposure time can be determined based on the n-potential well layer 1172 of the color pixel (as shown). Figure 4A As shown in the figure, this is used to determine the determination.

[0180] Please see Figure 33 This application provides a camera assembly 40. The camera assembly 40 includes an image sensor 10, a processing chip 20, and a lens 30 as described in any of the above embodiments. The image sensor 10 is electrically connected to the processing chip 20. The lens 30 is disposed in the optical path of the image sensor 10. The processing chip 20 may be packaged with the image sensor 10 and the lens 30 in the same housing of the camera assembly 40; or, the image sensor 10 and the lens 30 may be packaged in the housing, and the processing chip 20 may be disposed outside the housing.

[0181] This application also provides a method applicable to Figure 33 The image acquisition method of the camera component 40. For example... Figure 34 As shown, the image acquisition methods include:

[0182] 01: Control the exposure of the two-dimensional pixel array to obtain the panchromatic raw image and the color raw image;

[0183] 02: Process the original color image to treat all pixels of each sub-unit as a single-color large pixel corresponding to the single color in that sub-unit, and output the pixel value of the single-color large pixel to obtain the intermediate color image;

[0184] 03: Process the panchromatic original image to obtain a panchromatic intermediate image; and

[0185] 04: Process the color intermediate image and / or panchromatic intermediate image to obtain the target image.

[0186] Please see Figure 2 and Figure 33 The image acquisition method of this application can be implemented by the camera assembly 40. Step 01 can be implemented by the image sensor 10. Steps 02, 03, and 04 can be implemented by the processing chip 20. That is, the image sensor 10 can expose to acquire a pancolor original image and a color original image. The processing chip 20 can be used to process the color original image, treating all pixels of each sub-unit as monochrome large pixels corresponding to a single color in that sub-unit, and outputting the pixel values ​​of the monochrome large pixels to obtain a color intermediate image. The processing chip 20 can also be used to process the pancolor original image to obtain a pancolor intermediate image, and to process the color intermediate image and / or the pancolor intermediate image to obtain a target image.

[0187] Please combine Figure 35In related technologies, if the pixel array of an image sensor includes both panchromatic pixels and color pixels, then when the image sensor is working, it will fit the pixel value of each panchromatic pixel in the pixel array to the pixel values ​​of other color pixels, thereby outputting an original image that only includes color pixels. Specifically, taking pixel A as a red pixel R, pixel B as a green pixel G, and pixel C as a blue pixel Bu as an example, after the column processing unit in the image sensor reads the pixel values ​​of multiple red pixels R, multiple green pixels G, multiple blue pixels Bu, and multiple panchromatic pixels W, the image sensor will first fit the pixel value of each panchromatic pixel W to the red pixels R, green pixels G, and blue pixels Bu adjacent to that panchromatic pixel, and then convert the non-Bayer array arrangement image into a Bayer array arrangement original image output for the processing chip to perform subsequent processing on the original image, such as interpolating the original image to obtain a full-color image (the pixel value of each pixel in the full-color image is composed of three components: red, green, and blue). In this processing method, the image sensor needs to execute relatively complex algorithms, which involves a large amount of computation. Furthermore, since the Qualcomm platform does not support the processing of images with non-Bayer arrays, it may be necessary to add additional hardware (such as an additional processing chip) to the image sensor to perform the process of converting images with non-Bayer arrays into original images with Bayer arrays.

[0188] The image acquisition method and camera assembly 40 of this application can reduce the computational load of the image sensor and avoid adding extra hardware to the image sensor.

[0189] Specifically, please combine Figure 2 and Figure 36 When a user requests to take a picture, the vertical drive unit 12 in the image sensor 10 controls the exposure of multiple panchromatic pixels and multiple color pixels in the two-dimensional pixel array. The column processing unit 14 reads the pixel value of each panchromatic pixel and the pixel value of each color pixel. The image sensor 10 does not perform the operation of fitting the pixel values ​​of the panchromatic pixels to the pixel values ​​of the color pixels. Instead, it directly outputs a panchromatic original image based on the pixel values ​​of the multiple panchromatic pixels and a color original image based on the pixel values ​​of the multiple color pixels.

[0190] like Figure 36As shown, the panchromatic original image includes multiple panchromatic pixels W and multiple empty pixels N (NULL). Empty pixels are neither panchromatic pixels nor color pixels. The position of an empty pixel N in the panchromatic original image can be considered as having no pixel, or its pixel value can be considered zero. Comparing the two-dimensional pixel array and the panchromatic original image, it can be seen that for each sub-unit in the two-dimensional pixel array, the sub-unit includes two panchromatic pixels W and two color pixels (color pixel A, color pixel B, or color pixel C). The panchromatic original image also has a sub-unit corresponding to each sub-unit in the two-dimensional pixel array. The sub-unit of the panchromatic original image includes two panchromatic pixels W and two empty pixels N. The positions of the two empty pixels N correspond to the positions of the two color pixels in the two-dimensional pixel array sub-unit.

[0191] Similarly, the original color image includes multiple color pixels and multiple empty pixels N. Empty pixels are neither panchromatic pixels nor color pixels. The location of an empty pixel N in the original color image can be considered as having no pixel at that location, or its pixel value can be considered zero. Comparing the two-dimensional pixel array with the original color image, we can see that each sub-unit in the two-dimensional pixel array includes two panchromatic pixels W and two color pixels. The original color image also has a sub-unit corresponding to each sub-unit in the two-dimensional pixel array. The sub-unit of the original color image includes two color pixels and two empty pixels N. The locations of the two empty pixels N correspond to the locations of the two panchromatic pixels W in the two-dimensional pixel array sub-unit.

[0192] After receiving the panchromatic original image and the color original image output by the image sensor 10, the processing chip 20 can further process the panchromatic original image to obtain a panchromatic intermediate image, and further process the color original image to obtain a color intermediate image. For example, the color original image can be obtained through... Figure 37 The method shown transforms it into a color intermediate image. For example... Figure 37As shown, the original color image comprises multiple sub-units, each sub-unit including multiple empty pixels N and multiple single-color pixels (also called single-color pixels). Specifically, some sub-units include two empty pixels N and two single-color pixels A, some sub-units include two empty pixels N and two single-color pixels B, and some sub-units include two empty pixels N and two single-color pixels C. The processing chip 20 can treat all pixels in a sub-unit including empty pixels N and single-color pixels A as a single-color large pixel A corresponding to the single-color A in that sub-unit, treat all pixels in a sub-unit including empty pixels N and single-color pixels B as a single-color large pixel B corresponding to the single-color B in that sub-unit, and treat all pixels in a sub-unit including empty pixels N and single-color pixels C as a single-color large pixel C corresponding to the single-color C in that sub-unit. Thus, the processing chip 20 can form a color intermediate image based on the multiple single-color large pixels A, multiple single-color large pixels B, and multiple single-color large pixels C. If the original color image including multiple empty pixels N is considered as an image with a second resolution, then according to Figure 37 The color intermediate image obtained in the manner shown is an image with a first resolution, where the first resolution is smaller than the second resolution. After obtaining the pancolor intermediate image and the color intermediate image, the processing chip 20 can further process the pancolor intermediate image and / or the color intermediate image to obtain the target image. Specifically, the processing chip 20 can process only the pancolor intermediate image to obtain the target image; the processing chip 20 can also process only the color intermediate image to obtain the target image; or the processing chip 20 can process both the pancolor intermediate image and the color intermediate image simultaneously to obtain the target image. The processing chip 20 can determine the processing method for the two intermediate images according to actual needs.

[0193] In the image acquisition method of this application embodiment, the image sensor 10 can directly output a panchromatic original image and a color original image. The subsequent processing of the panchromatic original image and the color original image is performed by the processing chip 20. The image sensor 10 does not need to perform the operation of fitting the pixel value of the panchromatic pixel W to the pixel value of the color pixel. The computational load of the image sensor 10 is reduced, and there is no need to add new hardware to the image sensor 10 to support the image sensor 10 to perform image processing, which simplifies the design of the image sensor 10.

[0194] In some implementations, step 01, which controls the exposure of a two-dimensional pixel array to obtain a panchromatic original image and a color original image, can be achieved in a variety of ways.

[0195] Please see Figure 38 In one example, step 01 includes:

[0196] 011: Controls the simultaneous exposure of all panchromatic pixels and all color pixels in the two-dimensional pixel array;

[0197] 012: Output the pixel values ​​of all panchromatic pixels to obtain the original panchromatic image; and

[0198] 013: Output the pixel values ​​of all colored pixels to obtain the original color image.

[0199] Please see Figure 33 Steps 011, 012, and 013 can all be implemented by the image sensor 10. That is, all panchromatic pixels and all color pixels in the image sensor 10 are exposed simultaneously. The image sensor 10 can output the pixel values ​​of all panchromatic pixels to obtain a panchromatic raw image, and it can also output the pixel values ​​of all color pixels to obtain a color raw image.

[0200] Please combine Figure 2 and Figure 16 Panchromatic pixels and color pixels can be exposed simultaneously, with the exposure time of the panchromatic pixels being less than or equal to the exposure time of the color pixels. Specifically, when the first exposure time of the panchromatic pixels is equal to the second exposure time of the color pixels, the exposure start time and exposure end time of the panchromatic pixels are the same as those of the color pixels, respectively. When the first exposure time is less than the second exposure time, the exposure start time of the panchromatic pixels is later than or equal to the exposure start time of the color pixels, and the exposure end time of the panchromatic pixels is earlier than the exposure end time of the color pixels; or, when the first exposure time is less than the second exposure time, the exposure start time of the panchromatic pixels is later than the exposure start time of the color pixels, and the exposure end time of the panchromatic pixels is earlier than or equal to the exposure end time of the color pixels. After the panchromatic pixels and color pixels have been evenly exposed, the image sensor 10 outputs the pixel values ​​of all panchromatic pixels to obtain a panchromatic original image, and outputs the pixel values ​​of all color pixels to obtain a color original image. In this process, the panchromatic original image can be output before the color original image, or vice versa; or the color original image can be output before the panchromatic original image, or the panchromatic original image and the color original image can be output simultaneously. The output order is not specified here. Simultaneous exposure of panchromatic and color pixels can reduce the acquisition time of both panchromatic and color original images, thus accelerating the acquisition process. This method of simultaneous exposure of panchromatic and color pixels has significant advantages in modes such as fast shooting and continuous shooting, where high image output speed is required.

[0201] Please see Figure 39 In another example, step 01 includes:

[0202] 014: Controls the time-division exposure of all panchromatic pixels and all color pixels in the two-dimensional pixel array;

[0203] 015: Output the pixel values ​​of all panchromatic pixels to obtain the original panchromatic image; and

[0204] 016: Output the pixel values ​​of all colored pixels to obtain the original color image.

[0205] Please see Figure 33 Steps 014, 015, and 016 can all be implemented by the image sensor 10. That is, all panchromatic pixels and all color pixels in the image sensor 10 are exposed in a time-division manner. The image sensor 10 can output the pixel values ​​of all panchromatic pixels to obtain a panchromatic raw image, and it can also output the pixel values ​​of all color pixels to obtain a color raw image.

[0206] Specifically, panchromatic pixels and color pixels can be exposed in a time-sharing manner, wherein the exposure time of the panchromatic pixels can be less than or equal to the exposure time of the color pixels. Specifically, regardless of whether the first exposure time and the second exposure time are equal, the time-sharing exposure method for all panchromatic pixels and all color pixels can be: (1) all panchromatic pixels first perform the first exposure time, and after all panchromatic pixels have been exposed, all color pixels then perform the second exposure time; (2) all color pixels first perform the second exposure time, and after all color pixels have been exposed, all panchromatic pixels then perform the first exposure time. After the panchromatic pixels and color pixels have been exposed, the image sensor 10 outputs the pixel values ​​of all panchromatic pixels to obtain the panchromatic original image, and outputs the pixel values ​​of all color pixels to obtain the color original image. The output methods for the pancolor original image and the color original image are as follows: (1) When the pancolor pixels are exposed before the color pixels, the image sensor 10 can output the pancolor original image during the exposure of the color pixels, or it can output the pancolor original image and the color original image sequentially after the color pixels have been exposed; (2) When the color pixels are exposed before the pancolor pixels, the image sensor 10 can output the color original image during the exposure of the pancolor pixels, or it can output the color original image and the pancolor original image sequentially after the pancolor pixels have been exposed; (3) Regardless of which of the pancolor pixels and the color pixels is exposed first, the image sensor 10 can output the pancolor original image and the color original image simultaneously after all pixels have been exposed. The control logic for the time-division exposure of the pancolor pixels and the color pixels in this example is relatively simple.

[0207] Image sensor 10 can simultaneously have Figure 38 and Figure 39 The diagram illustrates the functions of controlling simultaneous exposure of panchromatic and color pixels, as well as controlling time-division exposure of panchromatic and color pixels. The image sensor 10 can autonomously select the specific exposure mode used during image acquisition based on actual needs. For example, simultaneous exposure can be used in fast shooting or continuous shooting modes to meet the need for rapid image output; while time-division exposure can be used in normal shooting modes to simplify control logic.

[0208] Figure 38 and Figure 39 In the two examples shown, the exposure order of the panchromatic pixels and the color pixels can be controlled by the control unit 13 in the image sensor 10.

[0209] Figure 38 and Figure 39 In the two examples shown, the exposure time of the panchromatic pixels can be controlled by the first exposure signal, while the exposure time of the colored pixels can be controlled by the second exposure signal.

[0210] Specifically, please combine Figure 16 As an example, the image sensor 10 can use a first exposure signal to control at least two adjacent panchromatic pixels in the first diagonal direction to be exposed for a first exposure time, and use a second exposure signal to control at least two adjacent color pixels in the second diagonal direction to be exposed for a second exposure time, wherein the first exposure time can be less than or equal to the second exposure time. Specifically, the vertical driving unit 12 in the image sensor 10 transmits the first exposure signal through the first exposure control line TX1 to control at least two adjacent panchromatic pixels in the first diagonal direction to be exposed for the first exposure time, and the vertical driving unit 12 transmits the second exposure signal through the second exposure control line TX2 to control at least two adjacent panchromatic pixels in the second diagonal direction to be exposed for the second exposure time. After all panchromatic pixels and all color pixels have been exposed, as... Figure 36 As shown, the image sensor 10 does not perform the process of fitting the pixel values ​​of multiple panchromatic pixels to the pixel values ​​of color pixels, but directly outputs a panchromatic original image and a color original image.

[0211] Please combine Figure 2 and Figure 17 As another example, image sensor 10 can use a first exposure signal to control the panchromatic pixels in rows 2n-1 and 2n to be exposed for a first exposure time, and use a second exposure signal to control the color pixels in rows 2n-1 and 2n to be exposed for a second exposure time, wherein the first exposure time can be less than or equal to the second exposure time. Specifically, the first exposure control line TX1 in image sensor 10 is connected to the control terminal TG of all panchromatic pixels in rows 2n-1 and 2n, and the second exposure control line TX2 is connected to the control terminal TG of all color pixels in rows 2n-1 and 2n. Vertical drive unit 12 transmits the first exposure signal through the first exposure control line TX1 to control the panchromatic pixels in rows 2n-1 and 2n to be exposed for the first exposure time, and transmits the second exposure signal through the second exposure control line TX2 to control the color pixels in rows 2n-1 and 2n to be exposed for the second exposure time. After all panchromatic pixels and all color pixels have been exposed, as... Figure 36As shown, the image sensor 10 does not perform the process of fitting the pixel values ​​of multiple panchromatic pixels to the pixel values ​​of color pixels, but directly outputs a panchromatic original image and a color original image.

[0212] In some implementations, the processing chip 20 can determine the relative relationship between the first exposure time and the second exposure time based on the ambient brightness. For example, the image sensor 10 can first control the exposure of the pancolor pixels and output a pancolor raw image. The processing chip 20 analyzes the pixel values ​​of multiple pancolor pixels in the pancolor raw image to determine the ambient brightness. When the ambient brightness is less than or equal to a brightness threshold, the image sensor 10 controls the pancolor pixels to expose at a first exposure time equal to the second exposure time; when the ambient brightness is greater than the brightness threshold, the image sensor 10 controls the pancolor pixels to expose at a first exposure time less than the second exposure time. When the ambient brightness is greater than the brightness threshold, the relative relationship between the first exposure time and the second exposure time can be determined based on the brightness difference between the ambient brightness and the brightness threshold. For example, the larger the brightness difference, the smaller the ratio of the first exposure time to the second exposure time. For example, when the brightness difference is within a first range [a, b), the ratio of the first exposure time to the second exposure time is V1:V2; when the brightness difference is within a second range [b, c), the ratio of the first exposure time to the second exposure time is V1:V3; when the brightness difference is greater than or equal to c, the ratio of the first exposure time to the second exposure time is V1:V4, where V1 <V2<V3<V4。

[0213] Please see Figure 40 In some implementations, step 02 includes:

[0214] 021: Merge the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of the monochrome large pixel; and

[0215] 022: A color intermediate image is formed based on the pixel values ​​of multiple monochrome large pixels, and the color intermediate image has a first resolution.

[0216] Please see Figure 33 In some embodiments, steps 021 and 022 can both be implemented by the processing chip 20. That is, the processing chip 20 can be used to merge the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of a monochrome large pixel, and to form a color intermediate image based on the pixel values ​​of multiple monochrome large pixels, wherein the color intermediate image has a first resolution. The color intermediate image has a first resolution.

[0217] Specifically, such as Figure 37As shown, for a single-color large pixel A, the processing chip 20 can add the pixel values ​​of all pixels in the sub-unit including empty pixel N and single-color pixel A, and use the sum as the pixel value of the single-color large pixel A corresponding to that sub-unit. The pixel value of empty pixel N can be considered zero, and the same applies below. The processing chip 20 can add the pixel values ​​of all pixels in the sub-unit including empty pixel N and single-color pixel B, and use the sum as the pixel value of the single-color large pixel B corresponding to that sub-unit. The processing chip 20 can add the pixel values ​​of all pixels in the sub-unit including empty pixel N and single-color pixel C, and use the sum as the pixel value of the single-color large pixel C corresponding to that sub-unit. Thus, the processing chip 20 can obtain the pixel values ​​of multiple single large pixels A, multiple single-color large pixels B, and multiple single-color large pixels C. The processing chip 20 then forms a color intermediate image based on the pixel values ​​of the multiple single-color pixels A, multiple single-color large pixels B, and multiple single-color large pixels C. Figure 37 As shown, when single color A is red (R), single color B is green (G), and single color C is blue (Bu), the intermediate color image is the image of the Bayer array arrangement. Of course, the processing chip 20 is not limited to this method of acquiring the intermediate color image.

[0218] In some implementations, please refer to Figure 33 and Figure 41 When the camera component 40 is in different modes, different modes correspond to different target images. The processing chip 20 first determines which mode the camera component 40 is in, and then processes the color intermediate image and / or panchromatic intermediate image according to the mode of the camera component 40 to obtain the target image corresponding to the mode. The target image includes at least four types of target images: first target image, second target image, third target image, and fourth target image. The modes of the camera component 40 include at least: (1) the mode is preview mode, and the target image in the preview mode can be the first target image or the second target image; (2) the mode is imaging mode, and the target image in the imaging mode can be the second target image, the third target image, or the fourth target image; (3) the mode is both preview mode and low power mode, and the target image is the first target image; (4) the mode is both preview mode and non-low power mode, and the target image is the second target image; (5) the mode is both imaging mode and low power mode, and the target image is the second target image or the third target image; (6) the mode is both imaging mode and non-low power mode, and the target image is the fourth target image.

[0219] Please see Figure 41 In one example, when the target image is the first target image, step 04 includes:

[0220] 040: Interpolate each large monochrome pixel in the intermediate color image to obtain pixel values ​​for the other two colors besides that single color, and output them to obtain a first target image with a first resolution.

[0221] Please see Figure 33 Step 040 can be implemented by the processing chip 20. That is, the processing chip 20 can be used to interpolate each monochrome large pixel in the color intermediate image to obtain pixel values ​​of two other colors besides that monochrome color and output them to obtain a first target image with a first resolution.

[0222] Specifically, please combine Figure 42 Assuming that a large monochrome pixel A is a red pixel R, a large monochrome pixel B is a green pixel G, and a large monochrome pixel C is a blue pixel Bu, then the intermediate color image is an image arranged in a Bayer array. The processing chip 20 needs to perform demosaicing (i.e., interpolation) on the intermediate color image so that the pixel value of each large monochrome pixel simultaneously has the three components R, G, and B. For example, linear interpolation can be used to calculate the pixel values ​​of the two colors other than the single color of the large monochrome pixel. Taking the large monochrome pixel C... 2,2 (“C 2,2 For example, C represents the pixel in the second row and second column from the top left. The large monochrome pixel C... 2,2 Pixel value P(C) that has only the component of color C 2,2 Furthermore, it is also necessary to calculate the pixel value P(A) of color A at position C of the monochrome large pixel. 2,2 ) and the pixel value P(B) of color B 2,2 ), then P(A) 2,2 )=α1·P(A 3,1 )+α2·P(A 3,3 )+α3·P(A 1,3 )+α4·P(A 1,1 ), P(B 2,2 )=β1·P(B 1,2 )+β2·P(B 2,1 )+β3·P(B 2,3 )+β4·P(B 3,2 ), where α 1～ α4 and β 1～ β4 are all interpolation coefficients, and α1+α2+α3+α4=1, β1+β2+β3+β4=1. The above P(A 2,2 ) and P(B 2,2 The calculation method for P(A) is only an example, P(A) 2,2 ) and P(B 2,2 It can also be calculated using other interpolation methods besides linear interpolation, and no restrictions are imposed here.

[0223] After calculating the pixel values ​​of the three components of each monochrome large pixel, the processing chip 20 can calculate the final pixel value corresponding to that monochrome large pixel, i.e., A+B+C. It should be noted that A+B+C here does not mean directly adding the three pixels together to obtain the final pixel value of the monochrome large pixel; it only indicates that the monochrome large pixel includes three color components: A, B, and C. The processing chip 20 can form a first target image based on the final pixel values ​​of multiple monochrome large pixels. Since the color intermediate image has a first resolution, and the first target image is obtained by interpolation of the color intermediate image, the processing chip 20 does not perform interpolation on the color intermediate image; therefore, the resolution of the first target image is also the first resolution. The processing algorithm of the processing chip 20 to obtain the first target image from the color intermediate image is relatively simple and fast. When the camera component 40 is in both preview mode and low-power mode, it uses the first target image as the preview image, which satisfies the image output speed requirement of preview mode while saving power consumption of the camera component 40.

[0224] Please refer to the following: Figure 41 In another example, when the target image is a second target image, step 03 includes:

[0225] 031: Process the original panchromatic image, take all pixels of each sub-unit as the panchromatic large pixel, and output the pixel value of the panchromatic large pixel to obtain the panchromatic intermediate image, which has a first resolution;

[0226] Step 04 includes:

[0227] 041: Separate the color and brightness of the intermediate color image to obtain a color-brightness separated image with a first resolution;

[0228] 042: Fusing the brightness of the panchromatic intermediate image and the brightness of the color-brightness separated image to obtain a brightness-corrected color image with a first resolution; and

[0229] 043: Interpolate each large monochrome pixel in the brightness-corrected color image to obtain pixel values ​​for the other two colors besides the single color, and output them to obtain a second target image with a first resolution.

[0230] Please see Figure 33Steps 031, 041, 042, and 043 can all be implemented by the processing chip 20. That is, the processing chip 20 can process the panchromatic original image, taking all pixels of each sub-unit as panchromatic large pixels and outputting the pixel values ​​of the panchromatic large pixels to obtain a panchromatic intermediate image, which has a first resolution. The processing chip 20 can also be used to separate the color and brightness of the color intermediate image to obtain a color-brightness separation image with a first resolution, fuse the brightness of the panchromatic intermediate image and the brightness of the color-brightness separation image to obtain a brightness-corrected color image with a first resolution, and perform interpolation processing on each monochrome large pixel in the brightness-corrected color image to obtain pixel values ​​of the other two colors besides that single color and output them to obtain a second target image with a first resolution.

[0231] Specifically, the panchromatic original image can be obtained through... Figure 43 The method shown transforms it into a panchromatic intermediate image. For example... Figure 43 As shown, the panchromatic original image comprises multiple sub-units, each sub-unit including two empty pixels N and two panchromatic pixels W. The processing chip 20 can use all pixels in each sub-unit including empty pixels N and panchromatic pixels W as the corresponding panchromatic large pixel W. Thus, the processing chip 20 can form a panchromatic intermediate image based on the multiple panchromatic large pixels W. If the panchromatic original image including multiple empty pixels N is considered an image with a second resolution, then according to... Figure 43 The panchromatic intermediate image obtained in the manner shown is an image with a first resolution, where the first resolution is smaller than the second resolution.

[0232] As an example, the processing chip 20 can use all pixels of each sub-unit in the panchromatic original image as the panchromatic large pixel W corresponding to that sub-unit in the following manner: The processing chip 20 first merges the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of the panchromatic large pixel W, and then forms a panchromatic intermediate image based on the pixel values ​​of multiple panchromatic large pixels W. Specifically, for each panchromatic large pixel, the processing chip 20 can add all pixel values ​​of the sub-unit including the empty pixel N and the panchromatic pixel W, and use the result of the addition as the pixel value of the panchromatic large pixel W corresponding to that sub-unit, where the pixel value of the empty pixel N can be regarded as zero. In this way, the processing chip 20 can obtain the pixel values ​​of multiple panchromatic large pixels W.

[0233] After the processing chip 20 obtains the pancolor intermediate image and the color intermediate image, it can perform a fusion process on the pancolor intermediate image and the color intermediate image to obtain the second target image.

[0234] For example, such as Figure 43 As shown, the processing chip 20 first separates the color and brightness of the intermediate color image to obtain a color-brightness separated image. Figure 43In the color-brightness separation image, L represents brightness, and CLR represents color. Specifically, assuming that single-color pixel A is red pixel R, single-color pixel B is green pixel G, and single-color pixel C is blue pixel Bu, then: (1) the processing chip 20 can convert the color intermediate image in RGB space into a color-brightness separation image in YCrCb space. At this time, Y in YCrCb is the brightness L in the color-brightness separation image, and Cr and Cb in YCrCb are the color CLR in the color-brightness separation image; (2) the processing chip 20 can also convert the color intermediate image in RGB space into a color-brightness separation image in Lab space. At this time, L in Lab is the brightness L in the color-brightness separation image, and a and b in Lab are the color CLR in the color-brightness separation image. It should be noted that Figure 43 In the color-luminance separation image shown, L+CLR does not mean that the pixel value of each pixel is the sum of L and CLR, but only that the pixel value of each pixel is composed of L and CLR.

[0235] Subsequently, the processing chip 20 fuses the brightness of the color-brightness separated image and the brightness of the panchromatic intermediate image. For example, the pixel value W of each panchromatic pixel in the panchromatic intermediate image is the brightness value of that panchromatic pixel. The processing chip 20 can add the L of each pixel in the color-brightness separated image to the W of the corresponding panchromatic pixel in the panchromatic intermediate image to obtain the brightness-corrected pixel value. The processing chip 20 forms a brightness-corrected color-brightness separated image based on multiple brightness-corrected pixel values, and then uses color space conversion to convert the brightness-corrected color-brightness separated image into a brightness-corrected color image.

[0236] When the large monochrome pixel A is a red pixel R, the large monochrome pixel B is a green pixel G, and the large monochrome pixel C is a blue pixel Bu, the brightness-corrected color image is an image arranged in a Bayer array. The processing chip 20 needs to perform interpolation processing on the brightness-corrected color image so that the pixel value of each brightness-corrected large monochrome pixel simultaneously has the three components R, G, and B. The processing chip 20 can perform interpolation processing on the brightness-corrected color image to obtain the second target image. For example, a linear interpolation method can be used to obtain the second target image. The linear interpolation process is similar to the interpolation process in step 040 above, and will not be described again here.

[0237] Since the luminance-corrected color image has a first resolution, and the second target image is obtained by interpolation of the luminance-corrected color image, the processing chip 20 does not perform interpolation on the luminance-corrected color image. Therefore, the resolution of the second target image is also the first resolution. Because the second target image is obtained by fusing the brightness of the intermediate color image and the panchromatic intermediate image, it has a better imaging effect. Using the second target image as the preview image in both preview mode and non-low-power mode can improve the preview effect. In both imaging mode and low-power mode, using the second target image as the image provided to the user, since it is obtained without interpolation calculations, can reduce the power consumption of the camera component 40 to some extent, meeting the usage requirements of low-power mode; at the same time, the brightness of the second target image is relatively bright, which can meet the user's brightness requirements for the target image.

[0238] Please refer to the following: Figure 41 In yet another example, when the target image is a third target image, step 04 includes:

[0239] 044: Interpolate the intermediate color image to obtain a color interpolated image with a second resolution, wherein the corresponding sub-units in the color interpolated image are arranged in a Bayer array, and the second resolution is greater than the first resolution; and

[0240] 045: Perform interpolation on all single-color pixels in the color interpolated image to obtain pixel values ​​for the other two colors besides the single color, and output them to obtain a third target image with a second resolution.

[0241] Please see Figure 33 Steps 044 and 045 can both be implemented by the processing chip 20. That is, the processing chip 20 can be used to interpolate the intermediate color image to obtain a color interpolated image with a second resolution. The corresponding sub-units in the color interpolated image are arranged in a Bayer array, and the second resolution is greater than the first resolution. The processing chip 20 can also be used to interpolate all single-color pixels in the color interpolated image to obtain pixel values ​​for two other colors besides the single color and output them to obtain a third target image with a second resolution.

[0242] Specifically, please combine Figure 44The processing chip 20 splits each large monochrome pixel in the intermediate color image into four color pixels. These four color pixels form a subunit in the color interpolated image. Each subunit includes three colors of color pixels: one color pixel A, two color pixels B, and one color pixel C. When color pixel A is a red pixel R, color pixel B is a green pixel G, and color pixel C is a blue pixel Bu, the multiple color pixels in each subunit are arranged in a Bayer array. Thus, the color interpolated image containing multiple subunits is an image arranged in a Bayer array. The processing chip 20 can perform interpolation processing on the color interpolated image to obtain a third target image. For example, linear interpolation can be used to obtain the second target image. The linear interpolation process is similar to the interpolation process in step 040 above and will not be described again here. The third target image is the image obtained after interpolation processing. The resolution of the third target image (i.e., the second resolution) is larger than the resolution of the intermediate color image (i.e., the first resolution). When the mode is both preview mode and non-low power mode, using the third target image as the preview image can obtain a clearer preview image. When the mode is both imaging mode and low power mode, the third target image is used as the image provided to the user. Since the third target does not need to be fused with the full-color intermediate image in the formation process, the power consumption of the camera component 40 can be reduced to a certain extent, while still meeting the user's requirements for the clarity of the captured image.

[0243] Please refer to the following: Figure 41 In yet another example, when the target image is the fourth target image, step 03 includes:

[0244] 032: Interpolate the panchromatic original image to obtain the pixel values ​​of all pixels in each sub-unit to obtain a panchromatic intermediate image with a second resolution;

[0245] Step 04 includes:

[0246] 046: The intermediate color image is interpolated to obtain a color interpolated image with a second resolution. The corresponding sub-units in the color interpolated image are arranged in a Bayer array, and the second resolution is greater than the first resolution.

[0247] 047: Separate the color and brightness of the color interpolated image to obtain a color-brightness separated image with a second resolution;

[0248] 048: Fusing the brightness of the panchromatic interpolated image and the brightness of the color-brightness separated image to obtain a brightness-corrected color image with a second resolution; and

[0249] 049: Interpolate all single-color pixels in the brightness-corrected color image to obtain pixel values ​​for the other two colors besides the single color, and output them to obtain a fourth target image with a second resolution.

[0250] Please see Figure 33 Steps 032, 046, 047, 048, and 049 can all be implemented by the processing chip 20. That is, the processing chip 20 can be used to interpolate the panchromatic original image, obtaining the pixel values ​​of all pixels in each sub-unit to obtain a panchromatic intermediate image with a second resolution. The processing chip 20 can also be used to interpolate the color intermediate image to obtain a color interpolated image with a second resolution, where the corresponding sub-units in the color interpolated image are arranged in a Bayer array, and the second resolution is greater than the first resolution. The processing chip 20 can also be used to separate the color and brightness of the color interpolated image to obtain a color-brightness separated image with a second resolution, fuse the brightness of the panchromatic interpolated image and the brightness of the color-brightness separated image to obtain a brightness-corrected color image with a second resolution, and perform interpolation processing on all single-color pixels in the brightness-corrected color image to obtain pixel values ​​of the other two colors besides the single color and output them to obtain a fourth target image with a second resolution.

[0251] Specifically, the processing chip 20 first performs interpolation processing on the first-resolution panchromatic original image to obtain the second-resolution panchromatic intermediate image. Please refer to... Figure 46 The original panchromatic image comprises multiple sub-units, each including two empty pixels N and two panchromatic pixels W. The processing chip 20 needs to replace each empty pixel N in each sub-unit with a panchromatic pixel W and calculate the pixel value of each panchromatic pixel W located at the position of the empty pixel N after replacement. For each empty pixel N, the processing chip 20 replaces the empty pixel N with a panchromatic pixel W and determines the pixel value of the replaced panchromatic pixel W based on the pixel values ​​of the other panchromatic pixels W adjacent to the replaced panchromatic pixel W. Figure 46 The empty pixel N in the panchromatic original image shown 1,8 ("empty pixel N") 1,8 Taking the empty pixel N (counting from the top left, first row, eighth column, the same below) as an example, empty pixel N 1,8 Replace with full-color pixel W 1,8 , with full-color pixels W 1,8 The adjacent pixels are the panchromatic pixels W in the original panchromatic image. 1,7 And full-color pixel W 2,8 As an example, the full-color pixel W can be... 1,7 Pixel values ​​and panchromatic pixels W 2,8 The average pixel value is used as the panchromatic pixel W. 1,8 The pixel value. Figure 46 The empty pixel N in the panchromatic original image shown 2,3 For example, empty pixel N 2,3 Replace with full-color pixel W 2,3 , with full-color pixels W 2,3Adjacent panchromatic pixels are panchromatic pixels W in the original panchromatic image. 1,3 Full-color pixel W 2,2 Full-color pixel W 2,4 and full-color pixel W 3,3 As an example, the processing chip 20 will process the full-color pixels W 1,3 Pixel values, full-color pixels W 2,2 Pixel values, full-color pixels W 2,4 The pixel values, and the full-color pixel W 3,3 The average pixel value is used as the replacement panchromatic pixel W. 2,3 The pixel value.

[0252] After the processing chip 20 obtains the pancolor intermediate image and the color intermediate image, it can perform a fusion process on the pancolor intermediate image and the color intermediate image to obtain the fourth target image.

[0253] First, the processing chip 20 can perform interpolation processing on the color intermediate image at the first resolution to obtain a color interpolated image at the second resolution, such as... Figure 45 As shown. The specific interpolation method is similar to that in step 045, and will not be described in detail here.

[0254] Subsequently, as Figure 45 As shown, the processing chip 20 can separate the color and brightness of the color interpolated image to obtain a color-brightness separated image. Figure 45 In the color-brightness separation image, L represents brightness and CLR represents color. Specifically, assuming that single-color pixel A is red pixel R, single-color pixel B is green pixel G, and single-color pixel C is blue pixel Bu, then: (1) the processing chip 20 can convert the color interpolated image in RGB space into a color-brightness separation image in YCrCb space. At this time, Y in YCrCb is the brightness L in the color-brightness separation image, and Cr and Cb in YCrCb are the color CLR in the color-brightness separation image; (2) the processing chip 20 can also convert the color interpolated image in RGB space into a color-brightness separation image in Lab space. At this time, L in Lab is the brightness L in the color-brightness separation image, and a and b in Lab are the color CLR in the color-brightness separation image. It should be noted that Figure 45 In the color-luminance separation image shown, L+CLR does not mean that the pixel value of each pixel is the sum of L and CLR, but only that the pixel value of each pixel is composed of L and CLR.

[0255] Subsequently, as Figure 46As shown, the processing chip 20 can fuse the brightness of the color-brightness separated image and the brightness of the pancolor intermediate image. For example, the pixel value W of each pancolor pixel in the pancolor intermediate image is the brightness value of that pancolor pixel. The processing chip 20 can add the brightness L of each pixel in the color-brightness separated image to the brightness W of the corresponding pancolor pixel in the pancolor intermediate image to obtain the brightness-corrected pixel value. The processing chip 20 forms a brightness-corrected color-brightness separated image based on multiple brightness-corrected pixel values, and then converts the brightness-corrected color-brightness separated image into a brightness-corrected color image, which has a second resolution.

[0256] When color pixel A is a red pixel R, color pixel B is a green pixel G, and color pixel C is a blue pixel Bu, the brightness-corrected color image is an image arranged in a Bayer array. The processing chip 20 needs to perform interpolation processing on the brightness-corrected color image so that the pixel value of each brightness-corrected color pixel simultaneously has the three components R, G, and B. The processing chip 20 can perform interpolation processing on the brightness-corrected color image to obtain a fourth target image. For example, a linear interpolation method can be used to obtain the fourth target image. The linear interpolation process is similar to the interpolation process in step 40 above, and will not be described again here.

[0257] Because the fourth target image is obtained by fusing the brightness of the color intermediate image and the panchromatic intermediate image, and because the fourth target image has a higher resolution, it has better brightness and clarity. Using the fourth target image as the image provided to the user when the mode is both imaging mode and non-low power mode can meet the user's requirements for the quality of the captured image.

[0258] In some embodiments, the image acquisition method may further include acquiring ambient brightness. This step can be implemented by the processing chip 20, and the specific implementation method is as described above and will not be repeated here. When the ambient brightness is greater than a brightness threshold, a first target image or a third target image can be used as the target image; when the ambient brightness is less than or equal to the brightness threshold, a second target image or a fourth target image can be used as the target image. It can be understood that when the ambient brightness is relatively bright, the brightness of the first and second target images obtained solely from the color intermediate image is sufficient to meet the user's brightness requirements for the target image. In this case, it is not necessary to fuse the brightness of the pancolor intermediate image to improve the brightness of the target image. This not only reduces the computational load of the processing chip 20 but also reduces the power consumption of the camera component 40. When the ambient brightness is low, the brightness of the first and second target images obtained solely from the color intermediate image may not meet the user's brightness requirements for the target image. Using the second or fourth target image obtained by fusing the brightness of the pancolor intermediate image as the target image can improve the brightness of the target image.

[0259] Please see Figure 47 This application also provides a mobile terminal 90. The mobile terminal 90 may be a mobile phone, tablet computer, laptop computer, smart wearable device (such as smartwatch, smart bracelet, smart glasses, smart helmet, etc.), head-mounted display device, virtual reality device, etc., and is not limited thereto.

[0260] The mobile terminal 90 includes an image sensor 50, a processor 60, a memory 70, and a housing 80, all of which are installed within the housing 80. The image sensor 50 is connected to the processor 60, and the image sensor 50 can be the image sensor 10 described in any of the above embodiments. Figure 33 (As shown). The processor 60 can perform operations with the camera assembly 40 ( Figure 33 The processor 60 has the same function as the processing chip 20 shown in the diagram; in other words, the processor 60 can perform the functions of the processing chip 20 described in any of the above embodiments. The memory 70 is connected to the processor 60 and can store data processed by the processor 60, such as target images. The processor 60 can be mounted on the same substrate as the image sensor 50, in which case the image sensor 50 and the processor 60 can be considered as a single camera assembly 40. Alternatively, the processor 60 can be mounted on a different substrate than the image sensor 50.

[0261] The image sensor 50 in the mobile terminal 90 of this application is provided with a condenser lens 1186 to converge light, so that more light can enter the photoelectric conversion element 117 of the corresponding pixel, thereby avoiding the problem of light crosstalk between adjacent pixels and improving the imaging quality of the image sensor 50.

[0262] In the description of this specification, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0263] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0264] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. An image sensor, characterized by, It includes a plurality of pixels, and at least a portion of the plurality of pixels includes: Isolation layer; A light guide layer, wherein the light guide layer is disposed within the insulating layer; and A photoelectric conversion element, wherein the photoelectric conversion element is used to receive light passing through the light guide layer; The light guide layer is embedded within the isolation layer. Along a direction perpendicular to the light-receiving direction of the image sensor, the isolation layer, light guide layer, and isolation layer are arranged sequentially for each pixel. The refractive index of the light guide layer is greater than that of the isolation layer. Along the light-receiving direction, the cross-sectional size of the light guide layer of each pixel gradually decreases, while the cross-sectional size of the isolation layer of each panchromatic pixel gradually increases, and the cross-sectional size of the isolation layer of each color pixel gradually decreases.

2. The image sensor of claim 1, wherein, The plurality of pixels includes a plurality of full-color pixels and a plurality of color pixels, each of the color pixels including the isolation layer, the light guide layer, and the photoelectric conversion element; or The plurality of pixels include a full-color pixel and a plurality of color pixels. Each full-color pixel includes the isolation layer, the light guide layer, and the photoelectric conversion element. Each color pixel also includes the isolation layer, the light guide layer, and the photoelectric conversion element.

3. The image sensor of claim 1, wherein, An anti-reflective film is provided on the side of the light guide layer opposite to the photoelectric conversion element.

4. The image sensor according to claim 1, characterized in that, The image sensor also includes an optical isolation layer disposed between the isolation layers of two adjacent pixels.

5. The image sensor according to claim 1, characterized in that, The plurality of pixels includes a plurality of panchromatic pixels and a plurality of color pixels, wherein the color pixels have a narrower spectral response than the panchromatic pixels and the panchromatic pixels have a larger full-well capacity than the color pixels.

6. The image sensor according to claim 5, characterized in that, Each pixel includes a photoelectric conversion element, and each photoelectric conversion element includes a substrate and an n-potential well layer formed within the substrate.

7. The image sensor according to claim 6, characterized in that, The cross-sectional dimensions of the n-potential well layer of the pancolor pixel are equal to those of the n-potential well layer of the color pixel; the depth of the n-potential well layer of the pancolor pixel is greater than the depth of the n-potential well layer of the color pixel.

8. The image sensor according to claim 6, characterized in that, The cross-sectional dimension of the n-potential well layer of the pancolor pixel is larger than that of the n-potential well layer of the color pixel, and the depth of the n-potential well layer of the pancolor pixel is greater than or equal to the depth of the n-potential well layer of the color pixel.

9. The image sensor according to claim 8, characterized in that, Along the light-receiving direction of the image sensor, the dimensions of each cross-section of the n-potential well layer of each pixel are equal.

10. The image sensor according to claim 6, characterized in that, Along the light-receiving direction of the image sensor, the cross-sectional size of the n-potential well layer of the pancolor pixel gradually increases, the cross-sectional size of the n-potential well layer of the color pixel gradually decreases, and the smallest cross-sectional size of the n-potential well layer of the pancolor pixel is greater than or equal to the largest cross-sectional size of the n-potential well layer of the color pixel.

11. The image sensor according to any one of claims 6-10, characterized in that, The depth of the photoelectric conversion element of the full-color pixel is equal to the depth of the photoelectric conversion element of the color pixel.

12. The image sensor according to any one of claims 6-10, characterized in that, Each pixel includes a microlens, a filter, and an isolation layer, and the microlens, the filter, the isolation layer, and the photoelectric conversion element are arranged sequentially along the light-receiving direction of the image sensor.

13. The image sensor according to claim 5, characterized in that, The panchromatic pixels and the colored pixels form a two-dimensional pixel array. The two-dimensional pixel array includes a minimum repeating unit. In the minimum repeating unit, the panchromatic pixels are arranged in a first diagonal direction, and the colored pixels are arranged in a second diagonal direction. The first diagonal direction is different from the second diagonal direction. The first exposure time of at least two adjacent panchromatic pixels in the first diagonal direction is controlled by a first exposure signal, and the second exposure time of at least two adjacent color pixels in the second diagonal direction is controlled by a second exposure signal.

14. The image sensor according to claim 13, characterized in that, The first exposure time is shorter than the second exposure time.

15. The image sensor according to claim 13, characterized in that, The image sensor also includes: The first exposure control line is electrically connected to the control terminals of the exposure control circuits in at least two of the panchromatic pixels adjacent to the first diagonal direction; and The second exposure control line is electrically connected to the control terminals of the exposure control circuits in at least two color pixels adjacent to the second diagonal direction; The first exposure signal is transmitted via the first exposure control line, and the second exposure signal is transmitted via the second exposure control line.

16. The image sensor according to claim 15, characterized in that, The first exposure control line is "W" shaped and is electrically connected to the control terminal of the exposure control circuit in the two adjacent rows of full-color pixels; The second exposure control line is "W" shaped and is electrically connected to the control terminal of the exposure control circuit in the adjacent two rows of color pixels.

17. The image sensor according to claim 15 or 16, characterized in that, The exposure control circuit is electrically connected to the photoelectric conversion element and is used to transfer the potential accumulated in the photoelectric conversion element after being exposed to light.

18. The image sensor according to claim 15, characterized in that, The exposure control circuit is a transfer transistor, and the control terminal of the exposure control circuit is the gate of the transfer transistor.

19. The image sensor according to claim 13, characterized in that, The smallest repeating unit is 4 rows and 4 columns, consisting of 16 pixels, arranged as follows: WAWB AWBW WBWC BWCW Wherein, W represents the full-color pixel; A represents the first color pixel among the plurality of color pixels; B represents the second color pixel among the plurality of color pixels; C represents the third color pixel among the plurality of color pixels.

20. The image sensor according to claim 13, characterized in that, The smallest repeating unit is 4 rows and 4 columns, consisting of 16 pixels, arranged as follows: AWBW WAWB BWCW WBWC Wherein, W represents the full-color pixel; A represents the first color pixel among the plurality of color pixels; B represents the second color pixel among the plurality of color pixels; C represents the third color pixel among the plurality of color pixels.

21. The image sensor according to claim 13, characterized in that, The smallest repeating unit is 6 rows and 6 columns, consisting of 36 pixels, arranged as follows: WAWBWB AWAWBW WAWBWB BWBWCW WBWCWC BWBWCW Wherein, W represents the full-color pixel; A represents the first color pixel among the plurality of color pixels; B represents the second color pixel among the plurality of color pixels; C represents the third color pixel among the plurality of color pixels.

22. The image sensor according to claim 13, characterized in that, The smallest repeating unit is 6 rows and 6 columns, consisting of 36 pixels, arranged as follows: AWAWBW WAWBWB AWAWBW WBWCWC BWBWCW WBWCWC Wherein, W represents the full-color pixel; A represents the first color pixel among the plurality of color pixels; B represents the second color pixel among the plurality of color pixels; C represents the third color pixel among the plurality of color pixels.

23. The image sensor according to claim 13, characterized in that, The smallest repeating unit is 8 rows and 8 columns, consisting of 64 pixels, arranged as follows: WAWAWBWB AWAWBWBW WAWAWBWB AWAWBWBW WBWBWCWC BWBWCWCW WBWBWCWC BWBWCWCW Wherein, W represents the full-color pixel; A represents the first color pixel among the plurality of color pixels; B represents the second color pixel among the plurality of color pixels; C represents the third color pixel among the plurality of color pixels.

24. The image sensor according to claim 13, characterized in that, The smallest repeating unit is 8 rows and 8 columns, consisting of 64 pixels, arranged as follows: AWAWBWBW WAWAWBWB AWAWBWBW WAWAWBWB BWBWCWCW WBWBWCWC BWBWCWCW WBWBWCWC Wherein, W represents the full-color pixel; A represents the first color pixel among the plurality of color pixels; B represents the second color pixel among the plurality of color pixels; C represents the third color pixel among the plurality of color pixels.

25. The image sensor according to any one of claims 19-24, characterized in that, The first color pixel A is the red pixel R; The second color pixel B is the green pixel G; The third color pixel C is the blue pixel Bu.

26. The image sensor according to any one of claims 19-24, characterized in that, The first color pixel A is the red pixel R; The second color pixel B is the yellow pixel Y; The third color pixel C is the blue pixel Bu.

27. The image sensor according to any one of claims 19-24, characterized in that, The first color pixel A is the magenta pixel M; The second color pixel B is the cyan pixel Cy; The third color pixel C is the yellow pixel Y.

28. The image sensor according to any one of claims 13, 19-24, characterized in that, The response band of the full-color pixel is the visible light band.

29. The image sensor according to any one of claims 13, 19-24, characterized in that, The response bands of the full-color pixels are the visible light band and the near-infrared band, which match the response bands of the photoelectric conversion elements in the image sensor.

30. A camera assembly, characterized in that, include: The image sensor according to any one of claims 1-29.

31. The camera assembly according to claim 30, characterized in that, The image sensor includes multiple pixels, which include multiple panchromatic pixels and multiple color pixels. The panchromatic pixels and the color pixels form a two-dimensional pixel array. The two-dimensional pixel array includes a minimum repeating unit, each of which contains multiple sub-units. Each sub-unit includes multiple single-color pixels and multiple panchromatic pixels. The image sensor is used for exposure to acquire a panchromatic raw image and a color raw image. The camera assembly also includes a processing chip, which is used for: The original color image is processed to treat all pixels of each sub-unit as a single-color large pixel corresponding to a single color in that sub-unit, and the pixel value of the single-color large pixel is output to obtain a color intermediate image. Process the panchromatic original image to obtain a panchromatic intermediate image; and Process the color intermediate image and / or the panchromatic intermediate image to obtain the target image.

32. The camera assembly according to claim 31, characterized in that, All the panchromatic pixels and all the color pixels in the image sensor are exposed simultaneously; The image sensor outputs the pixel values ​​of all the panchromatic pixels to obtain the panchromatic original image, and outputs the pixel values ​​of all the color pixels to obtain the color original image.

33. The camera assembly according to claim 31, characterized in that, All the panchromatic pixels and all the color pixels in the image sensor are exposed in a time-division manner; The image sensor outputs the pixel values ​​of all the panchromatic pixels to obtain the panchromatic original image, and outputs the pixel values ​​of all the color pixels to obtain the color original image.

34. The camera assembly according to claim 31, characterized in that, In the minimum repeating unit, the panchromatic pixels are arranged in the first diagonal direction, and the colored pixels are arranged in the second diagonal direction. The first diagonal direction is different from the second diagonal direction. The first exposure time of at least two adjacent panchromatic pixels in the first diagonal direction is controlled by a first exposure signal, and the second exposure time of at least two adjacent colored pixels in the second diagonal direction is controlled by a second exposure signal. The processing chip is also used to acquire ambient brightness. When the ambient brightness is greater than the brightness threshold, the first exposure time is less than the second exposure time.

35. The camera assembly according to claim 31, characterized in that, The processing chip is also used for: Merge the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of the monochrome large pixel; and The intermediate color image is formed based on the pixel values ​​of multiple monochrome large pixels, and the intermediate color image has a first resolution.

36. The camera assembly according to claim 35, characterized in that, The processing chip is also used for: Interpolation is performed on each of the monochrome large pixels in the color intermediate image to obtain pixel values ​​for the other two colors besides the monochrome color, and the results are output to obtain a first target image with the first resolution.

37. The camera assembly according to claim 35, characterized in that, The processing chip is also used for: The panchromatic original image is processed by taking all pixels of each sub-unit as a panchromatic large pixel and outputting the pixel value of the panchromatic large pixel to obtain a panchromatic intermediate image, wherein the panchromatic intermediate image has the first resolution; The color and brightness of the intermediate color image are separated to obtain a color-brightness separated image with the first resolution; The brightness of the panchromatic intermediate image and the brightness of the color-brightness separated image are fused to obtain a brightness-corrected color image with the first resolution; and Interpolation is performed on each of the large monochrome pixels in the brightness-corrected color image to obtain pixel values ​​for the other two colors besides the single color, and the results are output to obtain a second target image with the first resolution.

38. The camera assembly according to claim 37, characterized in that, The processing chip is also used for: Merge the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of the full-color large pixel; and The panchromatic intermediate image is formed based on the pixel values ​​of multiple panchromatic large pixels.

39. The camera assembly according to claim 35, characterized in that, The processing chip is also used for: The intermediate color image is interpolated to obtain an interpolated color image with a second resolution, wherein the corresponding sub-units in the interpolated color image are arranged in a Bayer array, and the second resolution is greater than the first resolution; and Interpolation is performed on all single-color pixels in the color interpolated image to obtain pixel values ​​for two other colors besides the single color, and the results are output to obtain a third target image with the second resolution.

40. The camera assembly according to claim 35, characterized in that, The processing chip is also used for: The panchromatic original image is interpolated to obtain the pixel values ​​of all pixels in each sub-unit to obtain a panchromatic intermediate image with a second resolution; The intermediate color image is interpolated to obtain an interpolated color image with a second resolution, wherein the corresponding sub-units in the interpolated color image are arranged in a Bayer array, and the second resolution is greater than the first resolution; The color and brightness of the color interpolated image are separated to obtain a color-brightness separated image with the second resolution; The brightness of the panchromatic interpolated image and the brightness of the color-brightness separated image are fused to obtain a brightness-corrected color image with the second resolution; and Interpolation is performed on all single-color pixels in the brightness-corrected color image to obtain pixel values ​​for the other two colors besides the single color, and the results are output to obtain a fourth target image with the second resolution.

41. The camera assembly according to any one of claims 36-40, characterized in that, The image sensor is applied to the camera assembly, and different modes correspond to different target images when the camera assembly is in different modes.

42. The camera assembly according to claim 41, characterized in that, When the mode is preview mode, the target image is either the first target image or the second target image; When the mode is imaging mode, the target image is a second target image, the third target image, or the fourth target image.

43. The camera assembly according to claim 41, characterized in that, When the mode is both preview mode and low power mode, the target image is the first target image; When the mode is both the preview mode and the non-low power mode, the target image is the second target image; When the mode is both the imaging mode and the low-power mode, the target image is either the second target image or the third target image; When the mode is both the imaging mode and the non-low power mode, the target image is the fourth target image.

44. The camera assembly according to any one of claims 36-40, characterized in that, The processing chip is also used to acquire ambient brightness; When the ambient brightness is greater than the brightness threshold, the target image is either the first target image or the third target image; When the ambient brightness is less than the brightness threshold, the target image is either the second target image or the fourth target image.

45. A mobile terminal, characterized in that, include: chassis; and The image sensor according to any one of claims 1-29, wherein the image sensor is mounted inside the housing.

46. ​​The mobile terminal according to claim 45, characterized in that, The image sensor includes multiple pixels, which include multiple panchromatic pixels and multiple color pixels. The panchromatic pixels and the color pixels form a two-dimensional pixel array. The two-dimensional pixel array includes a minimum repeating unit, each of which contains multiple sub-units. Each sub-unit includes multiple single-color pixels and multiple panchromatic pixels. The image sensor is used for exposure to obtain a panchromatic raw image and a color raw image. The mobile terminal also includes a processor, which is used for: The original color image is processed to treat all pixels of each sub-unit as a single-color large pixel corresponding to a single color in that sub-unit, and the pixel value of the single-color large pixel is output to obtain a color intermediate image. Process the panchromatic original image to obtain a panchromatic intermediate image; and Process the color intermediate image and / or the panchromatic intermediate image to obtain the target image.

47. The mobile terminal according to claim 46, characterized in that, All the panchromatic pixels and all the color pixels in the image sensor are exposed simultaneously; The image sensor outputs the pixel values ​​of all the panchromatic pixels to obtain the panchromatic original image, and outputs the pixel values ​​of all the color pixels to obtain the color original image.

48. The mobile terminal according to claim 46, characterized in that, All the panchromatic pixels and all the color pixels in the image sensor are exposed in a time-division manner; The image sensor outputs the pixel values ​​of all the panchromatic pixels to obtain the panchromatic original image, and outputs the pixel values ​​of all the color pixels to obtain the color original image.

49. The mobile terminal according to claim 46, characterized in that, In the minimum repeating unit, the panchromatic pixels are arranged in the first diagonal direction, and the colored pixels are arranged in the second diagonal direction. The first diagonal direction is different from the second diagonal direction. The first exposure time of at least two adjacent panchromatic pixels in the first diagonal direction is controlled by a first exposure signal, and the second exposure time of at least two adjacent colored pixels in the second diagonal direction is controlled by a second exposure signal. The processor is also used to acquire ambient brightness. When the ambient brightness is greater than the brightness threshold, the first exposure time is less than the second exposure time.

50. The mobile terminal according to claim 46, characterized in that, The processor is also used for: Merge the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of the monochrome large pixel; and The intermediate color image is formed based on the pixel values ​​of multiple monochrome large pixels, and the intermediate color image has a first resolution.

51. The mobile terminal according to claim 50, characterized in that, The processor is also used for: Interpolation is performed on each of the monochrome large pixels in the color intermediate image to obtain pixel values ​​for the other two colors besides the monochrome color, and the results are output to obtain a first target image with the first resolution.

52. The mobile terminal according to claim 50, characterized in that, The processor is also used for: The panchromatic original image is processed by taking all pixels of each sub-unit as a panchromatic large pixel and outputting the pixel value of the panchromatic large pixel to obtain a panchromatic intermediate image, wherein the panchromatic intermediate image has the first resolution; The color and brightness of the intermediate color image are separated to obtain a color-brightness separated image with the first resolution; The brightness of the panchromatic intermediate image and the brightness of the color-brightness separated image are fused to obtain a brightness-corrected color image with the first resolution; and Interpolation is performed on each of the large monochrome pixels in the brightness-corrected color image to obtain pixel values ​​for the other two colors besides the single color, and the results are output to obtain a second target image with the first resolution.

53. The mobile terminal according to claim 52, characterized in that, The processor is also used for: Merge the pixel values ​​of all pixels in each sub-unit to obtain the pixel value of the full-color large pixel; and The panchromatic intermediate image is formed based on the pixel values ​​of multiple panchromatic large pixels.

54. The mobile terminal according to claim 50, characterized in that, The processor is also used for: The intermediate color image is interpolated to obtain an interpolated color image with a second resolution, wherein the corresponding sub-units in the interpolated color image are arranged in a Bayer array, and the second resolution is greater than the first resolution; and Interpolation is performed on all single-color pixels in the color interpolated image to obtain pixel values ​​for two other colors besides the single color, and the results are output to obtain a third target image with the second resolution.

55. The mobile terminal according to claim 50, characterized in that, The processor is also used for: The panchromatic original image is interpolated to obtain the pixel values ​​of all pixels in each sub-unit to obtain a panchromatic intermediate image with a second resolution; The intermediate color image is interpolated to obtain an interpolated color image with a second resolution, wherein the corresponding sub-units in the interpolated color image are arranged in a Bayer array, and the second resolution is greater than the first resolution; The color and brightness of the color interpolated image are separated to obtain a color-brightness separated image with the second resolution; The brightness of the panchromatic interpolated image and the brightness of the color-brightness separated image are fused to obtain a brightness-corrected color image with the second resolution; and Interpolation is performed on all single-color pixels in the brightness-corrected color image to obtain pixel values ​​for the other two colors besides the single color, and the results are output to obtain a fourth target image with the second resolution.

56. The mobile terminal according to any one of claims 51-55, characterized in that, The image sensor is applied to the mobile terminal, and different target images correspond to different modes when the mobile terminal is in different modes.

57. The mobile terminal according to claim 56, characterized in that, When the mode is preview mode, the target image is either the first target image or the second target image; When the mode is imaging mode, the target image is a second target image, the third target image, or the fourth target image.

58. The mobile terminal according to claim 56, characterized in that, When the mode is both preview mode and low power mode, the target image is the first target image; When the mode is both the preview mode and the non-low power mode, the target image is the second target image; When the mode is both the imaging mode and the low-power mode, the target image is either the second target image or the third target image; When the mode is both the imaging mode and the non-low power mode, the target image is the fourth target image.

59. The mobile terminal according to any one of claims 51-55, characterized in that, The processor is also used to acquire ambient brightness; When the ambient brightness is greater than the brightness threshold, the target image is either the first target image or the third target image; When the ambient brightness is less than the brightness threshold, the target image is either the second target image or the fourth target image.

Images