Dual-core focusing image sensor based on large and small pixels

Abstract

The invention discloses a dual-core focusing image sensor based on large and small pixels, and belongs to the field of image sensors. According to the image sensor, a pixel detection unit is divided into a large unit and a small unit, so that light from different directions can be received in the horizontal direction and the vertical direction during focusing, namely defocusing information in the horizontal direction and defocusing information in the vertical direction are included at the same time, and then the phase difference and the defocusing distance in the horizontal direction and the vertical direction are calculated; therefore, focusing processing in the horizontal direction and the vertical direction can be carried out at the same time, and after focusing is completed, signal values of the large-pixel detection unit and the small-pixel detection unit are added during imaging to obtain complete image information. Focusing in the horizontal direction and the vertical direction can be carried out at the same time on the premise that the number of available pixels in a single direction is not lost.

Description

Technical Field

[0001] This invention relates to a dual-core focusing image sensor based on large and small pixels, belonging to the field of image sensors. Background Technology

[0002] In the imaging systems of smartphones, cameras, and other electronic devices, image sensors convert received light signals into electrical signals to achieve image formation. To obtain a clear image, focusing is required; this involves adjusting the distance between the lens and the image sensor to ensure the object being photographed is imaged on the focal plane. Traditional focusing methods include phase detection autofocus (PDAF).

[0003] In traditional phase detection autofocus technology, dual-core phase detection autofocus technology separates the light entering a pixel into two parts, left and right. When out of focus, incident light at different angles will produce imaging differences at different pixel positions of the image sensor. By comparing the phase difference between the images formed by the left and right parts of the light beam, focusing can be performed.

[0004] However, traditional dual-pixel autofocus technology can only achieve horizontal focusing and cannot provide vertical focusing information for the image. To address this issue, some designs propose setting some pixels as horizontally segmented focusing pixels and the remaining pixels as vertically segmented focusing pixels within the same image sensor. While this design can acquire both horizontal and vertical focusing information in a single image capture, the distribution of pixels in different directions increases the complexity of the readout circuitry. Furthermore, due to the horizontal and vertical pixel allocation, the number of usable pixels in a single direction is reduced by half, mitigating the impact on focusing performance in low-light conditions. Summary of the Invention

[0005] To achieve focusing in both the horizontal and vertical directions without affecting the sensor's focusing performance in low-light environments, this invention provides a dual-core focusing image sensor based on large and small pixels. The image sensor consists of multiple pixel units arranged in an array. Each pixel unit includes a small pixel detection unit and a large pixel detection unit, separated by a deep trench isolation region on the back. The area of ​​the small pixel detection unit is smaller than that of the large pixel detection unit. The small pixel detection unit includes a small pixel MOS-C capacitor and a small pixel MOSFET, separated by a shallow trench isolation region. The large pixel detection unit includes a large pixel MOS-C capacitor and a large pixel MOSFET, separated by a shallow trench isolation region. The shallow trench isolation regions of the small and large pixels are located at the same horizontal level.

[0006] Optionally, both the small pixel detection unit and the pixel unit are square, and the side length of the small pixel unit accounts for 35% to 50% of the side length of the overall pixel unit, and is located at the lower right corner of the overall pixel unit.

[0007] Optionally, the side length of the small pixel detection unit is half the side length of the overall pixel unit.

[0008] Optionally, the bottom layer of the large pixel detection unit is a large pixel substrate, and the large pixel MOS-C capacitor and the large pixel MOSFET are disposed above the large pixel substrate. The large pixel MOS-C capacitor and the large pixel MOSFET are connected through the floating gate above, and the floating gate is controlled by the control gate above.

[0009] Optionally, the bottom layer of the small pixel detection unit is a small pixel substrate, and the small pixel MOS-C capacitor and the small pixel MOSFET are disposed above the small pixel substrate. The small pixel MOS-C capacitor and the small pixel MOSFET are connected through the floating gate above, and the floating gate is controlled by the control gate above.

[0010] Optionally, the floating gate above the large pixel MOS-C capacitor and the floating gate above the small pixel MOS-C capacitor and the small pixel MOSFET are the same floating gate, and the control gate above the floating gate is the same control gate.

[0011] Optionally, the pixel units arranged in an array are isolated from each other by a deep trench isolation region on the back side.

[0012] The present invention also provides a method for using the above-mentioned dual-core focusing image sensor based on large and small pixels. When focusing, the method first reads the output value of the large pixel detection unit and the output value of the small pixel detection unit, then calculates the phase difference of the two output signals in the horizontal and vertical directions based on the output values ​​of the large pixel detection unit and the small pixel detection unit, and finally obtains the defocus distance in the two directions based on the phase difference in the horizontal and vertical directions, and performs focusing operation based on the defocus distance.

[0013] Optionally, after focusing, the method controls the arrayed pixel units to enter the imaging mode, and sequentially performs reset, exposure, and reads information values ​​to generate an image. The final image signal is obtained by adding the intensity values ​​of the large pixel detection unit and the small pixel detection unit on the same pixel.

[0014] The present invention also provides the application of the above-mentioned dual-core focusing image sensor based on large and small pixels in smartphones and cameras.

[0015] The beneficial effects of this invention are: By dividing the pixel detection unit into two units, one large and one small, light from different directions can be received in both the horizontal and vertical directions during focusing. This means that it simultaneously contains defocus information in both the horizontal and vertical directions. The phase difference and defocus distance in the horizontal and vertical directions can then be calculated, allowing for simultaneous focusing in both the horizontal and vertical directions. Furthermore, after focusing is completed, the signal values ​​of the large pixel detection unit and the small pixel detection unit are added together during imaging to obtain complete image information. This achieves simultaneous focusing in both the horizontal and vertical directions without losing the number of usable pixels in any single direction. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram illustrating the principle of phase detection autofocus.

[0018] Figure 2 This is a top view of the dual-core focusing sensor structure used in traditional dual-core focusing technology.

[0019] Figure 3 This is a schematic diagram of the longitudinal section of the dual-core focusing sensor structure used in traditional dual-core focusing technology.

[0020] Figure 4 This is a top view schematic diagram of the pixel unit of a dual-core focusing image sensor based on large and small pixels provided in an embodiment of the present invention.

[0021] Figure 5 for Figure 4 The diagram shows a longitudinal cross-section of the pixel unit of a dual-core focusing image sensor based on large and small pixels along the direction of the large pixel.

[0022] Figure 6 for Figure 4 The diagram shows a longitudinal cross-section of a pixel unit in a dual-core focusing image sensor based on large and small pixels, along the direction of the small pixel.

[0023] Figure 7 This is a top-view schematic diagram of a dual-core focusing image sensor based on large and small pixels provided in an embodiment of the present invention.

[0024] Figure 8 This is a flowchart of a focusing control method for a dual-core focusing image sensor based on large and small pixels, provided in an embodiment of the present invention.

[0025] Figure 9This is a flowchart of an imaging control method for a dual-core focusing image sensor based on large and small pixels, provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0027] To facilitate understanding of the proposed solution, the principle of Phase Detection Autofocus (PDAF) is explained below: like Figure 1 As shown, when focusing, the image sensor is set at the focal point 106. At this time, two imaging rays 102 and 103 reflected from the object 101 towards the lens converge at the focal point 106 of the image sensor and are imaged at the same position, namely the imaging position 110, where the image sensor produces the clearest image.

[0028] Defocus position 105 and defocus position 2107 represent the possible positions of the image sensor when out of focus, such as... Figure 1 As shown, when the image sensor is at defocus position one 105 or defocus position two 107, two imaging rays 102 and 103 reflected from the object 101 toward the lens will form images at different positions: When the image sensor is at defocus position 105, the imaging rays 102 and 103 reflected from the object 101 onto the lens are imaged at defocus point 108 and defocus point 2 109 respectively, resulting in an unclear image from the image sensor. When the image sensor is at defocus position 2 107, the imaging rays 102 and 103 reflected from the object 101 onto the lens are imaged at defocus point 3 111 and defocus point 4 112 respectively, resulting in an unclear image from the image sensor as well.

[0029] In traditional PDAF technology, the positional differences between light rays incident on the lens from different directions and their images formed on the image sensor can be obtained, and the defocus distance can be calculated. The lens movement is then adjusted according to the defocus distance to achieve focus.

[0030] For example, such as Figure 1 As shown, the difference between focus point 108 and focus point 2 109 can be obtained, or the difference between focus point 3 111 and focus point 4 112 can be obtained. Based on the geometric relationship between the difference and the image sensor, the defocus distance can be obtained. The defocus distance refers to the distance between the current position of the image sensor and the position that the image sensor should be in when it is in the focus state, that is, the distance between focus position 106 and defocus position 105 or defocus position 2 107.

[0031] When focusing, for a given object point, only one location on the sensor generates a light response; at this time, the calculated phase difference (PD) value is 0. When defocusing, two locations on the image sensor at a certain distance both generate a light response; at this time, the phase difference (PD) value between the two locations is not 0. A larger calculated PD value indicates a greater distance from the focusing position, and vice versa. When using PDAF for focusing, the defocus distance can be calculated based on the calculated PD value and the geometric relationship between the PD value and the defocus distance. The lens movement is then controlled according to the defocus distance to achieve focusing.

[0032] Traditional dual-core focusing technology uses dual-core focusing sensors such as Figure 2 and Figure 3 As shown, a pixel consists of two photodiodes, one on the left and one on the right, and a circular microlens. Figure 2 The circle in the middle Figure 3 The semicircle (representing a microlens) covers two diodes. During imaging, the values ​​of the left and right diodes are added together to obtain the light response value of the pixel unit. During focusing, the signal values ​​of the left and right diodes are read out separately to form the left and right images. The defocusing amount and state can be calculated by calculating the phase difference between the two images.

[0033] Therefore, in traditional dual-core focusing technology, since the left and right diodes have the same length in the vertical direction, they can only receive light with different incident angles in the horizontal direction and cannot obtain phase difference information in the vertical direction.

[0034] Based on the above analysis, in order to solve the problem that traditional dual-core focusing technology cannot achieve horizontal and vertical focusing simultaneously without increasing the complexity of the system, this invention proposes a dual-core focusing image sensor based on large and small pixels and its focusing method, which can maintain focusing sensitivity in both directions in low-light environments.

[0035] Example 1 This embodiment provides a dual-core focusing image sensor based on pixel size, such as... Figure 4 The image shown is a top view of a dual-core focusing image sensor based on large and small pixels. One pixel unit 400 of this image sensor includes a small pixel detection unit 401 and a large pixel detection unit 402. The pixel unit 400 is square, and the small pixel detection unit 401 is also square. The area of ​​the small pixel detection unit 401 is smaller than the area of ​​the large pixel detection unit 402, and the side length of the small pixel detection unit 401 is 35% to 50% of the side length of the overall pixel unit.

[0036] In pixel unit 400, the small pixel detection unit consists of a small pixel MOS-C capacitor 403 and a small pixel MOSFET 404; a small pixel shallow trench isolation region 405 is provided between the small pixel MOS-C capacitor 403 and the small pixel MOSFET 404. The large pixel detection unit consists of a large pixel MOS-C capacitor 406 and a large pixel MOSFET 407; a large pixel shallow trench isolation region 408 is provided between the large pixel MOS-C capacitor 406 and the large pixel MOSFET 407. In each pixel unit 400, the small pixel shallow trench isolation region 405 and the large pixel shallow trench isolation region 408 are located at the same horizontal position. A back-side deep trench isolation region 409 is provided between the small pixel detection unit 401 and the large pixel detection unit 402, and between the unit pixels.

[0037] In one implementation, the pixel unit 400 has a size of 2μm*2μm, and the maximum length and width of the small pixel detection unit 401 are half the size of the entire pixel unit 400, i.e., 1μm*1μm. However, the length and width of the small pixel detection unit 401 cannot be infinitely small; the minimum area is 0.7μm*0.7μm. The large pixel detection unit 402 can receive more light in low-light environments, while the small pixel detection unit 401 can prevent excessive light from entering in strong light environments. Therefore, different ratios of the area of ​​the small pixel detection unit 401 to the area of ​​the large pixel detection unit 402 can achieve focusing with a higher dynamic range. The specific area ratio can be set according to actual needs. For the sake of simplicity in subsequent phase difference calculations, the area of ​​the small pixel detection unit 401 can be set to occupy one-quarter of the entire pixel, i.e., the case where the length and width of the small pixel detection unit 401 are at their maximum. Figure 4 As shown.

[0038] This embodiment is illustrated by taking the ratio of the area of ​​the small pixel detection unit 401 to the area of ​​the large pixel detection unit 402 as 1:3. Figure 5 for Figure 4 A cross-sectional view along the 410° direction. (See attached image.) Figure 5 As shown, the bottom layer of the large pixel detection unit 402 is a large pixel substrate 412. Deep trench isolation regions 409 are provided on the left and right sides of the substrate. Above the substrate are a large pixel MOS-C capacitor 406 and a large pixel MOSFET 407. A shallow trench isolation region 408 is provided between the large pixel MOS-C capacitor 406 and the large pixel MOSFET 407. The large pixel MOS-C capacitor 406 and the large pixel MOSFET 407 are connected by an upper floating gate 413. The floating gate 413 is controlled by an upper control gate 414.

[0039] Figure 6 for Figure 4 A cross-sectional view along the 411 direction. (See figure.) Figure 6As shown, the pixel unit in this cross-sectional direction includes both a large pixel detection unit 401 and a small pixel detection unit 402. The bottom layer of the large pixel detection unit 401 is a large pixel substrate 412. The bottom layer of the small pixel detection unit 402 is a small pixel substrate 415. A back-side deep trench isolation region 409 is provided between the large pixel substrate 412 and the small pixel substrate 415. A large pixel MOS-C capacitor 406 is disposed above the large pixel substrate 412 of the large pixel detection unit. A small pixel MOS-C capacitor 403 and a small pixel MOSFET 404 are disposed above the small pixel substrate 415. A shallow trench isolation region 408 is provided between both the small pixel MOS-C capacitor 403 and the small pixel MOSFET 404. A back-side deep trench isolation region 409 is provided between the large pixel MOS-C capacitor 406 and the small pixel MOS-C capacitor 403.

[0040] Figure 7 This is a top view of the dual-core focusing image sensor based on large and small pixels according to the present invention. The overall pixel unit consists of a small pixel detection unit 1 and a large pixel detection unit 2.

[0041] based on Figures 4-7 The dual-core focusing image sensor structure with large and small pixels is described below, along with the focusing method of the dual-core image sensor with large and small pixels according to an embodiment of the present invention. Figure 8 This is a flowchart of a focusing control method for a dual-core focusing image sensor based on large and small pixels provided by the present invention. The method includes the following steps: Step 1: Enter focus mode.

[0042] Control the image sensor to enter focus mode and improve image clarity.

[0043] Step 2: Read the output values ​​of the large pixel detection unit and the small pixel detection unit of the focusing pixel.

[0044] In an embodiment of the present invention, after reading the output value of the large pixel detection unit, focus control can be performed based on its output value and that of the small pixel detection unit.

[0045] Step 3: Calculate the phase difference between the two output signals in the horizontal and vertical directions based on the output values ​​of the large pixel detection unit and the small pixel detection unit, and then obtain the defocus distance in the two directions. In traditional dual-core phase focusing technology, the phase difference is usually calculated from the output values ​​of two horizontally arranged photodiodes to obtain the defocus distance and defocus direction, but it is impossible to obtain focusing information in the vertical direction.

[0046] In the embodiments of the present invention, since the large pixel detection unit and the small pixel detection unit can simultaneously receive light from different directions in both the horizontal and vertical directions, and thus contain defocus information in both the horizontal and vertical directions, the defocus distance in both directions can be obtained by calculating the phase difference between the two output signals in the horizontal and vertical directions.

[0047] Specifically, after obtaining a small pixel image composed of small pixel detection units... Large pixel images composed of large pixel detection units Next, the regions of interest (ROIs) for focusing are defined in both images. Then, feature extraction is performed on both images to enhance image edges without amplifying noise. This is done using a large-pixel image. For example, in the feature extraction process, multi-scale mean interpolation filtering is used for processing:

[0048] Where (x, y) are the coordinates of the image. and These are filter factors at different scales. This is the processed feature image. In this step, by using different filtering factors on the two images, the different amounts of light received by the two images due to their different area ratios, and the resulting different intensity response amplitudes, are transformed to the same scale, possessing only spatial displacement correlation.

[0049] After obtaining two feature images, a two-dimensional Fourier transform is performed on the images to convert them to the frequency domain. During the two-dimensional Fourier transform, transformations are performed in the x and y directions respectively, which allows the separation of phase information in the horizontal and vertical directions. Finally, the cross-correlation of the power spectra of the two spectra is calculated, and the resulting value is the spatial phase difference PD between the two images.

[0050] Step 4: Focus according to the defocus distance in the horizontal and vertical directions; Specifically, the defocus distance has a linear relationship with the phase difference (PD), and this proportionality is related to the lens aperture and focal length. The lens needs to be calibrated beforehand to convert the PD into the required defocus distance.

[0051] Understandably, focusing is for improving the sharpness of the image. Therefore, after focusing, the imaging process needs to be completed. For example... Figure 9 As shown, in addition to the steps described above, the following steps are also included: Control the pixel array to enter imaging mode.

[0052] In an embodiment of the present invention, after focusing control is achieved, the pixel array is further controlled to enter the imaging mode.

[0053] Control the pixel array to reset, perform exposure, and read information values ​​to generate an image.

[0054] In one embodiment of the present invention, the final image signal is obtained by adding the intensity values ​​of the large pixel detection unit and the small pixel detection unit on the same pixel.

[0055] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dual-core focusing image sensor based on large and small pixels, characterized in that, The image sensor consists of multiple pixel units arranged in an array. Each pixel unit includes a small pixel detection unit and a large pixel detection unit, with a deep trench isolation region on the back side between them. The area of ​​the small pixel detection unit is smaller than that of the large pixel detection unit. The small pixel detection unit includes a small pixel MOS-C capacitor and a small pixel MOSFET, with a shallow trench isolation region between them. The large pixel detection unit includes a large pixel MOS-C capacitor and a large pixel MOSFET, with a shallow trench isolation region between them. The shallow trench isolation regions of the small and large pixels are located at the same horizontal level.

2. The image sensor according to claim 1, characterized in that, Both the small pixel detection unit and the pixel unit are square, and the side length of the small pixel unit accounts for 35% to 50% of the side length of the overall pixel unit, and it is located at the lower right corner of the overall pixel unit.

3. The image sensor according to claim 2, characterized in that, The bottom layer of the large pixel detection unit is a large pixel substrate. The large pixel MOS-C capacitor and the large pixel MOSFET are disposed above the large pixel substrate. The large pixel MOS-C capacitor and the large pixel MOSFET are connected through the floating gate above, and the floating gate is controlled by the control gate above.

4. The image sensor according to claim 3, characterized in that, The bottom layer of the small pixel detection unit is a small pixel substrate. The small pixel MOS-C capacitor and the small pixel MOSFET are disposed on the small pixel substrate. The small pixel MOS-C capacitor and the small pixel MOSFET are connected through the floating gate above, and the floating gate is controlled by the control gate above.

5. The image sensor according to claim 4, characterized in that, The floating gate above the large pixel MOS-C capacitor and the floating gate above the small pixel MOS-C capacitor and the small pixel MOSFET are the same floating gate, and the control gate above the floating gate is the same control gate.

6. The image sensor according to claim 5, characterized in that, The pixel units arranged in an array are isolated from each other by a deep trench isolation area on the back side.

7. A method of using the dual-core focusing image sensor based on large and small pixels as described in any one of claims 1-6, characterized in that, During focusing, the method first reads the output values ​​of the large pixel detection unit and the small pixel detection unit, then calculates the phase difference between the two output signals in the horizontal and vertical directions based on the output values ​​of the large pixel detection unit and the small pixel detection unit, and finally obtains the defocus distance in the two directions based on the phase difference in the horizontal and vertical directions, and performs focusing operation based on the defocus distance.

8. The method according to claim 7, characterized in that, After focusing, the method controls the arrayed pixel units to enter the imaging mode, and sequentially performs reset, exposure, and read information values ​​to generate an image. The final image signal is obtained by adding the intensity values ​​of the large pixel detection unit and the small pixel detection unit on the same pixel.

9. The application of the dual-core focusing image sensor based on large and small pixels as described in any one of claims 1-6 in smartphones and cameras.

Images