Image-sensing array with sparse autofocus pixels

Abstract

To provide image sensing arrays, and particularly, arrays including autofocus pixels and methods for their fabrication and use.SOLUTION: An image sensing device includes a semiconductor substrate and a first array of photodetectors disposed on the substrate at a predefined pitch. A color filter layer is disposed over the photodetectors and includes a matrix of red, green and blue regions, each region overlying a respective group of the photodetectors. A second array of microlenses is disposed over the color filter layer and includes first microlenses having a first transverse dimension less than or equal to the pitch disposed respectively over all the photodetectors within the green regions, and second microlenses having a second transverse dimension greater than the pitch disposed over at least some of the photodetectors within the red and blue regions. Each second microlens is configured to focus light onto two or more of the photodetectors in the respective group.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Patent Application No. 63 / 657,871, filed June 9, 2024, which is incorporated herein by reference.

[0002] The present invention relates generally to image sensing arrays, and more particularly to arrays including autofocus pixels and methods for their manufacture and use. [Background technology]

[0003] In many applications, camera systems use autofocus (AF) to ensure that relevant portions of a scene at various distances from the camera are captured in sharp focus. Some autofocus systems use image information output by the camera's image sensor in estimating the optimal distance of the image sensor from the camera lens. On-board electromechanical components then drive the lens position to the optimal distance from the image sensor.

[0004] To improve autofocus performance, some cameras use dual-pixel autofocus, specifically phase-difference-based autofocus, which relies on signals output by special pixels in the image sensing array that are divided into two subpixels. These special pixels can be created, for example, by fabricating a metal shield over certain pixels to obscure half of the sensing area of ​​each such pixel. The phase-difference autofocus logic compares the outputs of the divided subpixels to estimate whether the image is in focus, and thus provides feedback to drive the lens to rapidly converge to a position where the image is in focus.

[0005] U.S. Patent No. 11,563,910, the disclosure of which is incorporated herein by reference, describes an image capture device that includes an array of pixels. Each pixel includes a 2×2 array of photodetectors. The image capture device also includes an array of 1×2 on-chip lenses (OCLs) positioned over the array of pixels to purportedly improve phase detection autofocus (PDAF).

[0006] In this specification and claims, the terms "light" and "optical radiation" are used interchangeably to refer to electromagnetic radiation in any of the visible, infrared, and ultraviolet spectral ranges. Summary of the Invention

[0007] The following embodiments described herein provide improved image sensing arrays and image capture devices.

[0008] Thus, according to one embodiment of the present invention, there is provided an image sensing device including a semiconductor substrate and a first array of photodetectors arranged on the substrate at a predetermined pitch. A color filter layer is disposed over the photodetectors and includes a matrix of red, green, and blue regions, each region overlying a separate group of the photodetectors. A second array of microlenses is disposed over the color filter layer and includes first microlenses having a first lateral dimension less than or equal to the pitch and respectively disposed over all of the photodetectors in the green regions, and second microlenses having a second lateral dimension greater than the pitch and disposed over at least some of the photodetectors in the red and blue regions. Each second microlens is configured to focus light onto two or more of the photodetectors in a separate group.

[0009] In some embodiments, two or more of the photodetectors onto which light is focused by the second microlens define phase detection autofocus (PDAF) pixels.

[0010] According to one embodiment of the present invention, there is also provided an imaging apparatus including the above-described device and an objective optical system configured to image a target onto the device. An autofocus mechanism is configured to adjust focal characteristics of the objective optical system. A controller is configured to drive the autofocus mechanism in response to signals output by the PDAF pixels.

[0011] In one disclosed embodiment, the controller is further configured to reconstruct a full color image based on the signals output by the photodetectors by first calculating a green image by interpolating the photodetector signals in the green region across the red and blue regions, and then calculating red and blue images by interpolating the photodetector signals in the red and blue regions using the green image.

[0012] In some embodiments, the second lateral dimension is twice the pitch. In one disclosed embodiment, at least some of the second microlenses are each configured to focus light onto a respective 1×2 set of photodetectors. In one embodiment, for some of the second microlenses, the respective 1×2 sets of photodetectors are arranged along rows of the array, and for other of the second microlenses, the respective 1×2 sets of photodetectors are arranged along columns of the array.

[0013] Alternatively, each of at least some of the second microlenses is configured to focus light onto a respective 2x2 set of photodetectors.

[0014] Further alternatively or additionally, a second microlens is disposed over all of the photodetectors in the red and blue regions.

[0015] In another embodiment, the second microlens is positioned over only a first subset of the photodetectors in the red and blue regions, and the first microlens is also positioned over a second subset of the photodetectors in the red and blue regions.

[0016] In one disclosed embodiment, the separate groups of photodetectors overlaid by each of the red, green, and blue regions include a 4x4 set of photodetectors.

[0017] In yet another embodiment, in one or more of the red or blue regions, the color filter layer includes a filter over some of the photodetectors of a color other than red or blue.

[0018] According to one embodiment of the present invention, there is additionally provided an imaging method including providing a first array of photodetectors on a substrate with a predetermined pitch and depositing a color filter layer on the photodetectors, the color filter layer including a matrix of red, green, and blue regions, each region overlying a respective group of the photodetectors. A second array of microlenses is formed on the color filter layer, the first microlenses having a first lateral dimension less than or equal to the pitch and disposed over all of the photodetectors in the green regions, and second microlenses having a second lateral dimension greater than the pitch and disposed over at least some of the photodetectors in the red and blue regions. Each second microlens is configured to focus light onto two or more of the photodetectors in a respective group.

[0019] The present invention will be more fully understood from the following detailed description of the embodiments thereof taken in conjunction with the drawings in which: [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a simplified side view of a camera with an image sensing device having sparse autofocus pixels, according to one embodiment of the present invention. [Figure 2] 1 is a partial schematic front view of an image sensing device according to an embodiment of the present invention. [Figure 3] 1 is a partial schematic front view of an image sensing device according to an embodiment of the present invention. [Figure 4A] 1 is a partial schematic front view of an image sensing device according to an embodiment of the present invention. [Figure 4B] 1 is a partial schematic front view of an image sensing device according to an embodiment of the present invention. [Figure 4C] 1 is a partial schematic front view of an image sensing device according to an embodiment of the present invention. [Figure 5] 1 is a partial schematic front view of an image sensing device according to an embodiment of the present invention. [Figure 6] 1 is a partial schematic front view of an image sensing device according to an embodiment of the present invention. [Figure 7] 1 is a partial schematic front view of an image sensing device according to an embodiment of the present invention. [Figure 8] 3 is a flowchart that schematically illustrates a method for processing a mosaic image output by an image sensing device, according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0021] A conventional Bayer color image sensor comprises an array of photodetectors overlaid with a color filter layer, which comprises a matrix of red, green, and blue regions with the same pitch as the photodetector array, i.e., each color region in the matrix covers a single photodetector. For improved resolution and low-light performance (by binning signals from adjacent detectors together), in some image sensors, each colored region in the color filter layer overlies a group of four photodetectors. This type of configuration is described in the aforementioned U.S. Pat. No. 11,563,910. To further improve resolution and performance, image sensors have been proposed in which each colored region in the color filter layer overlies a 4x4 set of photodetectors (meaning a group of 16 photodetectors). This type of sensor allows for both high resolution and high dynamic range using either full resolution (no binning), 2x2 binning, or 4x4 binning, depending on the imaging conditions.

[0022] At the output of the camera, a mosaic of red, green, and blue raw color pixels is interpolated to reconstruct a full-color (RGB) image with red, green, and blue intensity values ​​for each pixel. Typically, the resolution of an RGB image depends primarily on the resolution of the green channel. As in U.S. Patent No. 11,563,910, if PDAF pixels are included in the green regions of the image sensor, a small amount of local blurring may be introduced by on-chip lenses (OCLs), potentially reducing the resolution in these regions.

[0023] The embodiments of the invention described herein ameliorate this problem by incorporating PDAF pixels only in the red and / or blue regions of the photodetector array. In these embodiments, an array of microlenses is disposed on top of the color filter layer of the image sensor. To achieve fine resolution, the microlenses disposed over all photodetectors in the green region have lateral dimensions equal to or less than the pitch of the photodetector array. (This type of microlens is referred to as a "1x1 OCL" because there is one on-chip microlens for each photodetector. However, in the red and / or blue regions, at least some of the microlenses have lateral dimensions greater than the pitch, thus focusing light onto two or more of the photodetectors in a group overlaid by the red or blue color filter. For example, either a 1x2 or 2x2 OCL, with lateral dimensions equal to twice the pitch of the photodetector array, may be used for this purpose in the red and / or blue regions.

[0024] The two or more photodetectors onto which light is focused by each of these latter microlenses define a PDAF pixel whose output signal can be used for focusing a camera in which the image sensing device is installed. All microlenses in the red and / or blue regions may be of this type, or alternatively, some of the photodetectors in the red and / or blue regions may have 1x1 OCLs to increase resolution. In either case, full resolution is available in the green region while maintaining good autofocus performance, regardless of the binning mode that may be applied. Limiting the distribution of PDAF pixels to certain regions of the photodetector array, such as the red and blue regions, is referred to herein as a sparse distribution of PDAF pixels.

[0025] Although the embodiments described herein are specifically based on a color matrix of red, green, and blue regions with a group of 16 photodetectors in each region for clarity and concreteness, the principles of the present invention may be applied to color filter layers containing other colors, as well as to devices with more or fewer photodetectors in each color region. For example, features of the following embodiments may be applied mutatis mutandis to an image sensor of the type described in the aforementioned U.S. Pat. No. 11,563,910, having four photodetectors in each region. As another example, the color filter array in the following embodiments may be modified to include clear (white) or gray regions. All such alternative embodiments are considered to be within the scope of the present invention.

[0026] 1 is a simplified side view of a camera 20 including an image sensing device 22 with sparse PDAF pixels, according to one embodiment of the present invention. Objective optics 24 images a target 26 onto the image sensing device 22. An autofocus mechanism 30 adjusts the focal characteristics of the objective optics 24, for example, by shifting the distance between the optics 24 and the device 22 or by adjusting the focal length of the optics 24, as is known in the art. A controller 28 processes signals output by the PDAF pixels and drives the autofocus mechanism 30 in response to these signals to form a well-focused image on the image sensing device.

[0027] The image sensing device 22 comprises a semiconductor substrate 32, such as a silicon wafer substrate. An array of photodetectors 34, such as silicon photodiodes, is formed on the substrate 32 at a predetermined pitch along with appropriate switching and readout circuitry (not shown). The photodetectors and associated circuitry may be formed using any suitable process of thin-film deposition and photolithography, such as a CMOS process. For example, the circuitry described in the aforementioned U.S. Pat. No. 11,563,910 may be adapted for this purpose. A color filter layer 36 is deposited over the photodetectors 34, and an array of microlenses 38 is disposed over the color filter layer 36. Typically (though not necessarily), the microlenses 38 comprise an optically conductive layer (OCL), which is also formed by material deposition, photolithography, and etching processes as known in the art.

[0028] The following figures illustrate various configurations of color filter layers 36 and microlenses 38 that may be used in accordance with embodiments of the present invention. These figures show only partial views of the color filter matrix and corresponding microlenses because patent drawings cannot practically show the entire image sensing device, which typically comprises many megapixels.

[0029] FIG. 2 is a partial schematic front view of an image sensing device 22 according to one embodiment of the present invention. The color filter layer 40 comprises a matrix of red, green, and blue regions, each overlying a separate 4×4 group of photodetectors 42. (Red, green, and blue are represented by different shading styles in FIG. 2 and subsequent figures, as indicated in the legend above FIG. 2.) The pitch of the array of photodetectors 42 is equal to the distance between adjacent gridlines in this and subsequent figures. In the green region of the color filter layer 40, the microlenses 44 comprise 1×1 OCLs with a lateral dimension (diameter) equal to or slightly smaller than the pitch of the photodetectors 42. In the red and blue regions, the microlenses 46 comprise 1×2 OCLs with a horizontal lateral dimension approximately twice the photodetector pitch. Each microlens 46 focuses light onto a pair of adjacent photodetectors, thereby functioning as a PDAF pixel.

[0030] The distribution of microlenses 44 and 46 in this embodiment enables the image sensing device to produce output images with resolution nearly as fine as that achieved when 1×1 OCLs are used across the entire array of photodetectors, while still achieving good autofocus performance. While the 1×2 OCLs in this embodiment are oriented horizontally along the rows of the photodetector array, in alternative embodiments, the 1×2 OCLs may be oriented vertically along the columns of the array, or a mix of horizontal and vertical OCLs may be used, as shown in subsequent figures. For even finer resolution, only a subset of the photodetectors in the red and / or blue regions of the color filter layer may be configured as PDAF pixels, e.g., with 1×2 or 2×2 OCLs, and the remaining subsets of photodetectors in the red and blue regions have 1×1 OCLs.

[0031] FIG. 3 is a partial schematic front view of an image sensing device 22 according to another embodiment of the present invention. In this embodiment, similar to the embodiment of FIG. 2, the microlenses 44 in the green region of the color filter layer 40 comprise 1×1 OCLs. However, in the red and blue regions, the microlenses 52 comprise 2×2 OCLs, with lateral dimensions in both the horizontal and vertical directions approximately twice the photodetector pitch. Each microlens 52 focuses light onto a 2×2 group of adjacent photodetectors and can thus function as a PDAF pixel. This embodiment may offer better autofocus performance at the expense of slightly reduced resolution.

[0032] 4A, 4B, and 4C are partial schematic front views of the image sensing device 22 illustrating other possible distributions of 1×2 microlenses according to alternative embodiments of the present invention. In the embodiment of FIG. 4A, the microlenses 54 in the red and blue regions comprise 1×2 OCLs oriented vertically along the columns of the photodetector array. In FIG. 4B, the red and blue regions are covered by a mixture of microlenses 46 and 54. The microlenses 46 are oriented along the rows of the photodetector array and focus light onto corresponding pairs of adjacent PDAF photodetectors arranged along the rows. Meanwhile, the microlenses 54 are oriented along the columns so that corresponding pairs of adjacent PDAF photodetectors are arranged along the columns. In FIG. 4C, the red and blue regions are partially covered by the microlenses 44 comprising 1×1 OCLs for high resolution and partially covered by the microlenses 46 comprising 1×2 OCLs for PDAF detection.

[0033] 5 is a partial schematic front view of image sensing device 22 according to yet another embodiment of the present invention, in which the red and blue regions are partially covered by microlenses 44 with 1×1 OCLs for high resolution and partially covered by microlenses 52 with 2×2 OCLs for PDAF detection.

[0034] 6 and 7 are partial schematic front views of an image sensing device 22 showing yet another possible configuration of color filters and microlenses according to alternative embodiments of the present invention. In the embodiment of FIG. 6, the red and blue regions of the color filter layer are covered by 1×2 OCLs, similar to the embodiment of FIG. 2. Meanwhile, in the embodiment of FIG. 7, the red and blue regions are covered by 2×2 OCLs, similar to the embodiment of FIG. 3. However, in some of the red regions, the color filter layer includes filters 60 or 62 of a color other than red on some of the photodetectors located below the 1×2 or 2×2 OCLs. Alternatively or additionally, the blue regions may include filters of a different color.

[0035] Filters 60 and 62 may be a variety of different colors depending on the requirements of the application. For example, filters 60 and 62 may be green to improve resolution of the green channel. Alternatively, filters 60 and 62 may be clear or gray (neutral density) to improve both the resolution and spectral sensing range of the device. A metal shield or grid can be formed over the red or blue photodetectors in these groups to provide PDAF information. This type of shield or grid is particularly useful for preserving PDAF information when the photodetector outputs in the arrays of FIGS. 6 and 7 are aggregated in 4×4 groups.

[0036] 8 is a flow chart that schematically illustrates a method for processing a mosaic image output by image sensing device 22, according to one embodiment of the present invention. Controller 28 (FIG. 1) applies this method to reconstruct a full-color RGB image based on signals output by photodetectors 34 in the red, green, and blue regions shown in the preceding figures, which are captured as a mosaic input image.

[0037] In the red and blue regions, at least some of the photodetectors share a common microlens for autofocus. Controller 28 aggregates the signals output by each group of two or four photodetectors to calculate extended pixel values. The controller calculates a high-resolution green image by interpolating the signals of the photodetectors in the green region across the red and blue regions. The controller then calculates red and blue images by interpolating the pixel values ​​of the photodetectors in the red and blue regions (including the aggregated autofocus group) using the green image to provide missing detail. The controller then combines the green, red, and blue images to provide a full RGB output.

[0038] In one alternative embodiment, a neural network can be trained to convert raw mosaic input to full RGB output without explicitly performing the interpolation steps described above.

[0039] While the above-described embodiments involve particular types and distributions of microlenses over the red and blue regions of an image sensing device, the principles of these embodiments can be applied to create other microlens patterns in accordance with the principles of the present invention. Accordingly, it will be understood that the above-described embodiments are given by way of example, and that the present invention is not limited to those specifically shown and described herein. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described herein, as well as variations and modifications thereof not disclosed in the prior art, which would occur to one skilled in the art upon reading the foregoing description.

Claims

1. 1. An image sensing device, comprising: a semiconductor substrate; a first array of photodetectors arranged on the substrate at a predetermined pitch; a color filter layer disposed over the photodetectors, the color filter layer comprising a matrix of red, green, and blue regions, each region overlying a respective group of the photodetectors; a second array of microlenses disposed over the color filter layer, first microlenses each disposed across all of the photodetectors in the green region and having a first lateral dimension less than or equal to the pitch; a second array of microlenses comprising: second microlenses having a second lateral dimension greater than the pitch disposed over at least some of the photodetectors in the red and blue regions, the second microlenses configured to focus light onto two or more of the photodetectors in the respective groups; A device comprising:

2. 10. The device of claim 1, wherein the two or more of the photodetectors onto which the light is focused by the second microlens define phase detection autofocus (PDAF) pixels.

3. 1. An imaging apparatus comprising: A device according to claim 2; an objective configured to image a target onto the device; an autofocus mechanism configured to adjust the focal characteristics of the objective optical system; a controller configured to drive the autofocus mechanism in response to signals output by the PDAF pixels.

4. 4. The apparatus of claim 3, wherein the controller is further configured to reconstruct a full-color image based on the signals output by the photodetectors by first calculating a green image by interpolating the signals of the photodetectors in the green region across the red and blue regions, and then calculating red and blue images by interpolating the signals of the photodetectors in the red and blue regions using the green image.

5. The device of claim 1 , wherein the second lateral dimension is twice the pitch.

6. The device of claim 5 , wherein each of at least some of the second microlenses is configured to focus the light onto a respective 1×2 set of the photodetectors.

7. 7. The device of claim 6, wherein for some of the second microlenses, the individual 1x2 sets of photodetectors are arranged along rows of the array, and for other of the second microlenses, the individual 1x2 sets of photodetectors are arranged along columns of the array.

8. The device of claim 5 , wherein each of at least some of the second microlenses is configured to focus the light onto a respective 2×2 set of the photodetectors.

9. The device of claim 1 , wherein the second microlens is disposed over all of the photodetectors in the red and blue regions.

10. 10. The device of claim 1, wherein the second microlens is disposed over only a first subset of the photodetectors in the red and blue regions, and the first microlens is also disposed over a second subset of the photodetectors in the red and blue regions.

11. 2. The device of claim 1, wherein the individual groups of photodetectors overlaid by each of the red, green, and blue regions comprise a 4x4 set of the photodetectors.

12. 10. The device of claim 1, wherein in one or more of the red or blue regions, the color filter layer comprises a filter over some of the photodetectors of a color other than red or blue.

13. 1. A method for imaging, comprising: providing a first array of photodetectors on a substrate at a predetermined pitch; depositing a color filter layer over the photodetectors, the color filter layer comprising a matrix of red, green, and blue regions, each region overlying a respective group of the photodetectors; On the color filter layer, first microlenses each disposed over all of the photodetectors in the green region and having a first lateral dimension less than or equal to the pitch; a second microlens having a second lateral dimension greater than the pitch disposed over at least some of the photodetectors in the red and blue regions, the second microlens configured to focus light onto two or more of the photodetectors in the respective groups; and forming a second array of microlenses comprising: device.

14. 14. The method of claim 13, wherein the two or more of the photodetectors onto which the light is focused by the second microlens define phase detection autofocus (PDAF) pixels.

15. imaging a target onto the photodetectors in the first array using objective optics; and driving an autofocus mechanism to adjust a focus characteristic of the objective optical system in response to the signal output by the PDAF pixel.

15. The method of claim 14.

16. 16. The method of claim 15, comprising reconstructing a full color image based on signals output by the photodetectors by first calculating a green image by interpolating the signals of the photodetectors in the green region across the red and blue regions, and then calculating red and blue images by interpolating the signals of the photodetectors in the red and blue regions using the green image.

17. The method of claim 13 , wherein the second lateral dimension is twice the pitch.

18. The method of claim 13 , wherein the second microlens is disposed over all of the photodetectors in the red and blue regions.

19. 14. The method of claim 13, wherein the second microlens is disposed over only a first subset of the photodetectors in the red and blue regions, and the first microlens is also disposed over a second subset of the photodetectors in the red and blue regions.

20. 14. The method of claim 13, wherein in one or more of the red or blue regions, the color filter layer comprises a filter over some of the photodetectors of a color other than red or blue.

Images