Polarization classification metasurface microlens array device

Abstract

A system and method comprising a polarization sorting metasurface microlens array device are disclosed herein. In a specific embodiment, a polarization imaging device is provided. The polarization imaging device comprises a metasurface lenslet array comprising a plurality of repeating metasurface lenslets, wherein the plurality of repeating metasurface lenslets comprises a plurality of first metasurface lenslets configured to diffract image light into a first polarized light of a first direction and a second polarized light of a second direction; and an image sensor positioned in the optical path of the first polarized light and the second polarized light, wherein the image sensor comprises a plurality of image sensing units, wherein the image sensor comprises a first image sensing unit positioned to detect the first polarized light and a second image sensing unit positioned to detect the second polarized light.

Description

Technology Field

[0001] Cross-reference for related applications

[0002] The present invention claims the benefit and priority under 35 USC § 119(e) to U.S. provisional patent application No. 63 / 362,285, filed on March 31, 2022, with the title of the invention “Polarization Sorting Metasurface Microlens Array Device,” the disclosure of which is incorporated herein by reference in its entirety for all purposes.

[0003] The present invention relates to a polarization sorting device. In particular, the present invention relates to a polarization sorting device comprising a polarization sorting metasurface microlens array and a method for manufacturing the polarization sorting device. Background Technology

[0004] Metasurface elements are diffractive optical elements that typically have a planar profile and each waveguide element has a sub-wavelength gap. Metasurface elements have recently been developed for application in the UV-IR band (300–10,000 nm). Compared to conventional refractive optics, metasurface elements introduce abrupt phase shifts into the light field. This allows metasurface elements to have a thickness at the wavelength level of the light for which they are designed to operate, whereas conventional refractive surfaces have a thickness 10 to 100 times (or more) greater than the wavelength of the light for which they are designed to operate. Additionally, metasurface elements may not have variations in the height of their components and thus can shape light without any curvature, as required for refractive optics. Compared to conventional diffractive optical elements (DOEs), such as binary diffractive optics, metasurface elements have the ability to impart a phase shift of a certain range to the incident light field. At the smallest scale, metasurface elements can have phase shifts between 0 and 2π and at least five discrete values ​​in that range, whereas binary DOEs can only provide two discrete values ​​of phase shift and are often limited to phase shifts of 0 or 1π. Compared to multilevel DOEs, metasurface elements do not require height variation along the optical axes of their components, and only the in-plane geometry of the metasurface elements changes. means of solving the problem

[0005] In some embodiments, the technology described herein relates to a polarization imaging device comprising: a metasurface lenslet array comprising a plurality of metasurface lenslets, wherein the plurality of metasurface lenslets comprises a plurality of first metasurface lenslets configured to diffract image light with an intensity proportional to a first polarized light in a first direction and an intensity proportionality with a second polarized light in a second direction; and an image sensor positioned in the optical paths of the first polarization and the second polarization, wherein the image sensor comprises a plurality of image sensing units, each comprising a first image sensing unit positioned to detect the first polarization and a second image sensing unit positioned to detect the second polarization.

[0006] In some embodiments, a plurality of image sensing units are different pixels and / or zones of an image sensor.

[0007] In some embodiments, a plurality of image sensing units are different pixels and / or zones of a plurality of image sensors.

[0008] In some embodiments, the plurality of metasurface lenslets further include a plurality of second metasurface lenslets interspersed among the first metasurface lenslets and configured to diffract the image light with an intensity proportional to a third polarization in a third direction and an intensity proportional to a fourth polarization in a fourth direction, and the plurality of image sensing units further include a third image sensing unit positioned to detect the third polarization and a fourth image sensing unit positioned to detect the fourth polarization.

[0009] In some embodiments, the first polarization and the third polarization are the same polarization, and the second polarization and the fourth polarization are the same polarization.

[0010] In some embodiments, the first metasurface lenslet is further configured to diffract the image light with an intensity proportional to a third polarization in a third direction and an intensity proportional to a fourth polarization in a fourth direction, and the plurality of image sensing units further include a third image sensing unit positioned to detect the third polarization and a fourth image sensing unit positioned to detect the fourth polarization.

[0011] In some embodiments, the first metasurface lenslet is further configured to transmit zero-order light in a third direction.

[0012] In some embodiments, the third direction is different from the first direction and the second direction.

[0013] In some embodiments, the plurality of image sensing units further include a third image sensing unit configured to detect the zero-order light.

[0014] In some cases, the third direction is the same direction as the first direction.

[0015] In some embodiments, the polarization imaging device further comprises one or more refractive lenses, and the metasurface lenslet array is positioned between the refractive lens and the image sensor.

[0016] In some embodiments, the first polarization and the second polarization are different polarizations, and

[0017] The first polarization and the second polarization are selected from the group consisting of linear polarization, diagonal polarization, elliptical polarization, and circular polarization.

[0018] In some embodiments, the polarization imaging device further comprises a microlens array, and the metasurface lenslet array comprises a polarization metasurface.

[0019] In some embodiments, the microlens array is configured to separate the image light into different pixels, and the polarizing metasurface is configured to diffract the first polarization in the first direction and the second polarization light in the second direction.

[0020] In some embodiments, the polarizing metasurface overlaps with a microlens in the microlens array, and a microlens not overlapped by the polarizing metasurface in the microlens array transmits undiffracted light to the image sensor, and the image sensor is configured to detect the undiffracted light and measure the intensity of the undiffracted light.

[0021] In some embodiments, the microlens array is a planar microlens array layer.

[0022] In some embodiments, the microlens array and the polarization metasurface are located on a single substrate.

[0023] In some embodiments, the primary image plane is configured on the surface of the single substrate opposite to the image sensor.

[0024] In some embodiments, the main image plane is configured within a single substrate.

[0025] In some embodiments, the microlens array is a metasurface element.

[0026] In some embodiments, the metasurface element is configured to accommodate a chief ray angle that varies across the imaging pupil of the polarization imaging device and to collimate light passing through each microlens.

[0027] In some embodiments, the metasurface elements of the microlens array are configured to provide a refractive microlens effect.

[0028] In some embodiments, the microlens array is a conventional refractive microlens array.

[0029] In some embodiments, the microlens array comprises a combination of at least one conventional refractive microlens and at least one metasurface element.

[0030] In some embodiments, the polarization imaging device further includes a color filter positioned above or below the metasurface lenslet array.

[0031] In some embodiments, the color filter is positioned over the metasurface lenslet array and filters light into different colors corresponding to each of the different metasurface lenslets of the plurality of first metasurface lenslets, and the metasurface lenslets receive the color-filtered light from the color filter.

[0032] In some embodiments, the color filter is positioned below the metasurface lenslet array and filters the diffracted first polarized light and second polarized light.

[0033] In some embodiments, the color filter includes different zones for filtering different wavelengths.

[0034] In some embodiments, a plurality of first metasurface lenslets are spaced apart on the cover substrate.

[0035] In some embodiments, the spaced first metasurface lenslet is configured to output light onto a zone of a color filter that outputs the same color of light.

[0036] In some embodiments, the spaced-apart first metasurface lenslet outputs light on the gap between the zone of the color filter that outputs green light and the zones of the color filter that output green light and red or blue light.

[0037] In some embodiments, the color filter includes different zones that output red, green, or blue light and zones that output black and white or near-infrared light, and the plurality of first metasurface lenslets are spaced apart on a cover substrate, and the spaced first metasurface lenslets output light on the zones of the color filter that output black and white or near-infrared light.

[0038] In some embodiments, the polarization imaging device further comprises a microlens array including a plurality of distinct microlenses that output collimated light into the metasurface lenslet array.

[0039] In some embodiments, the plurality of separate microlenses are positioned between the adjacent image sensing units such that light from the plurality of separate microlenses is diffracted into the adjacent image sensing units in opposite oblique directions by the plurality of first metasurface lenslets.

[0040] In some embodiments, the plurality of separate microlenses are positioned between the adjacent image sensing units such that light from the plurality of separate microlenses is diffracted into the adjacent image sensing units in opposite oblique directions by the plurality of first metasurface lenslets.

[0041] In some embodiments, the plurality of separated microlenses are each centered on one of the plurality of first metasurface lenslets so that the first polarization passes through one of the plurality of first metasurface lenslets and is transmitted to the first image sensing unit, and the second polarization is diffracted on the second image sensing unit in the second direction.

[0042] In some embodiments, the first polarization and the second polarization are orthogonal linear polarization, circular polarization, elliptical polarization, or any polarization.

[0043] In some embodiments, a plurality of first metasurface lenslets are spaced apart with a non-diffractive section between adjacent first metasurface lenslets.

[0044] In some embodiments, the plurality of image sensing units configured to detect the first polarization and the second polarization of the plurality of metasurface lenslets are positioned between the image sensing units of the image sensor configured to detect red, green, or blue light.

[0045] In some embodiments, the plurality of first metasurface lenslets are configured to diffract light into the plurality of image sensing units configured to detect the first polarization and the second polarization and the image sensing unit configured to detect red, green, or blue light.

[0046] In some embodiments, the plurality of image sensing units configured to detect the first polarization and the second polarization are further configured to detect black and white or near-infrared light.

[0047] In some embodiments, the image sensor is a complementary metal-oxide semiconductor (CMOS) image sensor, a charge-coupled element sensor, a silicon diode sensor, a cadmium-sulfide sensor diode, an RGB sensor, an indirect time of flight (iToF) sensor, or a direct time of flight (dToF) sensor.

[0048] In some embodiments, the technique described herein relates to a method for manufacturing a polarization imaging device, comprising the steps of: providing an image sensor wafer; depositing a spacing layer on the imaging sensor wafer; depositing a metasurface layer on the spacing layer; and patterning the metasurface layer to form individual metasurfaces on the spacing layer.

[0049] In some embodiments, the method further includes the step of directly depositing an encapsulation layer on the individual metasurface.

[0050] In some embodiments, the method further includes the step of flattening the encapsulation layer so that the encapsulation layer is at the same level as the individual metasurface.

[0051] In some embodiments, the method further comprises the step of forming one or more top layers on the encapsulation layer and / or the individual metasurface.

[0052] In some embodiments, the technique described herein relates to a method for manufacturing a polarization imaging device, comprising the steps of: providing an image sensor wafer; providing a metasurface substrate comprising individual metasurfaces; and depositing a spacing layer on the imaging sensor wafer and / or the individual metasurfaces on the metasurface substrate.

[0053] In some embodiments, the method further includes the step of bonding the metasurface substrate to the image sensor wafer.

[0054] In some embodiments, the metasurface substrate includes a transparent substrate.

[0055] In some embodiments, the method further includes the step of removing the metasurface substrate so that the individual metasurface remains attached to the image sensor wafer.

[0056] In some embodiments, the step of removing the metasurface substrate includes polishing, lapping, wet chemical etching, and / or dry chemical etching the metasurface substrate.

[0057] In some embodiments, the method further comprises the steps of singulating the metasurface substrate into separate metasurface dies, coupling at least one of the metasurface dies onto the image sensor wafer, and singulating the image sensor wafer into image sensor dies each having the coupled metasurface dies.

[0058] In some embodiments, the technique described herein is a method for manufacturing a polarization imaging device, comprising the steps of: providing a CMOS image sensor (CIS) wafer; positioning a first plurality of spacers on the CIS wafer; providing a first carrier wafer; positioning a first adhesive layer on the first carrier wafer; bringing the first plurality of spacers into contact with the first adhesive layer such that an adhesive is applied to the top of the first plurality of spacers; providing a nanopillar substrate having a plurality of nanopillars; positioning a second plurality of spacers on the nanopillar substrate; providing a second carrier wafer; positioning a second adhesive layer on the second carrier wafer; bringing the second plurality of spacers into contact with the second adhesive layer such that an adhesive is applied to the top of the second plurality of spacers; The present invention relates to a method for manufacturing a polarization imaging device, comprising the step of simultaneously contacting an adhesive on a second plurality of spacers with the CIS wafer and an adhesive on a first plurality of spacers with the nanofilament substrate so as to bond the CIS wafer and the nanofilament substrate to each other.

[0059] In some embodiments, after the first plurality of spacers are brought into contact with the first adhesive layer so that the adhesive is applied to the top portion of the first plurality of spacers, the adhesive is present only on the top portion of the first plurality of spacers and is not present in the area between the first plurality of spacers.

[0060] In some embodiments, after the second plurality of spacers are brought into contact with the second adhesive layer so that the adhesive is applied to the upper portion of the second plurality of spacers, the adhesive is present only on the upper portion of the second plurality of spacers and is not present on the area between the second plurality of spacers.

[0061] In some embodiments, the first plurality of spacers includes a pair of spacers forming a channel whose size is determined to accommodate one of the second plurality of spacers.

[0062] In some embodiments, the first plurality of spacers and the second plurality of spacers are interlocked with each other so that one of the second plurality of spacers is located within one of the channels formed by the pair of spacers.

[0063] In some embodiments, the first plurality of spacers form a plurality of rectangular shapes on the CIS wafer.

[0064] In some embodiments, the interior of the rectangular shape forms an imaging zone.

[0065] In some embodiments, the second plurality of spacers form a grid pattern on the nanopillar substrate.

[0066] In some embodiments, the first plurality of spacers and the second plurality of spacers form a void between the plurality of nanopillars and the CIS wafer.

[0067] In some embodiments, the technique described herein relates to a method for manufacturing a polarization imaging device, comprising the steps of: providing a CMOS image sensor (CIS) wafer having bonding pads; depositing a first dielectric layer on the CIS wafer and the bonding pads; flattening the first dielectric layer; providing a nanofilament substrate having a plurality of nanofilaments; providing a second dielectric layer on the plurality of nanofilaments; flattening the second dielectric layer; bringing the first dielectric layer and the second dielectric layer into contact so as to bond them together to form a bonded dielectric layer; removing the nanofilament substrate to expose the plurality of nanofilaments; partially etching the bonded dielectric layer to expose the bonding pads; and forming a conductive layer electrically connected to the bonding pads through the bonded dielectric layer.

[0068] In some embodiments, the first dielectric layer and the second dielectric layer are silicon dioxide layers.

[0069] In some embodiments, the step of depositing the first dielectric layer and the second dielectric layer is performed by a TEOS (tetraethylorthosilicate) process.

[0070] In some embodiments, the TEOS process is a plasma enhanced TEOS process.

[0071] In some embodiments, the step of flattening the first dielectric layer and the second dielectric layer is performed by a chemical mechanical polishing (CMP) process.

[0072] In some embodiments, the step of removing the nanopillar substrate is performed by a polishing, etching, or chemical mechanical polishing (CMP) process.

[0073] In some embodiments, the step of partially etching the bonded dielectric layer comprises patterning the bonded dielectric layer; and etching the bonded dielectric layer to expose the bonding pad.

[0074] In some embodiments, the method further comprises the step of growing a barrier seed layer on the bonding pad, the sidewall of the bonded dielectric layer, and the nanopillar.

[0075] In some embodiments, the technique described herein relates to a polarization imaging device comprising: a microlens array having at least two microlenses; a polarization filtering metasurface having two or more polarization filtering zones; and an imaging sensor having at least two zones, wherein imaging light having one or more polarization states is directed by the microlenses onto the polarization filtering metasurface, and the polarization filtering metasurface is configured to direct one or more polarization states onto one or more zones of the imaging sensor.

[0076] In some embodiments, the microlens array includes refractive microlenses.

[0077] In some embodiments, the microlens array further includes a metasurface configured to provide a refractive microlens effect.

[0078] In some embodiments, the microlens array includes a metasurface configured to provide a refractive microlens effect.

[0079] In some embodiments, each of the at least two microlenses is located over a single zone among one or more zones of the imaging sensor.

[0080] In some embodiments, each of the at least two microlenses is positioned over two or more sensor zones among one or more zones of the imaging sensor.

[0081] In some embodiments, each of at least two microlenses is located on a single polarization zone among one or more of two or more polarization filtering zones.

[0082] In some embodiments, each of the at least two microlenses is positioned over two or more sensor zones among the two or more polarization filtering zones. Brief explanation of the drawing

[0083] The detailed description will be more fully understood with reference to the following drawings, which are provided as exemplary embodiments of the invention and should not be understood as fully stating the scope of the invention. Figure 1 illustrates a schematic diagram of a conventional polarization imaging system as an example. Figure 2 illustrates a schematic diagram of a polarization imaging system as an example. FIGS. 3a and 3b illustrate schematic diagrams of a Polarization Sorting Metasurface Microlens Array (PSOMMA) according to an embodiment of the present invention. FIG. 3ca conceptually illustrates an optical path for a single-layer metasurface utilizing a blazed grating approach according to an embodiment of the present invention. FIG. 3cb conceptually illustrates the optical path for a single-layer metasurface that serves as a PSOMMA and a collimating lens according to an embodiment of the present invention. FIG. 3cc conceptually illustrates an optical path for two distinct layers of a metasurface that serves as a PSOMMA and a collimating lens according to an embodiment of the present invention. FIG. 4a illustrates a PSOMMA having two polarizations in each lenslet according to an embodiment of the present invention. FIG. 4b illustrates a PSOMMA having two polarizations in each lenslet according to an embodiment of the present invention. FIG. 5aa illustrates an optical system including a PSOMMA according to an embodiment of the present invention. FIGS. 5ab and FIGS. 5ac illustrate examples of optical systems including a PSOMMA according to various embodiments of the present invention. FIG. 5b illustrates an optical system including a PSOMMA according to an embodiment of the present invention. FIGS. 5ca, FIGS. 5cb, and FIGS. 5cc illustrate an optical system including a PSOMMA according to various embodiments of the present invention. FIGS. 6a and 6b illustrate an exemplary polarization distribution in an image sensor according to an embodiment of the present invention. FIG. 7 schematically illustrates an example of the function of an exemplary polarization imaging system including an MLA integrated with a PSOMMA according to an embodiment of the present invention. FIGS. 8a and FIGS. 8b illustrate various offset pixel architectures according to various embodiments of the present invention. FIGS. 9a and 9b illustrate various high-fidelity pixel architectures according to embodiments of the present invention. FIGS. 10a and FIGS. 10b illustrate various hybrid intensity / polarization architectures according to embodiments of the present invention. FIGS. 11a to 11c illustrate various hybrid architectures including MLA and PSOMMA according to various embodiments of the present invention. FIGS. 12a to 12g illustrate various steps of a direct manufacturing method according to an embodiment of the present invention. FIGS. 13a to 13d illustrate various steps of a wafer bonding method according to an embodiment of the present invention. FIG. 14a illustrates a plurality of second wafer dies coupled to an image sensor wafer according to an embodiment of the present invention. FIG. 14 illustrates an image sensor die (1404) after the image sensor wafer (1302) has been singulated, according to an embodiment of the present invention. FIG. 15 illustrates an exemplary PSOMMA sensor (1500) manufactured according to an embodiment of the present invention. FIGS. 16a to 16c illustrate an exemplary method for producing a CIS wafer (1502) according to an embodiment of the present invention. FIG. 17 illustrates a plan view of a CIS wafer (1602) according to an embodiment of the present invention. FIGS. 18a to 18c illustrate an exemplary method for producing a nanopillar wafer according to an embodiment of the present invention. FIG. 19 illustrates a plan view of a nanopillar substrate according to an embodiment of the present invention. FIGS. 20a and FIGS. 20b illustrate an exemplary method for fabricating a PSOMMA sensor according to an embodiment of the present invention. FIGS. 21a and FIGS. 21b illustrate a process flow for manufacturing a CIS wafer according to an embodiment of the present invention. FIGS. 22a and FIGS. 22b illustrate a process flow for fabricating a nanopillar wafer according to an embodiment of the present invention. FIGS. 23a to 23h is a fabrication process for fabricating an imaging sensor using a CIS wafer fabricated using the process described in FIGS. 21a and 21b and a nanofilament wafer fabricated using the process described in FIGS. 22a and 22b, according to an embodiment of the present invention. FIGS. 24a to 24c is a fabrication process for fabricating an imaging sensor using a CIS wafer fabricated using the process described in FIGS. 21a and 21b and a nanofilament wafer fabricated using the process described in FIGS. 22a and 22b, according to an embodiment of the present invention. FIGS. 25a to 25h are manufacturing processes for manufacturing an imaging sensor according to an embodiment of the present invention. FIG. 26 illustrates an exemplary operation of an exemplary metasurface microlens array (MLA) according to an embodiment of the present invention. FIG. 27a illustrates an exemplary pixel array for four polarized pixels according to an embodiment of the present invention. FIG. 27b illustrates an exemplary pixel array for two polarized pixels according to an embodiment of the present invention. FIG. 27c illustrates an exemplary pixel array for four polarized pixels according to an embodiment of the present invention. FIG. 28a is a cross-sectional view of an imaging device including a polarization separation function according to an embodiment of the present invention. FIG. 28b is a plan view of an imaging device including a polarization separation function according to an embodiment of the present invention. FIG. 28c is a plan view of an imaging device including a polarization separation function according to an embodiment of the present invention. FIG. 29a is a cross-sectional view of an imaging device including a polarization separation function according to an embodiment of the present invention. FIG. 29b is a plan view of an imaging device including a polarization separation function according to an embodiment of the present invention. FIG. 29c is a plan view of an imaging device including a polarization separation function according to an embodiment of the present invention. FIG. 30a illustrates a cross-sectional view of an imaging device including a polarization separation function according to an embodiment of the present invention. FIG. 30b illustrates a plan view of an imaging device including a polarization separation function according to an embodiment of the present invention. FIG. 30c is a plan view of an imaging device including a polarization separation function according to an embodiment of the present invention. FIG. 31a illustrates a cross-sectional view of an imaging device including a polarization filtering function according to an embodiment of the present invention. FIG. 31b illustrates a plan view of an imaging device including a polarization filtering function according to an embodiment of the present invention. FIG. 32a illustrates a cross-sectional view of an imaging device including a polarization filtering function according to an embodiment of the present invention. FIG. 32b illustrates a cross-sectional view of an imaging device including a polarization classification function according to an embodiment of the present invention. FIG. 32c illustrates a cross-sectional view of an imaging device including a polarization classification function according to an embodiment of the present invention. FIG. 33a illustrates a plan view of an exemplary polarization filtering nanograting according to an embodiment of the present invention. FIG. 33b illustrates a plan view of an exemplary polarization classification metasurface according to an embodiment of the present invention. FIG. 34a illustrates a schematic plan view of an exemplary linear polarization classification metasurface according to an embodiment of the present invention. FIG. 34b illustrates a schematic plan view of an exemplary circular polarization classification metasurface according to an embodiment of the present invention. FIG. 34c illustrates a schematic plan view of a random polarization classification metasurface according to an embodiment of the present invention. FIG. 35a illustrates a sparsely distributed PSOMMA sensor according to an embodiment of the present invention. FIG. 35b illustrates a cross-sectional view of a sparsely distributed PSOMMA sensor according to an embodiment of the present invention. FIG. 36a illustrates an exemplary PSOMMA sensor according to an embodiment of the present invention. FIG. 36b illustrates an exemplary PSOMMA sensor according to an embodiment of the present invention. Specific details for implementing the invention

[0084] An optical device comprising individual lenslets that classify polarization so that polarization information of the resulting scene can be reconstructed is disclosed herein. Specifically, the method described herein comprises an optical device comprising a metasurface microlens array comprising a set of lenses that can be coupled to an image sensor. The phase and polarization splitting at each lenslet can be uniquely defined. An individual lenslet can split incoming light into at least two polarizations directed toward a specific underlying pixel of the imaging sensor. Regardless of polarization control at each lenslet, the phase of each lenslet can also be uniquely defined. The phase response of each lenslet can be customized to accommodate a specific angle of incidence toward the lenslet. In some embodiments, the polarization-classifying metasurface microlens array may be customized for a specific optical system and may be directly integrated with the image sensor. In some embodiments, the image sensor may include a CMOS image sensor (CIS), a charge coupling element (CCD) sensor, a silicon diode, a cadmium-sulfide sensor diode, an RGB sensor, an indirect time-of-flight (iToF) sensor, a direct time-of-flight (dToF) sensor, and / or various other types of sensors.

[0085] A polarization classification metasurface can generate a sparse effect by including supercell or superpixel repeating structures and / or unpolarized open spaces between them.

[0086] Most conventional polarization imaging systems operate by absorbing specific polarization states, allowing unabsorbed states to pass through and reach the light-sensing sensor. By determining which polarization states are absorbed and which pass through, a polarization image of the scene can be constructed. However, this design can be inefficient because the absorbed light, which provides valuable polarization information, is bypassed and dissipated as heat rather than detected. Such designs may result in poor efficiency or signal-to-noise ratio (SNR). While this structure still preserves polarization information, there is a loss of efficiency due to the absorption of light. Additionally, common techniques for implementing this type of polarization measurement (e.g., wire grid polarizers) can only select linear polarization states and may therefore be unable to access elliptical polarization states.

[0087] FIG. 1 illustrates a schematic diagram of a conventional polarization imaging system as an example. As illustrated, incident light (106) may contain multiple polarizations, such as S polarization and P polarization. The incident light passes through a polarizer (102), which may allow only one polarization of light to pass through. In the case of incident light (106) containing S polarization and P polarization, the polarizer (102) may allow only the S polarization light (106a) to pass through and absorb the P polarization light. The amount of S polarization light (106a) may be measured by a light detection sensor (104), which may provide data related to the amount of S polarization light (106a). Adversely, the P polarization light absorbed by the polarizer (102) may produce unwanted byproducts, such as heat or scattered light from reflection.

[0088] The present invention relates to a metasurface polarization imaging system that spatially divides polarization by transmitting each of the designed polarizations at a specific diffraction angle, in contrast to conventional polarizers. Examples of metasurface optical elements comprising geometric structures and orientations that deflect light in a direction dependent on the polarization of the light are disclosed in U.S. Patent Application Publication No. 2021 / 0286188, filed February 22, 2019, with the title "Arbitrary polarization-switchable metasurfaces," which is incorporated herein by reference in its entirety. Additionally, examples of gratings configured to perform parallel polarization analysis of multiple polarization orders of incident light of arbitrary polarization are disclosed in International Publication No. WO 2020214615, filed April 14, 2020, with the title "System and method for parallel polarization analysis," which is incorporated herein by reference in its entirety. Furthermore, examples of polarimeters comprising elements of a metasurface that provide different responses based on the polarization of incident light are disclosed in U.S. Patent Application Publication No. 2021 / 0048569, filed July 22, 2020, with the title of the invention “Polarization state generation with a metasurface,” which is incorporated herein by reference in its entirety. All designs contained in these references may be incorporated herein by reference to split incident light into different polarizations directed in different directions.

[0089] In such a system, since light is not absorbed but instead diffracted in different directions so that different polarizations of light are detected by different image sensors, the efficiency can be twice that of a conventional polarization grating. However, in the case of such a system, the metasurface element divides the aperture of the imaging system, and consequently, an image having a specific polarization is formed within a separated region of the image sensor. For example, if the metasurface is designed to divide incident light into four polarization states, such a metasurface imaging system can form four distinct images of a single object, each of which is formed within a distinct spatial domain (e.g., a quadrant) of the image sensor, and a specific polarization state is associated with each image. FIG. 2 illustrates a schematic diagram of an exemplary polarization imaging system. The polarization imaging system includes a metasurface element (202) that divides incident light (206) into different polarizations. As illustrated, the incident light (206) may include a single mixed polarization state. The incident light (206) may be completely unpolarized, in a single pure polarized state, or in a partially mixed state. The incident light (206) may include four polarizations (Pol1, Pol2, Pol3, Pol4). Mathematically speaking, the incident light is represented by a Stokes vector S. Each outgoing diffraction direction may be associated with a different Stokes vector Mi, which may correspond to a row of the Müller matrix. Each outgoing direction corresponds to the Müller matrix U i It can be associated with. When light is captured by an image sensor, the total intensity of the light can be measured. Müller matrix U i Multiplying by the incident Stokes vector S gives the outgoing Stokes vector O in each direction. i is generated: O i = U i * S . When light is captured by the image sensor, Stokes vector Oi The first entry of corresponds to the intensity of light. Since there are four exponents (0, 1, 2, 3), this is O i [0] =(U i * S)[0] It corresponds to. [x] represents the x-th element within the vector and starts from 0. This is the Müller matrix U i It is equivalent to taking the first row of and calculating the inner product with the Stokes vector S: S(( U i * S)[0] = sum j (U {i, (0, j)} * S j ) ). Müller matrix U i This first row of is the Stokes vector M i It can be regarded as such because it follows the same algebraic properties as the Stokes vector. The intensity in each direction of diffraction is M i It is proportional to the inner product between and S. The diffracted order may not necessarily be the same as the polarization state being selected.

[0090] The metasurface element (202) can split incident light (206) into diffracted light (206a), each of which is oriented in a different direction. The diffracted light (206a) in each direction has an intensity proportional to the projection of the incident state of the polarization state of the corresponding light. The diffracted light (206a) can enter into a set (204) of four different image sensors. In some embodiments, the diffracted light (206a) can enter into four different zones of a single image sensor, two zones of two image sensors, or other combinations of zones and sensors. Although these image sensors (204) and the diffracted light (206a) are illustrated in different horizontal positions, this is merely an example, and the image sensors (204) may actually be located in different quadrants, and the diffracted light (206a) transmits each of the polarizations to a different quadrant.

[0091] The incident light (206) can be in any polarization state. The metasurface element can split the incident light into diffracted light, each of the four directions corresponding to a different target polarization state, and the intensity in the corresponding direction is determined by the superposition of the incident polarization state and the target polarization state.

[0092] Although the metasurface element (202) overcomes the absorption loss and / or complexity of a conventional polarization imaging system including the polarizer (102) described in relation to FIG. 1, some embodiments may be aided by an aperture or a light source of a specific field of view to prevent sub-images from overlapping on the image sensor (204).

[0093] Various embodiments of the present invention utilize a polarization sorting optical metasurface microlens array (PSOMMA) to spatially mix polarization states across an image sensor. In some embodiments, the PSOMMA may be the final element prior to the image sensor, and each lenslet containing the PSOMMA may direct a set of designed polarization states toward individual subpixels, imaging units, or zones of the sensor. Additionally, it may not be necessary to use a field of view of an illumination source or an additional aperture to prevent sub-images from overlapping on the image sensor. Finally, the image sensor and the PSOMMA may be integrated with any existing optical imaging system that may include refractive optical elements illustrated and described in conjunction with FIGS. 5aa through 5ac.

[0094] FIGS. 3a and 3b illustrate schematic diagrams of a Polarization Sorting Metasurface Microlens Array (PSOMMA) according to an embodiment of the present invention. In FIG. 3a, the PSOMMA comprises individual metasurface lenslets (302). The number of individual lenslets may vary depending on the optical system, but generally, there may be any number of lenslets. Each lenslet can split incident light into individual polarizations for each diffraction angle. Each lenslet can split light into at least two polarizations, but generally, it can split light into any number of polarizations. As illustrated, the first lenslet L1 can split incident light into four different polarizations (304), namely pol1, pol2, pol3, and pol4. Furthermore, the second lenslet L2 can split incident light into four different polarizations (306), namely pol5, pol6, pol7, and pol8. Light can be split into corresponding directions proportional to four different polarization states selected in each direction. Similarly, the nth lenslet Ln can split incident light into four polarizations (308) pol n, pol n+1, pol n+2, and pol n+3. As illustrated in the drawing, a specific polarization state may be unique to each lenslet (denoted as pol1, pol2...pol n), but two or more identical polarization states may be utilized in each lenslet. For example, both the first lenslet L1 and the second lenslet L2 can split incident light into the same four polarizations (pol1, pol2, pol3, pol4). Additionally, the phase function of each lenslet may be uniquely specified.

[0095] As illustrated in FIG. 3b, the PSOMMA can be coupled with an image sensor (308) having an arbitrary number of pixels (indicated as p1, p2, p3...pn). The pixels may be multiple image sensing units. In this case, the operation of the PSOMMA may be to diffract each polarization state to a specific pixel. The overall polarization state of a scene can be reconstructed on a pixel-by-pixel basis by designing the polarization state corresponding to the pixel reading. Although the drawing is shown in one dimension, it is understood that the same principle applies to two-dimensional lenslets and pixel arrays. The illustrated incident light has a chief ray angle (CRA) of 0 degrees, but the CRA of the incident light may be an arbitrary angle or even vary across the lens. Thus, the incident light may be collimated, but may also be incident on the metasurface lenslet at a different angle. In some embodiments, a metasurface may be used to provide a microlens array function and may be optimized so that the lenslets accommodate different incident light angles (e.g., non-zero CRA) across the image sensor field and to minimize effects that would otherwise occur in a lens optimized for a 0-degree CRA. In some embodiments, physical microlens shifting may be provided. This is an approach to matching the microlens CRA with the image sensor. In this approach, the period of the microlens array is reduced across the sensor and can partially offset the microlens CRA.

[0096] In some embodiments, the microlens array may use a metasurface to provide a microlens function (e.g., focusing light onto an image sensor) in a manner that provides a microlens movement similar to a physical microlens movement. The metasurface may be locally designed to match both the azimuth and elevation angles of the primary refractive lens to create CRA matching across the sensor or with different incident light angles. This is complex for conventional physical microlenses but can be achieved by utilizing a metasurface having a microlens function. In some embodiments, the metasurface may be utilized to provide a collimation function that collimates light received from the primary refractive lens or the incident pupil and directs it toward the center of the sensor area.

[0097] FIG. 3ca conceptually illustrates an optical path for a single-layer metasurface utilizing a blazed grating approach according to an embodiment of the present invention. Based on the generalized Snell's law, the relationship between the phase gradient, the transmitted azimuth, and the elevation angle is as follows:

[0098]

[0099]

[0100] In the case of PSOMMA with CRA matching, the interface phase function includes the CRA matching and the blazed gratin for the PSOMMA phase function. Consequently, Equation 1 can be rewritten as follows:

[0101]

[0102]

[0103] To simplify the problem, when CRA matching is fully satisfied ( In the absence of PSOMMA functionality, the blaze lattice phase can be defined as follows:

[0104]

[0105]

[0106] In Equation 3, the actual phase function of the blaze lattice can be calculated as follows:

[0107]

[0108]

[0109] If the metasurface locally satisfies Equation 5, this completely cancels out the refractive lens CRA and can send this back to the target sensor pixel:

[0110]

[0111] FIG. 3cb conceptually illustrates the optical path for a PSOMMA and a single-layer metasurface acting as a collimating lens according to an embodiment of the present invention. Formulas for a 2D scenario in which the metasurface is positioned along the y-axis are exemplified herein. Equation 6 describes trigonometric functions that can be utilized to calculate the equivalent effective focal length (EFL) for collimating functionality:

[0112]

[0113]

[0114]

[0115] This can be simplified to calculate the collimation phase by using the target EFL as follows:

[0116]

[0117]

[0118] or

[0119]

[0120] can be one-quarter of the sensor's physical dimensions. This is an approximation instead of optimizing the collimation phase across all received CRAs on the metasurface. Another estimate of the EFL is as follows:

[0121]

[0122] FIG. 3cc conceptually illustrates an optical path for two distinct layers of a metasurface that serves as a PSOMMA and a collimating lens according to an embodiment of the present invention. One metasurface (MS) layer serves as a collimating lens, and one MS layer serves as a PSOMMA.

[0123] FIG. 4a illustrates a PSOMMA having two polarizations at each lenslet according to an embodiment of the present invention. The PSOMMA comprises several lenslets (402) (L1, L2…LN, LN+1), and each lenslet can split incident light into two polarizations (pol). As illustrated, for example, L1 can split incident light into pol1 in one direction and pol2 in the other direction. Additionally, L2 can split incident light into pol3 and pol4. Each lenslet may be located over only two pixels. For example, as illustrated, L1 may be located over pixels p1 and p2, directing pol1 light to p1 and pol2 light to p2. In some embodiments, polarization for a set of two lenslets may be repeated across an image sensor (404). For example, L1 splits the light into pol1 and pol2, L2 splits the light into pol3 and pol4, and this can be repeated over the entire image sensor (404) or over one or several parts of the image sensor (404), so that LN splits the light into pol1 and pol2, and LN+1 splits the light into pol3 and pol4. In some embodiments, polarization may be split into only two states over the image sensor (404). For example, in the described PSOMMA associated with FIG. 4a, pol1 may be equal to pol3, and pol2 may be equal to pol4. Thus, both L1 and L2 can split the light into pol1 and pol2, which are the same polarization. Finally, it is understood that the polarization state for each lenslet may be completely unique and independent and is not limited to a linear polarization state. For example, pol1 can be right-turned circularly polarized light and pol2 can be left-turned circularly polarized light. Also, pol1 can be S-polarized light and pol2 can be P-polarized light.

[0124] FIG. 4b illustrates a PSOMMA having two polarizations in each lenslet according to an embodiment of the present invention. This configuration is similar to the configuration described in relation to FIG. 4a. However, in this configuration, the PSOMMA (406) includes a first lenslet L1, which directs the first polarized light pol1 into the first imaging unit p1 of the image sensor (408) and the second polarized light pol2 into the second imaging unit p2 of the image sensor (408). Similarly, the PSOMMA (406) includes a second lenslet L2, which directs the first polarized light pol1 into the first imaging unit p1 and the second polarized light pol2 into the second imaging unit p2. Thus, the same imaging units p1 and p2 can be shared by the first lenslet L1 and the second lenslet L2.

[0125] FIG. 5aa illustrates an optical system including a PSOMMA according to an embodiment of the present invention. As illustrated, the PSOMMA (502) may be coupled with one or more refractive lenses (504) within the optical system to provide a complete polarization imaging system. The PSOMMA (502) may be the final element of the imaging system in front of an image sensor (506). In some embodiments, each lenslet of the PSOMMA (502) may be customized to a specific polarization response. In some embodiments, each lenslet of the PSOMMA (502) may be customized to a specific intrinsic phase function. The intrinsic phase function may be used to improve the performance of the PSOMMA (502) for various principal ray angles of the optical system. In some embodiments, each lenslet of the PSOMMA (502) may be customized to both a specific intrinsic phase function and a specific polarization response. There may be a distance offset between the PSOMMA (502) and the image sensor (506). The material between the image sensor (506) and the PSOMMA (502) within the offset distance may be air or a solid material such as glass, quartz, SiO2, Si3N4, or a polymer. Many optical imaging systems have a microlens array (MLA) in front of the image sensor to couple light into individual pixels. In such systems, the microlens array may not distinguish polarization and may not provide a unique random phase function at each pixel. As previously discussed, the PSOMMA (502) can diffract different polarizations in the incident light in different directions and provide a specific unique phase function to the different polarizations of the incident light.

[0126] In some embodiments, the PSOMMA (502) may be positioned to re-image the image plane from the optical device. One or more refractive lenses (504) in FIG. 5aa are merely exemplary. FIG. 5ab and FIG. 5ac illustrate examples of optical systems including a PSOMMA according to various embodiments of the present invention. In FIG. 5ab, an 0P trace having several different fields of view is provided, but the central ray passes through the PSOMMA (502) and is diffracted into the image sensor (506). In FIG. 5ac, an 0F trace is provided with a nominal field of view, and several rays passing through the PSOMMA (502) and diffracted into the image sensor (506) are traced across the pupil.

[0127] FIG. 5b illustrates an optical system including a PSOMMA according to an embodiment of the present invention. As illustrated, a light source (508) may output optical light toward the PSOMMA (502a). Although the light from the light source (508) is illustrated as being collimated, such light may have various angles of incidence that are not fully collimated. The PSOMMA (502a) may include all the functionalities of the PSOMMA (502) described in relation to FIG. 5a. However, the PSOMMA (502) may have at least some of the functionalities of one or more refractive lenses (504) described in relation to FIG. 5a, or may include a microlens array function. In FIG. 5b, one or more refractive lenses (504) are not illustrated as being present, but while some of one or more refractive lenses (504) may be present to perform some of the functionalities of one or more refractive lenses (504), the PSOMMA (502a) may be included to perform the remainder of the functionalities. PSOMMA (502a) may be the last element of the imaging system in front of the image sensor (506).

[0128] The light source may be a fixed near-infrared light source, a vertical-cavity surface emitting laser (VCSEL), an LED, an edge-emitting laser (EEL), sunlight, and / or other ambient lighting. In some embodiments, the light source may be designed to have a specific fixed polarization across the lighting field or to have a point-to-point polarization pattern across the lighting field. For example, the lighting pattern projected onto the scene may have fixed circular polarization, e.g., left-turn or right-turn fixed circular polarization. In some embodiments, this may be fixed linear or elliptical polarization lighting. In some cases, two different fixed polarizations may be used simultaneously or sequentially. FIGS. 5ca, FIGS. 5cb, and FIGS. 5cc illustrate an optical system comprising a PSOMMA according to various embodiments of the present invention. The optical system includes a main lens (552) that refracts light toward the PSOMMA (554). The PSOMMA (554) distributes light refracted from the main lens (552) onto the image sensor (556). In FIG. 5ca, the main lens image plane (558a) of the main lens (552) overlaps with the PSOMMA (554) so ​​that the main lens (552) is focused onto the PSOMMA (554).

[0129] In FIG. 5cb, the main lens image plane (558b) of the main lens (552) is located behind the PSOMMA (554) and the image sensor (556) so that the PSOMMA (554) is focused on the image plane. In FIG. 5cc, the main lens image plane (558c) of the main lens (552) is located between the main lens (552) and the PSOMMA (554). Thus, in FIG. 5cb, the image plane (558b) of the main lens (552) is located behind the image sensor (556), whereas in FIG. 5cc, the image plane (558c) is located in front of the image sensor (556). The configuration of FIG. 5cc provides more flexibility in physical configuration.

[0130] In some embodiments, the main lens (552) is a bi-convex lens. In some embodiments, the main lens (552) may be a plano-convex lens. In some embodiments, the main lens (552) may be a multi-refractive lens configuration. In some embodiments, the PSOMMA (554) may be replaced by a combination of an adjacent polarization-dependent metasurface and a microlens array. The microlens array may be a bi-convex lens array or a plano-convex lens array. Any of the configurations disclosed in FIGS. 5ca through 5cc may be utilized, which may provide various advantages such as higher spatial resolution, a larger field of view, and better accuracy of directional sampling.

[0131] FIGS. 6a and 6b illustrate an exemplary polarization distribution in an image sensor according to an embodiment of the present invention. FIG. 6a is a top view of an exemplary polarization state at each pixel of the image sensor after light has passed through a PSOMMA. As illustrated, each lenslet (604) can divide the intensity of incident light in proportion to the polarization content of four distinct states (right-turned elliptical polarization (602a), vertical polarization (602b), diagonal polarization (602c), left-turned circular polarization (602d)). These four pixels with unique states can form a super pixel of the imaging system and can be repeated N times across the entire image sensor. Although four repeating polarization states are illustrated, the PSOMMA can be used to define a unique polarization state at each pixel on the image sensor, which may be a repetition of two, three, or more unique polarization states. Preferably, light may not be filtered (e.g., absorbed) before reaching the image sensor, and the PSOMMA is not limited to a linear polarization state only. FIG. 6b illustrates a superpixel of total polarization where the positioning of individual lenslets (604) is represented by a thick line. Each lenslet (604) can classify incoming light into four subpixels having different polarization states.

[0132] The image sensor can be a CMOS sensor, a Time of Flight (ToF) sensor, an Indirect Time of Flight (iToF) sensor, or an RGB sensor. The timing, method, and frame rate of the image sensor readouts may include binning. For example, in a 2x2 pixel configuration, a non-polarized readout is followed by a polarized readout that can provide efficient information capture. In other words, polarized pixels are binning together for signal capture and processing. For imaging or energy purposes, other pixels may be sampled at different times or somewhat more frequently. In the case of an iToF sensor, most pixels may be of the CMOS type, and a smaller percentage (e.g., 10% or 20%) may be iToF pixels; the iToF may be in a non-polarized 2D mode. Some configurations may have a sparse concept for the sensor (e.g., only some pixels are in 3D mode versus 2D mode), which can then save energy.

[0133] In some embodiments, a diffractive optical element (DOE) may be used for some segmented sides. For example, a conventional DOE may be applied to some lenslets within the PSOMMA instead of a metasurface.

[0134] An exemplary polarization imaging system including a microlens array

[0135] In some embodiments, the polarization imaging system may include two distinct components, a refractive microlens array (MLA) and a PSOMMA. The MLA may tile the projected image. FIG. 7 schematically illustrates an example of the function of an exemplary polarization imaging system comprising an MLA with an integrated PSOMMA according to an embodiment of the present invention. The polarization imaging system may include a separate refractive MLA (704) and a PSOMMA (702). The PSOMMA (702) may be a polarization meta-grating. In some embodiments, the refractive MLA (704) may be implemented within the same layer as the PSOMMA (702). In some embodiments, as illustrated, the MLA (704) may be implemented as a separate planar layer from the PSOMMA (702). In some embodiments, the MLA (704) may be a conventional refractive MLA, and thus the polarization imaging system may include a conventional refractive MLA and a PSOMMA (702). In some embodiments, the MLA (704) may be integrated with the PSOMMA (702) so that a metasurface can be used to fabricate both the layer containing the MLA and the layer containing the PSOMMA on a single substrate. The layer containing the MLA (704) may include metasurface elements.

[0136] The MLA (704) can be adjusted relative to the image plane of the imaging system so that the incident light is reduced by a factor greater than 2, thereby preventing the segmented images from overlapping during tiling. Examples of tiling are shown in FIG. 6b. It may be desirable to place the PSOMMA (702) as close as possible to the MLA (704), or in some embodiments, the MLA (704) and the PSOMMA (702) may be combined within a single layer. Pixels are schematically depicted as adjacent boxes. The active area of ​​a pixel may be smaller than schematically shown in the drawing, and the magnification may be greater than 2 to illuminate only the active area.

[0137] The MLA (704) includes a focal length. The distance (708) from the main image plane (707) may be greater than twice the focal length of the MLA (704). Imaging to pixels of a region (716) within the main image plane (707) may overlap with each microlens of the MLA (704). The PSOMMA (702) outputs light into an image sensor (710) containing a pixel grid. The distance (712) from the PSOMMA (702) to the image sensor (710) may be less than the focal length of the MLA (704). A schematic ray trace (714a) including a main ray (solid line) and an peripheral ray (dotted line) is illustrated in 0F. A schematic ray trace (714a) including a main ray (solid line) and an peripheral ray (dotted line) is illustrated in 1F.

[0138] FIGS. 8a and 8b illustrate offset pixel architectures according to various embodiments of the present invention. Instead of targeting a diffraction order such that diffraction is symmetric around a non-diffraction image, the non-diffraction image may be selected as one of the diffraction orders (e.g., zeroth order). Such architectures share many features numbered identically to the architecture of FIG. 7, and the description is applicable to these architectures. The description will not be repeated in detail.

[0139] As illustrated in FIG. 8a, the center of each lenslet of the microlens array can be directly aligned over the zero-order light between the pol1 and pol2 light. The zero-order light can have a negative effect on the fidelity of the reconstructed polarization. Therefore, it may be beneficial to have the zero-order light pass through the center of the photosensor between the pol1 and pol2 light. The active area of ​​the pixel (the photo-sensing region that generates a charge on the incident light) can be smaller than the microlens pitch divided by the number of pixels it is divided into. The zero-order light from the PSOMMA may not be incident on the active area of ​​the pixel because the active area of ​​the pixel array under the photosensor may be offset from the location where the zero-order light is incident. The zero-order light may coincide with the diffracted light. Because the zero-order light is "uncontrolled," it does not effectively parse the incident polarization and does not carry a polarization signal. Therefore, the zero-order light can be considered as noise. Therefore, by orienting this into the non-photosensitive zone, noise can be mitigated.

[0140] In some embodiments, the zero-order light may be diffused at least partially between the pol1 light and the pol2 light. When the active area of ​​the image sensor overlaps with the zero-order light, a portion of the zero-order light may be shared as noise among the pixels of the image sensor.

[0141] In some cases, it may be difficult to align the zero-order light with the inactive region of the pixel array of the image sensor. In such cases, it may be better for the zero-order light to diffract in the same direction as one of the polarizations of the light. Fig. 8b illustrates an offset pixel architecture in which the zero-order light is diffracted toward the side of the pol2 light in the same direction as the pol1 light. The target pixel can be offset with respect to the center of the lens. Only one of the pixels experiences noise from the zero-order light, which can then make subsequent polarization restoration easier. The zero-order image is displayed offset from the vertical axis. Mathematically, the lenslet of Fig. 8a can shorten all analyzer Stokes vectors on the Poincaré sphere, thereby reducing the degree of polarization of the pol1 and pol2 light. On the other hand, in the lenslet of Fig. 8b, only the vector containing the pol1 and zero-order light can be shortened, and the pol2 light can not be shortened.

[0142] FIGS. 9a and 9b illustrate various high-fidelity pixel architectures according to embodiments of the present invention. These architectures share many features numbered identically to the architectures of FIGS. 7 and 8, and the descriptions are applicable to these architectures. The descriptions will not be repeated in detail. As illustrated in FIG. 9a, the zero-order light may not be diffracted and thus may pass through the center of a microlens (e.g., a lenslet). The undiffracted zero-order light may be imaged onto a separate pixel on an image sensor where the diffraction order is not directed. In this case, the analyzer Stokes vector may not be shortened and thus the degree of polarization may not be reduced. The zero-order light may not affect the fidelity of the reconstructed polarization imaging. Furthermore, instead of becoming noise for the reconstructed polarization, the zero-order light may be an additional signal for the reconstructed intensity image. The zero-order light may not have a preferred polarization state and simply provides intensity information. In this configuration, the amount of light entering the zero order does not need to be minimized and can be optimized by design to increase overall optical efficiency. This configuration is advantageous in applications where a trade-off can be made between efficiency, signal-to-noise ratio (SNR), and angular resolution. FIG. 9b shows various top sensor views of exemplary implementations of a superpixel having separated zero-order pixels combined with a pixel offset. The pixel offset refers to the spatial translation of the lower pixel array relative to the microlens array. In some embodiments, pol3 may be equal to pol1, and pol4 may be equal to pol2. The illustrated combinations are not limited, and other combinations of subpixel positioning are considered.

[0143] FIGS. 10a and FIGS. 10b illustrate various hybrid intensity / polarization architectures according to embodiments of the present invention. These architectures share many features numbered in the same way as the architecture of FIG. 7, and the description is applicable to these architectures. The description will not be repeated in detail. In these architectures, the PSOMMA (706) may not be applied to all lenslets of the MLA (704), so that a subset of pixels may be imaged directly by the lenslets without the PSOMMA (706), and another subset of pixels may be imaged by adding the PSOMMA (706) to the lenslets of the MLA (704). This may be achieved by patterning a transparent window (1002) within the PSOMMA (706), selectively patterning the back surface of the MLA (704) with the PSOMMA (706), or by other methods suitable for various hybrid MLA-polarization meta-grating configurations. The offset pixel and high-fidelity architecture described above may also be applied to a subset of pixels including PSOMMA (706). This hybrid structure may be beneficial in balancing the trade-off between resolution, optical efficiency, polarization fidelity, and image sensor size. This involves implementing an interpolation scheme between a pixel receiving undiffracted light and a pixel receiving diffracted light from PSOMMA (706).

[0144] FIG. 10a illustrates a hybrid intensity / polarization architecture, where poln (e.g., pol1, pol2, pol3, pol4) are polarization states of diffracted light from the PSOMMA (706), and In (e.g., l1, l2, l3) are light that has not passed through the PSOMMA (706). The lenslets of the MLA (704) are named M1, M2, ..., Mn. As illustrated, Poln may be at least two distinct polarizations. Light that has not passed through the PSOMMA (706) can be used to measure intensity, time of flight (ToF), target distance, depth, distance to focus, defocus, and / or color, whereas polarization-separated light diffracted by the PSOMMA (706) can be used to measure various degrees of polarization. FIG. 10b illustrates various exemplary hybrid intensity / polarization subpixel configurations. The thick lines represent pixels where each square represents a subpixel. Poln may contain two or more distinct polarizations, 0 is the undiffracted zero-order pixel exiting the PSOMMA (706), and In is the light that did not pass through the PSOMMA (706). Preferably, capturing the light intensity can increase the resolution of the image intensity without increasing the sensor size. Capturing the intensity can also be used as a reference intensity in an interpolation scheme to extract some resolution for the intensity image and to manage noise arriving from the zero-order light. If some of the subpixels are dedicated to capturing the image intensity, the resolution of the polarization image may be reduced.

[0145] FIGS. 11a through 11c illustrate various hybrid architectures including MLA and PSOMMA according to various embodiments of the present invention. In FIG. 11a, a substrate (1102) includes a PSOMMA (1104) positioned on top of an MLA (1106). The MLA (1106) is positioned directly above an image sensor (1108), with an air gap between them. The PSOMMA (1104) may be embedded in a layer directly above the MLA (1106) within the substrate (1102). A main image plane (1110) may be positioned within the substrate (1102). In some embodiments, different microlenses of the MLA (1106) may have different sagittal planes (SAG) and different focal lengths or aspherical features. In some embodiments, the MLA (1106) may be implemented as a metasurface that can be used to customize various characteristics of the MLA (1106). In some embodiments, a metasurface lens may be used to customize asphericity on the MLA (1106) or to reduce aberrations from the MLA (1106).

[0146] In some embodiments, the MLA (1106) may include a variable focus having different focuses across the image. The MLA (1106) may include a phase-detection autofocus in which part of the pupil is obscured. The MLA (1106) may have a depth from defocus in a sparse format open window, or other autofocus that separates different focuses for autofocus calculation, for example, by using different MLA lenses by color.

[0147] In FIG. 11b, the MLA (1106) and PSOMMA (1104) are integrated within the substrate (1102), and the MLA (1106) is located on the surface opposite to the image sensor (1108). A main image plane (1110) may be located on the substrate (1102). The MLA (1106) integrated with the image sensor (1108) operates by directing light from the main image plane (1110) onto the image sensor (1108). The main image plane (1110) may be formed by an optical component, such as an optical system lens. In one configuration, the MLA (1106) takes the main image plane (1110) and reduces it so that the main object is imaged onto the active area of ​​pixels within the image sensor (1108). The main image plane (1110) may be an image plane formed by an optical system. The main image plane (1110) can be distinguished from the image plane formed by the MLA (1106) within the image sensor.

[0148] The hybrid architecture of FIG. 11c is similar to FIG. 11b, but the MLA (1106a) may be a planar MLA layer capable of performing an optical role similar to that of an MLA layer. This planar MLA layer may be an MLA containing a metasurface. Additional examples of polarization imaging systems including an MLA are discussed in relation to FIG. 28a to 28c, FIG. 29a to 29c, FIG. 30a to 30c, FIG. 31a and 31b, and FIG. 32a to 32c.

[0149] Exemplary method for fabricating PSOMMA and integrating it with an image sensor

[0150] Various fabrication methods for PSOMMA integrated with an image sensor were considered, and three specific methods are described below.

[0151] The first method is to fabricate the PSOMMA directly onto the image sensor. FIGS. 12a through 12g illustrate various steps of the direct fabrication method according to an embodiment of the present invention. In FIG. 12a, an image sensor wafer (1202) having a flattened or approximately flattened surface is provided. The image sensor wafer (1202) may contain an image sensor. In FIG. 12b, a spacer layer (1204) is deposited on the flattened surface of the image sensor wafer (1202). The thickness of the spacer layer (1204) may determine the distance between the PSOMMA and the image sensor. The spacer layer (1204) may include a dielectric material (e.g., SiO2, Si3N4, etc.), a polymer layer, or other suitable layer preferably having a refractive index of less than 1.6 at the operating wavelength of the image sensor. In FIG. 12c, one or more additional layers (1206) are deposited on the spacer layer (1204). In FIG. 12d, one or more additional layers (1206) are patterned and / or etched to form a nanoscale pillar (1206a). An example of a nanoscale pillar is described in U.S. Patent Application Publication No. 2018 / 0341090, filed May 22, 2018, with the title of the invention “Atomic layer deposition process for fabricating dielectric metasurfaces for wavelengths in the visible spectrum,” the entire contents of which are incorporated herein by reference. Patterning and / or etching may be performed using a photolithography process. Patterning and / or etching may include a wet or dry etching process. The nanoscale pillar (1206a) may be a metasurface pillar. In FIG. 12e, the nanoscale pillar (1206a) may be encapsulated by an encapsulation layer (1208).In FIG. 12f, the encapsulation layer (1208) can be flattened to form a layer (1210) of the same height as the nanoscale pillar (1206a). In FIG. 12g, one or more top layers (1212) can be deposited on the encapsulated nanoscale pillar (1206a).

[0152] A second method of PSOMMA fabrication and integration is to bond the PSOMMA metasurface to an image sensor by a wafer bonding method. FIGS. 13a through 13d illustrate various steps of a wafer bonding method according to an embodiment of the present invention. In FIG. 13a, an image sensor wafer (1302) having a flattened or approximately flattened surface is provided. In FIG. 13b, a spacer layer (1304) is deposited on the flattened surface of the image sensor wafer (1302). The thickness of the spacer layer (1304) can determine the distance between the PSOMMA and the image sensor. The spacer layer (1304) may comprise a dielectric material (e.g., SiO2, Si3N4, etc.), a polymer layer, or other suitable layer preferably having a refractive index of less than 1.6 at the operating wavelength of the image sensor. In FIG. 13c, a second wafer (1306) comprising a nanoscale pillar (1308) containing PSOMMA is provided. The nanoscale pillar (1308) on the second wafer (1306) may be fabricated using methods such as photolithography or wet or dry etching. The illustrated second wafer (1306) containing the nanoscale pillar (1308) is merely an example, and other configurations are considered. For example, there may be a layer between the second wafer (1306) and the nanoscale pillar (1308). An example of a nanoscale pillar fabricated on a substrate is described in U.S. Patent Application Publication No. 2019 / 0064532, filed on August 31, 2019, with the title of the invention “Transmissive Metasurface Lens Integration,” which is incorporated herein by reference in its entirety. The disclosed method of fabricating the nanoscale pillar on the substrate and various configurations may be utilized to create the nanoscale pillar (1308) on the second individual wafer (1306). The second wafer (1306) may include silicon, glass, polymer and / or other suitable materials.The second wafer (1306) may have an additional spacer layer placed on the front surface instead of the spacer on the first wafer, or in addition to the spacer on the image sensor wafer (1302).

[0153] After the image sensor wafer (1302) and the second wafer (1306) are fabricated separately, the two wafers can be joined together using a wafer joining method. The wafer joining method may include adhesive joining, direct oxide joining, eutectic joining, and / or thermal compression joining. FIG. 13d illustrates the image sensor wafer (1302) and the second wafer (1306) after the wafer joining method. The wafers (1302, 1306) can be aligned with each other so that individual metasurface lenslets can be aligned to corresponding pixels on the image sensor on the image sensor wafer (1302). The alignment tolerance of the lenslets to the pixels may be a small fraction (or equal size) of the pixel size. In some embodiments, the alignment tolerance of the lenslets to the pixels may be less than 10% of the pixel size. In some embodiments, the second wafer substrate (1306) may be transparent. In some embodiments, the second wafer substrate (1306) may be a transparent material such as glass. In the case where the second wafer substrate (1306) is transparent, the second wafer substrate (1306) can be part of the final integrated device.

[0154] In some embodiments, the second wafer (1306) may be opaque. Examples of opaque second wafer substrates include silicon. The second wafer substrate (1306) may be removed using methods including polishing, lapping, wet chemical etching and / or dry chemical etching. After removing the second wafer substrate (1306), the nanoscale pillar (1308) is transferred from the second wafer (1306) to the image sensor wafer (1302).

[0155] A third method for fabricating the PSOMMA and integrating it with an image sensor is a die attach method. The steps of the die attach method begin with steps similar to those previously exemplified for the wafer bonding method described in relation to FIGS. 13a through 13c. After fabricating the image sensor wafer (1302) and the second wafer (1306), the second wafer can be singulated into individual dies. The singulated individual dies can now be bonded to the image sensor wafer (1302) using a wafer-wafer bonding process. FIG. 14a illustrates a plurality of second wafer dies (1402) bonded to the image sensor wafer (1302) according to an embodiment of the present invention. Each individual second wafer die (1402) can be aligned to a suitable feature on the image sensor wafer (1302). The alignment tolerance of the lenslet for a pixel can be a small fraction of the pixel size or equal to a single pixel. In some embodiments, the alignment tolerance of the lenslet for the pixel may be less than 10% of the pixel size. After the individual dies are combined with the image sensor wafer (1302), the image sensor wafer (1302) may be singulated into individual image sensor dies, so as to result in an integrated image sensor, i.e., a PSOMMA sensor. FIG. 14 illustrates an image sensor die (1404) after the image sensor wafer (1302) has been singulated, according to an embodiment of the present invention. In some embodiments, a second wafer die (1402) may be combined with the previously formed individual image sensor dies to form an integrated image sensor, i.e., a PSOMMA sensor. Such individual image sensor dies may be formed by singulating the larger image sensor wafer (1302) before combining the second wafer dies (1402).

[0156] FIG. 15 illustrates an exemplary PSOMMA sensor (1500) fabricated according to an embodiment of the present invention. The PSOMMA sensor (1500) comprises a CMOS image sensor (CIS) wafer (1502). The PSOMMA sensor (1500) further comprises a transparent substrate (1504). The transparent substrate (1504) may be glass. A nanopillar (1506) is positioned on the transparent substrate (1504). A void (1508) is positioned between the nanopillar (1506) and the CIS wafer (1502), so that the nanopillar (1506) exists between the void (1508) and the transparent substrate (1504). The void (1508) is located between the CIS wafer (1502) and the transparent substrate (1504). The void (1508) may have a thickness between 6 μm and 7 μm. The gap (1508) may have a thickness between 3 μm and 4 μm. The PSOMMA sensor (1500) may include a die-die or wafer-wafer overlay accuracy of 0.3 μm, which may be an alignment tolerance during integration. The metasurface lens directs the classified polarized light toward the target image sensor pixel. Therefore, the metasurface lens needs to match / align the metasurface lens to the image sensor with subpixel-level accuracy. The gap (1508) may be used to maintain flatness between two wafers separated by less than 6 μm over a distance of 12 inches.

[0157] In some examples, a spacer layer may be fabricated on a CIS wafer (described later in FIGS. 16a to 16c). A spacer layer may also be fabricated on a nanopillar (1506) (described later in FIGS. 18a to 18c). FIGS. 20a and 20b show an alignment process for a CIS wafer (1602) and a nanopillar (1506).

[0158] FIGS. 16a through 16c illustrate an exemplary method for producing a CIS wafer (1502) according to an embodiment of the present invention. In FIG. 16a, a CIS wafer (1602) is provided together with a spacer layer (1604). The CIS wafer (1602) may include a previously fabricated image sensor. A layer of adhesive (1610) is provided on a secondary wafer (1608). In FIG. 16b, the secondary wafer (1608) contacts the CIS wafer (1602) such that the layer of adhesive (1610) contacts the spacer layer (1604). In FIG. 16c, the secondary wafer (1608) is removed and the layer of adhesive (1610) is transferred to the top of the spacer layer (1604) so ​​that the adhesive (1610a) remains only on the top of the spacer layer (1604) and not in the area between the spacer layers (1604). The spacer layer (1604) may have a thickness of 6 μm or more. The CIS wafer (1602) may have a diameter of 12 inches. The secondary wafer (1608) may also have a diameter of 12 inches. There may be a channel (1612) between adjacent portions of the spacer layer (1604). The CIS wafer (1602) may contain nanopillars. An example of such an adhesive delivery process is described in U.S. Patent No. 6,669,803, filed September 29, 2000, with the title of the invention “Simultaneous provision of controlled height bonding material at a wafer level and associated structures,” the entire contents of which are incorporated herein by reference.

[0159] FIG. 17 illustrates a plan view of a CIS wafer (1602) according to an embodiment of the present invention. The CIS wafer (1602) includes a spacer layer (1604). An imaging region (1702) is included in the internal portion of each unit of the spacer layer (1604).

[0160] FIGS. 18a through 18c illustrate an exemplary method for producing a nanopillar wafer according to an embodiment of the present invention. In FIG. 18a, a nanopillar substrate (1802) comprises a plurality of nanopillars (1804). A spacer layer (1806) is provided below the plurality of nanopillars (1804) so ​​that the spacer layer (1806) is on the opposite side of the nanopillar substrate (1802). An adhesive layer (1810) is provided on a secondary wafer (1808) that directly faces the spacer layer (1806). In FIG. 18b, the secondary wafer (1808) is in contact with the nanopillar substrate (1802) such that the layer of adhesive (1810) contacts the spacer layer (1806). In FIG. 18c, the secondary wafer (1808) is removed and a layer of adhesive (1810) is transferred to the top of the spacer layer (1806) so that the adhesive (1810a) remains only on the top of the spacer layer (1806) and not in the area between the spacer layers (1806). The spacer layer (1806) may be 6 μm wide. The nanopillar substrate (1802) may be 12 inches in diameter. The secondary wafer (1808) may also be 12 inches in diameter. The spacer layer (1806) is provided on the nanopillar substrate (1802). A layer of adhesive (1810) is provided on the secondary wafer (1808). An example of such an adhesive delivery process is described in U.S. Patent No. 6,669,803, filed on September 29, 2000, with the title of the invention “Simultaneous provision of controlled height bonding material at a wafer level and associated structures,” the entire contents of which are incorporated herein by reference.

[0161] FIG. 19 shows a plan view of a nanopillar substrate (1802) according to an embodiment of the present invention.

[0162] The nanofiller substrate (1802) includes a spacer layer (1806) that is lattice-oriented.

[0163] FIGS. 20a and FIG. 20b illustrate an exemplary method for fabricating a PSOMMA sensor according to an embodiment of the present invention. The PSOMMA sensor comprises combining a nanofilament substrate (1802) of FIG. 18c with a CIS wafer (1602) of FIG. 16c. In FIG. 20a, the nanofilament substrate (1802) of FIG. 18c and the CIS wafer (1602) of FIG. 16c are positioned close to each other. An adhesive (1810a) corresponding to a spacer layer (1806) on the nanofilament substrate (1802) has a predetermined width (2002). There is a channel (1612) between adjacent portions of the spacer layer (1604). The channel (1612) has a width (2004) greater than the width (2002) of the spacer layer (1806). In some examples, the width (2004) of the channel (1612) may be 0.6 μm larger than the width (2002) of the spacer layer (1806). In FIG. 20b, the two wafers (1602 and 1802) are joined together and the spacer layers (1604, 1806) are precisely interlocked to achieve a bond with accurate alignment.

[0164] FIGS. 21a and FIGS. 21b illustrate a process flow for fabricating a CIS wafer according to an embodiment of the present invention. In FIG. 21a, a CIS wafer (2102) including a bonding pad (2104) is provided. In FIG. 21b, a dielectric layer (2106) is deposited on the CIS wafer (2102) including the bonding pad (2104). The dielectric layer (2106) may be a SiO2 layer. The dielectric layer (2106) may be deposited using a plasma-enhanced tetraethyl orthosilicate (PETEOS) process. The dielectric layer (2106) may have a thickness of 5.5 μm. A planarization process may be utilized on the dielectric layer (2106). The planarization process may be a chemical mechanical polishing (CMP) process.

[0165] FIGS. 22a and 22b illustrate a process flow for fabricating a nanopillar wafer according to an embodiment of the present invention. In FIG. 22a, a nanopillar substrate (2202) comprising nanopillars (2204) is provided. In FIG. 22b, a dielectric layer (2206) is deposited on the nanopillar substrate (2202) comprising nanopillars (2204). The dielectric layer (2206) may be a SiO2 layer. The dielectric layer (2206) may be deposited using a plasma-enhanced tetraethyl orthosilicate (PETEOS) process. The dielectric layer (2206) may have a thickness of 5.5 μm. A planarization process may be utilized on the dielectric layer (2206). The planarization process may be a chemical mechanical polishing (CMP) process.

[0166] FIGS. 23a through 23h illustrate a fabrication process for fabricating an imaging sensor using a CIS wafer fabricated using the process described in FIGS. 21a and 21b and a nanopillar wafer fabricated using the process described in FIGS. 22a and 22b, according to an embodiment of the present invention. In FIG. 23a, a nanopillar substrate (2202) is bonded to a CIS wafer (2102) so that the nanopillar (2204) is separated from the bonding pad (2104) by a bonded dielectric layer (2302) comprising the dielectric layer (2106) of the CIS wafer (2102) and the dielectric layer (2206) of the nanopillar substrate (2202). Some wafer-wafer bonding machines can only process wafers with a warp not exceeding 250 μm. The dielectric layer (2106) of the CIS wafer (2102) and the dielectric layer (2206) on the nanopillar (2204), deposited using a tetraethyl orthosilicate (TEOS) process, can cause the wafer to bend, which will not allow wafer bonding. Advantageously, a flattening process for the dielectric layer (2106) of the CIS wafer (2102) and the dielectric layer (2206) on the nanopillar (2204) provides the ability to wafer bond the dielectric layer (2106) of the CIS wafer (2102) to the dielectric layer (2206) on the nanopillar (2204). The wafer bonding process can deliver an alignment accuracy of 200 nm.

[0167] In FIG. 23b, the nanopillar substrate (2202) is removed, leaving the nanopillar (2204) on the bonded dielectric layer (2302). The nanopillar substrate (2202) may be removed by grinding, etching, and / or planarization (e.g., CMP). In FIG. 23c, a photoresist layer (2304) is deposited on the nanopillar (2204). The photoresist layer may be spin-coated on the nanopillar (2204). In FIG. 23d, the photoresist layer (2304) is patterned, and then the bonded dielectric layer (2302) is etched to expose the bonding pad (2104). In FIG. 23e, the photoresist layer (2304) is removed, and a barrier seed layer (2306) may be deposited on the bonding pad (2104), the sidewall of the bonded dielectric layer (2302), and the nanopillar (2204). In some embodiments, the barrier seed layer (2306) may be omitted. In FIG. 23f, a conductive layer (2308) is deposited on the barrier seed layer (2306). The conductive layer (2308) may be a copper (Cu) layer. The conductive layer (2308) may be deposited using an electrochemical plating (ECP) process. In FIG. 23g, the conductive layer (2308) is flattened down to the nanopillar (2204). The flattening process may be a CMP process. In FIG. 23h, a chip on board (COB) process is performed. The COB process may involve directly wire-bonding a sensor to a printed circuit, which can be a printed circuit board. The printed circuit board can be flexible.

[0168] FIGS. 24a through 24c illustrate a fabrication process for fabricating an imaging sensor using a CIS wafer fabricated using the process described in FIGS. 21a and 21b and a nanopillar wafer fabricated using the process described in FIGS. 22a and 22b, according to an embodiment of the present invention. FIGS. 24a and 24b correspond to the steps of FIGS. 23a and 23b. The description of these steps of FIGS. 23a and 23b relates to the steps of FIGS. 24a and 24b. In FIG. 24c, a through-silicon via (TSV) process is performed. Instead of wire bonding to a bond pad on the front side of the chip, a hole may be formed to reach the bond pad on the front side from the back side of the chip / wafer, and then a metawire may be directly bonded from the hole using a solder ball on the back side of the wafer to achieve chip-scale packaging. In this case, there may be no bond pad open on the front side.

[0169] FIGS. 25a through 25h illustrate a fabrication process for fabricating an imaging sensor according to an embodiment of the present invention. In FIG. 25a, a P-type substrate (2502) is provided. The P-type substrate (2502) may be doped with a high concentration of a P-type dopant. In FIG. 25b, a nanopillar layer (2505) may be formed on the P-type substrate (2502). The nanopillar layer (2505) includes embedded nanopillars (2504). In FIG. 25c, a crystalline layer (2506) is grown on the nanopillar layer (2505). The crystalline layer (2506) may be a P-type layer containing a P-type dopant. The crystalline layer (2506) may be grown using epitaxial growth.

[0170] In FIG. 25d, a P-type doping process is performed to create a high-concentration P-type doping layer (2508) on a crystal layer (2506). The P-type doping process may be an implantation process (e.g., an ion implantation process). In FIG. 25e, another crystal layer (2510) is grown on the high-concentration P-type doping layer (2508). The crystal layer (2510) may be grown using epitaxial growth. In FIG. 25f, a CMOS layer (2512) may be formed on the crystal layer (2510). The CMOS layer (2512) may be a front circuit fabricated using a conventional CMOS process. The crystal layer (2510) may be a foundational layer for the CMOS layer (2512). In FIG. 25g, the device is bonded to a carrier wafer (2514) (e.g., wafer bonding), and the CMOS layer (2512) is in direct contact with the carrier wafer. In FIG. 25h, the P-type substrate (2502) is removed. The P-type substrate (2502) can be removed by grinding, lapping, wet chemical etching and / or dry chemical etching.

[0171] Various exemplary PSOMMA configurations

[0172] FIG. 26 illustrates an exemplary operation of an exemplary metasurface microlens array (MLA) according to an embodiment of the present invention. This design takes into account angular variation depending on the position of the MLA. Light entering from the metasurface MLA (2602) includes a principal ray angle (CRA) (2604) from an imaging optical device or aperture, which may vary in the plane of the metasurface MLA (2602). This variation may be calculated or measured as a function of the angle and / or position on the MLA, for example, as a distance from the center of the MLA, to provide the CRA (2604). Conventional refractive MLAs may not take into account such variation in angle. Conventional refractive MLAs may be designed to operate optically well or efficiently only for a CRA of 0 degrees. On the other hand, each metasurface lenslet of the metasurface MLA (2602) is optimized for a specific area of ​​illumination (AOI). In conventional refractive MLAs, each lenslet may include an aspherical term or a more complex phase function. This approach to conventional refractive MLAs would involve multiple refractive lenses / prisms, which would add complexity. Therefore, metasurface MLAs allow for solutions that include a varied CRA (2604) that may not be easily achievable using conventional refractive MLAs.

[0173] FIG. 27a illustrates an exemplary pixel array for four polarizing pixels according to an embodiment of the present invention. The pixel array comprises a plurality of sets (2702) of polarizing pixels, including a first pixel (2702a), a second pixel (2702b), a third pixel (2702c), and a fourth pixel (2702d) having different polarizations. There is one or more general pixels (2704) separating adjacent sets of polarizing pixels. The general pixels (2704) are essentially empty or are non-polarizing and / or non-dividing pixels to which optical power is simply applied for focusing or filtering. In some embodiments, the number of general pixels (2704) between adjacent sets (2702) of polarizing pixels may be greater or less than illustrated in the drawing.

[0174] FIG. 27b illustrates an exemplary pixel array for two polarizing pixels according to an embodiment of the present invention. The pixel array comprises a plurality of sets (2706) of polarizing pixels, including a first pixel (2706a) and a second pixel (2706b) having different polarizations. The first pixel (2706a) and the second pixel (2706b) may have orthogonal polarizations. The sets (2706) of polarizing pixels may not be identical pairs. For example, some pairs of sets (2706) of polarizing pixels may detect S1 polarization, some may detect S2 polarization, and others may detect S3 polarization. In some embodiments, depending on transmitted polarization, such as static polarization illumination or known polarization illumination, the detected polarization may match such a transmitted polarization source or sources. There is one or more normal pixels (2708) separating adjacent sets (2706) of polarizing pixels. In some embodiments, the number of normal pixels (2708) between adjacent sets (2706) of polarizing pixels may be greater or less than that illustrated in the drawing.

[0175] A microlens array (MLA) may exist that includes different microlenses associated with each of the pixels. The microlenses for the normal pixels (2704, 2708) and the microlenses for the polarizing pixels within the set of polarizing pixels (2702, 2706) may have the same principal focal plane. Since the aperture of the polarizing pixels within the set of polarizing pixels (2702) may differ between the 4-polarizing pixel array of FIG. 27a and the 2-polarizing pixel array of FIG. 27b, the f-number may differ for the different microlens arrays for the 4-polarizing pixel array of FIG. 27a and the 2-polarizing pixel array of FIG. 27b. In some embodiments, the MLA may be a metasurface or a refractive lens array. The functionality of the MLA may be implemented differently for the normal pixels and the polarizing pixels. An open non-polarizing simple-powered optical element or a filtered optical element may be present over the normal pixels. MLA may include multiple lenslets of different focal lengths for depth of field by defocusing.

[0176] FIG. 27c illustrates an exemplary pixel array for four polarizing pixels according to an embodiment of the present invention. The pixel array comprises a plurality of sets (2710) of polarizing pixels, including a first pixel (2710a), a second pixel (2710b), a third pixel (2710c), and a fourth pixel (2710d) having different polarizations. There is one or more normal pixels (2712) separating the first pixel (2710a), the second pixel (2710b), the third pixel (2710c), and the fourth pixel (2710d). In some embodiments, the number of normal pixels (2712) between adjacent sets (2702) of polarizing pixels may be greater or less than illustrated in the drawings. A set (2710) of polarizing pixels may also be separated by one or more normal pixels (2712).

[0177] The design described in relation to FIGS. 27a to 27c can be utilized in a classification or filtering implementation form as described below.

[0178] FIG. 28a is a cross-sectional view of an imaging device including a polarization separation function according to an embodiment of the present invention. As illustrated, the imaging device includes a microlens (2802) that directs light (2803) into a polarization splitting metasurface (2804). The microlens (2802) may be part of a microlens array. The polarization splitting metasurface (2804) splits the light (2803) into a first polarized light (2806a) and a second polarized light (2806b). The first polarized light (2806a) and the second polarized light (2806b) may have orthogonal polarizations. An image sensor including a first zone (2808a) and a second zone (2808b) is positioned below the polarization splitting metasurface (2804). The first polarized light (2806a) is directed into the first zone (2808a) and the second polarized light (2806b) is directed into the second zone (2808b). The microlens (2802) is offset from the first zone (2808a) and the second zone (2808b) such that the center of the microlens (2802) is between the first zone and the second zone.

[0179] Each microlens (2802) can cover at least half of the first zone (2808a) and the second zone (2808b) with overlapping metasurface lenslets (2804) in between, thereby refracting the first polarized light (2806a) in the first direction into the first zone and the second polarized light (2806b) in the second direction into the second zone. The first polarized light (2806a) may be orthogonal to the second polarized light (2806b).

[0180] FIG. 28b is a plan view of an imaging device including a polarization separation function according to an embodiment of the present invention. The imaging device includes a microlens and a polarization splitting metasurface described in relation to FIG. 28a. The polarization splitting metasurface splits light into a first polarized light (2810a), a second polarized light (2810b), a third polarized light (2810c), and a fourth polarized light (2810d). The imaging device includes an image sensor comprising a first zone (2812a), a second zone (2812b), a third zone (2812c), and a fourth zone (2812d). The first polarized light (2810a) is directed into the first zone (2812a), the second polarized light (2810b) is directed into the second zone (2812b), the third polarized light (2810c) is directed into the third zone (2812c), and the fourth polarized light (2810d) is directed into the fourth zone (2812d).

[0181] FIG. 28c is a plan view of an imaging device including a polarization separation function according to an embodiment of the present invention. The imaging device includes a first polarization splitting portion (2814a), a second polarization splitting portion (2814b), and a third polarization splitting portion (2814c), each configured similarly to the imaging device described in relation to FIG. 28a. As illustrated, in each of the first polarization splitting portion (2814a), the second polarization splitting portion (2814b), and the third polarization splitting portion (2814c), light is split so that different polarizations are split in different directions. Each of the first polarization splitting portion (2814a), the second polarization splitting portion (2814b), and the third polarization splitting portion (2814c) can operate for different polarizations. For example, the first polarization splitting part (2814a) can split light into the first polarization and the second polarization, and the second polarization splitting part (2814b) splits light into the third polarization and the fourth polarization.

[0182] FIG. 29a is a cross-sectional view of an imaging device including a polarization separation function according to an embodiment of the present invention. As illustrated, the imaging device includes an array of microlenses (2902a, 2902b) that direct light (2903) into a polarization splitting metasurface (2904). The polarization splitting metasurface (2904) splits the light (2903) into a first polarized light (2906a) and a second polarized light (2906b). The first polarized light (2906a) and the second polarized light (2906b) may have orthogonal polarizations. An image sensor including a first zone (2908a) and a second zone (2908b) is positioned below the polarization splitting metasurface (2904). The first polarized light (2906a) is directed into the first zone (2908a) and the second polarized light (2906b) is directed into the second zone (2908b). Light from a plurality of metal lenses (2902a, 2902b) can be split at different parts of the polarization splitting metasurface (2904) so ​​that the first polarized light (2906a) is directed into the first zone (2908a) and the second polarized light (2906b) is directed into the second zone (2908b). As illustrated, at some parts of the polarization splitting metasurface (2904), the first polarized light (2906a) may be zero-order light passing through the polarization splitting metasurface (2904), and the second polarized light (2906b) may be diffracted light redirected in a different direction. In another part of the polarization splitting metasurface (2904), the second polarized light (2906b) may be zero-order light passing through the polarization splitting metasurface (2904), and the first polarized light (2906a) may be diffracted light redirected in a different direction. One microlens (2902a) is centered in the first zone (2908a). One microlens (2902b) is centered in the second zone (2908b).

[0183] FIG. 29b is a plan view of an imaging device including a polarization separation function according to an embodiment of the present invention. The imaging device includes a microlens and a polarization splitting metasurface as described in relation to FIG. 29a. However, the polarization splitting metasurface splits light into a first polarized light (2910a), a second polarized light (2910b), a third polarized light (2910c), and a fourth polarized light (2910d) (vertically). As illustrated, the polarization splitting metasurface splits each polarized light in different directions, and the fourth polarized light (2910d) is zero-order light passing through the polarization splitting metasurface (2904). The imaging device includes an image sensor comprising a first zone (2912a), a second zone (2912b), a third zone (2912c), and a fourth zone (2912d). Various parts of the polarization splitting metasurface split the light such that all of the first polarized light (2910a) goes to the first zone (2912a), the second polarized light (2910b) goes to the second zone (2912b), the third polarized light (2910c) goes to the third zone (2912c), and the fourth polarized light (2910d) goes to the fourth zone (2912d).

[0184] FIG. 29c is a plan view of an imaging device including a polarization splitting function according to an embodiment of the present invention. The imaging device includes a first polarization splitting portion (2914a), a second polarization splitting portion (2914b), and a third polarization splitting portion (2914c), each configured similarly to the imaging device described in relation to FIG. 29a. As illustrated, in each of the first polarization splitting portion (2914a), the second polarization splitting portion (2914b), and the third polarization splitting portion (2914c), light is split so that different polarizations are split in different directions. Each of the first polarization splitting portion (2914a), the second polarization splitting portion (2914b), and the third polarization splitting portion (2914c) can operate for different polarizations. For example, the first polarization splitting part (2914a) can split light into the first polarization and the second polarization, and the second polarization splitting part (2914b) splits light into the third polarization and the fourth polarization.

[0185] FIG. 30a illustrates a cross-sectional view of an imaging device including a polarization separation function according to an embodiment of the present invention. Such a device operates similarly to the device described in relation to FIG. 28a. As illustrated, the imaging device includes an array of microlenses (3002a, 3002b, 3002c) that directs light (3003) into a polarization splitting metasurface (3004). The polarization splitting metasurface (3004) splits the light (3003) into a first polarized light (3006a) and a second polarized light (3006b). The first polarized light (3006a) and the second polarized light (3006b) may have orthogonal polarizations. An image sensor comprising a first zone (3008a), a second zone (3008b), a third zone (3008c), and a fourth zone (3008d) is positioned below a polarization splitting metasurface (3004). A first polarized light (3006a) is directed into the first zone (3008a), and a second polarized light (3006b) is directed into the second zone (3008b). Adjacent portions of the polarization splitting metasurface (3004) have a splitting function directed in opposite directions, thereby causing the second polarized light (3006b) from adjacent portions of the polarization splitting metasurface (3004) to be directed into the second zone (3008b). The polarization split metasurface (3004) is repeated so that the first polarized light (3006a) is directed into the third zone (3008c) and the second polarized light (3006b) is directed into the fourth zone (3008d). One microlens (3002a) is offset from the first zone (3008a) and the second zone (2808b) so that the center of the microlens (3002a) is between the first zone (3008a) and the second zone (3008b). One microlens (3002b) is offset from the second zone (3008b) and the third zone (3008c) so that the center of the microlens (3002b) is between the second zone (3008b) and the third zone (3008c).One microlens (3002c) is offset from the third zone (3008c) and the fourth zone (3008d) so that the center of the microlens (3002b) is between the third zone (3008c) and the fourth zone (3008d).

[0186] FIG. 30b illustrates a plan view of an imaging device including a polarization separation function according to an embodiment of the present invention. The imaging device includes a microlens and a polarization splitting metasurface as described in relation to FIG. 30a. However, the polarization splitting metasurface splits light into a first polarized light (3010a), a second polarized light (3010b), a third polarized light (3010c), and a fourth polarized light (3010d). The imaging device includes an image sensor comprising a first zone (3012a), a second zone (3012b), a third zone (3012c), and a fourth zone (3012d). The first polarized light (3010a) is directed into the first zone (3012a), the second polarized light (3010b) is directed into the second zone (3012b), the third polarized light (3010c) is directed into the third zone (3012c), and the fourth polarized light (3010d) is directed into the fourth zone (3012d).

[0187] FIG. 30c is a plan view of an imaging device including a polarization separation function according to an embodiment of the present invention. The imaging device includes a first polarization splitting portion (3014a), a second polarization splitting portion (3014b), and a third polarization splitting portion (3014c), each configured similarly to the imaging device described in relation to FIG. 30a. As illustrated, in each of the first polarization splitting portion (3014a), the second polarization splitting portion (3014b), and the third polarization splitting portion (3014c), light is split so that different polarizations are split in different directions. Each of the first polarization splitting portion (3014a), the second polarization splitting portion (3014b), and the third polarization splitting portion (3014c) can operate for different polarizations. For example, the first polarization splitting part (3014a) can split light into the first polarization and the second polarization, and the second polarization splitting part (3014b) splits light into the third polarization and the fourth polarization.

[0188] FIG. 31a illustrates a cross-sectional view of an imaging device including a polarization filtering function according to an embodiment of the present invention. As illustrated, the imaging device includes an array of microlenses (3102a, 3102b) that directs light (3103) into a polarization filtering metasurface (3104). A portion of the polarization filtering metasurface (3104) is configured to transmit first polarized light (3106a). A portion of the polarization filtering metasurface (3104) is configured to transmit second polarized light (3106b). An image sensor comprising a first zone (3108a) and a second zone (3108b) is positioned below the polarization splitting metasurface (3104). The first polarized light (3106a) is directed into the first zone (3108a), and the second polarized light (3106b) is directed into the second zone (3108b). One microlens (3102a) is aligned with the first zone (3108a), and one microlens (3102b) is aligned with the second zone (3108b).

[0189] FIG. 31b illustrates a plan view of an imaging device including a polarization filtering function according to an embodiment of the present invention. The imaging device includes a microlens and a polarization filtering metasurface as described in relation to FIG. 30a. However, different parts of the polarization filtering metasurface divide light into a first polarized light (3110a), a second polarized light (3110b), a third polarized light (3110c), and a fourth polarized light (3110d). The imaging device includes an image sensor comprising a first zone (3112a), a second zone (3112b), a third zone (3112c), and a fourth zone (3112d). The first polarized light (3110a) is directed into the first zone (3112a), the second polarized light (3110b) is directed into the second zone (3112b), the third polarized light (3110c) is directed into the third zone (3112c), and the fourth polarized light (3110d) is directed into the fourth zone (3112d).

[0190] FIG. 32a illustrates a cross-sectional view of an imaging device including a polarization filtering function according to an embodiment of the present invention. This imaging device shares many features with the imaging device described in relation to FIG. 31a. The imaging device includes an array of microlenses (3202a, 3202b) that direct light (3203) into a polarization filtering metasurface. The polarization filtering metasurface includes a first zone (3204a) that transmits a first polarization of light and a second zone (3204b) that transmits a second polarization of light. The first zone (3204a) filters all polarizations from the light (3203) except for the first polarization of light, and the second zone (3204b) filters all polarizations from the light (3203) except for the second polarization of light. Below the polarization filtering metasurface, there is an image sensor including a first pixel (3206a) and a second pixel (3206b). A first zone (3204a) corresponds to the first pixel (3206a) so that the first polarization of light is detected by the first pixel (3206a), and a second zone (3204b) corresponds to the second pixel (3206b) so that the second polarization of light is detected by the second pixel (3206b).

[0191] FIG. 32b illustrates a cross-sectional view of an imaging device including a polarization sorting function according to an embodiment of the present invention. This imaging device shares many features with the imaging device described in relation to FIG. 28a. The imaging device includes an array (3208) of microlenses that direct light into a polarization sorting metasurface (3210). The polarization sorting metasurface (3210) sorts light into a first polarization of light and a second polarization of light. The polarization sorting metasurface is separated from the optical sensor by a back focal length (3214). The back focal length may be 10 μm. The optical sensor includes a first pixel (3212a) configured to detect the first polarization light and a second pixel (3212b) configured to detect the second polarization light.

[0192] FIG. 32c illustrates a cross-sectional view of an imaging device including a polarization sorting function according to an embodiment of the present invention. This imaging device shares many features with the imaging device described in relation to FIG. 30a. The imaging device includes an array of microlenses (3216a, 3216b) that direct light into a polarization sorting metasurface. The polarization sorting metasurface includes a first zone (3218a) configured to sort a first polarized light in a first direction and a second polarized light in a second direction, and a second zone (3218b) configured to sort a second polarized light in a first direction and a first polarized light in a second direction. An image sensor is configured to receive light from the polarization sorting metasurface. The image sensor includes a first pixel (3220a) configured to receive first polarized light from a first zone (3218a), a second pixel (3220b) configured to receive second polarized light from the first zone (3218a) and the second zone (3218b), and a third pixel (3220c) configured to receive first polarized light from the second zone (3218b). As illustrated in FIG. 30a, there may be more zones of microlenses and polarization classification metasurfaces close to those illustrated in FIG. 32c. There may be one microlens for every pixel of the image sensor. The back focal length (3222) may be shorter than the back focal length (3214) of the image sensor in FIG. 32b. For example, the rear focal length (3222) may be 5 μm compared to the rear focal length (3214) of 10 μm.

[0193] FIG. 33a illustrates a top view of an exemplary polarization filtering nanograting according to an embodiment of the present invention. In some embodiments, the polarization filtering nanograting may be interspersed with the polarization filtering metasurface described in relation to FIG. 31a, FIG. 31b, and FIG. 32a. The polarization filtering nanograting comprises different zones (3302a, 3302b, 3302c, 3302d) corresponding to different pixels of the image sensor. The nanograting may have some advantages such as being easier to manufacture and less sensitive to angles, which may then make it advantageous to manufacture an imaging sensor comprising both the PSOMMA and the nanograting. In some embodiments, the microlens array functionality for the polarization pixels may be implemented as one or more metagratings, and the microlens array functionality for the general pixels may be implemented as one or more refractive lenses.

[0194] FIG. 33b illustrates a plan view of an exemplary polarization classification metasurface according to an embodiment of the present invention. Such a polarization classification metasurface may be utilized as the polarization classification metasurface described in relation to FIGS. 28a to 28c, FIGS. 29a to 29c, FIGS. 30a to 30c, FIG. 32b, and FIG. 32c. The polarization filtering metasurface includes different regions (3304a, 3304b, 3304c, 3304d) corresponding to different pixels of an image sensor.

[0195] FIG. 34a illustrates a schematic plan view of an exemplary linear polarization sorting metasurface according to an embodiment of the present invention. Such a linear polarization sorting metasurface may be utilized within the polarization sorting metasurface described in relation to FIG. 28a through 28c, FIG. 29a through 29c, FIG. 30a through 30c, FIG. 32b, and FIG. 32c. The linear polarization sorting metasurface diffracts different linear polarizations in different directions. In some embodiments, the different linear polarizations may be different linear polarizations that are orthogonal to each other. As illustrated, the linear polarization sorting metasurface comprises various metasurface elements (3402). The metasurface elements (3402) may have different dimensions (e.g., height and / or width). The metasurface elements (3402) may have the same orientation. The metasurface elements may be tilted at the same angle.

[0196] FIG. 34b illustrates a schematic plan view of an exemplary circular polarization sorting metasurface according to an embodiment of the present invention. Such a circular polarization sorting metasurface may be utilized within the polarization sorting metasurface described in relation to FIGS. 28a through 28c, FIGS. 29a through 29c, FIGS. 30a through 30c, FIG. 32b, and FIG. 32c. The circular polarization sorting metasurface diffracts different circular polarizations in different directions. In some embodiments, the different circular polarizations may be different circular polarizations that are opposite to each other. In some embodiments, the polarization sorting metasurfaces may sort different elliptical polarizations in different directions.

[0197] The circular polarization classification metasurface includes various metasurface elements (3404). The metasurface elements (3404) may have the same dimensions (e.g., height and / or width). The metasurface elements may have different tilt angles with respect to the horizontal axis and / or vertical axis.

[0198] FIG. 34c illustrates a schematic plan view of a random polarization sorting metasurface according to an embodiment of the present invention. Such a random polarization sorting metasurface may be utilized within the polarization sorting metasurface described in relation to FIGS. 28a through 28c, FIGS. 29a through 29c, FIGS. 30a through 30c, FIG. 32b, and FIG. 32c. A random polarization sorting metasurface diffracts different random polarizations in different directions. In some embodiments, the different polarizations may be different polarizations that are orthogonal or opposite to each other. Any polarization may be elliptical, circular, or linear. Linear polarization may be vertical and horizontal. Linear or elliptical polarization may be tilted. Circular polarization may be right-sided and left-sided polarization. Elliptical polarization may be right-sided and left-sided polarization. Any polarization sorting metasurface includes various metasurface elements (3406). The metasurface elements (3406) may have the same dimensions (e.g., height and / or width). The metasurface elements (3406) may have different tilt angles with respect to the horizontal axis and / or vertical axis.

[0199] FIG. 35a illustrates a sparsely distributed PSOMMA sensor according to an embodiment of the present invention. A polarization-splitting metasurface (3504) is sparsely distributed on a wafer (3502). The remaining area of ​​the wafer (3502) may be patterned only with MLA or metasurface-MLA without polarization sorting, or may not be patterned at all. The wafer (3502) may be transparent glass. In some embodiments, the remaining area of ​​the wafer (3502) may be patterned with a metasurface MLA that does not split polarization. Such a metasurface MLA may have identical lenslets that are repeated throughout, or may include lenslets having different focal lengths, which may be used for autofocusing or for distance and depth measurement and calculation. One or more different filters may be applied to some or all of the lenslets. The sparse configuration may be asymmetric, irregular, or variable across the entire sensor space. For example, to better capture points on the face, or perhaps to better capture images of the eyes, a sparse configuration in the center may be used, or a denser central configuration may be used. Additionally, different polarization modes may be used in different zones (e.g., linear polarization may be used at the edges and circular polarization at the center).

[0200] FIG. 35b shows a cross-sectional view of a sparsely distributed PSOMMA sensor according to an embodiment of the present invention. A PSOMMA wafer (3502) including a polarization sorting metasurface (3504) is integrated on an image sensor (3506). The image sensor (3506) is positioned below the polarization sorting metasurface (3504). The image sensor (3506) may include an R, G, B image sensor having R, G, B filters corresponding to their corresponding color sensors. The image sensor (3506) has a portion of the region where the RGB filters are removed to make it black and white within that region—and, now the PSOMMA wafer (3502) splits the light into pixels within that black and white region.

[0201] The image sensor (3506) may include a color filter capable of passing near-infrared light in a portion that receives polarized light from an empty, monochromatic, or polarization-dividing metasurface (3504). In some embodiments, the color from the sparse PSOMMA pixels may be interpolated from adjacent pixels. In this illustration, some of the pixels have a Bayer pattern, and there is a subset of pixels that are polarimetric pixels (PSOMMA pixels). When reconstructing the image, for the pixels below the PSOMMA pixels, the image can be reconstructed in full color by using nearby RGB pixels as values ​​for the color in the PSOMMA pixels, even if the PSOMMA pixels do not directly capture color.

[0202] In a sparse format, the open window or pixel may be for simply intensity measurement and / or depth from defocus. Alternatively, the sparse metasurface (3504) may have a metasurface for each pixel, but only some are polarization-based metasurface pixels and others have other characteristics such as focusing and deflection. In some cases, various lenslets with different focal powers may be used with point-size differentiation to determine distances—e.g., lenses of different focal lengths—so that image points are compared to find the most focused point for determining distances or other features, known as phase detection autofocus (PDAF).

[0203] FIG. 36a shows an example of a PSOMMA sensor according to an embodiment of the present invention. The PSOMMA (3602) is positioned below a color filter (3604). Each lenslet of the PSOMMA (3602) has a section of the color filter (3604) at its top. The PSOMMA lenslets are optimized for the color of the section corresponding to the color filter (3604). The PSOMMA (3602) directs light into different sections of the image sensor (3606) as described above. Different image modes can be distinguished for different pixels of the image sensor (3606).

[0204] FIG. 36b illustrates an exemplary PSOMMA sensor according to an embodiment of the present invention. The PSOMMA (3654) is positioned across the cover glass (3652). The PSOMMA (3654) may be positioned such that a specific area of ​​the cover glass (3652) has a PSOMMA lenslet and most of the area is bare, allowing light to pass through the cover glass. The PSOMMA (3654) directs light toward a specific color of the image sensor (3658). For example, the PSOMMA (3654) may direct light toward a green pixel of the image sensor (3658). The operation of the PSOMMA (3654) has been described throughout the specification. The image sensor (3658) may include RGB sensor pixels, which may have a built-in filter or an added separate color filter (3656). The image sensor (3658) may be a basic RGB sensor utilizing RGB sensor colors. Then, the part of the cover glass (3652) that does not contain the PSOMMA (3654) can be allowed to pass light through as is and be captured normally within the image sensor (3658).

[0205] Although several embodiments of the present invention have been described in detail, it should be understood that the present invention may be embodied in many other forms without departing from the spirit or scope of the invention. For example, the embodiments listed below are considered:

[0206] Section 1. As a polarization imaging device,

[0207] A metasurface lenslet array comprising a plurality of metasurface lenslets—the plurality of metasurface lenslets comprising a plurality of first metasurface lenslets configured to diffract image light with an intensity proportional to a first polarized light in a first direction and with an intensity proportionality with a second polarized light in a second direction—; and

[0208] An image sensor located in the optical paths of the first polarization and the second polarization.

[0209] Includes,

[0210] The image sensor is a polarization imaging device comprising a plurality of image detection units, each comprising a first image detection unit positioned to detect a first polarization and a second image detection unit positioned to detect a second polarization.

[0211] Section 2. In Section 1,

[0212] The above-mentioned plurality of image sensing units is a polarization imaging device, wherein the image sensing units are different pixels and / or zones of the image sensor.

[0213] Section 3. In Section 1,

[0214] The above-mentioned plurality of image sensing units is a polarization imaging device, wherein the plurality of image sensing units are different pixels and / or zones of a plurality of image sensors.

[0215] Section 4. In Section 1,

[0216] The above plurality of metasurface lenslets,

[0217] It further includes a plurality of second metasurface lenslets interspersed among the first metasurface lenslets and configured to diffract the image light with an intensity proportional to the third polarization in the third direction and an intensity proportional to the fourth polarization in the fourth direction,

[0218] The above plurality of image detection units,

[0219] A polarization imaging device further comprising a third image detection unit positioned to detect the third polarization and a fourth image detection unit positioned to detect the fourth polarization.

[0220] Section 5. In Section 4,

[0221] The first polarization and the third polarization are the same polarization, and

[0222] A polarization imaging device in which the second polarization and the fourth polarization light are the same polarization.

[0223] Section 6. In Section 1,

[0224] The above-mentioned first metasurface lenslet is,

[0225] The above image light is further configured to diffract with an intensity proportional to the third polarization in the third direction and an intensity proportional to the fourth polarization in the fourth direction, and

[0226] The above plurality of image detection units,

[0227] A polarization imaging device further comprising a third image detection unit positioned to detect the third polarization and a fourth image detection unit positioned to detect the fourth polarization.

[0228] Section 7. In Section 1,

[0229] The above-mentioned first metasurface lenslet is,

[0230] A polarization imaging device further configured to transmit zero-order light in a third direction.

[0231] Section 8. In Section 7,

[0232] The above third direction is different from the above first direction and the above second direction, a polarization imaging device.

[0233] Section 9. In Section 8,

[0234] The above plurality of image detection units,

[0235] A polarization imaging device further comprising a third image detection unit configured to detect the above zero-order light.

[0236] Section 10. In Section 7,

[0237] A polarization imaging device in which the third direction is the same as the first direction.

[0238] Section 11. In Section 1,

[0239] The above polarization imaging device further includes one or more refractive lenses, and

[0240] The above metasurface lenslet array is a polarization imaging device located between the refractive lens and the image sensor.

[0241] Section 12. In Section 1,

[0242] The first polarization and the second polarization are different polarizations, and

[0243] The first polarization and the second polarization are,

[0244] A polarization imaging device selected from the group consisting of linear polarization, diagonal polarization, elliptical polarization and circular polarization.

[0245] Section 13. In Section 1,

[0246] The above polarization imaging device further includes a microlens array, and

[0247] The above metasurface lenslet array is a polarization imaging device comprising a polarization metasurface.

[0248] Section 14. In Section 13,

[0249] The above microlens array is configured to separate the image light into different pixels, and

[0250] A polarization imaging device configured such that the polarization metasurface diffracts the first polarization in the first direction and the second polarization in the second direction.

[0251] Section 15. In Section 14,

[0252] The above polarization metasurface overlaps with the microlenses in the microlens array, and

[0253] Microlenses that are not superimposed by the polarization metasurface within the microlens array transmit undiffracted light to the image sensor, and

[0254] A polarization imaging device configured such that the image sensor detects the non-diffracted light and measures the intensity of the non-diffracted light.

[0255] Section 16. In Section 13,

[0256] The above-described microlens array is a planar microlens array layer, a polarization imaging device.

[0257] Section 17. In Section 13,

[0258] The above-described microlens array and the above-described polarization metasurface are polarization imaging devices located on a single substrate.

[0259] Section 18. In Section 17,

[0260] A polarization imaging device in which the primary image plane is configured on the surface of the single substrate opposite to the image sensor.

[0261] Section 19. In Section 17,

[0262] The above main image plane is a polarization imaging device configured within the single substrate.

[0263] Section 20. In Section 13,

[0264] The above-described microlens array is a polarization imaging device comprising metasurface elements.

[0265] Section 21. In Section 20,

[0266] The above metasurface element is,

[0267] A polarization imaging device configured to accommodate a chief ray angle that varies across the imaging pupil of the polarization imaging device and to collimate light passing through each microlens.

[0268] Section 22. In Section 20,

[0269] A polarization imaging device configured such that the metasurface elements of the above-described microlens array provide a refractive microlens effect.

[0270] Section 23. In Section 13,

[0271] The above-described microlens array is a polarization imaging device comprising a conventional refractive microlens array.

[0272] Section 24. In Section 13,

[0273] The above microlens array is,

[0274] A polarization imaging device comprising a combination of at least one conventional refractive microlens and at least one metasurface element.

[0275] Section 25. In Section 1,

[0276] The above polarization imaging device is,

[0277] A polarization imaging device further comprising a color filter positioned above or below the above metasurface lenslet array.

[0278] Section 26. In Section 25,

[0279] The above color filter is,

[0280] It is positioned on the above metasurface lenslet array and filters light into different colors corresponding to each of the different metasurface lenslets of the plurality of first metasurface lenslets, and

[0281] The above metasurface lenslet is a polarization imaging device that receives color-filtered light from the color filter.

[0282] Section 27. In Section 25,

[0283] The above color filter is,

[0284] A polarization imaging device positioned below the above metasurface lenslet array and filtering diffracted first polarized light and second polarized light.

[0285] Section 28. In Section 27,

[0286] The above color filter is,

[0287] A polarization imaging device comprising different zones for filtering different wavelengths of light.

[0288] Section 29. In Section 28,

[0289] The above plurality of first metasurface lenslets are spaced apart on a cover substrate, forming a polarization imaging device.

[0290] Section 30. In Section 29,

[0291] The separated first metasurface lenslet is,

[0292] A polarization imaging device configured to output light onto a region of the color filter that outputs the same color of light.

[0293] Section 31. In Section 30,

[0294] The above-mentioned spaced-apart first metasurface lenslet is,

[0295] A polarization imaging device that outputs light on the gap between a region of the color filter that outputs green light and regions of the color filter that output green light and red or blue light.

[0296] Section 32. In Section 27,

[0297] The above color filter is,

[0298] It includes different zones that output red, green, or blue light and zones that output black and white or near-infrared light, and

[0299] The plurality of first metasurface lenslets are spaced apart on the cover substrate, and

[0300] The above-described spaced first metasurface lenslet is a polarization imaging device that outputs light on a region of the color filter that outputs black and white or near-infrared light.

[0301] Section 33. In Section 1,

[0302] The above polarization imaging device is,

[0303] A polarization imaging device further comprising a microlens array comprising a plurality of distinct microlenses that output collimated light into the metasurface lenslet array.

[0304] Section 34. In Section 33,

[0305] A polarization imaging device wherein the plurality of segmented microlenses are positioned between adjacent image sensing units such that light from the plurality of separate microlenses is diffracted into adjacent image sensing units in opposite oblique directions by the plurality of first metasurface lenslets.

[0306] Section 35. In Section 34,

[0307] Multiple repeating metasurface lenslets,

[0308] It further includes a plurality of second metasurface lenslets configured to diffract the first polarization in the second direction and the second polarization in the first direction,

[0309] The first image detection unit above is,

[0310] A polarization imaging device further positioned to detect the first polarization diffracted from the plurality of second metasurface lenslets.

[0311] Section 36. In Section 33,

[0312] The above plurality of divided microlenses,

[0313] A center is positioned on each of the plurality of first metasurface lenslets so that the first polarization passes through one of the plurality of first metasurface lenslets and is transmitted to the first image sensing unit, and

[0314] The above second polarization is a polarization imaging device in which the second polarization is diffracted on the second image sensing unit in the above second direction.

[0315] Section 37. In Section 1,

[0316] A polarization imaging device in which the first polarization and the second polarization are orthogonal linear polarization, circular polarization, elliptical polarization, or any polarization.

[0317] Section 38. In Section 1,

[0318] A polarization imaging device in which the plurality of first metasurface lenslets are spaced apart, having a non-diffractive section between adjacent first metasurface lenslets.

[0319] Section 39. In Section 38,

[0320] The above plurality of metasurface lenslets,

[0321] The plurality of image sensing units configured to detect the first polarization and the second polarization are,

[0322] A polarization imaging device positioned between image detection units of the image sensor configured to detect red, green, or blue light.

[0323] Section 40. In Section 39,

[0324] The above plurality of first metasurface lenslets are,

[0325] A polarization imaging device configured to diffract light into the plurality of image sensing units configured to detect the first polarization and the second polarization, and the image sensing units configured to detect red, green, or blue light.

[0326] Section 41. In Section 39,

[0327] The plurality of image sensing units configured to detect the first polarization and the second polarization are,

[0328] A polarization imaging device further configured to detect black and white or near-infrared light.

[0329] Section 42. In Section 1,

[0330] The above image sensor is,

[0331] A polarization imaging device that is a complementary metal-oxide semiconductor (CMOS) image sensor, a charge-coupled element sensor, a silicon diode sensor, a cadmium-sulfide sensor diode, an RGB sensor, an indirect time of flight (iToF) sensor, and a direct time of flight (dToF) sensor.

[0332] Section 43. A method for manufacturing a polarization imaging device,

[0333] Step of providing an image sensor wafer;

[0334] A step of depositing a spacing layer on the imaging sensor wafer;

[0335] A step of depositing a metasurface layer on the above-mentioned spacing layer; and

[0336] Step of patterning the metasurface layer to form individual metasurfaces on the above-mentioned spacing layer

[0337] A method for manufacturing a polarization imaging device comprising

[0338] Section 44. In Section 43,

[0339] The above method is,

[0340] A method for manufacturing a polarization imaging device, further comprising the step of directly depositing an encapsulation layer on the individual metasurface.

[0341] Section 45. In Section 44,

[0342] The above method is,

[0343] A method for manufacturing a polarization imaging device, further comprising the step of flattening the encapsulation layer so that the encapsulation layer is at the same level as the individual metasurface.

[0344] Section 46. In Section 44,

[0345] The above method is,

[0346] A method for manufacturing a polarization imaging device, further comprising the step of forming one or more top layers on the encapsulation layer and / or the individual metasurface.

[0347] Section 47. A method for manufacturing a polarization imaging device,

[0348] Step of providing an image sensor wafer;

[0349] A step of providing a metasurface substrate comprising individual metasurfaces; and

[0350] Step of depositing a spacing layer on the imaging sensor wafer and / or the individual metasurface on the metasurface substrate

[0351] A method for manufacturing a polarization imaging device comprising

[0352] Section 48. In Section 47,

[0353] The above method is,

[0354] A method for manufacturing a polarization imaging device, further comprising the step of bonding the above metasurface substrate to the above image sensor wafer.

[0355] Section 49. In Section 48,

[0356] A method for manufacturing a polarization imaging device, wherein the above metasurface substrate includes a transparent substrate.

[0357] Section 50. In Section 48,

[0358] The above method is,

[0359] A method for manufacturing a polarization imaging device, further comprising the step of removing the metasurface substrate so that the individual metasurfaces remain attached to the image sensor wafer.

[0360] Section 51. In Section 50,

[0361] The step of removing the above metasurface substrate is,

[0362] A method for manufacturing a polarization imaging device, comprising polishing, lapping, wet chemical etching, and / or dry chemical etching the metasurface substrate.

[0363] Section 52. In Section 47,

[0364] The above method is,

[0365] Singulating the metasurface substrate into separate metasurface dies, coupling at least one of the metasurface dies onto the image sensor wafer, and singulating the image sensor wafer into image sensor dies each having the coupled metasurface die

[0366] A method for manufacturing a polarization imaging device, further comprising

[0367] Section 53. A method for manufacturing a polarization imaging device,

[0368] A step of providing a CMOS image sensor (CIS) wafer;

[0369] A step of positioning a first plurality of spacers on the CIS wafer;

[0370] Step of providing a first carrier wafer;

[0371] A step of positioning a first adhesive layer on the first carrier wafer;

[0372] A step of bringing the first plurality of spacers into contact with the first adhesive layer so that the adhesive is applied to the top of the first plurality of spacers;

[0373] A step of providing a nanofilament substrate having a plurality of nanofilaments;

[0374] A step of positioning a second plurality of spacers on the nanopillar substrate;

[0375] Step of providing a second carrier wafer;

[0376] A step of positioning a second adhesive layer on the second carrier wafer;

[0377] A step of bringing the second plurality of spacers into contact with the second adhesive layer so that an adhesive is applied to the top of the second plurality of spacers; and

[0378] A step of simultaneously contacting the adhesive on the second plurality of spacers with the CIS wafer and the adhesive on the first plurality of spacers with the nanofilament substrate so as to bond the CIS wafer and the nanofilament substrate to each other.

[0379] A method for manufacturing a polarization imaging device comprising

[0380] Section 54. In Section 53,

[0381] A method for manufacturing a polarization imaging device, wherein after contacting the first plurality of spacers with the first adhesive layer such that an adhesive is applied to the upper portions of the first plurality of spacers, the adhesive is present only on the upper portions of the first plurality of spacers and not on the zones between the first plurality of spacers.

[0382] Section 55. In Section 53,

[0383] A method for manufacturing a polarization imaging device, wherein after contacting the second plurality of spacers with the second adhesive layer such that an adhesive is applied to the upper portions of the second plurality of spacers, the adhesive is present only on the upper portions of the second plurality of spacers and not in the area between the second plurality of spacers.

[0384] Section 56. In Section 53,

[0385] The above-mentioned first plurality of spacers,

[0386] A method for manufacturing a polarization imaging device comprising a pair of spacers forming a channel whose size is determined to accommodate one of the second plurality of spacers.

[0387] Section 57. In Section 56,

[0388] A method for manufacturing a polarization imaging device, wherein the first plurality of spacers and the second plurality of spacers are interlocked with each other such that one of the second plurality of spacers is located within one of the channels formed by the pair of spacers.

[0389] Section 58. In Section 53,

[0390] A method for manufacturing a polarization imaging device, wherein the first plurality of spacers form a plurality of rectangular shapes on the CIS wafer.

[0391] Section 59. In Section 58,

[0392] A method for manufacturing a polarization imaging device in which the interior of the above rectangular shape forms an imaging zone.

[0393] Section 60. In Section 53,

[0394] A method for manufacturing a polarization imaging device, wherein the second plurality of spacers form a grid pattern on the nanopillar substrate.

[0395] Section 61. In Section 53,

[0396] A method for manufacturing a polarization imaging device, wherein the first plurality of spacers and the second plurality of spacers form a void between the plurality of nanopillars and the CIS wafer.

[0397] Section 62. A method for manufacturing a polarization imaging device,

[0398] A step of providing a CMOS image sensor (CIS) wafer having bonding pads;

[0399] A step of depositing a first dielectric layer on the CIS wafer and the bonding pad; a step of planarizing the first dielectric layer;

[0400] A step of providing a nanofilament substrate having multiple nanofilaments;

[0401] A step of providing a second dielectric layer on the plurality of nanopillars;

[0402] A step of flattening the second dielectric layer;

[0403] A step of bringing the first dielectric layer and the second dielectric layer into contact so that they are combined to form a combined dielectric layer;

[0404] A step of removing the nanofilament substrate to expose the plurality of nanofilaments;

[0405] A step of partially etching the bonded dielectric layer to expose the bonding pad; and

[0406] Step of forming a conductive layer electrically connected to the bonding pad through the combined dielectric layer

[0407] A method for manufacturing a polarization imaging device comprising

[0408] Section 63. In Section 62,

[0409] A method for manufacturing a polarization imaging device, wherein the first dielectric layer and the second dielectric layer are silicon dioxide layers.

[0410] Section 64. In Section 62,

[0411] The step of depositing the first dielectric layer and the second dielectric layer is,

[0412] A method for manufacturing a polarization imaging device, performed by a TEOS (tetraethylorthosilicate) process.

[0413] Section 65. In Section 64,

[0414] A method for manufacturing a polarization imaging device, wherein the above TEOS process is a plasma enhanced TEOS process.

[0415] Section 66. In Section 62,

[0416] The step of flattening the first dielectric layer and the second dielectric layer is,

[0417] A method for manufacturing a polarization imaging device, performed by a chemical mechanical polishing (CMP) process.

[0418] Section 67. In Section 62,

[0419] The step of removing the above nanofilament substrate is,

[0420] A method for manufacturing a polarization imaging device, performed by a polishing, etching, or chemical mechanical polishing (CMP) process.

[0421] Section 68. In Section 62,

[0422] The step of partially etching the combined dielectric layer is,

[0423] Patterning the combined dielectric layer above; and

[0424] Etching the bonded dielectric layer to expose the bonding pad

[0425] A method for manufacturing a polarization imaging device comprising

[0426] Section 69. In Section 62,

[0427] The above method is,

[0428] A method for manufacturing a polarization imaging device, further comprising the step of growing a barrier seed layer on the bonding pad, the sidewall of the bonded dielectric layer, and the nanopillar.

[0429] Section 70. As a polarization imaging device,

[0430] Microlens array having at least two microlenses;

[0431] A polarization filtering metasurface having two or more polarization filtering zones; and

[0432] Imaging sensor having at least two zones

[0433] Includes,

[0434] Imaging light including one or more polarization states is directed onto the polarization filtering metasurface by the microlens, and

[0435] A polarization imaging device, wherein the polarization filtering metasurface is configured to orient one or more polarization states onto one or more zones of the imaging sensor.

[0436] Section 71. In Section 70,

[0437] The above-described microlens array is a polarization imaging device comprising refractive microlenses.

[0438] Section 72. In Section 71,

[0439] The above microlens array is,

[0440] A polarization imaging device further comprising a metasurface configured to provide a refractive microlens effect.

[0441] Section 73. In Section 70,

[0442] The above microlens array is,

[0443] A polarization imaging device comprising a metasurface configured to provide a refractive microlens effect.

[0444] Section 74. In Section 70,

[0445] A polarization imaging device, wherein each of the above at least two microlenses is located over a single zone among one or more zones of the imaging sensor.

[0446] Section 75. In Section 70,

[0447] A polarization imaging device, wherein each of the above at least two microlenses is positioned over two or more sensor zones among one or more zones of the imaging sensor.

[0448] Section 76. In Section 70,

[0449] A polarization imaging device in which each of the above at least two microlenses is located on a single polarization zone among one or more zones of two or more polarization filtering zones.

[0450] Section 77. In Section 70,

[0451] A polarization imaging device, wherein each of the above at least two microlenses is positioned over two or more sensor zones among the above two or more polarization filtering zones.

[0452] Section 78. As a polarization imaging device,

[0453] A meta-grating array comprising a plurality of meta-grating lenslets—the plurality of meta-grating lenslets comprising a plurality of first meta-grating lenslets configured to filter image light into first polarized light in a first zone and second polarized light in a second zone—; and

[0454] An image sensor located in the optical paths of the first polarization and the second polarization.

[0455] Includes,

[0456] The image sensor is a polarization imaging device comprising a plurality of image detection units, each comprising a first image detection unit positioned to detect a first polarization and a second image detection unit positioned to detect a second polarization.

[0457] Section 79. In Section 34,

[0458] A polarization imaging device in which the first polarization is orthogonal to the second polarization.

[0459] Section 80. In Section 34,

[0460] Each microlens is,

[0461] A polarization imaging device that uses superimposed metasurface lenslets to cover half of two image sensing units, thereby diffracting a first polarization in the first direction into the first image sensing unit and a second polarization in the second direction into the second image sensing unit.

[0462] Section 81. In Section 80,

[0463] A polarization imaging device in which the first polarization is orthogonal to the second polarization.

[0464] Equality theory

[0465] Although the foregoing descriptions include many specific embodiments of the present invention, they should not be understood as a limitation to the scope of the present invention, but as examples of embodiments of the present invention. Therefore, it should be understood that the present invention may be practiced in a manner different from that specifically described without departing from the scope and spirit of the present invention. Accordingly, embodiments of the present invention should be regarded as illustrative and not limiting in any case. Accordingly, the scope of the invention should be determined by the appended claims and their equivalents, rather than by the illustrative embodiments.

Claims

Claim 1 A polarization imaging device comprising a metasurface lenslet array including a plurality of metasurface lenslets, wherein the plurality of metasurface lenslets comprises a plurality of first metasurface lenslets configured to diffract light with an intensity proportional to a first polarized light in a first direction and an intensity proportional to a second polarized light in a second direction, and a plurality of second metasurface lenslets interspersed between the first metasurface lenslets and configured to diffract the light with an intensity proportional to a third polarized light in a third direction and an intensity proportional to a fourth polarized light in a fourth direction. A polarization imaging device comprising: an image sensor positioned in the optical paths of the first polarization and the second polarization, wherein the image sensor comprises a plurality of image detection units, wherein the plurality of image detection units comprises: a first image detection unit positioned to detect the first polarization; a second image detection unit positioned to detect the second polarization; a third image detection unit positioned to detect the third polarization; a fourth image detection unit positioned to detect the fourth polarization; and a fifth image detection unit interspersed with one or more of the first image detection unit, the second image detection unit, the third image detection unit, and the fourth image detection unit, wherein the light incident on the fifth image detection unit has not passed through the first metasurface lenslet or the second metasurface lenslet, nor has it been diffracted by the first metasurface lenslet or the second metasurface lenslet. Claim 2 A polarization imaging device according to claim 1, wherein the plurality of image sensing units are different pixels and / or regions of the image sensor. Claim 3 delete Claim 4 A polarization imaging device according to claim 1, wherein the first polarization and the third polarization are the same polarization, and the second polarization and the fourth polarization light are the same polarization. Claim 5 delete Claim 6 A polarization imaging device according to claim 1, wherein the first metasurface lenslet is further configured to transmit zero-order light in a fifth direction. Claim 7 In claim 6, the polarization imaging device, wherein the third direction is different from the first direction and the second direction. Claim 8 In claim 7, the polarization imaging device further comprises a sixth image detection unit configured to detect the zero-order light, wherein the plurality of image detection units further comprises the sixth image detection unit. Claim 9 In claim 6, the polarization imaging device, wherein the third direction is the same as the first direction. Claim 10 In claim 1, the polarization imaging device further comprises a microlens array, and the metasurface lenslet array comprises a polarization metasurface. Claim 11 In claim 10, the polarization imaging device, wherein the microlens array is configured to separate image light into different pixels. Claim 12 A polarization imaging device according to claim 11, wherein a first subset of microlenses overlaps with a first metasurface lenslet and a second metasurface lenslet, and a second subset of microlenses does not overlap with the first metasurface lenslet and the second metasurface lenslet, so that the second subset of microlenses transmits undiffracted light to a fifth image sensing unit, and the fifth image sensing unit detects the undiffracted light and measures the intensity of the undiffracted light. Claim 13 In claim 10, the polarization imaging device wherein the microlens array is a planar microlens array layer. Claim 14 In claim 10, the polarization imaging device wherein the microlens array and the polarization metasurface are located on a single substrate. Claim 15 In claim 10, the polarization imaging device wherein the microlens array comprises metasurface elements. Claim 16 In claim 10, the polarization imaging device comprises a combination of at least one conventional refractive microlens and at least one metasurface element. Claim 17 The polarization imaging device of claim 1 further comprises a microlens array including a plurality of separate microlenses that output collimated light into the metasurface lenslet array. Claim 18 In claim 17, the plurality of separate microlenses is a polarization imaging device positioned between adjacent image sensing units such that light from the plurality of separate microlenses is diffracted into adjacent image sensing units in opposite oblique directions by the plurality of first metasurface lenslets and the plurality of second metasurface lenslets. Claim 19 delete Claim 20 delete Claim 21 A polarization imaging device according to claim 1, wherein the first metasurface lenslets are further configured to diffract light with an intensity proportional to the fifth polarized light in the fifth direction. Claim 22 In claim 21, the polarization imaging device, wherein the fifth direction is different from the first direction and the second direction.

Images