Polarization sorting metasurface microlens array device

Abstract

Systems and methods are disclosed herein that include a polarization sorting metasurface microlens array device. In a particular embodiment, a polarization imaging device is provided. A polarization imaging apparatus includes: an image light source; a metasurface lenslet array including a plurality of repeating metasurface lenslets, where the plurality of repeating metasurface lenslets include a plurality of first metasurface lenslets configured to diffract image light into first polarized light in a first direction and second polarized light in a second direction; and an image sensor in an optical path of the first polarized light and the second polarized light, and wherein the image sensor includes a plurality of image sensing units including a first image sensing unit positioned to sense the first polarized light and a second image sensing unit positioned to sense the second polarized light.

Description

[0001] Cross - reference to related applications

[0002] This application claims the benefit and priority of U.S. Provisional Patent Application 63 / 362,285, filed on March 31, 2022, entitled "Polarization Sorting Metasurface Microlens Array Device", the disclosure of which is incorporated herein by reference in its entirety for all purposes pursuant to 35 U.S.C.§119(e). Technical field

[0003] The present invention generally relates to a polarization sorting device. More specifically, the present invention relates to a polarization sorting device including a polarization sorting metasurface microlens array and a method of manufacturing the polarization sorting device. Background art

[0004] Metasurface elements are diffractive optical elements where individual waveguide elements have sub - wavelength spacing and typically have a planar profile. Metasurface elements for applications in the UV - IR band (300 - 10000 nm) have recently been developed. Compared to traditional refractive optical devices, metasurface elements abruptly introduce a phase shift into the light field. This allows metasurface elements to have a thickness on the order of the wavelength of the light they are designed to operate on, while traditional refractive surfaces have a thickness 10 - 100 times (or more) larger than the wavelength of the light they are designed to operate on. Additionally, metasurface elements can have no height variation in the constituent elements and can thus shape light without any curved surfaces (as required by refractive optical devices). Compared to traditional diffractive optical elements (DOEs), such as binary diffractive optical devices, metasurface elements have the ability to impart a range of phase shifts on the incident light field. At least metasurface elements can have a phase shift between 0 - 2π, with at least 5 different values within this range, while binary DOEs can only provide two different phase shift values and are typically limited to phase shifts of 0 or 1π. Compared to multi - level DOEs, metasurface elements do not require a height variation of their constituent elements along the optical axis, only an in - plane geometry variation of the metasurface element features. Summary of the invention

[0005] In some aspects, the techniques described herein relate to a polarization imaging device, including: a metasurface microlens array including a plurality of metasurface microlenses, wherein the plurality of metasurface microlenses includes a plurality of first metasurface microlenses configured to diffract image light in a first direction with an intensity proportional to a first polarization and in a second direction with an intensity proportional to a second polarization; and an image sensor located in the optical paths of the first polarization and the second polarization, and wherein the image sensor includes a plurality of image sensing units, the plurality of image sensing units including a first image sensing unit positioned to sense the first polarization and a second image sensing unit positioned to sense the second polarization.

[0006] In some aspects, the plurality of image sensing units are different pixels and / or different regions of the image sensor.

[0007] In some aspects, the plurality of image sensing units are different pixels and / or different regions of a plurality of image sensors.

[0008] In some aspects, the plurality of metasurface microlenses further includes a plurality of second metasurface microlenses dispersed among the first metasurface microlenses and configured to diffract image light in a third direction with an intensity proportional to a third polarization and in a fourth direction with an intensity proportional to a fourth polarization, and wherein the plurality of image sensing units further includes a third image sensing unit positioned to sense the third polarization and a fourth image sensing unit positioned to sense the fourth polarization.

[0009] In some aspects, the first polarization and the third polarization are the same polarization, and wherein the second polarization and the fourth polarization light are the same polarization.

[0010] In some aspects, the first metasurface microlens is further configured to diffract image light in a third direction with an intensity proportional to a third polarization and in a fourth direction with an intensity proportional to a fourth polarization, and wherein the plurality of image sensing units further includes a third image sensing unit positioned to sense the third polarization and a fourth image sensing unit positioned to sense the fourth polarization.

[0011] In some aspects, the first metasurface microlens is further configured to transmit zero-order light in the third direction.

[0012] In some aspects, the third direction is different from the first direction and the second direction.

[0013] In some aspects, the plurality of image sensing units further includes a third image sensing unit configured to sense zero-order light.

[0014] In some aspects, the third direction is the same as the first direction.

[0015] In some aspects, the polarization imaging device further comprises one or more refractive lenses, wherein the metasurface microlens array is positioned between the refractive lens and the image sensor.

[0016] In some aspects, the first polarization and the second polarization are different polarizations, and wherein the first polarization and the second polarization are selected from the group consisting of: linearly polarized light, diagonally polarized light, elliptically polarized light, and circularly polarized light.

[0017] In some aspects, the polarization imaging device further comprises a microlens array, and wherein the metasurface microlens array comprises a polarization metasurface.

[0018] In some aspects, the microlens array is configured to separate image light into different pixels and the polarization metasurface is configured to diffract the first polarization in a first direction and diffract the second polarized light in a second direction.

[0019] In some aspects, the polarization metasurface overlaps with the microlenses in the microlens array, and wherein the microlenses in the microlens array that do not overlap with the polarization metasurface transmit non-diffracted light to the image sensor, wherein the image sensor is configured to sense the non-diffracted light to measure the intensity of the non-diffracted light.

[0020] In some aspects, the microlens array is a planar microlens array layer.

[0021] In some aspects, the microlens array and the polarization metasurface are positioned on a single substrate.

[0022] In some aspects, the main image plane is configured above the surface of the single substrate opposite the image sensor.

[0023] In some aspects, the main image plane is configured within the single substrate.

[0024] In some aspects, the microlens array comprises metasurface elements.

[0025] In some aspects, the metasurface elements are configured to receive the chief ray angles that vary across the imaging pupil of the polarization imaging device and collimate the light passing through each microlens.

[0026] In some aspects, the metasurface elements of the microlens array are configured to provide a refractive microlens effect.

[0027] In some aspects, the microlens array comprises a conventional refractive microlens array.

[0028] In some aspects, the microlens array comprises a combination of at least one conventional refractive microlens and at least one metasurface element.

[0029] In some aspects, the polarization imaging device further includes a color filter positioned above or below the metasurface microlens array.

[0030] In some aspects, the color filter is positioned above the metasurface microlens array and filters light into different colors, each of the different colors corresponding to a different metasurface microlens among the plurality of first metasurface microlenses, and wherein the metasurface microlens receives the filtered light from the color filter.

[0031] In some aspects, the color filter is positioned below the metasurface microlens array and filters the diffracted first polarized light and second polarized light.

[0032] In some aspects, the color filter includes different regions that filter light of different wavelengths.

[0033] In some aspects, the plurality of first metasurface microlenses are spaced apart on the cover substrate.

[0034] In some aspects, the spaced-apart first metasurface microlenses are configured to output light onto a region of the color filter that outputs light of the same color.

[0035] In some aspects, the spaced-apart first metasurface microlenses output light onto a region of the color filter that outputs green light, and the gaps between the regions of the color filter that output green light output red light or blue light.

[0036] In some aspects, the color filter includes different regions that output red light, green light, or blue light and a region that outputs monochromatic or near-infrared light, wherein the plurality of first metasurface microlenses are spaced apart on the cover substrate, and the spaced-apart first metasurface microlenses output light onto the region of the color filter that outputs monochromatic or near-infrared light.

[0037] In some aspects, the polarization imaging device further includes a microlens array that includes a plurality of separate microlenses that output collimated light into the metasurface microlens array.

[0038] In some aspects, the plurality of separate microlenses are positioned between adjacent image sensing units such that light from the plurality of separate microlenses is diffracted by the plurality of first metasurface microlenses into adjacent image sensing units in opposite tilting directions.

[0039] In some aspects, the plurality of repeated metasurface microlenses further includes a plurality of second metasurface microlenses that are configured to diffract the first polarization into a second direction and diffract the second polarization into a first direction, and wherein a first image sensing unit is further positioned to sense the first polarization diffracted from the plurality of second metasurface microlenses.

[0040] In some aspects, each of the plurality of separated microlenses is centered on one of the plurality of first metasurface microlenses such that a first polarization passes through the one of the plurality of first metasurface microlenses to reach a first image sensing unit, and wherein a second polarization is diffracted in a second direction to reach a second image sensing unit.

[0041] In some aspects, the first polarization and the second polarization are orthogonal linear polarizations, circular polarizations, elliptical polarizations, or any polarizations.

[0042] In some aspects, the plurality of first metasurface microlenses are spaced apart by non-diffracting portions between adjacent first metasurface microlenses.

[0043] In some aspects, the plurality of metasurface microlenses and the plurality of image sensing units configured to sense the first polarization and the second polarization are positioned between the image sensing units of an image sensor configured to sense red, green, or blue light.

[0044] In some aspects, the plurality of first metasurface microlenses are configured to diffract light into the plurality of image sensing units configured to sense the first polarization and the second polarization and into the image sensing units of an image sensor configured to sense red, green, or blue light.

[0045] In some aspects, the plurality of image sensing units configured to sense the first polarization and the second polarization are further configured to sense monochromatic or near-infrared light.

[0046] In some aspects, the image sensor is a complementary metal-oxide-semiconductor (CMOS) image sensor, a charge-coupled device sensor, a silicon diode sensor, a cadmium sulfide sensor diode, an RGB sensor, an indirect time-of-flight (iToF) sensor, a direct time-of-flight (dToF) sensor.

[0047] In some aspects, the techniques described herein relate to a method of manufacturing a polarization imaging device, the method comprising: providing an image sensor wafer; depositing a spacer layer over the imaging sensor wafer; depositing a metasurface layer over the spacer layer; and patterning the metasurface layer to form separated metasurfaces over the spacer layer.

[0048] In some aspects, the method further comprises directly depositing a packaging layer over the separated metasurfaces.

[0049] In some aspects, the method further comprises planarizing the packaging layer such that the packaging layer is at the same level as the individual metasurfaces.

[0050] In some aspects, the method further comprises forming one or more top layers over the packaging layer and / or the individual metasurfaces.

[0051] In some aspects, the techniques described herein relate to a method of manufacturing a polarization imaging device, the method comprising: providing an image sensor wafer; providing a metasurface substrate comprising individual metasurfaces; and depositing a spacer layer on the image sensor wafer and / or on each of the metasurfaces on the metasurface substrate.

[0052] In some aspects, the method further comprises bonding the metasurface substrate to the image sensor wafer.

[0053] In some aspects, the metasurface substrate comprises a transparent substrate.

[0054] In some aspects, the method further comprises removing the metasurface substrate such that the separated metasurfaces remain attached to the image sensor wafer.

[0055] In some aspects, removing the metasurface substrate comprises grinding, wet chemical etching, and / or dry chemical etching of the metasurface substrate.

[0056] In some aspects, the method further comprises singulating the metasurface substrate into separate metasurface die, bonding at least one of the metasurface die to the image sensor wafer, and singulating the image sensor wafer into image sensor die each having a bonded metasurface die.

[0057] In some aspects, the techniques described herein relate to a method of manufacturing a polarization imaging device, the method comprising: providing a complementary metal-oxide-semiconductor (CMOS) image sensor (CIS) wafer; positioning a first plurality of spacers on the CIS wafer; providing a first carrier wafer; positioning a first adhesive layer on the first carrier wafer; contacting the first plurality of spacers with the first adhesive layer such that the adhesive is coated on top of the first plurality of spacers; providing a nanopillar substrate having a plurality of nanopillars; positioning a second plurality of spacers on the nanopillar substrate; providing a second carrier wafer; positioning a second adhesive layer on the second carrier wafer; contacting the second plurality of spacers with the second adhesive layer such that the adhesive is coated on top of the second plurality of spacers; and simultaneously contacting the adhesive on the second plurality of spacers with the CIS wafer and contacting the adhesive on the first plurality of spacers with the nanopillar substrate to bond the CIS wafer and the nanopillar substrate together.

[0058] In some aspects, after contacting the first plurality of spacers with the first adhesive layer such that the adhesive is coated on top of the first plurality of spacers, the adhesive is present only on top of the first plurality of spacers and not on the regions between the first plurality of spacers.

[0059] In some aspects, after bringing the second plurality of spacers into contact with the second adhesive layer such that the adhesive is coated onto the tops of the second plurality of spacers, the adhesive is present only on the tops of the second plurality of spacers and not on the areas between the second plurality of spacers.

[0060] In some aspects, the first plurality of spacers includes pairs of spacers that form channels sized to accommodate spacers from the second plurality of spacers.

[0061] In some aspects, the first plurality of spacers and the second plurality of spacers interlock with each other such that one of the second plurality of spacers is positioned within one of the channels formed by a pair of spacers.

[0062] In some aspects, the first plurality of spacers forms a plurality of rectangular shapes on the CIS wafer.

[0063] In some aspects, an imaging area is formed inside the rectangular shapes.

[0064] In some aspects, the second plurality of spacers forms a grid pattern on the nanopillar substrate.

[0065] In some aspects, the first plurality of spacers and the second plurality of spacers form an air gap between the plurality of nanopillars and the CIS wafer.

[0066] In some aspects, the techniques described herein relate to a method of manufacturing a polarization imaging device, the method comprising: providing a CMOS image sensor (CIS) wafer having bonding pads; depositing a first dielectric layer on the CIS wafer and the bonding pads; planarizing the first dielectric layer; providing a nanopillar substrate having a plurality of nanopillars; depositing a second dielectric layer on the plurality of nanopillars; planarizing the second dielectric layer; bringing the first dielectric layer into contact with the second dielectric layer such that they bond together to form a combined dielectric layer; removing the nanopillar substrate to expose the plurality of nanopillars; partially etching the combined dielectric layer to expose the bonding pads; and forming a conductive layer that is electrically connected to the bonding pads through the combined dielectric layer.

[0067] In some aspects, the first dielectric layer and the second dielectric layer are silicon dioxide layers.

[0068] In some aspects, depositing the first dielectric layer and the second dielectric layer is performed by a tetraethyl orthosilicate (TEOS) process.

[0069] In some aspects, the TEOS process is a plasma enhanced TEOS (PETEOS) process.

[0070] In some aspects, planarizing the first dielectric layer and the second dielectric layer is performed by a chemical mechanical polishing (CMP) process.

[0071] In some aspects, removing the nanocolumn substrate is performed by a grinding, etching, or chemical mechanical polishing (CMP) process.

[0072] In some aspects, partially etching the combined dielectric layer includes: patterning the combined dielectric layer; and etching the combined dielectric layer to expose the bonding pad.

[0073] In some aspects, the method further includes growing a barrier seed layer on the bonding pad, sidewalls of the combined dielectric layer, and the nanocolumns.

[0074] In some aspects, the techniques described herein relate to a polarization imaging device including: a microlens array having at least two microlenses; a polarization filtering metasurface having two or more polarization filtering regions; and an imaging sensor having at least two regions, wherein imaging light including one or more polarization states is guided by the microlenses onto the polarization filtering metasurface, and the polarization filtering metasurface is configured to direct the one or more polarization states onto one or more regions of the imaging sensor.

[0075] In some aspects, the microlens array includes refractive microlenses.

[0076] In some aspects, the microlens array further includes a metasurface configured to provide a refractive microlens effect.

[0077] In some aspects, the microlens array includes a metasurface configured to provide a refractive microlens effect.

[0078] In some aspects, each of the at least two microlenses is over a single region of the one or more regions of the imaging sensor.

[0079] In some aspects, each of the at least two microlenses is over two or more sensor regions of the one or more regions of the imaging sensor.

[0080] In some aspects, each of the at least two microlenses is over a single polarization region of one or more regions of the two or more polarization filtering regions.

[0081] In some aspects, each of the at least two microlenses is over two or more polarization filtering regions of the two or more polarization filtering regions. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The present description will be more fully understood with reference to the following drawings, which are presented as exemplary embodiments of the invention and should not be construed as a complete recitation of the scope of the invention, wherein:

[0083] Figure 1A schematic diagram of a conventional polarization imaging system is shown by way of example.

[0084] Figure 2 A schematic diagram of a polarization imaging system is shown by way of example.

[0085] Figure 3A And Figure 3B A schematic diagram of PSOMMA according to an embodiment of the present invention is shown.

[0086] Figure 3C-1 The optical path of a single-layer metasurface using a blazed grating method according to an embodiment of the present invention is conceptually shown.

[0087] Figure 3C-2 The optical path of a single-layer metasurface serving as a PSOMMA and a collimating lens according to an embodiment of the present invention is conceptually shown.

[0088] Figure 3C-3 The optical path of two separate layers of metasurfaces serving as a PSOMMA and a collimating lens according to an embodiment of the present invention is conceptually shown.

[0089] Figure 4A A PSOMMA having two polarizations on each microlens according to an embodiment of the present invention is shown.

[0090] Figure 4B A PSOMMA having two polarizations on each microlens according to an embodiment of the present invention is shown.

[0091] Figure 5A-1 An optical system incorporating a PSOMMA according to an embodiment of the present invention is shown.

[0092] Figure 5A-2 And Figure 5A-3 Examples of optical systems incorporating a PSOMMA according to various embodiments of the present invention are shown.

[0093] Figure 5B An optical system incorporating a PSOMMA according to an embodiment of the present invention is shown.

[0094] Figure 5C-1 、 Figure 5C-2 And Figure 5C-3 Optical systems incorporating a PSOMMA according to various embodiments of the present invention are shown.

[0095] Figure 6A And Figure 6B An example polarization distribution on an image sensor according to an embodiment of the present invention is shown.

[0096] Figure 7An example of the functionality of an exemplary polarization imaging system including an MLA integrated with a PSOMMA is schematically shown in accordance with an embodiment of the present invention.

[0097] Figure 8A and Figure 8B Various offset pixel architectures in accordance with various embodiments of the present invention are shown.

[0098] Figure 9A and Figure 9B Various high-fidelity pixel architectures in accordance with an embodiment of the present invention are shown.

[0099] Figure 10A and Figure 10B Various hybrid intensity / polarization architectures in accordance with an embodiment of the present invention are shown.

[0100] Figure 11A - Figure 11C Various hybrid architectures including an MLA and a PSOMMA in accordance with various embodiments of the present invention are shown.

[0101] Figure 12A - Figure 12G The respective steps of a direct manufacturing method in accordance with an embodiment of the present invention are shown.

[0102] Figure 13A - Figure 13D The respective steps of a wafer bonding method in accordance with an embodiment of the present invention are shown.

[0103] Figure 14A Multiple second wafer dice bonded to an image sensor wafer in accordance with an embodiment of the present invention are shown.

[0104] Figure 14B An image sensor die 1404 after the image sensor wafer 1302 has been singulated in accordance with an embodiment of the present invention is shown.

[0105] Figure 15 An exemplary manufactured PSOMMA sensor 1500 in accordance with an embodiment of the present invention is shown.

[0106] Figure 16A - Figure 16C An exemplary method for manufacturing a CIS wafer 1502 in accordance with an embodiment of the present invention is shown.

[0107] Figure 17 A plan view of a CIS wafer 1602 in accordance with an embodiment of the present invention is shown.

[0108] Figure 18A - Figure 18C An exemplary method for manufacturing a nanowire wafer in accordance with an embodiment of the present invention is shown.

[0109] Figure 19 A plan view of a nanowire substrate in accordance with an embodiment of the present invention is shown.

[0110] Figure 20A and Figure 20B illustrates an example method for manufacturing a PSOMMA sensor according to an embodiment of the present invention.

[0111] Figure 21A and Figure 21B illustrates a process flow for manufacturing a CIS wafer according to an embodiment of the present invention.

[0112] Figure 22A and Figure 22B illustrates a process flow for manufacturing a nanocolumn wafer according to an embodiment of the present invention.

[0113] Figure 23A - Figure 23H is a manufacturing process for an imaging sensor using a CIS wafer manufactured by the process described in Figure 21A and Figure 21B and a nanocolumn wafer manufactured by the process described in Figure 22A and Figure 22B according to an embodiment of the present invention.

[0114] Figure 24A - Figure 24C is a manufacturing process for an imaging sensor using a CIS wafer manufactured by the process described in Figure 21A and Figure 21B and a nanocolumn wafer manufactured by the process described in Figure 22A and Figure 22B according to an embodiment of the present invention.

[0115] Figure 25A - Figure 25H is a manufacturing process for an imaging sensor according to an embodiment of the present invention.

[0116] Figure 26 illustrates an example operation of an example metasurface microlens array (MLA) according to an embodiment of the present invention.

[0117] Figure 27A illustrates an example pixel arrangement for four polarization pixels according to an embodiment of the present invention.

[0118] Figure 27B illustrates an example pixel arrangement for two polarization pixels according to an embodiment of the present invention.

[0119] Figure 27C illustrates an example of an example pixel arrangement for four polarization pixels according to an embodiment of the present invention.

[0120] Figure 28A is a cross-sectional view of an imaging device including polarization splitting ability according to an embodiment of the present invention.

[0121] Figure 28B is a plan view of an imaging device including polarization splitting ability according to an embodiment of the present invention.

[0122] Figure 28C is a plan view of an imaging device including polarization splitting ability according to an embodiment of the present invention.

[0123] Figure 29A is a cross-sectional view of an imaging device including polarization splitting ability according to an embodiment of the present invention.

[0124] Figure 29B is a plan view of an imaging device including polarization splitting ability according to an embodiment of the present invention.

[0125] Figure 29C is a plan view of an imaging device including polarization splitting ability according to an embodiment of the present invention.

[0126] Figure 30A shows a cross-sectional view of an imaging device including polarization splitting ability according to an embodiment of the present invention.

[0127] Figure 30B shows a plan view of an imaging device including polarization splitting ability according to an embodiment of the present invention.

[0128] Figure 30C is a plan view of an imaging device including polarization splitting ability according to an embodiment of the present invention.

[0129] Figure 31A shows a cross-sectional view of an imaging device including polarization filtering ability according to an embodiment of the present invention.

[0130] Figure 31B shows a plan view of an imaging device including polarization filtering ability according to an embodiment of the present invention.

[0131] Figure 32A shows a cross-sectional view of an imaging device including polarization filtering ability according to an embodiment of the present invention.

[0132] Figure 32B shows a cross-sectional view of an imaging device including polarization sorting ability according to an embodiment of the present invention.

[0133] Figure 32C shows a cross-sectional view of an imaging device including polarization sorting ability according to an embodiment of the present invention.

[0134] Figure 33A shows a plan view of an exemplary polarization filtering nanograting according to an embodiment of the present invention.

[0135] Figure 33B shows a plan view of an exemplary polarization sorting metasurface according to an embodiment of the present invention.

[0136] Figure 34AShows a schematic plan view of an exemplary linearly polarized sorting metasurface according to an embodiment of the present invention.

[0137] Figure 34B Shows a schematic plan view of an exemplary circularly polarized sorting metasurface according to an embodiment of the present invention.

[0138] Figure 34C Shows a schematic plan view of an arbitrary polarized sorting metasurface according to an embodiment of the present invention.

[0139] Figure 35A Shows a sparsely distributed PSOMMA sensor according to an embodiment of the present invention.

[0140] Figure 35B Shows a cross-sectional view of a sparsely distributed PSOMMA sensor according to an embodiment of the present invention.

[0141] Figure 36A Shows an exemplary PSOMMA sensor according to an embodiment of the present invention.

[0142] Figure 36B Shows an exemplary PSOMMA sensor according to an embodiment of the present invention. Detailed Description

[0143] Disclosed herein is an optical device including a single small lens that sorts polarization such that the resulting polarization information of a scene can be reconstructed. Specifically, the invention described herein includes an optical device that includes a metasurface microlens array that includes small lenses that can be coupled to an image sensor. The phase and polarization separation at each small lens can be uniquely specified. Each small lens can split the incident light into at least two polarizations that are directed to specific underlying pixels of the imaging sensor. Independent of the polarization control of each small lens, the phase of each small lens can also be uniquely specified. The phase response of each small lens can be customized such that it adapts to the specific incident angle of the small lens. In some embodiments, the polarization sorting metasurface microlens array can also be customized for a specific optical system and integrated directly with the image sensor. In some embodiments, the image sensor can include a complementary metal oxide semiconductor (CMOS) image sensor, a charge coupled device sensor, a silicon diode, a cadmium sulfide sensor diode, an RGB sensor, an indirect time of flight (iToF) sensor, a direct time of flight (dToF) sensor, and / or various other types of sensors.

[0144] The polarization sorting metasurface can include a supercell or superpixel repeating structure and / or non-polarized open spaces therebetween to create a sparse effect.

[0145] Most traditional polarization imaging systems work by absorbing a specific polarization state while allowing the unabsorbed polarization state to pass through to a photosensitive sensor. By knowing which polarization state is absorbed and which passes through, a polarization image of the scene can be constructed. However, this design can be inefficient because the absorbed light is dissipated as heat rather than being redirected and sensed to potentially provide valuable polarization information. This design may have poor efficiency or signal-to-noise ratio. This configuration still retains the polarization information, but there is an efficiency loss because the light is absorbed. Additionally, common techniques for implementing this type of polarization measurement (e.g., wire grid polarizers) can only select linear polarization states, so elliptical polarization states may not be available.

[0146] Figure 1 A schematic diagram of a traditional polarization imaging system is shown by way of example. As shown, incident light 106 can include multiple polarizations, such as S-polarized light and P-polarized light. The incident light passes through a polarizer 102, which can transmit only light of one polarization. In the case where the incident light 106 includes S-polarized light and P-polarized light, the polarizer 102 can transmit only the S-polarized light 106a and absorb the P-polarized light. The amount of S-polarized light 106a can be measured by a photosensitive sensor 104, which can provide data on the amount of S-polarized light 106a. Unfortunately, the P-polarized light is absorbed by the polarizer 102, which can be an unwanted byproduct, such as heat or scattered light from reflections.

[0147] Embodiments of the present invention relate to a metasurface polarization imaging system. Compared with traditional polarizers, the metasurface polarization imaging system spatially separates polarizations by sending each of the designed polarizations to a specific diffraction angle. Examples of metasurface optical elements that deflect light in a direction depending on the polarization of light are disclosed in U.S. Patent Application Publication No. 2021 / 0286188, titled "Arbitrary polarization-switchable metasurfaces", filed on February 22, 2019, which is incorporated herein by reference in its entirety for all purposes. Additionally, examples of gratings configured to perform parallel polarization analysis on multiple polarization levels of incident light of arbitrary polarization are disclosed in International Publication No. WO2020214615, titled "System and Method for Parallel Polishing Analysis", filed on April 14, 2020, which is incorporated herein by reference in its entirety for all purposes. Furthermore, examples of polarimeters including metasurface elements that provide different responses based on the polarization of incident light are disclosed in U.S. Patent Application Publication No. 2021 / 0048569, titled "Polarization state generation with a metasurface", filed on July 22, 2020, which is incorporated herein by reference in its entirety for all purposes. The designs in these references can all be incorporated into the present disclosure to split incident light into different polarizations directed in different directions.

[0148] In such a system, the efficiency can be two times higher than that of traditional polarization gratings because the light is not absorbed, but rather different polarized lights are diffracted into different directions to be detected by different image sensors. However, for these systems, the metasurface elements divide the aperture of the imaging system and result in the formation of images with specific polarizations in separate regions of the image sensor. For example, if the metasurface is designed to split incident light into four polarization states, the metasurface imaging system can form four separate images of a single object, where each of the four images is formed in a separate spatial domain (e.g., quadrant) of the image sensor, and a specific polarization state is associated with each image. Figure 2A schematic diagram of a polarization imaging system is shown by way of example. The polarization imaging system includes a metasurface element 202 that splits incident light 206 into different polarizations. As shown, the incident light 206 can include a single mixed polarization state. The incident light 206 can be completely unpolarized, or it can be a single pure polarization state or some mixed state. The incident light 206 can include four polarizations: Pol1, Pol2, Pol3, Pol4. In more mathematical terms, the incident light is represented by a Stokes vector S. Each outgoing diffraction direction is associated with another Stokes vector Mi, which can correspond to a row of the Mueller matrix. Each outgoing direction can be associated with a Mueller matrix Ui. When the light is captured by an image sensor, the total intensity of the light can be measured. The Mueller matrix Ui multiplied by the incident Stokes vector S gives the outgoing Stokes vector Oi in each direction, Oi = Ui * S. When the light is captured by an image sensor, the first entry of the Stokes vector Oi corresponds to the intensity of the light. Since there are four indices 0, 1, 2, 3, this corresponds to O i [0] = (U i *S)[0]. [x] represents the x-th element in the vector starting from 0. This is equivalent to taking the first row of the Mueller matrix Ui and calculating its dot product with the Stokes vector S ((U i *S)[0] = sum j (U {i,(0,j)} *S j ). The first row of the Mueller matrix Ui can be considered the Stokes vector Mi because it also obeys the same algebra as the Stokes vector. The intensity in each outgoing diffraction direction is proportional to the dot product between Mi and S. The diffraction orders do not have to be the same polarization state that is selected.

[0149] The metasurface element 202 can split the incident light 206 into diffracted light 206a, where each of the four polarizations is directed in a different direction. The diffracted light 206a in each direction has an intensity proportional to the projection of the polarization state of that particular light on the incident state. The diffracted light 206a can enter a set of four different image sensors 204. In some embodiments, the diffracted light 206a can enter four different regions of a single image sensor, or two regions of two image sensors or other combinations of regions and sensors. Although these image sensors 204 and the diffracted light 206a are shown in different horizontal positions, this is merely exemplary, and the image sensors 204 can actually be in different quadrants, where the diffracted light 206a sends each polarization to a different quadrant.

[0150] The incident light 206 can be in any polarization state. The metasurface element can split the incident light into diffracted light, where each of four directions corresponds to a different target polarization state, and the intensity in that direction is determined by the overlap of the incident polarization state and the target polarization state.

[0151] Although the metasurface element 202 overcomes the absorption losses and / or complexities of conventional polarization imaging systems including the polarizer 102 discussed in Figure 1 connection, some embodiments can benefit from an aperture or illumination source for a particular field of view to prevent sub-images from overlapping on the image sensor 204.

[0152] Various embodiments of the present invention utilize a polarization sorting optical metasurface microlens array (PSOMMA) to spatially mix polarization states on an image sensor. In some embodiments, the PSOMMA can be the last element before the image sensor, and each microlens of the PSOMMA can direct a set of designed polarization states to respective sub-pixels or imaging units or regions of the sensor. Additionally, it may not be necessary to use a field of view of the illumination source or an additional aperture to prevent sub-images from overlapping on the image sensor. Finally, the image sensor and the PSOMMA can be combined with any existing optical imaging system, which can include refractive optical elements as shown and described in Figure 5A-1 to Figure 5A-3 connection.

[0153] Figure 3A and Figure 3B shows a schematic diagram of a PSOMMA according to an embodiment of the present invention. In Figure 3AIn it, the PSOMMA includes separated metasurface microlenses 302. The number of separated microlenses can depend on the optical system, but can generally be any number of microlenses. Each microlens can split the incident light into separate polarizations for each diffraction angle. Each microlens can split the light into at least two polarizations, but can generally split the light into any number of polarizations. As shown, the first microlens L1 can split the incident light into four different polarizations 304: pol1, pol2, pol3, and pol4. In addition, the second microlens L2 can split the incident light into four different polarizations 306: pol5, pol6, pol7, pol8. The light can be split into four different directions proportionally to the selected polarization state in that direction in each direction. Similarly, the nth microlens Ln can split the incident light into four different polarizations 308: pol n, pol n+1, pol n+2, and pol n+3. As shown here, for each microlens, a specific polarization state can be unique (labeled pol 1, pol 2,..., pol n), but the same two or more polarization states can be used on each microlens. For example, both the first microlens L1 and the second microlens L2 can split the incident light into the same four different polarizations: pol 1, pol 2, pol 3, pol 4. Additionally, the phase function of each microlens can be uniquely specified.

[0154] As Figure 3B shown, the PSOMMA can be coupled to an image sensor 308 having any number of pixels (labeled p1, p2, p3... pn). The pixels can be multiple image sensing units. In this case, the role of the PSOMMA can be to diffract each polarization state to a specific pixel. By designing the polarization state corresponding to the pixel readout, the complete polarization state of the scene can be reconstructed on a pixel-by-pixel basis. Although the drawings here are shown in one dimension, it should be understood that the same principle applies to two-dimensional microlens and pixel arrays. The incident light shown has a 0-degree chief ray angle (CRA), but the CRA of the incident light can be any angle or even vary across the lens. Thus, the incident light can be collimated, but can also be incident on the metasurface microlens at other angles. In some embodiments, the metasurface can be used to provide a microlens array function and is designed such that the microlenses can be optimized to accept different incident light angles (e.g., non-zero CRA) across the image sensor field and minimize the effects that would otherwise be caused for lenses designed for a zero-CRA function. In some embodiments, a physical microlens shift can be provided. This is a method of matching the CRA of the microlens and the image sensor. In this method, the periodicity of the microlens array is reduced across the sensor, and the microlens CRA can be partially canceled.

[0155] In some embodiments, the microlens array may use a metasurface to provide microlens functionality (e.g., focusing light at an image sensor) in a way that provides microlens shift (as if a physical microlens were shifted). The metasurface may be locally designed such that they can match both the azimuth and elevation angles of the main refractive lens in order to produce a CRA match across the sensor or have different incident imaging light angles. This is complex for conventional physical microlenses, but can be achieved by utilizing a metasurface with microlens functionality. In some embodiments, the metasurface may be used to provide a collimation function such that light received from the main refractive lens or the entrance pupil is collimated and / or directed to the center of the sensor area.

[0156] Figure 3C-1 Conceptually shows the optical path of a single-layer metasurface using a blazed grating method according to an embodiment of the present invention. Based on the generalized Snell's law, the relationship between the phase gradient, emission azimuth, and elevation angle can be written as follows:

[0157]

[0158] In the case of a PSOMMA with CRA matching, the phase function of the interface includes a blazed grating for CRA matching and a PSOMMA phase function. Therefore, Equation (1) can be rewritten as:

[0159]

[0160] To simplify the problem, when the CRA match is fully satisfied, θ t = φ t = 0°, the blazed grating phase can be defined as follows in the absence of PSOMMA functionality:

[0161]

[0162] The actual phase function of the blazed grating in Equation (3) can be calculated as:

[0163] φ 闪耀光栅 (x,y) = -k0n′ i ·x·sinθ′ CRA cosφ′ CRA -k0n′ i ·y·sinθ′ CRA sinφ′ CRA ;

[0164] φ 闪耀光栅 (x,y) = -k0n′ i ·sinθ′ CRA (x·cosφ′ CRA +y·sinφ′CRA ) (4)

[0166] If the metasurface locally satisfies Equation (5), the refractive lens CRA can be completely eliminated and brought back to the target sensor pixel:

[0167] φ 闪耀光栅 (x,y) = -k eff ·sinθ′ CRA (x·cosφ′ CRA +y·sinφ′ CRA ) (5)

[0169] Figure 3C-2 Conceptually shows the optical path of a single-layer metasurface acting as a PSOMMA and a collimating lens according to an embodiment of the present invention. The equations for a 2D scenario where the metasurface is positioned along the y-axis are shown. Equation (6) describes the trigonometric functions that can be used to calculate the equivalent effective focal length (EFL) for the collimation function:

[0170]

[0171] This can be simplified to calculate the collimation phase by using the target EFL as follows:

[0172]

[0173] Or

[0174]

[0175] y′ 传感器 can be a quarter of the physical size of the sensor. This is an approximation rather than optimizing the collimation phase for all received CRAs at the metasurface. Another estimate of the EFL can be:

[0176] EFL = h 衬底 +h′·n′ (8)

[0177] Figure 3C-3 Conceptually shows the optical path of two separate layers of a metasurface acting as a PSOMMA and a collimating lens according to an embodiment of the present invention. One metasurface (MS) layer acts as a collimating lens and one MS layer acts as a PSOMMA.

[0178] Figure 4AShows a PSOMMA with two polarizations on each microlens according to an embodiment of the present invention. The PSOMMA includes a plurality of microlenses 402 (L1, L2, …, LN, LN+1), and each microlens can split the incident light into two polarizations (pol). As shown in the figure, for example, L1 can split the incident light into pol1 in one direction and pol2 in another direction. In addition, L2 can split the incident light into pol3 and pol4. Each microlens can be over only two pixels. For example, as shown in the figure, L1 can be over pixels p1 and p2, guiding the light of pol1 into p1 and the light of pol2 into p2. In some embodiments, the polarization of a set of two microlenses can be repeated on the image sensor 404. For example, L1 can split the light into pol1 and pol2, L2 can split the light into pol3 and pol4, and this pattern can be repeated over the entire image sensor 404 or a part or several parts of the image sensor 404, such that LN splits the light into pol1 and pol2, and LN+1 splits the light into pol3 and pol4. In some embodiments, the polarization is split into only two states on the image sensor 404. For example, in the PSOMMA described in conjunction with Figure 4A pol1 can be equal to pol3, and pol2 can be equal to pol4. Thus, both L1 and L2 can split the light into the same polarizations, namely pol1 and pol2. Finally, it should be understood that the polarization state of each microlens can be completely unique and independent and is not limited to only linear polarization states. For example, pol1 can be right-handed circularly polarized light, and pol2 can be left-handed circularly polarized light. In addition, pol1 can be S-polarized light, while pol2 can be P-polarized light.

[0179] Figure 4B Shows a PSOMMA with two polarizations on each microlens according to an embodiment of the present invention. This configuration is similar to the one described in conjunction with Figure 4A However, in this configuration, the PSOMMA 406 includes a first microlens L1 that guides the first polarized light pol1 into the first imaging unit p1 of the image sensor 408 and guides the second polarized light pol2 into the second imaging unit p2 of the image sensor 408. Similarly, the PSOMMA 406 includes a second microlens L2 that guides the first polarized light 1 into the first imaging unit p1 and guides the second polarized light 2 into the second imaging unit p2. Thus, the first microlens L1 and the second microlens L2 can share the same imaging units p1, p2.

[0180] Figure 5A-1An optical system incorporating a PSOMMA according to an embodiment of the present invention is shown. As shown, the PSOMMA 502 can be coupled with one or more refractive lenses 504 in the optical system to provide a complete polarization imaging system. The PSOMMA 502 can be the last element in the imaging system before the image sensor 506. In some embodiments, each microlens of the PSOMMA 502 can be customized for a specific polarization response. In some embodiments, each microlens of the PSOMMA 502 can be customized for a specific unique phase function. This unique phase function can be used to improve the performance of the PSOMMA 502 to change the chief ray angle of the optical system. In some embodiments, each microlens of the PSOMMA 502 can be customized for both a specific unique phase function and a specific polarization response. There can be some distance offset between the image sensor 506 and the PSOMMA 502. The material in the offset distance between the image sensor 506 and the PSOMMA 502 can be air or a solid material such as glass, quartz, SiO2, Si3N4, or a polymer. Many optical imaging systems have a microlens array (MLA) before the image sensor to couple light into individual pixels. In these systems, the microlens array may not be able to distinguish polarization and may not provide a unique, arbitrary phase function at each pixel. As previously mentioned, the PSOMMA 502 can diffract different polarizations of incident light in different directions and provide a specific unique phase function for different polarizations of the incident light.

[0181] In some embodiments, the PSOMMA 502 can be positioned to re-image the image plane from the optical device. Figure 5A-1 The one or more refractive lenses 504 are merely exemplary. Figure 5A-2 and Figure 5A-3 Examples of optical systems incorporating a PSOMMA according to various embodiments of the present invention are shown. In Figure 5A-2 , the 0P trajectory has several different fields of view, but only the central ray diffracted into the image sensor 506 by the PSOMMA 502. In Figure 5A-3 , the 0F trajectory has a normal field of view, where some of the rays are traced through the pupil, which is diffracted into the image sensor 506 by the PSOMMA 502.

[0182] Figure 5BShows an optical system incorporating PSOMMA according to an embodiment of the present invention. As shown, a light source 508 can output light to PSOMMA 502a. Although the light from the light source 508 is illustrated as being collimated, the light can also have different incident angles that are not fully collimated. PSOMMA 502A can include all the functions of PSOMMA 502 described in conjunction with FIG. 5A. However, PSOMMA 502 can also have at least some of the functions of one or more refractive lenses 504 described in conjunction with FIG. 5A, or it can include a microlens array function. In Figure 5B although the presence of one or more refractive lenses 504 is not shown, a portion of one or more refractive lenses 504 can be present to perform some of the functions of one or more refractive lenses 504, while the remaining functions can be incorporated into PSOMMA 502a. PSOMMA 502a can be the last element in the imaging system before the image sensor 506.

[0183] The light source can be a fixed near-infrared light source, a vertical-cavity surface-emitting laser (VCSEL), an LED, an edge-emitting laser (EEL), sunlight, and / or other ambient lighting. In some embodiments, the light source can be designed to have a specific fixed polarization across the illumination field, or it can be designed to have a point-by-point polarization pattern across the illumination field. For example, the illumination pattern projected onto the scene can have a fixed circular polarization, such as left-handed circular or right-handed circular. In some embodiments, it can be fixed linear or elliptical polarization illumination. In some cases, two different fixed polarizations can be used simultaneously or sequentially. Figure 5C-1 , Figure 5C-2 and Figure 5C-3 Shows an optical system incorporating PSOMMA according to various embodiments of the present invention. The optical system includes a main lens 552 that refracts light onto PSOMMA 554. PSOMMA 554 distributes the refracted light from the main lens 552 onto an image sensor 556. In Figure 5C-1 the main lens image plane 558a of the main lens 552 overlaps with PSOMMA 554 such that the main lens 552 focuses on PSOMMA 554.

[0184] In Figure 5C-2 the main lens image plane 558b of the main lens 552 is behind PSOMMA 554 and the image sensor 556 such that PSOMMA 554 focuses on the image plane. In Figure 5C-3 the main lens image plane 558C of the main lens 552 is between the main lens 552 and PSOMMA 554. Thus, in Figure 5C-2 the image plane 558b of the main lens 552 is behind the image sensor 556, while Figure 5C-3The image plane 558C therein is located in front of the image sensor 556. Figure 5C-3 This arrangement allows for a more flexible physical configuration.

[0185] In some embodiments, the main lens 552 is a biconvex lens. In some embodiments, the main lens 552 can be a plano - convex lens. In some embodiments, the main lens 552 can be a multi - refractive lens configuration. In some embodiments, the PSOMMA 554 can be replaced by a combination of a polarization - dependent metasurface adjacent to the microlens array. The microlens array can be a biconvex lens array or a plano - convex lens array. Any configuration disclosed in Figure 5C-1 to Figure 5C-3 can be used, which can provide different benefits, such as higher spatial resolution, a larger viewing angle, and better direction sampling accuracy.

[0186] Figure 6A and Figure 6B shows an example polarization distribution on an image sensor according to an embodiment of the present invention. Figure 6A is a top view of the example polarization states at each pixel on the image sensor after the light has passed through the PSOMMA. As shown, each microlens 604 can divide the intensity of the incident light proportionally into the polarization content of four separate states: right - hand elliptical 602a, vertical 602b, diagonal 602c, and left - hand circular 602d. These four pixels with unique states can form a super - pixel of the imaging system and can be repeated N times across the entire image sensor. Although four repeated polarization states are shown, the PSOMMA can be used to specify a unique polarization state at each of the pixels on the image sensor, and it can be two, three, or more unique polarization states repeated. Advantageously, the light can not be filtered (e.g., absorbed) before reaching the image sensor, and the PSOMMA is not limited to linear polarization states. Figure 6B shows a fully polarized super - pixel, where the positioning of each microlens 604 is outlined in thick lines. Each microlens 604 can divide the incident light into four sub - pixels, each with a different polarization state.

[0187] The image sensor can be a CMOS sensor, a time-of-flight (ToF) sensor, an indirect time-of-flight (iToF) sensor, and / or an RGB sensor. The readout timing, method, and frame rate of the image sensor can include pixel binning. For example, in a 2×2 pixel configuration, readout is performed without polarization first and then with polarization, which can provide effective information capture. In other words, polarization pixels are grouped together in terms of signal capture and processing. Different pixels can be sampled at different times, or more or less frequently, for imaging or energy purposes. For an iToF sensor, most pixels can be of the CMOS type, and a smaller percentage can be iToF pixels (e.g., 10% or 20%); the iToF can be in 2D mode without polarization. Some configurations can have a sparse concept regarding the sensor (e.g., only some pixels are in 3D mode relative to the 2D mode), which can save energy.

[0188] In some embodiments, a diffractive optical element (DOE) can be used for some spectroscopic aspects. For example, a conventional DOE can be applied to some small lenses in PSOMMA instead of a metasurface.

[0189] Example polarization imaging system including a microlens array

[0190] In some embodiments, a polarization imaging system can include two separate components: a refractive microlens array (MLA) and a PSOMMA. The MLA can tile the projected image. Figure 7 An example of the function of an example polarization imaging system including an MLA integrated with a PSOMMA according to an embodiment of the present invention is schematically shown. The polarization imaging system can include a separate refractive MLA 704 and a PSOMMA 702. The PSOMMA 702 can be a polarization metagrating. In some embodiments, the refractive MLA 704 can be implemented in the same layer as the PSOMMA 702. In some embodiments, as shown, the MLA 704 can be implemented as a separate planar layer from the PSOMMA 702. In some embodiments, the MLA 704 can be a conventional refractive MLA, so the polarization imaging system can include a conventional refractive MLA and a PSOMMA 702. In some embodiments, the MLA 704 can be integrated with the PSOMMA 702 such that a metasurface can be used to fabricate a layer including the MLA and a layer including the PSOMMA on a single substrate. The layer including the MLA 704 can include metasurface elements.

[0191] The MLA 704 can be adjusted relative to the image plane of the imaging system such that it reduces the incident light by a factor greater than 2, so that when tiling, the separate images do not overlap. Examples of tiling are shown in Figure 6BShown in. Preferably, the PSOMMA 702 can be made as close as possible to the MLA 704, or in some embodiments, can include combining the MLA 704 and the PSOMMA 702 in a single layer. The pixels are schematically shown as adjacent boxes. The active area of the pixels can be smaller than the area schematically shown here, and the magnification can be greater than 2 so as to illuminate only the active area.

[0192] The MLA 704 includes a focal length. The distance 708 from the main image plane 707 can be greater than twice the focal length of the MLA 704. The imaging of the region 716 in the main image plane 707 onto the pixels can overlap with each microlens of the MLA 704. The PSOMMA 702 outputs light into an image sensor 710 including a pixel grid. The distance 712 from the PSOMMA 702 to the image sensor 710 can be less than the focal length of the MLA 704. Schematic ray trajectories 714A are shown at 0F, including the principal (solid line) and marginal (dashed line) rays. The schematic ray trajectory 714b at 1F includes the principal (solid line) and marginal (dashed line) rays.

[0193] Figure 8A and Figure 8B Shows various offset pixel architectures according to various embodiments of the present invention. Instead of aiming at a diffraction order to make the diffraction symmetric about the non-diffracted image, the non-diffracted image can be selected as one of the diffraction orders (e.g., the zero order). These architectures share many identically numbered features with the Figure 7 architecture, and the description applies to these architectures. The description will not be repeated in detail.

[0194] As Figure 8A shown, the center of each small lens of the microlens array can be directly aligned with the zero-order light between the pol1 light and the pol2 light. The zero-order light may have a negative impact on the fidelity of the reconstructed polarization. Therefore, it may be advantageous to direct the zero-order light through the center of the photosensor between the pol1 light and the pol2 light. The active area of the pixels (the photosensitive area that generates electron charge under incident light) can be smaller than the microlens pitch divided by the number of pixels it is divided into. The zero-order light from the PSOMMA may not be incident on the active area of the pixels because the active area of the lower pixel array of the photosensor may be offset from the position where the zero-order light is incident. The zero-order light can coincide with the diffracted light. Since the zero-order light is "uncontrolled", it cannot effectively resolve the incident polarization and does not carry a polarization signal. Therefore, the zero-order light can be considered as noise. Therefore, the noise can be reduced by guiding it into a non-photosensitive area.

[0195] In some embodiments, the zero-order light can propagate at least partially between the pol1 light and the pol2 light. If the active area of the image sensor overlaps with the zero-order light, some of the zero-order light can be shared as noise between the pixels of the image sensor.

[0196] In some cases, it may be difficult to align the zero-order light with the non-active area of the pixel array of the image sensor. In such cases, it may be better if the zero-order light is diffracted into the same direction as one of the light polarizations. Figure 8B An offset pixel architecture is shown in which the zero-order light is diffracted into the side of the pol2 light in the same direction as the pol1 light. The target pixel can be offset relative to the lens center. Only one pixel experiences noise from the zero-order, which can make subsequent polarization recovery easier. The zero-order image is shown offset on the vertical axis. Mathematically, Figure 8A The small lens of can shorten all the Stokes vectors of the polarizers on the Poincare sphere, which can reduce the degree of polarization of the pol1 light and the pol2 light. While in Figure 8B the small lens of, only the vectors including the pol1 light and the zero-order light can be shortened, rather than the pol2 light.

[0197] Figure 9A and Figure 9B show various high-fidelity pixel architectures according to embodiments of the present invention. These architectures share many features with the same numbers as the architectures of Figure 7 and FIG. 8, and the description can be applied to these architectures. The description will not be repeated in detail. As Figure 9A shown, the zero-order light can be non-diffracted and thus pass through the middle of the microlens (e.g., the small lens). The non-diffracted zero-order light can be imaged onto a separate pixel on the image sensor, where no diffracted orders are directed. In this case, none of the analyzer Stokes vectors can be shortened, so the degree of polarization may not be reduced. The zero-order light may not affect the fidelity of the reconstructed polarization imaging. In addition, the zero-order light can be an additional signal for the intensity image for reconstruction, rather than noise for the polarization for reconstruction. The zero-order light may not have a preferred polarization and only provide intensity information. In this structure, the amount of light entering the zero-order does not need to be minimized and can be optimized by design to increase the overall optical efficiency. This configuration is beneficial in applications where a trade-off can be made between efficiency and signal-to-noise ratio (SNR) and angular resolution. Figure 9B Shows various top sensor views of an example superpixel implementation with a separate zero-order pixel combined with pixel offset. Pixel offset refers to the spatial translation of the underlying pixel array and the microlens array. In some embodiments, pol3 can be equal to pol1 and pol4 can be equal to pol2. The combinations shown are not restrictive, and other combinations of sub-pixel positioning can be expected.

[0198] Figure 10A and Figure 10B show various hybrid intensity / polarization architectures according to embodiments of the present invention. These architectures share many features with the same numbers as the architectures of Figure 7The architectures share many features with the same numbers, and the description applies to these architectures. The description will not be repeated in detail. In these architectures, PSOMMA 706 may not be applied to all the microlenses of MLA 704, such that a subset of pixels can be directly imaged by the microlenses without PSOMMA 706, while another subset of pixels is imaged by the microlenses of MLA 704 plus PSOMMA 706. This can be achieved by patterning a transparent window 1002 in PSOMMA 706, or selectively patterning the back side of MLA 704 with PSOMMA 706, or other methods suitable for various hybrid MLA - polarization metasurface grating configurations. The offset pixel and high - fidelity architectures described above can be further applied to the subset of pixels including PSOMMA 706. Such a hybrid configuration can be beneficial for balancing the trade - offs between resolution, optical efficiency, polarization fidelity, and image sensor size. This includes implementing an extrapolation scheme between pixels receiving non - diffracted light and pixels receiving diffracted light from PSOMMA 706.

[0199] Figure 10A A hybrid intensity / polarization architecture is shown, where poln (e.g., pol1, pol2, pol3, pol4) is the polarization state of the diffracted light from PSOMMA 706, and ln (e.g., l1, l2, l3) is the light that does not pass through PSOMMA 706. The microlenses of MLA 704 are labeled as M1, M2, ……, Mn. As shown, Poln can be at least 2 different polarizations. The light that does not pass through PSOMMA 706 can be used to measure intensity, time - of - flight (ToF), distance to a target, depth, depth to a focus, defocus, and / or color, while the polarization - separated light diffracted by PSOMMA 706 can be used to measure various degrees of polarization. Figure 10B Various example hybrid intensity / polarization sub - pixel configurations are shown. The thick lines show the pixels, and each square represents a sub - pixel. Poln can include 2 or more different polarizations, 0 is the non - diffracted zero - order pixel exiting PSOMMA 706, and In is the light that does not pass through PSOMMA 706. Advantageously, capturing the intensity of light can increase the resolution of the image intensity without increasing the sensor size. The captured intensity can also be used as a reference intensity in an interpolation scheme to retrieve a certain resolution of the intensity image and manage the noise from the zero - order light. When some sub - pixels are dedicated to capturing the image intensity, the resolution of the polarization image may be reduced.

[0200] Figure 11A - Figure 11C Various hybrid architectures including MLA and PSOMMA according to various embodiments of the present invention are shown. In Figure 11AIn it, the substrate 1102 includes PSOMMA 1104 located above the MLA 1106. The MLA 1106 is directly located above the image sensor 1108 with an air gap therebetween. The PSOMMA 1104 can be embedded in a layer directly above the MLA 1106 within the substrate 1102. The main image plane 1110 can be located within the substrate 1102. In some embodiments, different microlenses of the MLA 1106 can have different sagittal heights (SAGs) and different focal lengths or aspherical surfaces. In some embodiments, the MLA 1106 can be implemented as a metasurface, which can be used to customize various properties of the MLA 1106. In some embodiments, the metasurface lens can be used to customize the aspherical surface above the MLA 1106 or to reduce the aberration from the MLA 1106.

[0201] In some embodiments, the MLA 1106 can include a variable focal length with different focal points across the image. The MLA 1106 can include phase detection autofocus with a portion of the pupil masked. The MLA 1106 can have a depth from the defocus or other autofocus from the open window in the sparse format, e.g., using different MLA microlenses for each color to separate different focal points for autofocus calculation.

[0202] In Figure 11B it, with the MLA 1106 in a state located on the surface opposite to the image sensor 1108, the MLA 1106 and the PSOMMA 1104 are integrated into the substrate 1102. The main image plane 1110 can be located above the substrate 1102. The MLA 1106 integrated with the image sensor 1108 works by guiding the light from the main image plane 1110 to the underlying image sensor 1108. The main image plane 1110 can be formed by an optical component such as an optical system lens. In one configuration, the MLA 1106 acquires the main image plane 1110 and reduces it such that the main object is imaged onto the effective area of the pixels within the image sensor 1108. The main image plane 1110 can be an image plane formed by an optical system. The main image plane 1110 can be distinguished from the image plane formed by the MLA 1106 within the image sensor.

[0203] Figure 11C The hybrid architecture of Figure 11B is similar, however the MLA 1106a can be a planar MLA layer, which can achieve the same optical effect as the MLA layer. The planar MLA layer can be an MLA including a metasurface. Other examples of polarization imaging systems including MLA are discussed in conjunction with Figure 28A - Figure 28C , Figure 29A - Figure 29C , Figure 30A - Figure 30C , Figure 31A - Figure 31B and Figure 32A - Figure 32C .

[0204] Example methods for fabricating PSOMMA and integrating it with an image sensor

[0205] Various fabrication methods for PSOMMA integrated with an image sensor have been considered, and three specific methods are described below.

[0206] The first method is to fabricate PSOMMA directly on top of the image sensor. Figure 12A - Figure 12G The various steps of the direct fabrication method according to an embodiment of the present invention are shown. In Figure 12A , an image sensor wafer 1202 having a planarized or substantially planarized surface is provided. The image sensor wafer 1202 may include an image sensor. In Figure 12B , a spacer layer 1204 is deposited on the planarized surface of the image sensor wafer 1202. The thickness of the spacer layer 1204 can determine the distance between the PSOMMA and the image sensor. The spacer layer 1204 may include a dielectric material (e.g., SiO2, Si3N4, etc.), a polymer layer, or other suitable layers, which preferably have a refractive index less than 1.6 at the operating wavelength of the image sensor. In Figure 12C , one or more additional layers 1206 are deposited on top of the spacer layer 1204. In Figure 12D , the one or more additional layers 1206 are patterned and / or etched to form nanoscale pillars 1206a. Examples of nanoscale pillars are disclosed in U.S. Patent Application Publication No. 2018 / 0341090, filed on May 22, 2018, entitled "Atomic layer deposition process for fabricating dielectric metasurfaces for wavelengths in the visible spectrum", which is incorporated herein by reference in its entirety for all purposes. The patterning and / or etching can be performed using a lithography process. The patterning and / or etching may include wet or dry etching processes. The nanoscale pillars 1206a may be metasurface pillars. In Figure 12E , the nanoscale pillars 1206a may be encapsulated by an encapsulation layer 1208. In Figure 12F , the encapsulation layer 1208 may be planarized to form a layer 1210 flush with the nanoscale pillars 1206a. In Figure 12G , one or more top layers 1212 may be deposited on the encapsulated nanoscale pillars 1206a.

[0207] The second method of PSOMMA fabrication and integration is to bond the PSOMMA metasurface to the image sensor by a wafer bonding method. Figure 13A - Figure 13D The various steps of the wafer bonding method according to an embodiment of the present invention are shown. In Figure 13ATherein, an image sensor wafer 1302 having a planarized or mostly planarized surface is provided. In Figure 13B Therein, a spacer layer 1304 is deposited on the planarized surface of the image sensor wafer 1302. The thickness of the spacer layer 1304 can determine the distance between the PSOMMA and the image sensor. The spacer layer 1304 can include a dielectric material (such as SiO2, Si3N4, etc.), a polymer layer, or other suitable layers, which preferably have a refractive index less than 1.6 at the operating wavelength of the image sensor. In Figure 13C Therein, a second wafer 1306 including nanoscale pillars 1308 is provided, and the nanoscale pillars include PSOMMA. Methods such as lithography and / or wet or dry etching can be used to fabricate the nanoscale pillars 1308 on the second wafer 1306. The shown second wafer 1306 including the nanoscale pillars 1308 is merely exemplary, and other configurations can be considered. For example, there may be a layer between the second wafer 1306 and the nanoscale pillars 1308. Examples of nanoscale pillars fabricated on a substrate are disclosed in U.S. Patent Application Publication No. 2019 / 0064532, entitled "Transmissional MetaSurface Lens Integration", filed on August 31, 2018, the entire content of which is incorporated herein by reference for all purposes. The disclosed manufacturing methods and various configurations of nanoscale pillars on a substrate can be used to produce the nanoscale pillars 1308 on the second separate wafer 1306. The second wafer 1306 can include silicon, glass, polymer, and / or another suitable material. The second wafer 1306 can also have a spacer layer disposed on the front surface, which can replace the spacer on the first wafer or supplement the spacer on the image sensor wafer 1302.

[0208] After separately manufacturing the image sensor wafer 1302 and the second wafer 1306, the two wafers can be bonded together using a wafer bonding method. The wafer bonding method can include adhesive bonding, direct oxide bonding, eutectic bonding, and / or thermocompression bonding. Figure 13D The image sensor wafer 1302 and the second wafer 1306 after the wafer bonding method are shown. The wafers 1302, 1306 can be aligned with each other such that each metasurface microlens can be aligned with a corresponding pixel on the image sensor on the image sensor wafer 1302. The alignment tolerance of the microlens relative to the pixel can be a fraction of the pixel size (or equal to the pixel size). In some embodiments, the alignment tolerance of the microlens with respect to the pixel can be less than 10% of the pixel size. In some embodiments, the second wafer substrate 1306 can be transparent. In some embodiments, the second wafer substrate 1306 can be a transparent material, such as glass. In the case where the second wafer substrate 1306 is transparent, the second wafer substrate 1306 can be part of the final integrated device.

[0209] In some embodiments, the second wafer 1306 can be opaque. Examples of an opaque second wafer substrate include silicon. The second wafer substrate 1306 can be removed using methods including grinding, polishing, wet chemical etching, and / or dry chemical etching. After removing the second wafer substrate 1306, the nanoscale pillars 1308 are transferred from the second wafer 1306 to the image sensor wafer 1302.

[0210] A third method of PSOMMA fabrication and integration with an image sensor is the die attach method. The steps of the die attach method are the same as those described above with reference to Figure 13A - Figure 13C the wafer bonding method described. After fabricating the image sensor wafer 1302 and the second wafer 1306, the second wafer can be singulated into separate dies. Then, the separated singulated dies can be bonded to the image sensor wafer 1302 using a die-to-wafer bonding process. Figure 14A Shown are a plurality of second wafer dies 1402 bonded to an image sensor wafer 1302 in accordance with an embodiment of the present invention. Each individual second wafer die 1402 can be aligned with appropriate features on the image sensor wafer 1302. The alignment tolerance of the microlens to the pixel can be a fraction of the pixel size or equal to one pixel. In some embodiments, the alignment tolerance of the microlens to the pixel can be less than 10% of the pixel size. After bonding the individual dies to the image sensor wafer 1302, the image sensor wafer 1302 can be singulated into individual image sensor dies, thereby producing a fully integrated image sensor-PSOMMA sensor. Figure 14B Shown is an image sensor die 1404 after the image sensor wafer 1302 has been singulated in accordance with an embodiment of the present invention. In some embodiments, the second wafer die 1402 can be bonded to a previously formed separate image sensor die to form an integrated image sensor-PSOMMA sensor. These individual image sensor dies can be formed by singulating a larger image sensor wafer 1302 prior to bonding the second wafer die 1402.

[0211] Figure 15Shows a PSOMMA sensor 1500 fabricated as an example according to an embodiment of the present invention. The PSOMMA sensor 1500 includes a CMOS image sensor (CIS) wafer 1502. The PSOMMA sensor 1500 also includes a transparent substrate 1504. The transparent substrate 1504 can be glass. The nanocolumns 1506 are located on the transparent substrate 1504. An air gap 1508 is located between the nanocolumns 1506 and the CIS wafer 1502, such that the nanocolumns 1506 are between the air gap 1508 and the transparent substrate 1504. The air gap 1508 is located between the CIS wafer 1502 and the transparent substrate 1504. The air gap 1508 can have a thickness between 6 μm and 7 μm. The air gap 1508 can have a thickness between 3 μm and 4 μm. The PSOMMA sensor 1500 can include a die-to-die or wafer-to-wafer overlay accuracy of 0.3 μm, which can be the alignment tolerance during integration. The metasurface lens guides the sorted polarized light to the target image sensor pixels. Thus, the metasurface lens needs to match / align the metasurface lens with the image sensor with sub-pixel level accuracy. The air gap 1508 can be used to maintain the planarity between two wafers separated by less than 6 μm at a 12″ distance.

[0212] In some examples, a spacer layer can be fabricated on the CIS wafer (described below in Figure 16A - Figure 16C . The spacer layer can also be fabricated on the nanocolumns 1506 (described below in Figure 18A - Figure 18C . Figure 20A And FIG. 20B shows the alignment process of the CIS wafer 1602 and the nanocolumns 1506.

[0213] Figure 16A - Figure 16C Shows an example method for fabricating the CIS wafer 1502 according to an embodiment of the present invention. In Figure 16A , the CIS wafer 1602 is provided with a spacer layer 1604. The CIS wafer 1602 can include a previously fabricated image sensor. The secondary wafer 1608 is provided with an adhesive layer 1610. In Figure 16B , the secondary wafer 1608 contacts the CIS wafer 1602 such that the adhesive layer 1610 contacts the spacer layer 1604. In Figure 16CIn this case, the secondary wafer 1608 is removed, and the adhesive layer 1610 is transferred onto the top of the spacer layer 1604 such that the adhesive 1610a resides only on the top of the spacer layer 1604 and not on the regions between the spacer layers 1604. The spacer layer 1604 can be 6 μm thick or thicker. The diameter of the CIS wafer 1602 can be 12 inches. The diameter of the secondary wafer 1608 can be 12 inches. The channels 1612 can be located between adjacent portions of the spacer layer 1604. The CIS wafer 1602 can include nanocolumns. An example of such an adhesive transfer process is disclosed in U.S. Patent No. 6,669,803, titled "Simultaneous provision of controlled height bonding material at a wafer level and associated structures," filed on September 29, 2000, the entire content of which is incorporated herein by reference in its entirety for all purposes.

[0214] Figure 17 A plan view of a CIS wafer 1602 according to an embodiment of the present invention is shown. The CIS wafer 1602 includes a spacer layer 1604. An imaging region 1702 is included inside each cell of the spacer layer 1604.

[0215] Figure 18A - Figure 18C An example method for manufacturing a nanocolumn wafer according to an embodiment of the present invention is shown. In Figure 18A this case, the nanocolumn substrate 1802 includes a plurality of nanocolumns 1804. A spacer layer 1806 is provided under the plurality of nanocolumns 1804 such that the spacer layer 1806 is on opposite sides of the nanocolumn substrate 1802. The secondary wafer 1808 has an adhesive layer 1810 that directly faces the spacer layer 1806. In Figure 18B this case, the secondary wafer 1808 contacts the nanocolumn substrate 1802 such that the adhesive layer 1810 contacts the spacer layer 1806. In Figure 18CIn this process, the secondary wafer 1808 is removed, and the adhesive layer 1810 is transferred to the top of the spacer layer 1806 such that the adhesive 1810a is present only on the top of the spacer layer 1806 and not on the area between the spacer layers 1806. The spacer layer 1806 can be 6 μm wide. The nanocolumn substrate 1802 can be 12 inches in diameter. The diameter of the secondary wafer 1808 can be 12 inches. The nanocolumn substrate 1802 is provided with the spacer layer 1806. The secondary wafer 1808 is provided with a layer of adhesive 1810. An example of such an adhesive transfer process is disclosed in U.S. Patent 6,669,803, entitled "Simultaneous provision of controlled height bonding material at a wafer level and associated structures," filed on September 29, 2000, the entire content of which is incorporated herein by reference in its entirety for all purposes.

[0216] Figure 19 FIG. shows a plan view of the nanocolumn substrate 1802 according to an embodiment of the present invention. The nanocolumn substrate 1802 includes spacer layers 1806 in a grid orientation.

[0217] Figure 20A and Figure 20B FIG. shows an example method of manufacturing a PSOMMA sensor according to an embodiment of the present invention. The PSOMMA sensor includes combining Figure 18C the nanocolumn substrate 1802 with Figure 16C the CIS wafer 1602. In Figure 20A this process, Figure 18C the nanocolumn substrate 1802 and Figure 16C the CIS wafer 1602 are positioned close to each other. The spacer layer 1806 and the corresponding adhesive 1810a on the nanocolumn substrate 1802 have a width of 2002. The channel 1612 is between adjacent portions of the spacer layer 1604. The width 2004 of the channel 1612 is greater than the width 2002 of the spacer layer 1806. In some examples, the width 2004 of the channel 1612 can be 0.6 μm greater than the width 2002 of the spacer layer 1806. In Figure 20B this process, the two wafers 1602 and 1802 are bonded to each other, where the spacer layers 1604, 1806 are precisely interlocked to achieve a precisely aligned bond.

[0218] Figure 21A and Figure 21B FIG. shows a process flow for manufacturing a CIS wafer according to an embodiment of the present invention. In Figure 21A this process, a CIS wafer 2102 including bonding pads 2104 is provided. InFigure 21B In Figure 21B , a dielectric layer 2106 is deposited on a CIS wafer 2102 including a bonding pad 2104. The dielectric layer 2106 can be a SiO2 layer. A plasma enhanced tetraethyl orthosilicate (PETEOS) process can be utilized to deposit the dielectric layer 2106. The dielectric layer 2106 can have a thickness of 5.5 μm. A planarization process can be utilized on the dielectric layer 2106. The planarization process can be a chemical mechanical polishing (CMP) process.

[0219] Figure 22A and Figure 22B shows a process flow for manufacturing a nanowire wafer according to an embodiment of the present invention. In Figure 22A Figure 22A , a nanowire substrate 2202 including nanowires 2204 is provided. In Figure 22B Figure 22B , a dielectric layer 2206 is deposited on the nanowire substrate 2202 including the nanowires 2204. The dielectric layer 2206 can be a SiO2 layer. The dielectric layer 2206 can be deposited using a plasma enhanced tetraethyl orthosilicate (PETEOS) process. The dielectric layer 2206 can have a thickness of 5.5 μm. A planarization process can be utilized on the dielectric layer 2206. The planarization process can be a chemical mechanical polishing (CMP) process.

[0220] Figure 23A - Figure 23H is according to an embodiment of the present invention using Figure 21A and Figure 21B Figure 21B and the process described in Figure 22A and Figure 22B Figure 22B to manufacture an imaging sensor using a CIS wafer manufactured by the process described in Figure 23A Figure 23A , the nanowire substrate 2202 is bonded to the CIS wafer 2102 such that the nanowires 2204 are separated from the bonding pads 2104 by a combined dielectric layer 2302, which includes the dielectric layer 2106 of the CIS wafer 2102 and the dielectric layer 2206 of the nanowire substrate 2202. Some wafer-to-wafer bonders can only process wafers with a warp of no more than 250 μm. The dielectric layer 2106 of the CIS wafer 2102 and the dielectric layer 2206 deposited on the nanowires 2204 using a tetraethyl orthosilicate (TEOS) process may cause the wafers to bend, which will not allow wafer bonding. Advantageously, the planarization process on the dielectric layer 2106 of the CIS wafer 2102 and the dielectric layer 2206 on the nanowires 2204 provides the ability to bond the dielectric layer 2106 of the CIS wafer 2102 and the dielectric layer 21 of the wafer to the dielectric layer 2206 on the nanowires 2204. The wafer bonding process can provide an alignment accuracy of 200 nm.

[0221] In Figure 23BIn [description], the nanocolumn substrate 2202 is removed, leaving the nanocolumns 2204 on the combined dielectric layer 2302. The nanocolumn substrate 2202 can be removed by grinding, etching, and / or planarization (such as CMP). In Figure 23C In [description], a photoresist layer 2304 is deposited onto the nanocolumns 2204. The photoresist layer can be spin-coated onto the nanocolumns 2204. In Figure 23D In [description], the photoresist layer 2304 is patterned and then the combined dielectric layer 2302 is etched to expose the bond pads 2104. In Figure 23E In [description], the photoresist layer 2304 is removed and a barrier seed layer 2306 can be deposited on the bond pads 2104, the sidewalls of the combined dielectric layer 2302, and the nanocolumns 2204. In some embodiments, the barrier seed layer 2306 can be omitted. In Figure 23F In [description], a conductive layer 2308 is deposited on the barrier seed layer 2306. The conductive layer 2308 can be a copper (Cu) layer. The conductive layer 2308 can be deposited using an electrochemistry plating (ECP) process. In Figure 23G In [description], the conductive layer 2308 is planarized to the nanocolumns 2204. The planarization process can be a CMP process. In Figure 23H In [description], a chip on board (COB) process is performed. The COB process can be directly wire bonding a sensor to a printed circuit, which can be a printed circuit board. The printed circuit board can be flexible.

[0222] Figure 24A - Figure 24C is a manufacturing process of an imaging sensor using a CIS wafer manufactured by the process described in Figure 21A and Figure 21B and a nanocolumn wafer manufactured by the process described in Figure 22A and Figure 22B In [description]. Figure 24A and Figure 24B correspond to the steps of Figure 23A and Figure 23B . Figure 23A and Figure 23B The description of these steps in [description] is related to the steps of Figure 24A and Figure 24B . In Figure 24C In [description], a through-silicon via (TSV) process is performed. Instead of wire bonding to the bond pads on the front surface of the chip, holes can be formed from the back side of the chip / wafer to reach the bond pads on the front side, and then metal wires can be directly bonded from the holes using solder balls on the back side of the wafer to achieve chip-level packaging. In this case, there may be no openings in the bond pads in the front.

[0223] Figure 25A - Figure 25H is a manufacturing process for manufacturing an imaging sensor according to an embodiment of the present invention. In Figure 25ATherein, a P-type substrate 2502 is provided. The P-type substrate 2502 can be highly doped with a P-type dopant. In Figure 25B Therein, a nanocolumn layer 2505 can be formed on the P-type substrate 2502. The nanocolumn layer 2505 includes embedded nanocolumns 2504. In Figure 25C Therein, a crystal layer 2506 is grown on top of the nanocolumn layer 2505. The crystal layer 2506 can be a P-type layer including a P-type dopant. Epitaxial growth can be utilized to grow the crystal layer 2506.

[0224] In Figure 25D Therein, a P-type doping process is performed, which generates a highly P-doped layer 2508 on the crystal layer 2506. The P-type doping process can be an implantation process (e.g., an ion implantation process). In Figure 25E Therein, another crystal layer 2510 is grown on top of the highly P-doped layer 2508. Epitaxial growth can be utilized to grow the crystal layer 2510. In Figure 25F Therein, a CMOS layer 2512 can be formed on the crystal layer 2510. The CMOS layer 2512 can be a front-end circuit fabricated using a typical CMOS process. The crystal layer 2510 can be a base layer for the CMOS layer 2512. In Figure 25G Therein, the device is bonded (e.g., wafer bonding) to a carrier wafer 2514, wherein the CMOS layer 2512 is in direct contact with the carrier wafer. In Figure 25H Therein, the P-type substrate 2502 is removed. The P-type substrate 2502 can be removed by grinding, wet chemical etching, and / or dry chemical etching.

[0225] Various examples of the PSOMMA configuration

[0226] Figure 26Shows an example operation of an exemplary metasurface microlens array (MLA) according to an embodiment of the present invention. This design takes into account the variation of the angle with the position of the MLA. The light entering the metasurface MLA 2602 includes the chief ray angle (CRA) 2604 from the imaging optics or an aperture that can vary in the plane of the metasurface MLA 2602. This variation can be calculated or measured to give the CRA 2604 as a function of the angle and / or position on the MLA, e.g., as a distance from the center of the MLA. Conventional refractive MLAs may not be able to address this variation in angle. Conventional refractive MLAs can be designed to work well or be effective optically only for a zero-degree CRA. Each metasurface microlens of the metasurface MLA 2602 is optimized for a specific area of illumination (AOI). In a conventional refractive MLA, each microlens may include aspherical terms or more complex phase functions. This approach for a conventional refractive MLA would involve multiple refractive lenses / prisms, which would increase complexity. Thus, the metasurface MLA allows for solutions that include various CRAs 2604 that may not be easily achievable with a conventional refractive MLA.

[0227] Figure 27A Shows an example pixel arrangement for four polarization pixels according to an embodiment of the present invention. The pixel arrangement includes a group 2702 of multiple polarization pixels, including a first pixel 2702a, a second pixel 2702b, a third pixel 2702c, and a fourth pixel 2702d having different polarizations. There are one or more vertical pixels 2704 that separate the adjacent groups 2702 of polarization pixels. The normal pixels 2704 are non-polarized and / or non-dispersive pixels, which are substantially blank or simply optically provide the ability for focusing or filtering. In some embodiments, the number of normal pixels 2704 between adjacent groups 2702 of polarization pixels can be more or less than the number shown.

[0228] Figure 27BShows an example pixel arrangement for two polarization pixels according to an embodiment of the present invention. The pixel arrangement includes a group 2706 of a plurality of polarization pixels, including a first pixel 2706a and a second pixel 2706b having different polarizations. The first pixel 2706a and the second pixel 2706b may include orthogonal polarizations. The groups 2706 of polarization pixels may not be the same pair. For example, some pairs of the groups 2706 of polarization pixels may detect S1 polarization, some may detect S2 polarization, and some may detect S3 polarization. In some embodiments, depending on the transmitted polarization, such as static polarization illumination or a known polarization illumination source, the detected polarization may be consistent with one or more such transmitted polarization sources. There are one or more vertical pixels 2708 that separate adjacent groups 2706 of polarization pixels. In some embodiments, the number of normal pixels 2708 between adjacent groups 2706 of polarization pixels may be more or less than the number shown.

[0229] There may be a microlens array (MLA) including different microlenses associated with each pixel. The microlenses of the normal pixels 2704, 2708 and the microlenses of the polarization pixels within the groups 2702, 2706 of polarization pixels may have the same principal image plane. Because Figure 27A in the four-polarization pixel arrangement of Figure 27B and the two-polarization pixel arrangement of Figure 27A in the four-polarization pixel arrangement of Figure 27B the apertures of the polarization pixels within the groups 2702, 2706 of polarization pixels may be different, so for the four-polarization pixel arrangement of

[0230] Figure 27C Shows an example of an example pixel arrangement for four polarization pixels according to an embodiment of the present invention. The pixel arrangement includes a group 2710 of a plurality of polarization pixels, which includes a first pixel 2710a, a second pixel 2710b, a third pixel 2710c, and a fourth pixel 2710d having different polarizations. There is one or more normal pixels 2712 that separate the first pixel 2710a, the second pixel 2710b, the third pixel 2710c, and the fourth pixel 2710d. In some embodiments, the number of normal pixels 2712 between adjacent groups 2702 of polarization pixels may be more or less than the number shown. The groups 2710 of polarization pixels may also be separated by one or more normal pixels 2712.

[0231] In combination with Figures 27A - 27CThe described design can be used in sorting or filtering embodiments as described below.

[0232] Figure 28A is a cross-sectional view of an imaging device including polarization splitting ability according to an embodiment of the present invention. As shown, the imaging device includes a microlens 2802 that guides light 2803 into a polarization splitting metasurface 2804. The microlens 2802 can be part of a microlens array. The polarization splitting metasurface 2804 splits the light 2803 into a first polarized light 2806a and a second polarized light 2806b. The first polarized light 2806a and the second polarized light 2806b can have orthogonal polarizations. An image sensor including a first region 2808a and a second region 2808b is positioned below the polarization splitting metasurface 2804. The first polarized light 2806a is guided into the first region 2808a, and the second polarized light 2806b is guided into the second region 2808b. The microlens 2802 is offset from the first region 2808a and the second region 2808b such that the center of the microlens 2802 is between the first region 2808a and the second region 2808b.

[0233] Each microlens 2802 can cover at least half of the first region 2808a and the second region 2808b, with the overlapping metasurface microlenses 2804 therebetween to diffract the first polarized light 2806a in a first direction into the first region and the second polarized light 2806b in a second direction into the second region. The first polarized light 2806a can be polarized orthogonally to the second polarized light 2806b.

[0234] Figure 28B is a plan view of an imaging device including polarization splitting ability according to an embodiment of the present invention. The imaging device includes the combination Figure 28A described microlenses and polarization splitting metasurfaces. The polarization splitting metasurface divides light into a first polarized light 2810a, a second polarized light 2810b, a third polarized light 2810c, and a fourth polarized light 2810d. The imaging device includes an image sensor that includes a first region 2812a, a second region 2812b, a third region 2812c, and a fourth region 2812d. The first polarized light 2810a is guided into the first region 2812a, the second polarized light 2810b is guided into the second region 2812b, the third polarized light 2810c is guided into the third region 2812c, and the fourth polarized light 2810d is guided into the fourth region 2812d.

[0235] Figure 28Cis a plan view of an imaging device including polarization beam splitting ability according to an embodiment of the present invention. The imaging device includes a first polarization beam splitting part 2814A, a second polarization beam splitting part 2814b, and a third polarization beam splitting part 2814c, each of which has a structure similar to the imaging device described in conjunction with Figure 28A As shown, in each of the first polarization beam splitting part 2814a, the second polarization beam splitting part 2814b, and the third polarization beam splitting part 2814c, light is split such that different polarizations are directed in different directions. Each of the first polarization beam splitting part 2814a, the second polarization beam splitting part 2814b, and the third polarization beam splitting part 2814c can operate on different polarizations. For example, the first polarization beam splitting part 2814a can split light into a first polarization and a second polarization, while the second polarization beam splitting part 2814b can split light into a third polarization and a fourth polarization.

[0236] Figure 29A is a cross-sectional view of an imaging device including polarization beam splitting ability according to an embodiment of the present invention. As shown, the imaging device includes an array of microlenses 2902a, 2902b that direct light 2903 into a polarization separation metasurface 2904. The polarization separation metasurface 2904 separates the light 2903 into a first polarized light 2906a and a second polarized light 2906b. The first polarized light 2906a and the second polarized light 2906b can have orthogonal polarizations. An image sensor including a first region 2908a and a second region 2908b is located below the polarization separation metasurface 2904. The first polarized light 2906a is directed into the first region 2908a, and the second polarized light 2906b is directed into the second region 2908b. Light from the plurality of metalenses 2902a, 2902b can be separated in different parts of the polarization separation metasurface 2904 such that the first polarized light 2906a is directed into the first region 2908a and the second polarized light 2906b is directed into the second region 2908b. As shown, in some parts of the polarization separation metasurface 2904, the first polarized light 2906a can be a zero-order light that passes through the polarization separation metasurface 2904, and the second polarized light 2906b can be a diffracted light that is redirected in a different direction. In other parts of the polarization separation metasurface 2904, the second polarized light 2906b can be a zero-order light that passes through the polarization separation metasurface 2904, and the first polarized light 2906a can be a diffracted light that is redirected in a different direction. One microlens 2902a is centered on the first region 2908a. One microlens 2902b is centered on the second region 2908b.

[0237] Figure 29B is a plan view of an imaging device including polarization beam splitting ability according to an embodiment of the present invention. The imaging device includes the combination of Figure 29AThe described microlens and polarization splitting metasurface. However, the polarization splitting metasurface separates light into a first polarized light 2910a, a second polarized light 2910b, a third polarized light 2910c, and a fourth polarized light 2910d (in the vertical direction). As shown, the polarization splitting metasurface separates each polarized light into different directions, where the fourth polarized light 2910d is the zero-order light passing through the polarization splitting metasurface 2904. The imaging device includes an image sensor, which includes a first region 2912a, a second region 2912b, a third region 2912c, and a fourth region 2912d. Different parts of the polarization splitting metasurface separate light such that all of the first polarized light 2910a enters the first region 2912a, the second polarized light 2910b enters the second region 2912b, the third polarized light 2910c enters the third region 2912c, and the fourth polarized light 2910d enters the fourth region 2912d.

[0238] Figure 29C is a plan view of an imaging device including polarization splitting ability according to an embodiment of the present invention. The imaging device includes a first polarization splitting part 2914A, a second polarization splitting part 2914b, and a third polarization splitting part 2914c, each of which is constructed similarly to the imaging device described in combination Figure 29A The imaging device described. As shown, in each of the first polarization splitting part 2914a, the second polarization splitting part 2914b, and the third polarization splitting part 2914c, light is separated such that different polarizations are directed to different directions. Each of the first polarization splitting part 2914a, the second polarization splitting part 2914b, and the third polarization splitting part 2914c can operate with different polarizations. For example, the first polarization splitting part 2914a can split light into a first polarization and a second polarization, while the second polarization splitting part 2914b can split light into a third polarization and a fourth polarization.

[0239] Figure 30A shows a cross-sectional view of an imaging device including polarization splitting ability according to an embodiment of the present invention. The function of the device is similar to that described in combination Figure 28AThe device described above. As shown in the figure, the imaging device includes an array of microlenses 3002a, 3002b, 3002c that direct light 3003 into a polarization splitting metasurface 3004. The polarization splitting metasurface 3004 splits the light 3003 into a first polarized light 3006a and a second polarized light 3006b. The first polarized light 3006a and the second polarized light 3006b may have orthogonal polarizations. An image sensor including a first region 3008a, a second region 3008b, a third region 3008c, and a fourth region 3008d is located below the polarization splitting metasurface 3004. The first polarized light 3006a is directed into the first region 3008a, while the second polarized light 3006b is directed into the second region 3008b. Adjacent portions of the polarization splitting metasurface 3004 have opposite separation capabilities such that the second polarized light 3006b from adjacent portions of the polarization splitting metasurface 3004 is directed into the second region 3008b. The polarization splitting metasurface 3004 continues to repeat such that the first polarized light 3006a is directed into the third region 3008c and the second polarized light 3006b is directed into the fourth region 3008d. One microlens 3002a is offset from the first region 3008a and the second region 3008b such that the center of the microlens 3002a is between the first region 3008a and the second region 3008b. One microlens 3002b is offset from the second region 3008b and the third region 3008c such that the center of the microlens 3002b is between the second region 3008b and the third region 3008c. One microlens 3002c is offset from the third region 3008c and the fourth region 3008d such that the center of the microlens 3002 is between the third region 3008c and the fourth region 3008d.

[0240] Figure 30B A plan view of an imaging device including polarization splitting capabilities according to an embodiment of the present invention is shown. The imaging device includes a combination of Figure 30A the microlenses and polarization splitting metasurface described above. However, the polarization splitting metasurface divides the light into a first polarized light 3010a, a second polarized light 3010b, a third polarized light 3010c, and a fourth polarized light 3010d. The imaging device includes an image sensor that includes a first region 3012a, a second region 3012b, a third region 3012c, and a fourth region 3012d. The first polarized light 3010a is directed into the first region 3012a, the second polarized light 3010b is directed into the second region 3012b, the third polarized light 3010c is directed into the third region 3012c, and the fourth polarized light 3010d is directed into the fourth region 3012d.

[0241] Figure 30Cis a plan view of an imaging device including polarization splitting ability according to an embodiment of the present invention. The imaging device includes a first polarization splitting section 3014A, a second polarization splitting section 3014b, and a third polarization splitting section 3014c, each of which is configured similarly to the imaging device described in conjunction with Figure 30A As shown, in each of the first polarization splitting section 3014a, the second polarization splitting section 3014b, and the third polarization splitting section 3014c, the splitting is such that different polarizations are directed in different directions. Each of the first polarization splitting section 3014a, the second polarization splitting section 3014b, and the third polarization splitting section 3014c can operate on different polarizations. For example, the first polarization splitting section 3014a can split light into a first polarization and a second polarization, while the second polarization splitting section 3014b can split light into a third polarization and a fourth polarization.

[0242] Figure 31A shows a cross-sectional view of an imaging device including polarization filtering ability according to an embodiment of the present invention. As shown, the imaging device includes microlens arrays 3102a, 3102b that direct light 3103 into a polarization filtering metasurface 3104. A portion of the polarization filtering metasurface 3104 is configured to transmit first polarized light 3106A. A portion of the polarization filtering metasurface 3104 is configured to transmit second polarized light 3106b. An image sensor including a first region 3108a and a second region 3108b is located below the polarization separation metasurface 3104. The first polarized light 3106a is directed to the first region 3108a, and the second polarized light 3106b is directed to the second region 3108b. One microlens 3102a is aligned with the first region 3108a, and one microlens 3102b is aligned with the second region 3108b.

[0243] Figure 31B shows a plan view of an imaging device including polarization filtering ability according to an embodiment of the present invention. The imaging device includes the microlenses and the polarization filtering metasurface described in conjunction with Figure 30A However, different portions of the polarization filtering metasurface split light into first polarized light 3110a, second polarized light 3110b, third polarized light 3110c, and fourth polarized light 3110d. The imaging device includes an image sensor that includes a first region 3112a, a second region 3112b, a third region 3112c, and a fourth region 3112d. The first polarized light 3110a is directed into the first region 3112a, the second polarized light 3110b is directed into the second region 3112b, the third polarized light 3110c is directed into the third region 3112c, and the fourth polarized light 3110d is directed into the fourth region 3112d.

[0244] Figure 32AA cross-sectional view of an imaging device including polarization filtering capabilities according to an embodiment of the present invention is shown. The imaging device shares many features with the imaging device described in conjunction with Figure 31A The imaging device includes an array of microlenses 3202a, 3202b that direct light 3203 into a polarization filtering metasurface. The polarization filtering metasurface includes a first region 3204a that transmits first polarized light and a second region 3204b that transmits second polarized light. The first region 3204a filters out all polarizations of the light 3203 except for the first polarized light, while the second region 3204b filters out all polarizations of the light 3203 except for the second polarized light. Below the polarization filtering metasurface is an image sensor that includes a first pixel 3206a and a second pixel 3206b. The first region 3204a corresponds to the first pixel 3206a such that the first polarized light is detected by the first pixel 3206a, while the second region 3204b corresponds to the second pixel 3206b such that the second polarized light is detected by the second pixel 3206b.

[0245] Figure 32B A cross-sectional view of an imaging device including polarization sorting capabilities according to an embodiment of the present invention is shown. The imaging device shares many features with the imaging device described in conjunction with Figure 28A The imaging device includes a microlens array 3208 that directs light into a polarization sorting metasurface 3210. The polarization sorting metasurface 3210 sorts the light into first polarized light and second polarized light. The polarization sorting metasurface is separated from the light sensor by a back focal length 3214. The back focal length can be 10 μm. The light sensor includes a first pixel 3212a configured to detect the first polarized light and a second pixel 3212b configured to detect the second polarized light.

[0246] Figure 32C A cross-sectional view of an imaging device including polarization sorting capabilities according to an embodiment of the present invention is shown. The imaging device shares many features with the imaging device described in conjunction with Figure 30A The imaging device includes microlens arrays 3216a, 3216b that direct light into a polarization sorting metasurface. The polarization sorting metasurface includes a first region 3218a configured to sort the first polarized light in a first direction and the second polarized light in a second direction and a second region 3218b configured to sort the second polarized light in the first direction and the first polarized light in the second direction. The image sensor is configured to receive light from the polarization sorting metasurface. The image sensor includes a first pixel 3220a configured to receive the first polarized light from the first region 3218a, a second pixel 3220b configured to receive the second polarized light from the first region 3218a and the second region 3218b, and a third pixel 3220c configured to receive the first polarized light from the second region 3218b. As Figure 30AAs shown, there can be more microlenses and regions of the polarization sorting metasurface adjacent to the region of the polarization sorting metasurface shown in Figure 32C For each pixel of the image sensor, there can be one microlens. The back focal length 3222 can be shorter than the back focal length 3214 of the Figure 32B image sensor. For example, compared to the back focal length 3214 of 10 μm, the back focal length 3222 can be 5 μm.

[0247] Figure 33A A plan view of an exemplary polarization filtering nanograting according to an embodiment of the present invention is shown. In some embodiments, the polarization filtering nanograting can be interspersed with a polarization filtering metasurface as described in Figures 31A - 31B and Figure 32A . The polarization filtering nanograting includes different regions 3302a, 3302b, 3302c, 3302d corresponding to different pixels of the image sensor. The nanograting has some advantages, namely, it is easier to fabricate and has less angular sensitivity, which can make an imaging device including a PSOMMA and a nanograting on an imaging sensor more advantageous. In some embodiments, the microlens array function for polarization pixels can be implemented as one or more metagrating, and the microlens array function can be implemented as one or more refractive lenses for normal pixels.

[0248] Figure 33B A plan view of an exemplary polarization sorting metasurface according to an embodiment of the present invention is shown. The polarization sorting metasurface can be used as a polarization sorting metasurface as described in Figures 28A - 28C , Figures 29A - 29C , Figures 30A - 30C , Figure 32B and Figure 32C . The polarization filtering metasurface includes different regions 3304a, 3304b, 3304c, 3304d corresponding to different pixels of the image sensor.

[0249] Figure 34A A schematic plan view of an exemplary linear polarization sorting metasurface according to an embodiment of the present invention is shown. The linear polarization sorting metasurface can be used in a polarization sorting metasurface as described in Figures 28A - 28C , Figures 29A - 29C , Figures 30A - 30C , Figure 32B and Figure 32C . The polarization sorting metasurface diffracts different linear polarizations in different directions. In some embodiments, the different linear polarizations can be different orthogonal linear polarizations. As shown, the linear polarization sorting metasurface includes various metasurface elements 3402. The metasurface elements 3402 can have different sizes (e.g., height and / or width). The metasurface elements 3402 can have the same orientation. The metasurface elements can be tilted at the same angle.

[0250] Figure 34B shows a schematic plan view of an exemplary circular polarization sorting metasurface according to an embodiment of the present invention. The circular polarization sorting metasurface can be used in combination with Figures 28A - 28C , Figures 29A - 29C , Figures 30A - 30C , Figure 32B and Figure 32C described polarization sorting metasurfaces. The circular polarization sorting metasurface diffracts different circular polarizations in different directions. In some embodiments, the different circular polarizations can be different opposite circular polarizations. In some embodiments, the polarization sorting metasurface can sort different elliptical polarizations in different directions. The circular polarization sorting metasurface includes various metasurface elements 3404. The metasurface elements 3404 can have the same size (e.g., height and / or width). The metasurface elements can have different tilt angles relative to the horizontal axis and / or the vertical axis.

[0251] Figure 34C shows a schematic plan view of an arbitrary polarization sorting metasurface according to an embodiment of the present invention. The arbitrary polarization sorting metasurface can be used in combination with Figures 28A - 28C , Figures 29A - 29C , Figures 30A - 30C , Figure 32B and Figure 32C described polarization sorting metasurfaces. The arbitrary polarization sorting metasurface diffracts different arbitrary polarizations in different directions. In some embodiments, the different polarizations can be different orthogonal or opposite polarizations. The arbitrary polarization can be elliptical, circular or linear. The linear polarization can be vertical and horizontal. The linear or elliptical polarization can be tilted. The circular polarization can be right-handed and left-handed. The elliptical polarization can be right-handed and left-handed. The arbitrary polarization sorting metasurface includes various metasurface elements 3406. The metasurface elements 3406 can have the same size (e.g., height and / or width). The metasurface elements 3406 can have different tilt angles relative to the horizontal axis and / or the vertical axis.

[0252] Figure 35AA sparsely distributed PSOMMA sensor according to an embodiment of the present invention is shown. The polarization separation metasurface 3504 is sparsely distributed on the wafer 3502. The remaining area of the wafer 3502 may be unpatterned or patterned only with MLA or metasurface MLA without polarization sorting. The wafer 3502 may be transparent glass. In some embodiments, the remaining area of the wafer 3502 may be patterned with a metasurface MLA that does not split light by polarization. Such a metasurface MLA may have the same small lenses repeating from end to end, or it may include small lenses with different focal lengths, which may be used for autofocus or distance and depth measurement and / or calculation. One or more different filters may be applied to some or all of the small lenses. The sparse configuration may be asymmetric, irregular, or vary across the sensor space. In particular, a sparser configuration in the center may be used, or a denser center configuration may be used, for example, to better capture facial points or to better capture an eye image at a location. Also, different polarization aspects may be used for different regions (e.g., linear polarization at the edge and circular polarization at the center).

[0253] Figure 35B A cross-sectional view of a sparsely distributed PSOMMA sensor according to an embodiment of the present invention is shown. The PSOMMA wafer 3502 including the polarization separation metasurface 3504 is integrated on the image sensor 3506. The image sensor 3506 is located below the polarization separation metasurface 3504. The image sensor 3506 may include R, G, B image sensors with R, G, B filters corresponding to their respective color sensors. The image sensor 3506 has some areas where the RGB filters are removed so that it is monochromatic in those areas, and then the PSOMMA wafer 3502 splits the light to the pixels in that monochromatic area.

[0254] The image sensor 3506 may include a color filter, which may be blank, monochromatic, or pass near-infrared light on the part that receives polarized light from the polarization separation metasurface 3504. In some embodiments, the color from the sparse PSOMMA pixels may be interpolated from adjacent pixels. In this figure, there is a Bayer pattern on a part of the pixels, and there is a subset of pixels that are polarization determination (PSOMMA) pixels. When reconstructing the image, for the pixels on the surface that are below the PSOMMA pixels, even if the PSOMMA pixels do not directly capture color, the image can be reconstructed in full color by using the neighboring RGB pixels as the color values at the PSOMMA pixels.

[0255] In the sparse format, open windows or pixels can be used only for intensity measurement and / or depth from defocus. Alternatively, the sparse metasurface 3504 can have metasurfaces for each pixel, but only some are polarization-based metasurface pixels and the other metasurfaces have other properties such as focusing and deflection. In some cases, various different light power microlenses can be used together with spot size differences to determine distance - for example, lenses with different focal lengths such that the image points are compared to find the spot with the most focused focus for determining distance or other features, which is called phase detection autofocus (PDAF).

[0256] Figure 36A An example PSOMMA sensor according to an embodiment of the present invention is shown. The PSOMMA 3602 is located below the color filter 3604. Each PSOMMA microlens of the PSOMMA 3602 has an area of the color filter 3604 on top. The PSOMMA microlens is optimized for the color of its corresponding area of the color filter 3604. As described above, the PSOMMA 3602 guides light to different areas of the image sensor 3606. Different image aspects can be distinguished for different pixels of the image sensor 3606.

[0257] Figure 36B An example PSOMMA sensor according to an embodiment of the present invention is shown. The PSOMMA 3654 is located in the entire cover glass 3652. The PSOMMA 3654 can be positioned such that certain areas of the cover glass 3652 have PSOMMA microlenses and most areas allow light to pass through the bare cover glass. The PSOMMA 3654 guides light to a specific color of the image sensor 3658. For example, the PSOMMA 3654 can guide light into the green pixels of the image sensor 3658. The operation of the PSOMMA 3654 is described throughout the above disclosure. The image sensor 3658 can include RGB sensor pixels, which can have built-in filters or can have separate color filters 3656 added. The image sensor 3658 can be a basic RGB sensor that utilizes the RGB sensor colors. Then, the part of the cover glass 3652 that does not include the PSOMMA 3654 can simply let light pass through to be normally captured in the image sensor 3658.

[0258] Although only a few embodiments of the present invention have been described in detail, it should be understood that the present invention can be implemented in many other forms without departing from the spirit or scope of the present invention. For example, the following embodiments are considered:

[0259] Clause 1. A polarization imaging device, comprising: a metasurface microlens array including a plurality of metasurface microlenses, wherein the plurality of metasurface microlenses includes a plurality of first metasurface microlenses configured to diffract image light to a first direction with an intensity proportional to a first polarization and to a second direction with an intensity proportional to a second polarization; and an image sensor located in the optical paths of the first polarization and the second polarization, and wherein the image sensor includes a plurality of image sensing units, the plurality of image sensing units including a first image sensing unit positioned to sense the first polarization and a second image sensing unit positioned to sense the second polarization.

[0260] Clause 2. The polarization imaging device according to Clause 1, wherein the plurality of image sensing units are different pixels and / or different regions of the image sensor.

[0261] Clause 3. The polarization imaging device according to Clause 1, wherein the plurality of image sensing units are different pixels and / or different regions of a plurality of image sensors.

[0262] Clause 4. The polarization imaging device according to Clause 1, wherein the plurality of metasurface microlenses further includes a plurality of second metasurface microlenses scattered between the first metasurface microlenses and configured to diffract image light to a third direction with an intensity proportional to a third polarization and to a fourth direction with an intensity proportional to a fourth polarization, and wherein the plurality of image sensing units further includes a third image sensing unit positioned to sense the third polarization and a fourth image sensing unit positioned to sense the fourth polarization.

[0263] Clause 5. The polarization imaging device according to Clause 4, wherein the first polarization and the third polarization are the same polarization, and wherein the second polarization and the fourth polarization light are the same polarization.

[0264] Clause 6. The polarization imaging device according to Clause 1, wherein the first metasurface microlens is further configured to diffract image light to a third direction with an intensity proportional to a third polarization and to a fourth direction with an intensity proportional to a fourth polarization, and wherein the plurality of image sensing units further includes a third image sensing unit positioned to sense the third polarization and a fourth image sensing unit positioned to sense the fourth polarization.

[0265] Clause 7. The polarization imaging device according to Clause 1, wherein the first metasurface microlens is further configured to transmit zero-order light in the third direction.

[0266] Clause 8. The polarization imaging device according to Clause 7, wherein the third direction is different from the first direction and the second direction.

[0267] Clause 9. The polarization imaging device according to Clause 8, wherein the plurality of image sensing units further includes a third image sensing unit configured to sense zero-order light.

[0268] Clause 10. The polarization imaging device according to Clause 7, wherein the third direction is the same as the first direction.

[0269] Clause 11. The polarization imaging device according to Clause 1, further comprising one or more refractive lenses, wherein the metasurface microlens array is positioned between the refractive lens and the image sensor.

[0270] Clause 12. The polarization imaging device according to Clause 1, wherein the first polarization and the second polarization are different polarizations, and wherein the first polarization and the second polarization are selected from the group consisting of: linearly polarized light, diagonally polarized light, elliptically polarized light, and circularly polarized light.

[0271] Clause 13. The polarization imaging device according to Clause 1, further comprising a microlens array, and wherein the metasurface microlens array includes a polarization metasurface.

[0272] Clause 14. The polarization imaging device according to Clause 13, wherein the microlens array is configured to separate image light into different pixels and the polarization metasurface is configured to diffract the first polarization in a first direction and diffract the second polarized light in a second direction.

[0273] Clause 15. The polarization imaging device according to Clause 14, wherein the polarization metasurface overlaps with the microlenses in the microlens array, and wherein the microlenses in the microlens array that do not overlap with the polarization metasurface transmit non-diffracted light to the image sensor, wherein the image sensor is configured to sense the non-diffracted light to measure the intensity of the non-diffracted light.

[0274] Clause 16. The polarization imaging device according to Clause 13, wherein the microlens array is a planar microlens array layer.

[0275] Clause 17. The polarization imaging device according to Clause 13, wherein the microlens array and the polarization metasurface are positioned on a single substrate.

[0276] Clause 18. The polarization imaging device according to Clause 17, wherein the main image plane is configured above the surface of the single substrate opposite to the image sensor.

[0277] Clause 19. The polarization imaging device according to Clause 17, wherein the main image plane is configured within the single substrate.

[0278] Clause 20. The polarization imaging device according to Clause 13, wherein the microlens array includes metasurface elements.

[0279] Clause 21. The polarization imaging device according to Clause 20, wherein the metasurface element is configured to receive a chief ray angle that varies across the imaging pupil of the polarization imaging device and collimate the light passing through each microlens.

[0280] Clause 22. The polarization imaging device according to Clause 20, wherein the metasurface element of the microlens array is configured to provide a refractive microlens effect.

[0281] Clause 23. The polarization imaging device according to Clause 13, wherein the microlens array includes a conventional refractive microlens array.

[0282] Clause 24. The polarization imaging device according to Clause 13, wherein the microlens array includes a combination of at least one conventional refractive microlens and at least one metasurface element.

[0283] Clause 25. The polarization imaging device according to Clause 1, further comprising a color filter positioned above or below the metasurface microlens array.

[0284] Clause 26. The polarization imaging device according to Clause 25, wherein the color filter is positioned above the metasurface microlens array and filters light into different colors, each of the different colors corresponding to a different metasurface microlens among the plurality of first metasurface microlenses, and wherein the metasurface microlens receives the filtered light from the color filter.

[0285] Clause 27. The polarization imaging device according to Clause 25, wherein the color filter is positioned below the metasurface microlens array and filters the diffracted first polarized light and second polarized light.

[0286] Clause 28. The polarization imaging device according to Clause 27, wherein the color filter includes different regions that filter light of different wavelengths.

[0287] Clause 29. The polarization imaging device according to Clause 28, wherein the plurality of first metasurface microlenses are spaced apart on a cover substrate.

[0288] Clause 30. The polarization imaging device according to Clause 29, wherein the spaced-apart first metasurface microlenses are configured to output light onto a region of the color filter that outputs light of the same color.

[0289] Clause 31. The polarization imaging device according to Clause 30, wherein the spaced-apart first metasurface microlenses output light onto a region of the color filter that outputs green light, and a gap between regions of the color filter that output green light outputs red light or blue light.

[0290] Clause 32. The polarization imaging device according to Clause 27, wherein the color filter includes different regions that output red light, green light, or blue light and a region that outputs monochromatic or near-infrared light, and wherein the plurality of first metasurface microlenses are spaced apart on the cover substrate, and the spaced-apart first metasurface microlenses output light onto the region of the color filter that outputs monochromatic or near-infrared light.

[0291] Clause 33. The polarization imaging device according to Clause 1, further comprising a microlens array that includes a plurality of separate microlenses that output collimated light into the metasurface microlens array.

[0292] Clause 34. The polarization imaging device according to Clause 33, wherein the plurality of separate microlenses are positioned between adjacent image sensing units such that light from the plurality of separate microlenses is diffracted by the plurality of first metasurface microlenses into adjacent image sensing units in opposite tilting directions.

[0293] Clause 35. The polarization imaging device according to Clause 34, wherein the plurality of repeating metasurface microlenses further includes a plurality of second metasurface microlenses that are configured to diffract a first polarization into a second direction and diffract a second polarization into a first direction, and wherein a first image sensing unit is further positioned to sense the first polarization diffracted from the plurality of second metasurface microlenses.

[0294] Clause 36. The polarization imaging device according to Clause 33, wherein each of the plurality of separate microlenses is centered on one of the plurality of first metasurface microlenses such that a first polarization passes through the one of the plurality of first metasurface microlenses and reaches a first image sensing unit, and wherein a second polarization is diffracted into a second direction and reaches a second image sensing unit.

[0295] Clause 37. The polarization imaging device according to Clause 1, wherein the first polarization and the second polarization are orthogonal linear polarizations, circular polarizations, elliptical polarizations, or arbitrary polarizations.

[0296] Clause 38. The polarization imaging device according to Clause 1, wherein the plurality of first metasurface microlenses are spaced apart by non-diffracting portions between adjacent first metasurface microlenses.

[0297] Clause 39. The polarization imaging device according to Clause 38, wherein the plurality of metasurface microlenses and the plurality of image sensing units configured to sense the first polarization and the second polarization are positioned between the image sensing units of an image sensor configured to sense red light, green light, or blue light.

[0298] Clause 40. The polarization imaging device according to Clause 39, wherein the plurality of first metasurface microlenses are configured to diffract light into the plurality of image sensing units configured to sense the first polarization and the second polarization, and the image sensing units configured to sense red, green, or blue light.

[0299] Clause 41. The polarization imaging device according to Clause 39, wherein the plurality of image sensing units configured to sense the first polarization and the second polarization are further configured to sense monochromatic or near-infrared light.

[0300] Clause 42. The polarization imaging device according to Clause 1, wherein the image sensor is a complementary metal oxide semiconductor (CMOS) image sensor, a charge-coupled device sensor, a silicon diode sensor, a cadmium sulfide sensor diode, an RGB sensor, an indirect time-of-flight (iToF) sensor, a direct time-of-flight (dToF) sensor.

[0301] Clause 43. A method of manufacturing a polarization imaging device, the method comprising: providing an image sensor wafer; depositing a spacer layer over the imaging sensor wafer; depositing a metasurface layer over the spacer layer; and patterning the metasurface layer to form discrete metasurfaces over the spacer layer.

[0302] Clause 44. The method according to Clause 43, further comprising directly depositing a packaging layer over the discrete metasurfaces.

[0303] Clause 45. The method according to Clause 44, further comprising planarizing the packaging layer such that the packaging layer is at the same level as the individual metasurfaces.

[0304] Clause 46. The method according to Clause 44, further comprising forming one or more top layers over the packaging layer and / or the individual metasurfaces.

[0305] Clause 47. A method of manufacturing a polarization imaging device, the method comprising: providing an image sensor wafer; providing a metasurface substrate comprising discrete metasurfaces; and depositing a spacer layer over each of the metasurfaces on the image sensor wafer and / or the metasurface substrate.

[0306] Clause 48. The method according to Clause 47, further comprising bonding the metasurface substrate to the image sensor wafer.

[0307] Clause 49. The method according to Clause 48, wherein the metasurface substrate comprises a transparent substrate.

[0308] Clause 50. The method according to Clause 48, further comprising removing the metasurface substrate such that the discrete metasurfaces remain attached to the image sensor wafer.

[0309] Clause 51. The method according to Clause 50, wherein removing the metasurface substrate includes grinding, wet chemical etching, and / or dry chemical etching of the metasurface substrate.

[0310] Clause 52. The method according to Clause 47, further comprising singulating the metasurface substrate into separate metasurface die, bonding at least one of the metasurface die to an image sensor wafer, and singulating the image sensor wafer into image sensor die each having a bonded metasurface die.

[0311] Clause 53. A method of manufacturing a polarization imaging device, the method comprising: providing a CMOS image sensor (CIS) wafer; positioning a first plurality of spacers on the CIS wafer; providing a first carrier wafer; positioning a first adhesive layer on the first carrier wafer; contacting the first plurality of spacers with the first adhesive layer such that the adhesive is coated on top of the first plurality of spacers; providing a nanopillar substrate having a plurality of nanopillars; positioning a second plurality of spacers on the nanopillar substrate; providing a second carrier wafer; positioning a second adhesive layer on the second carrier wafer; contacting the second plurality of spacers with the second adhesive layer such that the adhesive is coated on top of the second plurality of spacers; and simultaneously contacting the adhesive on the second plurality of spacers with the CIS wafer and contacting the adhesive on the first plurality of spacers with the nanopillar substrate to bond the CIS wafer and the nanopillar substrate together.

[0312] Clause 54. The method according to Clause 53, wherein after contacting the first plurality of spacers with the first adhesive layer such that the adhesive is coated on top of the first plurality of spacers, the adhesive is present only on top of the first plurality of spacers and not in the area between the first plurality of spacers.

[0313] Clause 55. The method according to Clause 53, wherein after contacting the second plurality of spacers with the second adhesive layer such that the adhesive is coated on top of the second plurality of spacers, the adhesive is present only on top of the second plurality of spacers and not in the area between the second plurality of spacers.

[0314] Clause 56. The method according to Clause 53, wherein the first plurality of spacers includes pairs of spacers that form channels, the size of the channels being designed to accommodate the spacers in the second plurality of spacers.

[0315] Clause 57. The method according to Clause 56, wherein the first plurality of spacers and the second plurality of spacers interlock with each other such that one of the second plurality of spacers is positioned within one of the channels of a pair of spacers.

[0316] Clause 58. The method according to Clause 53, wherein the first plurality of spacers form a plurality of rectangular shapes on the CIS wafer.

[0317] Clause 59. The method according to Clause 58, wherein an imaging area is formed inside the rectangular shape.

[0318] Clause 60. The method according to Clause 53, wherein the second plurality of spacers form a grid pattern on the nanowire substrate.

[0319] Clause 61. The method according to Clause 53, wherein the first plurality of spacers and the second plurality of spacers form an air gap between the plurality of nanowires and the CIS wafer.

[0320] Clause 62. A method of manufacturing a polarization imaging device, the method comprising: providing a CMOS image sensor (CIS) wafer having bonding pads; depositing a first dielectric layer on the CIS wafer and the bonding pads; planarizing the first dielectric layer; providing a nanowire substrate having a plurality of nanowires; depositing a second dielectric layer on the plurality of nanowires; planarizing the second dielectric layer; bringing the first dielectric layer into contact with the second dielectric layer such that they are bonded together to form a combined dielectric layer; removing the nanowire substrate to expose the plurality of nanowires; partially etching the combined dielectric layer to expose the bonding pads; and forming a conductive layer that is electrically connected to the bonding pads through the combined dielectric layer.

[0321] Clause 63. The method according to Clause 62, wherein the first dielectric layer and the second dielectric layer are silicon dioxide layers.

[0322] Clause 64. The method according to Clause 62, wherein the deposition of the first dielectric layer and the second dielectric layer is performed by a tetraethyl orthosilicate (TEOS) process.

[0323] Clause 65. The method according to Clause 64, wherein the TEOS process is a plasma-enhanced TEOS (PETEOS) process.

[0324] Clause 66. The method according to Clause 62, wherein the planarization of the first dielectric layer and the second dielectric layer is performed by a chemical mechanical polishing (CMP) process.

[0325] Clause 67. The method according to Clause 62, wherein the removal of the nanowire substrate is performed by grinding, etching, or a chemical mechanical polishing (CMP) process.

[0326] Clause 68. The method according to Clause 62, wherein the partial etching of the combined dielectric layer comprises: patterning the combined dielectric layer; and etching the combined dielectric layer to expose the bonding pads.

[0327] Clause 69. The method according to Clause 62 further includes growing a barrier seed layer on the bonding pads, the sidewalls of the combined dielectric layer, and the nanocolumns.

[0328] Clause 70. A polarization imaging device includes: a microlens array having at least two microlenses; a polarization filtering metasurface having two or more polarization filtering regions; and an imaging sensor having at least two regions, wherein imaging light including one or more polarization states is guided by the microlenses onto the polarization filtering metasurface, and the polarization filtering metasurface is configured to guide the one or more polarization states onto one or more regions of the imaging sensor.

[0329] Clause 71. The polarization imaging device according to Clause 70, wherein the microlens array includes refractive microlenses.

[0330] Clause 72. The polarization imaging device according to Clause 71, wherein the microlens array further includes a metasurface configured to provide the effect of refractive microlenses.

[0331] Clause 73. The polarization imaging device according to Clause 70, wherein the microlens array includes a metasurface configured to provide the effect of refractive microlenses.

[0332] Clause 74. The polarization imaging device according to Clause 70, wherein each of the at least two microlenses is above a single region among the one or more regions of the imaging sensor.

[0333] Clause 75. The polarization imaging device according to Clause 70, wherein each of the at least two microlenses is above two or more sensor regions among the one or more regions of the imaging sensor.

[0334] Clause 76. The polarization imaging device according to Clause 70, wherein each of the at least two microlenses is above a single polarization region among one or more regions of the two or more polarization filtering regions.

[0335] Clause 77. The polarization imaging device according to Clause 70, wherein each of the at least two microlenses is above two or more polarization filtering regions among the two or more polarization filtering regions.

[0336] Clause 78. A polarization imaging device, comprising: a metasurface grating array including a plurality of metasurface microlenses, wherein the plurality of metasurface microlenses include a plurality of first metasurface microlenses configured to filter image light having a first polarized light in a first region and a second polarization in a second region; and an image sensor located in the optical paths of the first polarization and the second polarization, and wherein the image sensor includes a plurality of image sensing units, the plurality of image sensing units including a first image sensing unit positioned to sense the first polarization and a second image sensing unit positioned to sense the second polarization.

[0337] Clause 79. The polarization imaging device according to Clause 34, wherein the first polarization is orthogonal to the second polarization.

[0338] Clause 80. The polarization imaging device according to Clause 34, wherein each microlens covers half of the two image sensing units with an overlapping metasurface microlens to diffract the first polarization into a first direction into the first image sensing unit and diffract the second polarization into a second direction into the second image sensing unit.

[0339] Clause 81. The polarization imaging device according to Clause 80, wherein the first polarization is orthogonal to the second polarization.

[0340] Doctrine of equivalents

[0341] Although the foregoing description includes many specific embodiments of the present invention, these embodiments should not be construed as limiting the scope of the present invention, but rather as examples of one embodiment thereof. Accordingly, it should be understood that the present invention may be practiced in a manner different from that specifically described without departing from the scope and spirit of the present invention. Therefore, the embodiments of the present invention should be considered illustrative rather than restrictive in all respects. Therefore, the scope of the present invention should not be determined by the illustrated embodiments, but by the appended claims and their equivalents.

Claims

1. A polarization imaging device, comprising: A metasurface microlens array including a plurality of metasurface microlenses, wherein the plurality of metasurface microlenses includes a plurality of first metasurface microlenses configured to diffract image light in a first direction with an intensity proportional to a first polarization and in a second direction with an intensity proportional to a second polarization; and an image sensor located in the optical paths of the first polarization and the second polarization, and wherein the image sensor includes a plurality of image sensing units, the plurality of image sensing units including a first image sensing unit positioned to sense the first polarization and a second image sensing unit positioned to sense the second polarization.

2. The polarization imaging device according to claim 1, wherein, The plurality of image sensing units are different pixels and / or different regions of the image sensor.

3. The polarization imaging device according to claim 1, wherein, The plurality of metasurface microlenses further includes a plurality of second metasurface microlenses scattered among the first metasurface microlenses and configured to diffract image light in a third direction with an intensity proportional to a third polarization and in a fourth direction with an intensity proportional to a fourth polarization, and wherein the plurality of image sensing units further includes a third image sensing unit positioned to sense the third polarization and a fourth image sensing unit positioned to sense the fourth polarization.

4. The polarization imaging device according to claim 3, wherein, The first polarization and the third polarization are the same polarization, and wherein the second polarization and the fourth polarization are the same polarization.

5. The polarization imaging device according to claim 1, wherein, The first metasurface microlens is further configured to diffract image light in a third direction with an intensity proportional to a third polarization and in a fourth direction with an intensity proportional to a fourth polarization, and wherein the plurality of image sensing units further includes a third image sensing unit positioned to sense the third polarization and a fourth image sensing unit positioned to sense the fourth polarization.

6. The polarization imaging device according to claim 1, wherein, The first metasurface microlens is further configured to transmit zero-order light in the third direction.

7. The polarization imaging device according to claim 6, wherein, The third direction is different from the first direction and the second direction.

8. The polarization imaging device according to claim 7, wherein, The plurality of image sensing units further includes a third image sensing unit configured to sense the zero-order light.

9. The polarization imaging device according to claim 6, wherein, The third direction is the same as the first direction.

10. The polarization imaging device according to claim 1, further comprising a microlens array, and wherein, The metasurface microlens array includes a polarization metasurface.

11. The polarization imaging device according to claim 10, wherein, The microlens array is configured to separate image light into different pixels and the polarization metasurface is configured to diffract the first polarization in a first direction and the second polarization in a second direction.

12. The polarization imaging device according to claim 11, wherein, The polarization metasurface overlaps with the microlenses in the microlens array, and wherein the microlenses in the microlens array that do not overlap with the polarization metasurface transmit non-diffracted light to the image sensor, wherein the image sensor is configured to sense the non-diffracted light to measure the intensity of the non-diffracted light.

13. The polarization imaging device according to claim 10, wherein, The microlens array is a planar microlens array layer.

14. The polarization imaging device according to claim 10, wherein, The microlens array and the polarization metasurface are located on a single substrate.

15. The polarization imaging device according to claim 10, wherein, The microlens array includes metasurface elements.

16. The polarization imaging device according to claim 10, wherein, The microlens array includes a combination of at least one conventional refractive microlens and at least one metasurface element.

17. The polarization imaging device according to claim 1, further comprising a microlens array, the microlens array including a plurality of separate microlenses that output collimated light to a metasurface microlens array.

18. The polarization imaging device according to claim 17, wherein,The plurality of separated microlenses are positioned between adjacent image sensing units such that light from the plurality of separated microlenses is diffracted by the plurality of first metasurface microlenses in opposite inclined directions into adjacent image sensing units.

19. The polarization imaging device according to claim 18, wherein, The plurality of repeating metasurface microlenses further includes a plurality of second metasurface microlenses configured to diffract a first polarization in a second direction and a second polarization in a first direction, and wherein the first image sensing unit is further positioned to sense the first polarization diffracted from the plurality of second metasurface microlenses.

20. The polarization imaging device according to claim 1, wherein, The plurality of first metasurface microlenses are spaced apart by non-diffracting portions between adjacent first metasurface microlenses.

Images