Wide-field-of-view metasurface optical device, sensor, camera and projector

Abstract

A wide field of view sensor or projector includes a transparent substrate having one or more apertures on one side and one or more chip stacks bonded to opposite sides. Each chip stack includes a flat optic layer (e.g., a metasurface), at least one spacer layer, optionally a filter layer, and an image sensor or light source. In one example, two apertures and two corresponding chip stacks are provided; the two chip stacks each include an image sensor but different metasurfaces and filters to capture different information from the scene. In an alternative embodiment, the two chip stacks include a light source and an image sensor to serve as a light projector and a light receiver, respectively. In other examples, two apertures and a single chip stack are provided, where the single chip stack includes two metasurfaces and / or two filters corresponding to the two apertures.

Description

Background Art

[0001] The present invention relates to optical devices and optical systems, and more particularly to metasurfaces, metamaterials, imaging, sensing, projection, 3-D sensors, and depth sensing.

[0002] 3-D depth sensing is an important technology with applications in biometrics, automotive sensing, AR / VR, robotics, and industrial automation. Existing 3-D optical sensors based on structured light, time-of-flight (TOF), or stereo techniques are typically assembled from traditional bulk refractive optical devices and discrete components. The sensor optics involve multiple stacked lenses, which increases module thickness and assembly complexity. In addition, current 3-D sensors are limited by a small field of view (FOV, the angular range of scene objects that can be imaged by an optical system), typically less than 90°, and a lateral / spatial resolution typically limited to about 1,000 × 1,000 or lower. When using traditional optical devices, further performance improvement necessarily requires adding more components. The trade-off between optical performance and thickness poses an increasing challenge to the miniaturization of embedded sensors in mobile phones and other small electronic devices and the implementation of new architectures. The current multi-component optical architectures thus impede further control of 3-D sensor performance and cost. Summary of the Invention

[0003] Accordingly, the present invention is directed to imaging, sensing, and / or projection systems (e.g., 3-D sensing systems) and related methods that substantially eliminate one or more problems caused by the limitations and deficiencies of the related art.

[0004] Embodiments of the present invention provide novel imaging and sensing systems based on flat lens optics, which feature excellent optical performance, simple structure, and a compact footprint compared to traditional corresponding systems based on bulk refractive optics.

[0005] Compared to existing sensors, the imager, sensor, or projector architecture, design, and module configuration according to embodiments of the present invention utilize a novel optical metasurface architecture and packaging process to avoid complex multi-component assembly while enhancing optical performance. The imager, sensor, or projector design also requires a large FOV (>90°), high resolution, and a compact form factor. In addition, it features a simple architecture with a minimum number of components and is fully compatible with wafer-level assembly and packaging, thus offering significant advantages in scalable manufacturing and cost reduction.

[0006] Additional features and advantages of the present invention will be set forth in the following description, and in part will be obvious from the description, or may be learned by practice of the present invention. The objectives and other advantages of the present invention will be realized and attained by the structure particularly pointed out in the written description and claims, as well as the accompanying drawings.

[0007] To achieve the above object, the present invention provides a sensor device, which includes: a transparent substrate; an opaque material layer located above the first side of the substrate, the opaque material layer including at least one aperture; a chip stack, which is bonded to the second side of the substrate opposite to the first side, the chip stack including: a flat optical device layer; at least one spacer layer; and an image sensor, which is located at a defined distance from the flat optical device layer.

[0008] In another aspect, the present invention provides a projector device, which includes: a transparent substrate; an opaque material layer located above the first side of the substrate, the opaque material layer including at least one aperture; a chip stack, which is bonded to the second side of the substrate opposite to the first side, the chip stack including: a flat optical device layer; at least one spacer layer; and a light source or a light source array, which is located at a defined distance from the flat optical device layer.

[0009] In some embodiments of the sensor or projector device, the flat optical device layer includes a metasurface, a diffractive optical element, a hologram, or a gradient index (GRIN) optical element. In some embodiments, the flat optical device layer is formed by a metasurface structure configured to control the optical phase, amplitude, polarization, and / or spectrum, angle, and / or polarization-related transmission, reflection, and / or beam shaping profile of incident light.

[0010] In some embodiments of the sensor or projector device, the chip stack further includes a filter layer.

[0011] In some embodiments of the sensor or projector device, the device further includes a printed circuit board, wherein the image sensor or the light source or the light source array and other related electronic components are mounted on the printed circuit board.

[0012] In some embodiments of the sensor device, at least one aperture includes a first aperture and a second aperture, and the sensor further includes a second chip stack, which includes: a second flat optical device layer; at least one second spacer layer; and a second image sensor, which is located at a defined distance from the second flat optical device layer, wherein the chip stack and the second chip stack spatially correspond to the first aperture and the second aperture respectively, and the flat optical device layer and the second flat optical device layer are configured to perform different functions according to different attributes of light.

[0013] In some embodiments of the sensor device, at least one aperture includes a first aperture and a second aperture, and a single chip stack corresponds to the two apertures.

[0014] In another aspect, the present invention provides a sensor device including a transparent substrate; an opaque material layer located above a first side of the substrate, the opaque material layer including at least one aperture; a first chip stack and a second chip stack, the first chip stack and the second chip stack being bonded to a second side of the substrate opposite to the first side, the first chip stack including: a first flat optical device layer; at least one first spacer layer; and an image sensor located at a defined distance from the first flat optical device layer; the second chip stack including: a second flat optical device layer; at least one second spacer layer; and a light source or a light source array located at a defined distance from the second flat optical device layer. In some embodiments, the sensor device further includes a printed circuit board, wherein the image sensor and the light source or the light source array are mounted on the printed circuit board.

[0015] It should be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the claimed invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1A and Figure 1B Schematically illustrate two sensors / imagers for 3-D sensors or other imaging, sensing, or projection applications according to two embodiments of the present invention, each of which has a plurality of chip stacks corresponding to a plurality of apertures.

[0017] Figures 2A to 2D Schematically illustrate sensors / imagers for 3-D sensors or other imaging, sensing, or projection applications according to additional embodiments of the present invention, each of which has a single chip stack corresponding to a plurality of apertures.

[0018] Figure 3 Schematically illustrate a substrate having four apertures used in some embodiments of the present invention.

[0019] Figure 4A and Figure 4B Illustrate an exemplary structure and simulated performance of a micro metasurface imager according to an embodiment of the present invention.

[0020] Figures 5A to 5C Schematically illustrate a sensor device module integrated with a metasurface flat optical device according to an embodiment of the present invention.

[0021] Figure 6is illustrated in a perspective view similar to Figure 5B the exemplary sensor / imager module shown for 3-D sensor or other imaging, sensing, or projection applications.

[0022] Figure 7 Another exemplary camera module according to an embodiment of the present invention is illustrated.

[0023] Figure 8 Schematically illustrates the manufacturing and assembly process of a sensor module according to an embodiment of the present invention. Detailed Description

[0024] Embodiments of the present invention can be widely applied to imaging, sensing, and projection optical systems, as well as systems combining such functions.

[0025] Figure 1A and Figure 1B Schematically illustrates two sensors / imagers (which can be used for 3-D sensors or other imaging, sensing, or projection applications) according to two embodiments of the present invention. Each sensor includes a transparent substrate 11, which serves as a mechanical support for other components. The substrate 11 can also be used as a micro-optical platform, and in this case its surface is patterned to form mechanical alignment features to assist in the passive assembly of components.

[0026] In Figure 1AIn the illustrated embodiment, the front side (object side) of the substrate 11 is covered with an opaque material, which is patterned to form one, two, or more apertures 12. This can be achieved by defining the apertures in an opaque layer such as metal or black ink using, for example, lithography or printing methods, or by assembling a separate opaque layer, window, or light blocker containing the apertures to the substrate. On the back side, one, two, or more chip stacks 13 are assembled (e.g., bonded) to the substrate 11 and spatially correspond to the apertures 12. Each stack 13 includes a flat optical device layer 31, one or more spacers (which may be air gaps) 32 and 34 (two are shown in this example, but the second is optional), a filter 33, and an image sensor 35. The flat optical device layer 31 can be a metasurface, a diffractive optical element, a hologram, a gradient index (GRIN) optical element, etc. The following description uses a metasurface as an example. Herein, a metasurface is defined as including subwavelength structures (i.e., meta-atoms) fabricated or assembled on a substrate for imposing a spatially varying optical phase delay and / or amplitude or polarization modulation on an incident wavefront. The meta-atoms and the substrate can be made of the same or different optical materials. The meta-atoms are designed to change the phase, amplitude, and / or polarization of the incident light. The meta-atoms can have the same or different geometries, sizes, orientations, and / or spacings. Exemplary geometries can include rectangular, cylindrical, freeform, or any other suitable shape, or a combination of different shapes, etc. The spacing or lattice of the meta-atoms can have any suitable shape and period (e.g., square, rectangular, or hexagonal). The lattice can also be aperiodic, with a varying or random distance between adjacent meta-atoms. In some examples, the gaps between adjacent meta-atoms can be designed to have a constant gap distance. One or both sides of the substrate can be flat or curved. Both the metasurface and the substrate can be rigid, flexible, or stretchable. The substrate can also include spacers.

[0027] The geometries, sizes, and layouts of the meta-atoms and the substrate are designed to provide a target optical function. The metasurface can be designed to operate at a single wavelength, multiple wavelengths, or a continuous spectral range. The metasurface can be designed to provide different functions depending on the properties of the incident light (e.g., polarization, wavelength, angle of incidence / angle of emergence, intensity, etc.).

[0028] Each metasurface 31 combined with the corresponding aperture 12 forms a wide field-of-view (FOV) metalens capable of high-resolution imaging within a FOV of up to 180°. In its baseline form, light transmitted through the aperture 12 is focused or redirected by the metasurface 31 (with or without an additional optical filter) onto an image sensor 35 within the wide FOV. The metasurface lens (or metalens) can be designed to operate at infrared wavelengths (e.g., 850 nm or 940 nm) such that it is invisible to the human eye and can also operate at other wavelengths (e.g., within the visible spectrum). It can also be designed for broadband operation.

[0029] The filter 33 can be a spectral, angular, and / or polarization filter. The filter can be in the form of a multilayer filter, a cavity structure, a diffractive optical element, a tilted grating, or a metasurface that performs the above (multiple) filtering functions. An angular filter (e.g., certain cavity structures, diffractive optical elements, or metasurfaces that exhibit angular selectivity) can be used to block or reduce stray light or to form a self-limiting aperture, depending on the incident or exit angle of the light. A polarization filter is also useful when the metalens is designed to be polarization-sensitive. The metasurface can also be used as a filter.

[0030] One feature of this embodiment is that it allows angular-selective filtering of ambient background light to enhance the signal-to-noise ratio (SNR), which is not achievable when traditional multilayer filters are applied to wide-field imaging. This is achieved through the (near) telecentric configuration of the metalens, which means that light from different angles of incidence (AOI) on the object side exits the metalens only within its surface normal (or near normal, e.g., within 20 degrees of the normal) exit cone. In other words, at any AOI, the chief ray of the incident light exits the metasurface in the direction of the normal (or near normal, e.g., within 20 degrees of the normal) of the metasurface. Therefore, the tight distribution of the light angles on the image side allows the use of a single bandpass filter to effectively suppress ambient background light from all AOIs. At the same time, the meta-atoms located at different positions on the metasurface can be designed differently (e.g., according to AOI) to provide enhanced angular or spatially correlated responses.

[0031] The metasurface 31 and the filter 33 can be further assembled (e.g., bonded) to the image sensor 35 via an optical adhesive, which also serves as a spacer that controls the separation between different components. The sensor is the outermost layer of the chip stack; the spatial order of the metasurface 31, the filter 33, the spacers or air gaps 32 and 34 can be different from that Figure 1A shown, as long as the image sensor is located at a defined distance from the metasurface. For example, the spacer or air gap 32 can be located between the filter layer 33 and the image sensor 35 and not present between the metasurface and the image sensor.

[0032] Spacers or air gaps 32 and 34 for defining the distance between the respective layers can be made of glass, plastic, etc., or a stack of materials or an air gap. For example, in the structure of a wafer-level optical device, the spacer can be a layer of material with a desired thickness, having a hollow cutout in the region corresponding to the aperture position to achieve an air gap. Alternatively, the entire layer can be air, and an external mechanical structure can be used to hold the other components to form an air gap. In the present application, the term "spacer" includes an air gap, which can be formed by either of the above two structures or any other suitable structure.

[0033] In an alternative embodiment, the two metasurfaces 31 corresponding to the two apertures 12 can perform different functions according to different properties of light (e.g., AOI, polarization, wavelength, etc.), and the two filters 33 can accordingly have different filtering properties. Thus, different image sensors 35 can capture different information from the scene (e.g., different FOV, AOI, polarization, and / or spectral information, etc.), depending on the corresponding metasurface and filter structures.

[0034] In Figure 1B In the illustrated embodiment, the two chip stacks 13-1 and 13-2 are used as a light projector and a light receiver, respectively. In this case, in the second stack 13-2, the image sensor is replaced by a light source 36 (e.g., a light-emitting diode LED or a vertical-cavity surface-emitting laser VCSEL, an LED / VCSEL array, or a display array, etc.). In this stack, the filter 33 is optional. The second metasurface 31-2 in the second chip stack 13-2 is designed as a pattern projector or illuminator. The basic function of the second metasurface 31-2 is to convert the emission from the (multiple) light sources into a defined 2D or 3D light intensity distribution (e.g., an array of separated beams densely filling the angular space, a structured pattern, etc.). Then, the light beam exiting from the second aperture 12-2 in front of the second metasurface 31-2 forms a projected dot pattern. In one example, the solid immersion structure of the second metasurface 31-2 (i.e., without any air gap) enables the output light beam to span the entire front hemisphere when refracting at the top surface of the substrate 11. In addition to the dot array, the second metasurface 31-2 can also be easily designed to output other structures or diffused light patterns with a desired intensity distribution (e.g., an image, a dot, a line, a random pattern, or any other light intensity distribution pattern). In the imager module, the first metasurface 31-1 in the first chip stack 13-1 together with the integrated image sensor 35 then forms a light receiver for capturing the scene illuminated by the light source or ambient light. In the 3-D sensor module, the first metasurface 31-1 in the first chip stack 13-1 together with the integrated image sensor 35 then forms a light receiver for capturing the reflection of the dot pattern, from which depth information can be inferred using structured light or TOF techniques.

[0035] In other embodiments (e.g., Figures 2A to 2D ), multiple apertures can be integrated with a single chip stack and / or a single image sensor to enable stereoscopic imaging. A single flat (planar) optical device layer can carry one or more metasurfaces that are designed to modulate the properties (phase, amplitude, and / or polarization) of light from multiple corresponding apertures. For example, different regions of the metasurface can be designed for different apertures. In another example, the metasurface can be designed to have a multiplexing function such that light incident from different apertures is modulated differently.

[0036] Figure 2A FIG. schematically illustrates an example in which two metasurfaces 41-1 and 41-2 corresponding to two apertures 12-1 and 12-2 in a single chip stack 14 couple light to two different regions of a single image sensor 45. The image sensor 45 is integrated with a filter 43; both are large enough to spatially overlap with the regions of the two metasurfaces. The captured images can then be used to generate a 3-D image. One or two spacers 42 and 44 can also be provided in the chip stack 14. Different from existing 3-D sensing technologies that require multiple discrete assembled optical modules, the highly integrated all-planar architecture in this embodiment can achieve a significantly simplified assembly process, improved performance with minimal misalignment, a super-compact module configuration, and reduced costs.

[0037] Figure 2B FIG. schematically illustrates another example. In this embodiment, a single chip stack 15 includes multiple filters / filter regions 53-1 and 53-2 (referred to as a pixelated filter array) coupled to corresponding metasurfaces 51-1 and 51-2 or an image sensor 55, rather than a single filter. In some examples, the metasurfaces 51-1 and 51-2 can be configured according to filter properties such that different regions or pixels of the image sensor 55 can capture different information about the scene (e.g., different AOI, polarization, and / or spectral information, etc.). The pixelated filter arrays 53-1 and 53-2 can be directly integrated on the image sensor and aligned with the pixels. One or two spacers 52 and 54 can also be provided in the chip stack 15.

[0038] In Figure 2CIn another example of the schematic illustration, within a single chip stack 16, a single metasurface 61 can be designed to provide a multiplexing function, enabling it to perform different optical modulations according to optical characteristics (polarization, wavelength, AOI, etc.). Depending on the metasurface function, two or more filters or pixelated filter arrays 63-1 and 63-2 with different filtering characteristics can be coupled to the image sensor 65. Thus, different regions or pixels of the image sensor 65 can capture different information of the scene (e.g., different AOI, polarization, and / or spectral information, etc.), depending on the corresponding metasurface and filter structures. Similarly, one or two spacers 62 and 64 can also be provided in the chip stack 16.

[0039] In Figure 2D Another example of the schematic illustration, filters 73-1 and 73-2 can be integrated on the front side of the substrate 11, respectively located on or near the apertures 12-1 and 12-2 and covering these apertures. The metasurface can be used as each filter. A single chip stack 17 on the back side of the substrate 11 includes a metasurface 71, a spacer 72, and an image sensor 75.

[0040] In another embodiment, more than two apertures can be used to provide a full range of stereoscopic sensing in both the horizontal and vertical directions, as Figure 3 depicted in the top view of the lower part (showing four apertures), while using a single planar optical layer and a single image sensor according to the embodiments of Figure 2A , Figure 2B , Figure 2C or Figure 2D .

[0041] Figure 4A And Figure 4B illustrate the exemplary structure and simulated performance of a micro metasurface imager (camera) according to the embodiments of Figure 1A ( Figure 4A(not all layers of the chip stack are shown). This camera structure accommodates the ultra-compact commercial CMOS image sensor 35. The metasurface 31 includes an array of amorphous Si nanocolumns on a glass substrate. This type of metasurface structure has been previously described. In the example of FIG. 4, the meta-atoms located at different positions on the metasurface (depending on different AOIs) are designed differently to provide enhanced angular or spatially correlated responses. The metasurface substrate is assembled (e.g., bonded) to the image sensor and mounted on a custom ultra-small printed circuit board for image output. In this particular example, the diameter of the input aperture is 0.18 mm, the diameter of the metasurface is 0.5 mm, and the area of the image sensor is 0.36 square millimeters. The distance between the aperture and the metasurface is 0.21 mm, and the distance between the metasurface and the sensor surface is 0.16 mm. This metalens and the sensor module achieve an FOV of up to 180 degrees, with a combined diameter as small as 0.5 mm, a total thickness of less than 0.4 mm, and a resolution better than 100 line pairs / mm.

[0042] Figure 4A A ray tracing simulation of the imager is shown. Figure 4B is a graph of the simulated modulation transfer function (MTF) of the camera, indicating excellent resolution within a nearly 180-degree FOV. As Figure 4A shown, this metalens has a telecentric configuration, where for all angles of incidence within the FOV, the chief ray of the incident light exits the metasurface in the direction normal to the metasurface. As previously mentioned, the telecentric or near-telecentric configuration of the metalens enables an FOV of 180 degrees and good image quality even at large angles. In the present disclosure, near-telecentric means a configuration where for all angles of incidence, the chief ray angle (i.e., the angle at which the chief ray of the incident light exits the metasurface) is less than 20 degrees. The telecentric (including near-telecentric) configuration is achieved by selecting the metasurface design and the aperture size (i.e., f-number) relative to the substrate thickness. Numerical optimization techniques can be used to achieve such a configuration. In Figure 4A various specific examples of sensors with a general structure as shown, the f-number of the metalens ranges from 0.8 to 8.0.

[0043] An important advantage of the 3-D sensors according to the various embodiments of the present invention is that the single-chip stacked structure can be fabricated using wafer-level processes to achieve excellent alignment accuracy, high manufacturing throughput, and low manufacturing cost. Using an image sensor stack as an example, the process begins with a complementary metal oxide semiconductor (CMOS) image sensor wafer, followed by sequentially bonding a filter and a metasurface. The filter and the metasurface are preferably fabricated or assembled on a separate substrate (e.g., glass or semiconductor wafer or polymer substrate) with dimensions matching those of the CMOS sensor wafer. The filter can be made of a multi-layer thin film acting as an interference coating, or made of a patterned metasurface layer with customized spectral, angular, and / or polarization-dependent transmission characteristics. In another embodiment, the metasurface and the filter can be combined on one substrate. For example, by designing the metasurface structure to control the optical phase and / or spectral, angular, and / or polarization-dependent transmission profile, the metasurface itself can also perform the filtering function. Alternatively, the metasurface can also be patterned on top of a multi-layer interference coating filter. The metasurface can also be sandwiched between two multi-layer interference coating filters. In yet another embodiment, a multi-layer stacked metasurface structure can be employed to impose optical phase delay, amplitude modulation, polarization sensitivity, and / or spectral and / or angular filtering on the incident light. An optical adhesive or a transparent epoxy resin can be used as a bonding agent and simultaneously as a spacer layer with a controllable thickness.

[0044] Figure 8 Another alternative manufacturing and assembly process is illustrated. In this process, the metasurface is fabricated as a single piece and encapsulated (e.g., using epoxy resin or dielectric) (step S81); the aperture is fabricated as a single piece on a substrate (step S82); and the spacer and the filter are fabricated as a single piece (step S83). Each of the above includes a plurality of related structural units. The metasurface piece is assembled (e.g., bonded) to the aperture and the substrate piece (step S84); and then the spacer and the filter pieces are assembled (e.g., bonded) to the metasurface piece (step S85). Then, the assembled structure is cut into individual units, each unit containing one (or two) sets of aperture, metasurface, and filter forming a metalens (or other meta-optical device) (step S86). Then, the metalens is assembled (e.g., bonded) to the image sensor (step S87). Each of the bonding steps can use an optical adhesive, which can itself also be used as an additional spacer. The edges of the metalens and the spacer can be blackened or roughened. In an alternative method, the aperture and the metasurface can be combined on one substrate, i.e., the aperture and the metasurface are patterned on two sides of a single substrate. The aperture can be an optical aperture or a mechanical aperture (e.g., a frame for encapsulation, support, etc.).

[0045] A metasurface or a portion of the metasurface can be designed to be sensitive to the incident / exit angle or position of light. For example, different regions of a metalens are configured to have different modulation behaviors (e.g., transmittance, phase delay, polarization modulation, spectral response, etc.), depending on the incident or exit angle or position of light on the metasurface. In one example, the metasurface, one or more regions of the metasurface, or the surrounding region of a metalens can be designed to deflect, reflect, absorb, diffract, or block light and thus act as an aperture or light-blocking element, which is useful for restricting the propagation of light or reducing / blocking stray light. For example, a metasurface grating or other meta-optical device structures / components can be designed to deflect light or improve the optical efficiency of the metasurface for light within or outside certain incident angle ranges / exit angle ranges or diffraction orders. Such metasurface optical components can be optimized for different incident / exit angles or diffraction orders. The size, geometry, and / or spacing of meta-atoms are optimized to improve the diffraction efficiency of a target diffraction angle θ at a specific incident / exit angle α (or range of angles). The design starts with a full-wave simulation of the unit cell of the meta-atom, where the period and incident angle are determined by the requirements. In one example, an 8-meta-atom library is generated, where the phase delay covers the range of 0 - 2π. Subsequently, a meta-optical device is constructed from this library, and full-wave simulations (e.g., RCWA, FDTD, FEM, etc.) are used to perform simulations of the diffraction efficiency of the meta-optical device (e.g., in the form of a super-grating). In one example, the AOI-related design shows a diffraction efficiency three times higher than that of the meta-atoms designed for normal incidence and can thus be used for metasurface regions with off-axis incident light or for effectively deflecting stray light. This design method can be used not only for grating structures but also generally for other metasurfaces (e.g., metalenses).

[0046] Figures 5A to 5C Schematically illustrates a sensor device module integrating a metasurface flat optical device according to an embodiment of the present invention.

[0047] As Figure 5ASchematic illustration. An exemplary stereo 3-D sensor module includes two sub-modules, namely, a first camera 100-1 and a second camera 100-2. Each of the first camera and the second camera includes a wide FOV meta-lens (meta-surfaces 103-1 and 103-2 formed on a substrate 101 respectively, having corresponding apertures 102-1 and 102-2) and an imager sensor 106. The image sensor 106 is further integrated with a corresponding optical filter 105. In some embodiments, spacers 104 may be included between the filter 105 and the meta-surfaces 103-1 and 103-2. In the illustrated embodiment, the two meta-surfaces 103-1 and 103-2 are integrated into a single piece, while the two image sensors (with corresponding filters and optional spacers) are separate components. The image sensor 106, as well as the image processor 109 and other electronic components, are mounted on a printed circuit board (PCB) 108. In the illustrated embodiment, a front glass 107 is also provided to cover the front side of the substrate.

[0048] Figure 5B An exemplary imager / sensor with active illumination (e.g., structured light or TOF 3-D sensor module or other imager / sensors coupled to an illumination source) is illustrated. The module includes two sub-modules, namely, a projector 200-1, which includes a light source 211 (e.g., VCSEL or LED, or an array of lights), a driver chip 210 for the light source, and a meta-projection optics (a first meta-surface 203-1 formed on a substrate 201 having a corresponding aperture 202-1) to generate a high-resolution illumination pattern (such as a dot array); and an imager (camera) 200-2, which includes a wide FOV meta-lens (a second meta-surface 203-2 formed on the substrate 201 having a corresponding aperture 202-2) to capture an image of the pattern and map it onto the imager sensor 206. The filter 205 is integrated with the image sensor 206. Optional spacers 204 may be provided in the projector 200-1 and the camera 200-2. The projector meta-optics can be reverse-designed and co-optimized with the imager meta-optics to provide a large FOV of up to 180 degrees and high-resolution illumination and detection. The image sensor 206, its associated image processor 209, the light source 211 and its driver 210, and other suitable electronic components are mounted on a printed circuit board (PCB) 208. In the illustrated embodiment, an optional front glass 207 is also provided to cover the front side of the substrate.

[0049] Figure 5C Illustrated is associated with Figure 5BA similar imager / sensor module as shown, but the spacer 204 is omitted, and the filter 205 is formed on the second metasurface 203-2 instead of on the image sensor 206. Thus, there are air gaps between the light source (VCSEL) 211 and the first metasurface 203-1 and between the image sensor 206 and the filter 205. The air gaps are maintained by mechanical mounting and / or spacer structures ( Figure 5C not shown in the figure) that mount the substrate 201 (with the metasurface) on the PCB 208. Additionally, an optional backplane 212 is provided below the PDB 208.

[0050] In Figures 5A to 5C 's example, the front aperture can be further integrated with another optical component (e.g., a flat optical device or a filter) to provide additional modulation of the incident light. Refractive or reflective optical devices (e.g., lenses or mirrors) can also be used to form a hybrid optical system. Different from existing imagers / sensors (e.g., 3-D sensors) involving discrete packaging of optical modules and opto-mechanical modules, Figure 5A and Figure 5B 's metadevices (metalenses and metaprojection optical devices) in the embodiments are coplanarly integrated on a single common substrate, which simplifies the assembly and allows for further functional expansion.

[0051] Figure 6 A perspective view shows an exemplary imager / sensor module (e.g., a 3-D sensor module) similar to that shown in Figure 5B . Subfigure (a) is an exploded view, and subfigure (b) is a perspective view of the sensor module; subfigure (c) is a top view of the PCB.

[0052] Figure 7 Another exemplary camera module is shown. A metalens mounting and spacer structure 213 is shown, which is located between the substrate and the filter, surrounding the area of the metasurface and the filter. The metalens mounting and spacer structure 213 and the substrate are surrounded by sidewalls 214. Figure 7 The various dimensions shown in

[0053] Figure 5A , Figure 5B and Figure 6 's 3-D sensor module overcome the traditional trade-off between performance, complexity, and size and uniquely combine high resolution, panoramic FOV, enhanced SNR, and a thin and lightweight device structure.

[0054] Figure 5A and Figure 5BThe structure of the sensor device module shown can be used to implement other devices, such as near-IR or short-wave IR cameras that can be similar to cameras 100-1, 100-2, or 200-2 but without a processor, or illuminators that are similar to projector 200-1 but without a driver.

[0055] In the above embodiments, the metasurface can more generally be a flat optical device layer, and in addition to the metasurface, the flat optical device layer can also be a diffractive optical element, a hologram, a gradient refractive index (GRIN) optical element, etc. The front aperture can be further integrated with another optical component (e.g., a flat optical device or a filter) to provide additional modulation of light. Refractive or reflective optical devices (e.g., lenses or mirrors) can also be used to form a hybrid optical system.

[0056] Various other alternative embodiments are possible. For example, an aperture or a light stop can be patterned, integrated, or assembled on one or more optical component layers in the above various embodiments, and its function is to limit light propagation or reduce or block stray light. The light stop can be, for example, an absorptive and / or reflective coating such as a black or metal coating, or a deflecting optical structure.

[0057] In another alternative embodiment, the space between the front aperture 12 and the metasurface 31 can be an air gap.

[0058] The above wide FOV sensor structure can be designed and optimized for different types of applications such as driver monitoring systems (DMS), eye trackers, AR / VR (augmented reality / virtual reality) devices, etc.

[0059] It will be apparent to those skilled in the art that various modifications and variations can be made to the wide field of view metasurface 3-d sensor and camera and related methods of the present invention without departing from the spirit or scope of the present invention. Accordingly, it is intended that the present invention cover the modifications and variations that fall within the scope of the appended claims and their equivalents.

Claims

1. A sensor device, comprising: A transparent substrate; An opaque material layer located above a first side of the substrate, the opaque material layer comprising at least one aperture; A chip stack bonded to a second side of the substrate opposite the first side, the chip stack comprising: A flat optical device layer; At least one spacer layer; and An image sensor located at a defined distance from the flat optical device layer.

2. The sensor device according to claim 1, wherein, The flat optical device layer comprises a metasurface, a diffractive optical element, a hologram, or a gradient index (GRIN) optical element.

3. The sensor device according to claim 1, wherein, The flat optical device layer comprises a metasurface that spatially corresponds to one of the at least one aperture to form a near-aplanatic superlens.

4. The sensor device according to claim 1, wherein, The flat optical device layer is formed of a metasurface structure configured to control the optical phase, amplitude, polarization, and / or spectrum, angle, and / or polarization-dependent transmission, reflection, and / or beam shaping profile of incident light.

5. The sensor device according to claim 1, wherein, The at least one aperture comprises a first aperture and a second aperture, The sensor further comprises a second chip stack, the second chip stack comprising: A second flat optical device layer; At least one second spacer layer; and A second image sensor located at a defined distance from the second flat optical device layer, Wherein the chip stack and the second chip stack spatially correspond to the first aperture and the second aperture respectively, and wherein the flat optical device layer and the second flat optical device layer are configured to perform different functions according to different properties of light.

6. The sensor device according to claim 5, wherein, The chip stack further comprises a first filter layer, and the second chip stack further comprises a second filter layer, wherein the first filter layer and the second filter layer are configured to have different filtering properties according to the different functions of the flat optical device layer and the second flat optical device layer respectively.

7. The sensor device according to claim 5, further comprising: A printed circuit board, wherein the image sensor of the chip stack and the second image sensor of the second chip stack are mounted on the printed circuit board; and An image processor mounted on the printed circuit board.

8. The sensor device according to claim 1, wherein, The at least one aperture comprises a first aperture and a second aperture, The sensor further comprises a second chip stack, the second chip stack comprising: A second flat optical device layer; At least one second spacer layer; and A light source located at a defined distance from the second flat optical device layer, Wherein the chip stack and the second chip stack spatially correspond to the first aperture and the second aperture respectively, and Wherein the second flat optical device layer is configured to convert light emitted by the light source into a defined two-dimensional or three-dimensional light intensity distribution.

9. The sensor device according to claim 8, wherein, The second chip stack further comprises a second filter layer.

10. The sensor device according to claim 8, further comprising: A printed circuit board, wherein the image sensor of the chip stack and the light source of the second chip stack are mounted on the printed circuit board; A light source driver chip, the light source driver chip being mounted on the printed circuit board; and An image processor, the image processor being mounted on the printed circuit board.

11. The sensor device according to claim 1, wherein, The chip stack further includes a filter layer.

12. The sensor device according to claim 11, wherein, The at least one aperture includes a first aperture and a second aperture, wherein the flat optical device layer includes a first metasurface and a second metasurface, the first metasurface and the second metasurface spatially correspond to the first aperture and the second aperture respectively, and wherein both the filter layer and the image sensor spatially overlap with regions of the first metasurface and the second metasurface.

13. The sensor device according to claim 11, wherein, The at least one aperture includes a first aperture and a second aperture, wherein the flat optical device layer includes a first metasurface and a second metasurface, the first metasurface and the second metasurface spatially correspond to the first aperture and the second aperture respectively, wherein the filter layer includes a first filter and a second filter, the first filter and the second filter spatially correspond to the first aperture and the second aperture respectively, and wherein the image sensor spatially overlaps with regions of the first metasurface and the second metasurface and the first filter and the second filter.

14. The sensor device according to claim 11, wherein, The at least one aperture includes a first aperture and a second aperture, wherein the filter layer includes a first filter and a second filter, the first filter and the second filter spatially correspond to the first aperture and the second aperture respectively, wherein both the flat optical device layer and the image sensor spatially overlap with regions of the first filter and the second filter.

15. The sensor device according to claim 11, wherein, The flat optical device layer is located between the image sensor and the substrate, the at least one spacer layer includes a first spacer layer located between the flat optical device layer and the image sensor, and the filter layer is located between the first spacer layer and the image sensor.

16. The sensor device according to claim 15, wherein, The at least one spacer layer further includes a second spacer layer located between the filter layer and the image sensor.

17. The sensor device according to claim 11, wherein, The flat optical device layer is located between the image sensor and the substrate, the filter layer is located between the flat optical device layer and the image sensor, and the at least one spacer layer includes a first spacer layer located between the filter layer and the image sensor.

18. The sensor device according to claim 11, wherein, The image sensor is a complementary metal oxide semiconductor (CMOS) image sensor wafer, wherein the filter layer is a multi-layer thin film or a patterned metasurface layer formed on a substrate and assembled to the image sensor wafer, and wherein the flat optical device layer is a metasurface formed on a substrate and assembled to the filter layer.

19. The sensor device according to claim 11, wherein, The flat optical device layer and the filter layer are formed of a metasurface structure configured to control the optical phase, amplitude, polarization, and / or spectral, angular, and / or polarization-dependent transmission, reflection, and / or beam shaping profiles of incident light.

20. The sensor device according to claim 11, wherein, The filter layer includes one or more multilayer interference coating filters, and the flat optical metasurface layer is a metasurface patterned on top of the one or more multilayer interference coating filters or a metasurface sandwiched between two of the one or more multilayer interference coating filters.

21. The sensor device according to claim 11, wherein, The flat optical device layer and the filter layer are formed of a multilayer stacked metasurface or optical structure configured to impart optical phase, amplitude modulation, polarization sensitivity, and / or spectral and / or angular-dependent filtering and / or beam shaping to incident light.

22. The sensor device according to claim 1, further comprising a filter covering each of the at least one aperture.

23. The sensor device according to claim 1, wherein, The at least one spacer layer includes a first spacer layer that is an air gap.

24. A projector device, comprising: A transparent substrate; An opaque material layer located above a first side of the substrate, the opaque material layer including at least one aperture; A chip stack bonded to a second side of the substrate opposite the first side, the chip stack including: A flat optical device layer; At least one spacer layer; and A light source or light source array located at a defined distance from the flat optical device layer.

Images