Spectral routers for snapshot multispectral imaging

Abstract

The present disclosure is directed toward systems and methods for performing spectral imaging. Embodiments include a spectral imager having spectral router that is formed on a light-detection layer that includes a plurality of photodetectors. The spectral router has a structure that includes a first plurality of scatterers that is arranged within the spectral router in a first arrangement. The spectral router also includes a first material having a first dielectric constant. Each scatterer includes at least a second material having a second dielectric constant that is different than the first dielectric constant.

Description

SPECTRAL ROUTERS FOR SNAPSHOT MULTISPECTRAL IMAGINGGovernment Funding[oooi] This invention was made with Government support under contract FA9550-21-1 -0312 awarded by the Air Force Office of Scientific Research. The Government has certain rights in the invention.Cross Reference to Related Applications[ooo2] This case claims priority of U.S. Provisional Patent Application Serial Number 63 / 438,850, filed January 13, 2023 (Attorney Docket: 146- 091 PR1 ), which is incorporated herein by reference. If there are any contradictions or inconsistencies in language between this application and one or more of the cases that have been incorporated by reference that might affect the interpretation of the claims in this case, the claims in this case should be interpreted to be consistent with the language in this case.Technical Field

[0003] The present disclosure relates to image sensing in general, and, more particularly, to spectral image sensing applications.Background

[0004] The ability to separate light into spectral components without loss of photons is of great importance in many optical applications, including imaging spectroscopy and multispectral imaging. Snapshot spectral imaging, in particular, which aims to collect spatial and spectral information simultaneously, has many applications, including microcopy, precision agriculture, food inspection, machine vision, bio / medical imaging, forensics, and counterfeit detection.[ooos] While related to color imaging, spectral imaging differs in goal and applications. In color imaging, the goal is to create imaging pleasing photographs or videos for human consumption, which is done with a few, i.e. , typically three (3), color channels, e.g., Red-Green-Blue (RGB), that cover the visible wavelength range. By contrast, in spectral imaging the goal is to collect spectral data, in addition to spatial data, to locate and identify objects, items or features in an imaged scene, which can be done with various computational methods, e.g., (non)linear statistical methods, machine learning, deep learning, convolutional neural networks, and the like. To collect spectral and spatial information, a spectral imaging system comprises M spectral channels, where M is larger than four (4), that cover the wavelength range of interest, which can be the visible range, but other ranges, such as the ultraviolet, infrared (e.g., near-infrared, short-wave infrared, mid-wave infrared, long-wave infrared, very long-wave infrared), terahertz ranges and beyond, are equally of interest. The spectral decomposition should ideally be performed efficiently and without sacrificing spatial resolution. That is, however, not the case with current technology.

[0006] Multispectral filter arrays are currently the enabling technology for compact snapshot spectral imaging systems. While they can be integrated on- chip with the pixel arrays of solid-state image sensors to form the most compact spectral imaging systems, they suffer from two fundamental limitations: low photon efficiency and low spatial resolution. Regarding photon efficiency, the spectral channel of interest is selected as with any filtering approach, for example, by absorbing unwanted light using dye-based absorbing filters or by reflection using interference filters (e.g., Fabry-Perot resonators). When using filters, the spectral channel of interest is filtered by rejecting any light with out-of- channel spectral content. Filters are hence intrinsically photon inefficient. An M- channel spectral imaging system using multispectral filters therefore has a theoretical maximum efficiency of 1 / M for each spectral channel.

[0007] Regarding spatial resolution, a multispectral filter array consists of a periodic arrangement of repeat units that contain the filter set for the different spectral channels. Current filter technology, however, limits the scaling of filters and the smallest filters in practice are about a = / um in size. The period for the repeat units in a two-dimensional array therefore scales as which ismuch larger than the diffraction limit. Hence, the spatial resolution achieved in single-chip multispectral imagers is currently significantly lower than the resolution allowed by diffraction in the optical system. This issue becomes increasingly severe with larger number of spectral channels M .

[0008] The need for a spectral decomposition capability for snapshot spectral imaging that enables both the collection of spectral information with high photon efficiency and the collection of spatial information with high spatial resolution, using state-of-the-art pixels, remains unmet.Summary

[0009] In one aspect, embodiments in accordance with the present disclosure are spectral routers that address the aforementioned problems by addressing the fundamental limitations of multispectral filter arrays, such as low photon efficiency and low spatial resolution. Such spectral routers therefore enable single-chip snapshot spectral imaging sensors and systems that are highly photon efficient to provide spectral information without sacrificing spatial information.[ooio] Spectral routers in accordance with the present disclosure address the fundamental limitations of multispectral filter arrays, which are the key enabling technology for compact single-chip multispectral imaging systems. A spectral router is a device that routes all light incident on its entire surface directly and without loss to the photodetector of the proper channel based on its spectral content. Since a spectral router, unlike spectral filters, does not reject light by absorption and / or reflection to achieve spectral selectivity, it can ideally exploit100% of the incident light. Spectral routers can also be designed with a wavelength size footprint, which allows routers to break the size barrier that exists for filter array repeat units due to scaling limitations on the size of filters. This enables spectral routers to simultaneously allow for much higher spatial resolution as well, and provide spectral information without sacrificing the spatial information in an imaged scene.[ooii] While spectral routers benefit all current spectral imaging applications, this disclosure enables spectral routers that are extremely compact and photon efficient, which can further increase use cases by enabling photonefficient, high-spatial resolution multispectral imaging systems on highly portable platforms (e.g., smart phones, tablets).[ooi2] Spectral routers in accordance with the present disclosure are particularly well suited for use in applications such as spectral imaging and imaging spectroscopy applications: (bio)medical imaging, microscopy, precision agriculture, food inspection, machine vision, forensics and counterfeit detection, art, and the like. Furthermore, they enable single-chip spectral imaging systems based on solid state image sensors, CMOS image sensors, CCD image sensors that can be used in smartphone cameras, security cameras, automotive cameras, and the like.

[0013] Embodiments in accordance with the present disclosure afford significant advantages over systems known in the prior art.

[0014] First, a spectral router does not reject any light, through absorption or reflection, when separating light into its spectral components, and can ideally exploit all the incident light.

[0015] Second, since spatial resolution in the system is already limited by diffraction in the imaging lens (FIG. 1 (a)), the entire spectral router is smaller than the diffraction limit of the imaging lens to prevent loss of spatial resolution by the router. Specifically, unlike spectral filters, spectral routers can be designedwith wavelength size footprint. This enables spectral routers to simultaneously allow for much higher spatial resolution and provide spectral information without sacrificing the spatial information in an imaged scene.

[0016] This enables spectral routers to simultaneously allow for much higher spatial resolution and provide spectral information without sacrificing the spatial information in an imaged scene.

[0017] Third, while spectral routers break the scaling barrier that exists for multispectral filter arrays and can be designed for the smallest pixels of image sensors today as well as for future smaller pixels, it is of course possible to design larger spectral routers matched to larger pixels as well and take advantage of the photon efficiency of the spectral router.[ooi8] Fourth, spectral routers can operate in different parts of the optical spectrum, e.g., in the ultraviolet, infrared (e.g., near-infrared, short-wave infrared, mid-wave infrared, long-wave infrared, very long-wave infrared), terahertz ranges and beyond. In the visible and the short-wave infrared ranges, for example, spectral routers with 6 and 9 spectral channels have been demonstrated, since this corresponds to the number of spectral channels existing snapshot spectral imaging systems based on filter arrays. It should be noted, however, that a spectral router in accordance with the present disclosure is not limited to 6 or 9 channels and can have any practical number of channels. For example, spectral routers can be designed with fewer or many more channels, if an application requires it.

[0019] In another aspect, a method of forming a spectral imager for use in one or more spectral imaging applications is presented. The method includes: selecting a spectral operating range of interest and a number of spectral channels to be imaged by the spectral imager within the spectral operating range such that a plurality of different spectral signatures within the spectral operating range are able to be identified by the spectral imager, the different spectral signatures including spectral signatures that are able to distinguish between oneor more items or features in a scene being imaged; selecting, for each of the selected spectral channels, a plurality of specified spectral characteristics, the specified spectral characteristics including a center wavelength of the respective channel, a bandwidth or full-width at half maximum (FWHM) of the respective channel, an optical efficiency and optical crosstalk of the respective optical channel, and a spectral response shape of the respective channel to thereby define selected spectral channels having the plurality of specified characteristics; selecting a spatial layout, representing the spatial location and the size, of spatial areas located at an output plane, each of the spatial areas receiving one of the selected spectral channels with the respective plurality of specified spectral characteristics; providing at the output plane a light-detection layer comprising a plurality of photodetectors that are arranged in accordance with the spatial layout that is selected; forming a spectral router on the light-detection layer, the spectral router having a structure comprising a first plurality of scatterers that is arranged within the spectral router in a first arrangement, and wherein the spectral router comprises a first material having a first dielectric constant, and wherein each scatterer of the first plurality thereof comprises at least a second material having a second dielectric constant that is different than the first dielectric constant; and defining the first arrangement such that the spectral router directly routes each of the selected spectral channels with their respective plurality of specified characteristics to the plurality of photodetectors in the light-detection layer such that each photodetector of the plurality thereof selectively receives one of the spectral channels.

[0020] In yet another aspect, a method of performing spectral imaging is provided. The method includes: selecting one or more items or features to be identified by a spectral imager; selecting a spectral signature of the one or more items or features to be identified by the spectral imager, the spectral signature being able to identify the one or more items or features or distinguish between the one or more items or features in a scene being imaged; selecting one ormore spectral channels of the spectral imager that are collectively able to identify the spectral signature; imaging the scene using the spectral imager to obtain an imaged scene that is represented by a datacube having a set with images for each spectral channel; identifying a location of the spectral signature in the imaged scene; and wherein the spectral imager includes a spectral router having a structure comprising a first plurality of scatterers that is arranged within the spectral router in a first arrangement, and wherein the spectral router comprises a first material having a first dielectric constant, and wherein each scatterer of the first plurality thereof comprises at least a second material having a second dielectric constant that is different than the first dielectric constant, the spectral imager further including a light-detection layer comprising a plurality of photodetectors that are arranged in accordance with a specified spatial layout, the first arrangement of the first plurality of scatters being arranged such that the spectral router directly routes each of the selected spectral channels to the plurality of photodetectors in the light-detection layer such that each photodetector of the plurality thereof selectively receives one of the spectral channels.Brief Description of the Drawings

[0021] FIG. 1 (a) depicts a schematic drawing of a cross-sectional view of a snapshot multispectral imaging system; FIG. 1 (b) shows the unit cell of a snapshot multispectral imaging system that employs a photon-inefficient multispectral filter array; and FIG. 1 (c) shows the unit cell of a snapshot multispectral imaging system that employs a spectral router of the type described herein.

[0022] FIG. 2 depicts a schematic drawing of a cross-sectional view of an individual pixel-repeat unit of an illustrative embodiment of an image sensor in accordance with the present disclosure.

[0023] FIG. 3A depicts operations of an exemplary method suitable for forming a color router in accordance with the present disclosure.

[0024] FIG. 3B depicts operations of an exemplary alternative method suitable for forming a color router having multiple scatterer types, each having a different material composition, in accordance with the present disclosure.

[0025] FIG. 4 depicts sub-operations of a non-limiting example of a submethod suitable for designing a color router in accordance with the present disclosure.

[0026] FIG. 5A shows a cross-sectional view of nascent pixel-repeat unit 100’ after formation of void pattern 404-1 .

[0027] FIG. 5B shows a cross-sectional view of nascent pixel-repeat unit 100’ after formation of scatterer pattern 410-1 in media layer 402-1 .

[0028] FIG. 5C shows a cross-sectional view of pixel-repeat unit 100 after completion of sub-layer 412-N and, therefore, color router 104.

[0029] FIG. 5D shows a cross-sectional view of nascent pixel-repeat unit 100’ after formation of scatterer patterns 410-1-1 through 410-1-2 in media layer 402-1 .

[0030] FIG. 6A depicts a plot of the dielectric constant throughout the structure of pixel-repeat unit 100.

[0031] FIG. 6B depicts field intensity plots showing the passage of each of the wavelength signals of light signal 106 through color router 104.

[0032] FIG. 7 depicts spectral characteristics of the target design and corresponding optimized design response for wavelength signals 106B, 106G,106R, and 106NIR routed by fully binarized color router in accordance with the present disclosure.

[0033] FIG. 8 depicts results demonstrating the effect of design-element size on the performance of a color router in accordance with the present disclosure.

[0034] FIG. 9 depicts a schematic drawing of a cross-sectional view of an individual pixel-repeat unit of an alternative embodiment of an image sensor in accordance with the present disclosure.

[0035] FIG. 10 depicts spectral characteristics of a target design and corresponding optimized design response for wavelength signals 106B, 106G, and 106R as directly routed by color router 804.

[0036] FIG. 11 depicts spectral characteristics of target designs and corresponding optimized design responses for two color routers designed for different incidence angles in accordance with the present disclosure.

[0037] FIG. 12 depicts a schematic drawing of a cross-sectional view of an individual pixel-repeat unit of another alternative embodiment of an image sensor in accordance with the present disclosure.

[0038] FIG. 13 depicts field intensity plots for the routing of the photons of the wavelength signals in light 106 by color router 1104.

[0039] FIG. 14 depicts perspective views of field intensity plots showing the passage of each of the wavelength signals of light signal 106 through color router 1104.

[0040] FIGS. 15A-15B depict spectral characteristics of an optimized broadband response for unpolarized and polarized wavelength signals as directly routed in pixel-repeat unit.

[0041] FIG. 16 depicts a schematic drawing of a cross-sectional view of an individual pixel-repeat unit of another alternative embodiment of an image sensor in accordance with the present disclosure.

[0042] FIG. 17 shows a plot of the measured spectral response of surface- optimized color router in accordance with the present disclosure.

[0043] FIG. 18(a) shows a vertical cross-section through the middle of the spectral router structure; and FIG. 18(b) shows the vertical cross-sections taken along the dashed horizontal lines shown in the subpanel below each crosssection.

[0044] FIG. 19(a) shows a schematic diagram of a vertical cross-section of an optimized spectral router structure integrated on a visible image sensor with 330 nm wide photodetecting pixels and having a sub-micrometer device size of 990 nm; and FIG. 19(b) shows the detailed layouts of the design elements for the individual layers of the structure.

[0045] FIG. 20 shows the spectral response across the visible wavelength range for an optimized 6-channel spectral router design.

[0046] FIG. 21 (a) (left) shows a schematic perspective view of the spectral router structure integrated on a SWIR imager with 990 nm wide photodetectors and FIG. 21 (a) (right) shows the layout of the photodetectors of the 9 spectral channels; and FIG. 21 (b) shows the horizontal cross-section of the optical power flow at the output plane of the spectral router.

[0047] FIG. 22 shows the spectral response across the SWIR wavelength range for an optimized 9-channel spectral router design.Detailed Description

[0048] The following merely illustrates the principles of the disclosure. It will thus be appreciated that those skilled in the art will be able to devise variousarrangements which, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its spirit and scope.

[0049] Furthermore, all examples and conditional language recited herein are principally intended expressly to be only for pedagogical purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor(s) to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions.

[0050] Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

[0051] Unless otherwise explicitly specified herein, the figures comprising the drawing are not drawn to scale.Definitions

[0052] The following terms are defined for use in the present Specification, including the appended claims:• “Disposed on” or “Formed on” is defined as “exists on” an underlying material or layer. This layer may comprise intermediate layers, such as transitional layers, necessary to ensure a suitable surface. For example, if a material is described to be “disposed (or grown) on a substrate,” this can mean that either (1 ) the material is in intimate contact with the substrate; or (2) the material is in contact with one or more transitional layers that reside on the substrate.“Lossless material” is defined as a material that is substantially non- absorptive for the wavelength components included in a light signal.• “Wavelength component” is defined as a light signal that is centered at a particular wavelength.• “Wavelength signal” is defined as an optical signal that includes at least one wavelength component. A wavelength signal in accordance with the present disclosure can include only one wavelength component, or multiple wavelength components. It should be noted that multiple wavelength components included in a wavelength signal can be separated by uniform or nonuniform wavelength spacing without departing from the scope of the present disclosure.• “Directly routed” is defined as routing the photons of an optical signal through a color router such that they remain within the structure of the color router and emerge from the color router at or within one wavelength of their intended spatial location.• “Optical efficiency” is defined as the fraction (or percentage) of the optical power incident on the entire entry surface of the color router or pixel-repeat unit that is routed to the photodetector of the intended color channel.

[0053] FIG. 1 (a) depicts a schematic drawing of a cross-sectional view of a snapshot multispectral imaging system, which includes an imaging lens and an on-chip spectrally selective layer integrated with a light-detection layer comprising a photodetector array. The rectangle indicated by the dashed line in FIG. 1 (a) delineates the unit cell or pixel-repeat unit of the spectrally selective layer. FIG. 1 (b) shows the unit cell of a snapshot multispectral imaging system that employs a photon-inefficient multispectral filter array, which has filters larger than the diffraction limit imposed by the imaging lens. In contrast, FIG. 1 (c) shows the unit cell of a snapshot multispectral imaging system that employs aspectral router of the type described herein, which is based on lossless dielectric materials (illustrated by the dark rectangle shown in the spectral router of FIG. 1 (c)), and which can be made smaller than the diffraction limit imposed by the imaging lens. As shown, the spatial router may directly contact the photodetector layer without any intervening layers in between. The arrows in FIGs. 1 (b) and1 (c) represent the diffraction limit of the imaging lens shown in FIG. 1 (a).

[0054] FIG. 1 (c) illustrates that light within a certain spectral range that is incident over the entire spectral router surface is routed to a specific photodetector placed directly beneath the spectral router. The spectral router does not reject any light through absorption or reflection, when separating the light into its spectral components, and can ideally exploit all the incident light. Since spatial resolution in the imaging system is already limited by diffraction in the imaging lens, the challenge is to make the entire spectral router smaller than the diffraction limit of the imaging lens to prevent loss of spatial resolution by the spectral router. Unlike spectral filters, spectral routers can be designed with a wavelength size footprint. The unit cell, having a size of , where a ' is thesize of a photodetector, can therefore be far smaller as compared with the size of the spectral filter N As a result, the spectral router can provide a much higher spatial resolution.Color Routers

[0055] Co-pending U.S. Appl. Serial No [Docket No. 146-085WO1], which is herein incorporated by reference in its entirety, describes a color router that may be used to perform color imaging. As described therein, the color router is able to selectively directly route virtually all photons of each given wavelength signal in a received light signal to only the photodetector of a plurality of photodetectors meant to receive that wavelength signal. This allows the use of smaller photodetectors because they collect more light than larger photodetectors coupled with color filters.

[0056] The various techniques described in the aforementioned patent application for the design and fabrication of color routers for color imaging have certain applicability to the design and fabrication of the spectral routers described herein, which may be used to perform spectral imaging. Accordingly, the design and fabrication of color routers will be described below, followed by a discussion of the design and fabrication of spectral routers.

[0057] FIG. 2 depicts a schematic drawing of a cross-sectional view of an individual pixel-repeat unit of an illustrative embodiment of an image sensor in accordance with the present disclosure. Pixel-repeat unit 100 includes lightdetection layer 102 and color router 104, which is disposed directly on the lightdetection layer. In some embodiments, a spacer layer is included between color router 104 and light detection layer 102. Pixel-repeat unit 100 is a spectrally selective detection system configured to individually detect the wavelength signals included in light signal 106. In the depicted example, light signal 106 contains wavelength signals 106R, 106G, 106B, and 106NIR, and pixel-repeat unit 100 provides output signals 108-1 through 108-4, which are based on the intensity of wavelength signals 106R, 106G, 106B, and 106NIR, respectively.

[0058] In the depicted example, each of wavelength signals 106R, 106G, 106B, and 106NIR includes only one individual wavelength component. Specifically, wavelength signal 106B includes only blue light ( / .e., a narrow spectral band centered at 450 nm), wavelength signal 106G includes only green light ( / .e., a narrow spectral band centered at 550 nm), wavelength signal 106R includes only red light ( / .e., a narrow spectral band centered at 650 nm), and wavelength signal 106NIR includes only near-infrared light ( / .e., a narrow spectral band centered at 750 nm. In some embodiments, at least one wavelength signal routed by a color router includes more than one wavelength component.

[0059] Light-detection layer 102 (hereinafter referred to as “LD layer 102”) includes photodetectors 110B, 110G, 110R, and 110NIR, which are arranged in a linear array having uniform spacing or pitch P1 and size W1.

[0060] Photodetectors 11 OB, 110G, 11 OR, and 110NIR (referred to, collectively, as photodetectors 110) are conventional photodetectors suitable for detecting light having any wavelength component within the spectral range of light signal 106.[oo6i] In the depicted example, each of photodetectors 110 has a size W1 of approximately 280 nm and pitch P1 is approximately 400 nm. It should be noted that this photodetector pitch is approximately half the photodetector pitch of state-of-the-art image sensors that employ absorption color filters, which have a photodetector pitch of approximately 800 nm. In some embodiments, the spacing between photodetectors 110B, 110G, 11 OR, and 110NIR is non-uniform. In some embodiments, pitch P1 is less than or equal to the wavelength of the longest wavelength component included in light signal 106. In some embodiments, the size of photodetectors 110B, 11 OG, 110R, and 110NIR is non- uniform. As will be apparent to one skilled in the art, after reading this Specification, the size, W1 , of photodetectors 110 and the spacing between them ( / .e., pitch P1 ) is a matter of design and any practical size and / or pitch can be used without departing from the scope of the present disclosure.

[0062] It should be noted that, although the depicted example operates over only a spectrum that visible and near-infrared light, embodiments in accordance with the present disclosure can be configured for operation at wavelengths within virtually any electromagnetic spectral range, such as infrared, ultraviolet, multiple spectral ranges, and the like.

[0063] Color router 104 has a structure that includes a layer of background medium having thickness t1 , throughout which a three-dimensional arrangement of nanoscale scatterers is present. In the depicted example, thickness t1 is approximately 2 microns; however, color router 104 can have any practical thickness without departing from the scope of the present disclosure. Color router 104 is described in more detail below and with respect to FIGS. 3 through 6A-B. As discussed below, the material of the medium of color router 104 has a firstdielectric constant, while the material of the scatterers comprises a different material that has a second dielectric constant that is different (higher or lower) than the first dielectric constant. As will be apparent to one skilled in the art, the dielectric constant and refractive index of a material are related, where the dielectric constant sris the refractive index squared, 8r=n2

[0064] Color routers in accordance with the present disclosure afford significant advantages over prior-art optical stack elements (which can include, e.g., lenses, absorptive color filters, cavity spectral filters, anti-reflection coatings, etc.) and prior-art color-separation elements (which can include, e.g., plasmonic color filters, plasmonic photon sorters, diffractive optical filters, color splitters, etc.) , including: i. little or no optical crosstalk, enabling each photodetector to selectively receive virtually all of the optical energy of its respective wavelength signal; or ii. mitigation of reflections between the color router and the photodetectors and / or from the top surface of the color router by forming the color router directly on the photodetector surfaces; or ill. smaller photodetector sizes are possible due to the fact that the optical efficiency of the color router is significantly higher than prior-art elements; or iv. the use of photodetectors that are sub-wavelength in size because diffraction effects are mitigated by the fact that light does not need to propagate outside of a color router enroute to its corresponding photodetector; or v. significantly smaller size ( / .e. , color routers can be wavelength scale or smaller in height and / or width); orvi. they can have a wider spectral range and / or a variety of different spectral configurations, including the visible spectrum, ultraviolet, infrared spectrum, and the like; or vii. compatibility with conventional CMOS manufacture or other advanced nanofabrication methods; or viii. substantially perfect spectral color photon efficiency, substantially perfect broadband photon efficiency, substantially perfect spectral shape matching, and angular robustness are possible; or ix. any combination of i, ii, iii, iv, v, vi, vii, and viii.

[0065] Furthermore, it is an aspect of the present disclosure that the teachings herein enable a color router that directly routes the photons of a wavelength signal from the entry surface of the color router to their intended photodetector located at or within a wavelength of the exit surface of the color router. It should be noted that the direct-routing capability of color routers in accordance with the present disclosure is in direct contrast to meta-optics-based light-scattering structures, which rely on propagation of light over several wavelengths beyond the structures to focus and sort light spatially at the pixel photodetectors, such as those disclosed in U.S. Patent Publication No. 2020 / 0124866 or by P. Camayd-Munoz, et al., in “Multifunctional volumetric meta-optics for color and polarization image sensors,” in Optica, Vol. 7, pp. 280- 282 (2020), each of which is incorporated herein by reference.

[0066] FIG. 3A depicts operations of an exemplary method suitable for forming a color router in accordance with the present disclosure. Method 200 is described with continuing reference to FIG. 2, as well as reference to FIGS. 5A- C, which depict cross-sectional views of pixel-repeat unit 100 at different stages of fabrication.

[0067] Method 200 begins with operation 201 , wherein a three- dimensional design for color router 104 is generated. In the depicted example, the design of color router 104 is realized via a computational approach that aims for systematic optimization of an objective function. This objective function can be related to the photon efficiency with which the light incident on the color router is separated and redirected to different photodetectors depending on its spectral content (color).

[0068] FIG. 4 depicts sub-operations of a non-limiting example of a submethod suitable for designing a color router in accordance with the present disclosure. Operation 201 begins with sub-operation 301 , wherein a candidate design for color router 104 is established.

[0069] At sub-operation 302, the candidate design for color router 104 is simulated using an electromagnetic simulation. Examples of electromagnetic simulations suitable for use in accordance with the present disclosure include, without limitation, finite-difference time-domain, finite-difference frequencydomain, finite element, and the like.

[0070] At sub-operation 303, for the proposed color-router-structure design, an adjoint variable method is used to estimate the gradient of the objective function with respect to a plurality of design parameters. Preferably, this adjoint variable method includes at least two full electromagnetic simulations of the structure.

[0071] At sub-operation 304, the structure of the color router is updated along the direction of the gradient. It should be noted that any or all degrees of freedom in the color-router structure can be adjusted in parallel. Degrees of freedom are adjusted to respect bound constraints of the material dielectrics available for use in design region.

[0072] At sub-operation 305, this performance of the given color-router structure is evaluated against a set of performance metrics. Typically, thesemetrics primarily include routing efficiency into the desired channel, which, preferably, should be maximized while also measuring reflection and cross talk to attempt to minimize them. If the performance is deemed less than satisfactory, sub-method 300 returns to sub-operation 303. Note that sub-operations 303 through 305 can be iterated as many times as necessary to realize a color router design that is satisfactory.

[0073] If the performance satisfies the design criteria, however, submethod 300 continues with sub sub-operation 306, in which a scatterer-pattern 410- / is established for each of sub-layers 412-1 through 412-N, where each scatterer pattern includes a two-dimensional arrangement of scatterers that is based upon the position of its respective sub-layer within the arrangement of the N sub-layers along the z-direction.

[0074] The value of N depends on the desired performance for the color router, among other factors. Typically, has a value within the range of approximately 5 to approximately 40; however, any practical number of sublayers, from just one layer to a few sub-layers to many dozen sub-layers or more, can be used in a color router without departing from the scope of the present disclosure.

[0075] It should be noted that, in some embodiments, the number, N, of sub-layers is established as part of sub-operation 301 .

[0076] Returning now to method 200, for each of / =1 through N:

[0077] At operation 202, media layer 402- / is formed. Media layer 402- / comprises material M1 , which has a first dielectric constant sr=1 . In the depicted example, material M1 is silica having a dielectric constant of approximately 2.1 . It should be noted that, preferably, photodetectors 110 are passivated by encasing them in dielectric material. In the depicted example, photodetectors 110 are encased in material M1 , which also forms spacer layer 112. As a result, media layer 402- / is formed on space layer 112.

[0078] At operation 203, a two-dimensional void pattern 404- / of voids 406 is formed in media layer 402- / . The arrangement of void pattern 404- / corresponds to the position of sub-layer 412- / within the arrangement of the N sub-layers of color router 104, as determined in sub-operation 305, as discussed above. In the depicted example, voids 406 are approximately 10 nm by 10 nm in size ( / .e., the design-element size is 10 nm by 10 nm), with abutting individual voids collectively defining larger features. It should be noted, however, that any suitable dimension for voids 406 can be used without departing from the scope of the present disclosure.

[0079] FIG. 5A shows a cross-sectional view of nascent pixel-repeat unit 100’ after formation of void pattern 404-1 .

[0080] At operation 204, voids 406 are filled with material M2, which has a second dielectric constant, sr=2, to form scatterers 408, which have substantially the same dimensions as voids 406. Within each of layers 412- / , scatterers 408 are arranged in scatterer pattern 410- / , which matches void pattern 404- / . In the depicted example, material M2 is silicon nitride having a dielectric constant of approximately 4.0.[oo8i] FIG. 5B shows a cross-sectional view of nascent pixel-repeat unit 100’ after formation of scatterer pattern 410-1 in media layer 402-1 .

[0082] Once voids 406A contain material M2, media layer 402- / and scatterer pattern 410A- / collectively define sub-layer 412- / .

[0083] It should be noted that any of a wide range of suitable materials can be used for the materials included in color router 104. However, embodiments in accordance with the present disclosure derive significant advantages over the prior art by using lossless dielectric materials for the materials of color router 104. For example, the use of lossless materials enables all of the light incident on each pixel-repeat unit to be collected, while each wavelength signal is directly routed to a corresponding photodetector with nearly perfect optical efficiency. Asa result, substantially lossless dielectric materials, such as silica, silicon oxides, silicon nitrides, silicon oxynitrides, titanium dioxide, hafnium oxide and the like, are typically preferred for operation in the visible spectrum. As will be apparent to one skilled in the art, after reading this Specification, the loss of a material is dependent upon the wavelength of operation; therefore, different dielectric materials, such as silicon, germanium, gallium phosphide, magnesium fluoride, zinc selenide, zinc sulfide, barium fluoride, calcium fluoride, sapphire, amorphous silicon dioxide are preferable for color routers designed for operation in other wavelength ranges, such as the ultraviolet wavelength range, mid-infrared wavelength range, long-infrared wavelength range, and the like.

[0084] At optional operation 205, sub-layer 412- / is planarized.

[0085] Operations 202 through 205 are repeated N times such that sublayers 412-1 through 412-N collectively define color router 104 and scatterer patterns 410-1 through 410-N collectively define three-dimensional scatterer arrangement 414.

[0086] FIG. 5C shows a cross-sectional view of pixel-repeat unit 100 after completion of sub-layer 412-N and, therefore, color router 104.

[0087] In the depicted example, color router 104 al of scatterers 408 comprise the same material. However, in some embodiments, scatterers in color router 104 are made of more than one material.

[0088] FIG. 3B depicts operations of an exemplary alternative method suitable for forming a color router having multiple scatterer types, each having a different material composition, in accordance with the present disclosure. Alternative method 200B is analogous to method 200A and begins with operations 201 and 202 described above.

[0089] It should be noted, however, that in method 200B, the design established for color router 104 includes arrangements for each of the m scatterer types included in the color router. As such, each of sub-layers 412-1through 412-N generated via sub-method 300 includes a different scatterer pattern for each scatterer type.

[0090] For each of / =1 through m:

[0091] At operation 203B, void pattern 404- / - comprising voids 406-j is defined in media layer 402- / .

[0092] At operation 204B, voids 406- / ' are filled with material M2- / to form scatterers 408- where each of materials M2- / has a different dielectric constant that is also different than material M1 . Scatterers 408-j are arranged in scatterer pattern 410- / - / which substantially matches void pattern 404- / - / In the depicted example, m=2 and material M2-1 is silicon nitride and material M2-2 is silicon oxynitride.

[0093] FIG. 5D shows a cross-sectional view of nascent pixel-repeat unit 100’ after formation of scatterer patterns 410-1-1 through 410-1-2 in media layer 402-1 .

[0094] Upon completion of all m scatterer patterns in media layer 402- / , method 200B continues with operation 205 of method 200A, as described above.

[0095] In some embodiments, a three-dimensional design of a color router is established without employing a topographic approach in which it is segmented into a plurality of vertically arranged sub-layers. In some such embodiments, the three-dimensional arrangement of scatterers throughout color router 104 is fabricated using gray-scale lithographic and / or fabrication methods.

[0096] In some embodiments, color router 104 is fabricated by inducing a dielectric constant change at each of a plurality of voxel locations distributed throughout the three-dimensional volume of the layer. In some such embodiments, material M1 is a material whose dielectric constant can be altered by exposing it to an optical signal (e.g., an ultraviolet light beam, etc.) and each scatterer is produced by intersecting two light beams at the position of the scatterer. The intensity of each individual light beam is too low to induce adielectric-constant change in material M1 . However, at the crossing point of the light beams, the combined optical intensity is sufficient to induce a desired dielectric-constant change in material M1 , thereby producing a scatterer at that point.

[0097] FIG. 6A depicts a plot of the dielectric constant throughout the structure of pixel-repeat unit 100.

[0098] FIG. 6B depicts field intensity plots showing the passage of each of the wavelength signals of light signal 106 through color router 104.

[0099] Plots 502, 504, 506, and 508 are intensity maps that show the paths of the photons of wavelength signals 106R, 106G, 106B, and 106NIR, respectively, through color router 104.[ooioo] Table 1 shows the optical efficiency, and reflection for pixelrepeat unit 100.[ooioi] As can be seen from Table 1 , highly efficient routing and excellent optical efficiency can be achieved by color routers in accordance with the present disclosure. In fact, color router 104 exhibits substantially perfect optical routing of each of the wavelength signals of light signal 106 into their intended photodetectors, enabling pixel-repeat unit 100 to operate with substantially perfect optical efficiency (>99%) and substantially no optical crosstalk.

[0102] It should be noted that, in addition to directly routing the wavelength signals to their intended destinations, color router 104 is a multifunctional element that also replaces an anti-reflection coating and a microlens array. As a result, a color router in accordance with the present disclosure mitigates optical loss due to light reflection at the top surface of an image sensor, as well as from photons erroneously directed toward non-photosensitive regions of the image sensor (e.g., the regions between photodetectors 110). Such multifunctional capability is heretofore unknown in the prior art.

[0103] As will be apparent to one skilled in the art, absorptionbased color filters are designed to achieve images with accurately rendered or pleasing colors. In fact, significant effort is directed to tuning the spectral response of absorbing color filters to facilitate color processing and digital image processing.

[0104] It is yet another aspect of the present disclosure that color router 104 can be designed such that its spectral response substantially matches a desired spectral template.

[0105] Specifically, in some embodiments, color router 104 is designed such that each of the wavelength components it routes has a desired spectral shape and center wavelength.

[0106] FIG. 7 depicts spectral characteristics of the target design and corresponding optimized design response for wavelength signals 106B, 106G, 106R, and 106NIR routed by fully binarized color router in accordance with the present disclosure. The spectral characteristics for each wavelength component in wavelength signals 106B, 106G, 106R, and 106NIR is routed such that it has a desired spectral shape having a full-width at half-maximum (FWHM) about a center wavelength.

[0107] Plot 602 shows the performance of a second exemplary color router that routes wavelength signals 106B, 106G, 106R, and 106NIR suchthat they have a wider Gaussian spectral shape that has a FWHM of approximately 125 nm about the same center wavelengths.

[0108] It should be noted that, for each color router design, peak optical efficiency is within the range of 80-90%. As will be apparent to one skilled in the art, the theoretical limit, TL, of an absorbing-color-filter-based image sensor is 1 / M, where M is the number of wavelength signals detected by a system. For a four-color image sensor that employs absorbing color filters, therefore, TL is 25%, which is well below the optical efficiency color channels in pixel-repeat unit 100.

[0109] FIG. 8 depicts results demonstrating the effect of designelement size on the performance of a color router in accordance with the present disclosure. Plot 700 shows spectra for wavelength signals 106B, 106G, 106R, and 106NIR for color routers having scatterers whose sizes are based on minimum feature sizes ( / .e., design-element sizes) of 20 nm and 50 nm.[oono] As is readily seen from plot 700, the spectral responses are nearly identical for both design-element sizes. Varying design-element size over the range of 10 nm to 50 nm, therefore, has little effect on overall color router efficiency or spectral bandwidth. It should be noted that the design-elements can have any practical size and design-element size is not limited to the range from 10 nm to 50 nm.[oom] FIG. 9 depicts a schematic drawing of a cross-sectional view of an individual pixel-repeat unit of an alternative embodiment of an image sensor in accordance with the present disclosure. Pixel-repeat unit 800 includes light-detection layer 802 and color router 804, which is disposed on the lightdetection layer. Pixel-repeat unit 800 is analogous to pixel-repeat unit 100; however, pixel-repeat unit 800 is a three-photodetector arrangement for detecting only the blue, green, and red wavelength signals 106R, 106G, 106B of light signal 106.[ooii2] FIG. 10 depicts spectral characteristics of a target design and corresponding optimized design response for wavelength signals 106B, 106G, and 106R as directly routed by color router 804.[ooii3] Plot 900 shows that wavelength signals 106B, 106G, and 106R are routed by color router 804 with peak optical efficiency that approaches 100%. Furthermore, it can be seen that the optimized design response and the design target are a nearly perfect match.[ooii4] In some embodiments, an imaging system includes an imaging lens and a focal-plane array comprising a large number of pixel-repeat units arranged in a two-dimensional arrangement. The imaging lens, however, directs light toward each pixel-repeat unit, such that light received is characterized by a chief ray OR, which has an incidence angle 0CR (as indicated in FIG. 9) on the color router, the magnitude of which is based upon the position of that pixel-repeat unit in the arrangement.[ooii5] Furthermore, in some embodiments, one or more color routers of an array of pixel-repeat units is designed such that its functionality is tailored to the position of that pixel-repeat unit within a two-dimensional arrangement, which dictates the angle of incidence, 0CR, of light signal 106 at that pixel-repeat unit.

[0116] FIG. 11 depicts spectral characteristics of target designs and corresponding optimized design responses for two color routers designed for different incidence angles in accordance with the present disclosure.[ooii7] Plots 1000 and 1002 depicts the optical efficiency as a function of wavelength for color routers configured to route wavelength components 106B, 106G, 106R, and 106NIR when light signal 106 is received at incidence angles of 15° and 30°, respectively.

[0118] As can be seen from plots 1000 and 1002, the teachings of the present disclosure enable excellent spectral shape matching and high opticalefficiency ( / .e., > 80%) to be realized even when light is received at incidence angles as large as 30 degrees, which provides significant flexibility in color-router design.[ooii9] FIG. 12 depicts a schematic drawing of a cross-sectional view of an individual pixel-repeat unit of another alternative embodiment of an image sensor in accordance with the present disclosure. Pixel-repeat unit 1100 includes light-detection layer 1102 and color router 1104, which is disposed on the light-detection layer. Pixel-repeat unit 1100 is analogous to pixel-repeat unit 100; however, pixel-repeat unit 1100 is a four-photodetector arrangement whose light-detection layer and color router are arranged in a two-dimensional pattern. In the depicted example, the two-dimensional pattern of photodetectors defines a red-green-green-blue Bayer mosaic pattern; however, a plurality of photodetectors can be arranged in any practical two-dimensional arrangement (e.g., regular array, irregular array, irregular arrangement, etc.) without departing from the scope of the present disclosure.

[0120] Light-detection layer 1102 (hereinafter referred to as “LD layer 1102”) includes photodetectors 110B, 110G1 , 110G2, and 11 OR, which are arranged in a 2x2 array having pitch P2 in the x-dimension and P3 in the y- dimension. It should be noted that, in some embodiments, photodetector pitch can be less than the wavelength of the light upon which the color router operates. In the depicted example, P2=P3=320 nm; however, in some embodiments, P2 and P3 are unequal and / or at least one of P2 and P3 has a value other than 320 nm.

[0121] Color router 1104 is analogous to color router 104; however, color router 1104 is configured to route the wavelength signals of light 106 according to a 2x2 Bayer color mosaic geometry.

[0122] FIG. 13 depicts field intensity plots for the routing of the photons of the wavelength signals in light signal 106 by color router 1104.

[0123] Plot 1202 is a field intensity plot showing green photons as redirected by color router 1104 and captured by photodetectors 110G1 and 110G2.

[0124] Plot 1204 is a field intensity plot showing blue photons as redirected by color router 1104 and captured by photodetector 110B.

[0125] Plot 1206 is a field intensity plot showing red photons as redirected by color router 1104 and captured by photodetector 110R.

[0126] FIG. 14 depicts perspective views of field intensity plots showing the passage of each of the wavelength components of light signal 106 through color router 1104. Plots 1300, 1302, and 1304 depict the three- dimensional field distributions for the routing of the blue, green, and red wavelength components through color router 1104 to photodetectors 110B, 110G1 and 110G2, and 11 OR, respectively.

[0127] It should be noted that the size of pixel-repeat unit 1000 is approximately 640 nm x 640 nm, which is significantly smaller that can be realized using prior-art color-absorbing-filter based approaches.

[0128] FIGS. 15A-B depict spectral characteristics of an optimized broadband response for unpolarized and polarized wavelength signals 106B, 106G, 106R, respectively, as directly routed in pixel-repeat unit 1100.

[0129] Plot 1400 shows that unpolarized wavelength components 106B, 106G, and 106R are routed by color router 1104 with peak optical efficiency that approaches 100%.

[0130] Plot 1402 shows that the routing of x- and y-polarized wavelength components 106B, 106G, and 106R are also routed by color router 1104 with peak optical efficiency that approaches 100% and with virtually no difference in the shapes of their spectra.

[0131] It is yet another aspect of the present disclosure that a color router can be configured such that it has a large acceptable included-angle-of- incidence (IAOI) for light signal 106, within which the color router enables very high optical efficiency.

[0132] Table 2 shows the optical efficiency of pixel-repeat unit 1100 for each of wavelength signals 106B, 106G, and 106R as a function of incidence angle range 9, corresponding to imaging lenses with different f / #.

[0133] As can be seen from Table 2, each wavelength signal is directly routed through color router 1104 to their intended photodetectors with greater than 91 % efficiency even at incident angles of up to 14° and with greater than 95% efficiency at incident angles of up to 4°. It should be noted that, even for large incident angles of up to 26° (which corresponds to an imaging lens with f / 1 .0, the lower limit of f / # for a typical practical imaging lens), color routers in accordance with the present disclosure still directly route wavelength signals to their intended photodetectors with better than 60% efficiency. It should be noted that these efficiencies are significantly higher than even the theoretical limit (25%) of absorbing-color-filter-based image sensors.

[0134] In some cases, a less complex fabrication process would be advantageous, as it would potentially enable lower cost image sensors. In someembodiments, therefore, surface optimization is used to produce a color router having only one layer.

[0135] FIG. 16 depicts a schematic drawing of a cross-sectional view of an individual pixel-repeat unit of another alternative embodiment of an image sensor in accordance with the present disclosure. Pixel-repeat unit 1500 includes light-detection layer 1502 and color router 1504, which is disposed directly on the light-detection layer.[ooi36] Light-detection layer 1502 is analogous to light-detection layer 102; however, the photodetectors of light-detection layer 1502 are encased in material M3. In the depicted example, material M3 is silicon dioxide; however, any suitable dielectric material can be used without departing from the scope of the present disclosure.

[0137] Color router 1504 has maximum thickness, t2, and comprises a single layer of scatterers that are pillars of material M4, which has a fourth dielectric constant sr=4. The pillars of color router 1504 collectively define scatterer arrangement 1506, in which each pillar has the same lateral dimensions but a height that depends on the location of that pillar within the lateral extent of the scatterer arrangement. In the depicted example, material M4 is titanium dioxide having a dielectric constant of approximately 7.0.[ooi38] The design of color router 1504 is realized via the adjoint variable method in which the dielectric function of the color router is expressed, in one dimension, as:where sris the dielectric constant of material M4, sair is 1 .0, and h(x) is the pillar height along the x direction.

[0139] In such a color router, the boundary of the surface normal to z is graded smoothly using a sigmoid function. By choosing a large enough G, the pillar edges approach a sharp boundary between the design material and air. In some embodiments, a Gaussian filter is used to smooth the function h(x) to realize reasonable aspect ratios.

[0140] FIG. 17 shows a plot of the measured spectral response of surface-optimized color router in accordance with the present disclosure.

[0141] As is clear from plot 1600, the peak performance a surface- optimized color router is reduced relative to that of color router 104. However, as will be apparent to one skilled in the art, after reading this Specification, the performance of color router 1504 is significantly better than the absolute theoretical limit, TL, of 25% for absorbing-filter-based approaches.Spectral Routers

[0142] As previously mentioned, spectral routers may be produced using some of the same techniques described above to produce color routers. Before describing the spectral routers, the different goals that are to be achieved by spectral imaging in contrast to color imaging will be described, followed by a discussion of the functional and structural features that spectral routers require that differ from color routers in order to achieve these goals.

[0143] Color imaging involves capturing and reproducing visual information in a way that simulates the perception of color by the human eye. It is typically based on the three primary colors: red, green, and blue (RGB), which can be combined in various proportions to produce a wide range of colors. The final image is usually represented in a color space, such as RGB or CMYK (cyan, magenta, yellow, and black), which defines how colors are mixed and displayed and / or printed. Color imagers are commonly used in applications such as consumer displays, cameras, and printers to produce realistic and visually appealing images.

[0144] In contrast to color imaging, spectral imaging involves capturing and analyzing a range of wavelengths across the electromagnetic spectrum to collect spectral data in addition to spatial data. This collected data has the format of a datacubehaving a set with images, which represent the spatial data (x, ) , for each spectral channel (2) . The data is used to locate and identify objects or features in an imaged scene based on their spectral signature (i.e., particular features in its spectrum). Instead of relying on a limited set of primary colors, spectral imaging considers a broader range of wavelengths or spectral channels, i.e., ( ) consists of , where 7 =and M is the number of spectral channels. This allows for a more detailed analysis of the composition of the scene or object being imaged. The spectral image is generally represented by more wavelengths or spectral channels than is usually employed in the color space of a color router (M > 4 ), which can be used to extract spectral signatures for different materials. Spectral imaging can be used in applications such as microscopy, precision agriculture, food inspection, machine vision, bio / medical imaging, forensics, and counterfeit detection. It enables the identification of materials based on their unique spectral fingerprints or signatures.

[0145] In many cases spectral imaging is used to identify items or features or to distinguish between items or features in a scene that is being imaged. Accordingly, the spectral signatures that are to be obtained are signatures that are able to identify those items or features or to differentiate between those items or features. That is, for a spectral router, the spectral signatures (and hence the spectral channels) are selected based on what is to be detected in the scene. In contrast, in color imaging the spectral channels that are used are those that are visible to the human visual system. That is, the spectral channels that are used in a color imager are spectral channels that mimic those that can be seen by a human. In many cases, the spectral signatures that are selected for spectral imaging are chosen to distinguish between items or featuresthat cannot be distinguished by the human visual system. In other words, spectral imaging is often used to search for objects in a scene that are invisible to or indistinguishable by humans or color imagers. For example, in a food inspection application, spectral imaging may be used to distinguish between food and plastic material that may be the same color as the food. In medical imaging, for example, spectral imaging may be used to distinguish between healthy and unhealthy tissue or cells that may have the same color or look the same as healthy tissue or cells using color imaging.

[0146] In summary, color imaging is focused on reproducing visual information with a limited set of primary colors to mimic human perception, while spectral imaging involves capturing and analyzing a broader range of spectral channels or wavelengths for detailed information about the spectral characteristics of a scene or object and items in a scene, some or all which cannot be determined by the human visual system or by color imagers that generate images intended for the human visual system. Spectral imaging can therefore provide more insights into the composition of materials, whereas color imaging is primarily concerned with creating visually appealing representations.

[0147] Due to their different goals and applications, the functionality required of a spectral router is different from that required of a color router. For instance, to locate and identify objects and items in the image scene based on their spectral signatures, a spectral router that is sufficiently flexible to be suitable for multiple applications may require more than four spectral channels (wavelengths). These channels generally also need to have a narrower spectral bandwidth than required of a color router so that it is possible to distinguish objects based on their spectral features or signatures. The spectral channels may also have spectral shapes that differ from one another. The channel spacing in the spectral router can also be arbitrary and, unlike a color router, there may be gaps between the channels e.g., if there is no useful spectral data in the gap. Furthermore, unlike color routers, spectral routers often need to operate beyond the visible range of wavelengths, i.e., the operating range of a spectral routermay extend beyond the visible spectrum and may include, for example, ultraviolet, near-infrared, short-wave infrared, mid-wave infrared, long-wave infrared and very long-wave infrared spectral ranges. For example, most materials exhibit spectral signatures or “molecular fingerprints,” which can be identified at much longer infrared wavelengths.

[0148] In contrast to color routers, spectral routers may also be designed to operate with a wide variety of different spatial layouts, which refers to the spatial location and size, of the underlying photodetectors that detect different spectral channels. This spatial layout dictates the necessary routing of spectral content by the spectral router. Accordingly, spectral routers need to be more flexible than color routers to accommodate the different spatial layouts of the spectral channels that are to be detected. In contrast, color routers typically have a 2 by 2 channel kernel with an RGGB Bayer pattern so that they are compatible with current color image sensors and image processing pipelines.

[0149] To realize the spectral router, a design region is defined in the spectrally selective layer above the photodetectors where spectral filters are currently placed. In some embodiments this region may be a few micrometers tall. The spectral router design region is patterned with (sub)wavelength size design-elements made of lossless dielectric materials to form a system that consists of a large number of dielectric scatterers embedded in a dielectric background. The operating principle of the spectral router is the strong multiple scattering of light, which includes evanescent waves. Both the background and the scatterers are made of lossless materials and hence there is no absorption loss. Moreover, the multiple scattering of light in the router is used to cancel reflections and to route different spectral components to their respective photodetectors. The design elements may be formed using the techniques described above in connection with the color router, in which two-dimensional void patterns are formed in a media material. The voids are then filled with material having a different dielectric constant from the surrounding media material.[oolso] For a spectral router having design elements in the sub-100 nm size range, the design region can contain thousands or more of design elements and therefore thousands or more of scatterers. In some embodiments, the number of scatterers is at least ten. In other embodiments, the number of scatterers is at least a thousand, and in yet other embodiments the number of scatterers is at least a million. Even if only one of two lossless dielectric materials is assigned to each element, this makes for an extremely large design space. Of course, various embodiments of the spectral router may employ more than two lossless dielectric material to further increase the design space. To take advantage of the large number of degrees of freedom, efficient computational design can be used based on the adjoint variable method and gradient-based optimization, as discussed above in connection with color routers. Illustrative dielectric materials that may be employed include, without limitation, silica, silicon oxides, silicon nitrides, silicon oxynitrides, titanium dioxide, hafnium oxide silicon, germanium, gallium phosphide, magnesium fluoride, zinc selenide, zinc sulfide, barium fluoride, calcium fluoride, sapphire, amorphous silicon dioxide, and the like.

[0151] The spectral router response to incident light may be calculated using a first-principles electromagnetic field approach based on rigorous coupled wave analysis, such as described, for example, in V. Liu and S. H. Fan, Computer Physics Communications 183, 2233 (2012). The gradients required to optimize the structural design parameters, i.e. , the dielectric properties of each design element, which are initially continuous but converge in the optimization process to discrete values, are calculated with an adjoint variable method. A gradient-based optimizer using the method of moving asymptotes maximizes the optical power routed to the photodetector of each spectral channel based on the spectral content of incident light. An example of this method may be found in K. Svanberg, International Journal for Numerical Methods in Engineering 24, 359 (1987). This is performed mathematically by the maximization of an objective functionefficiency of the ithspectral channel, OXis the optical crosstalk from the jthspectral channel into the ithchannel, and a is a weighting factor between 0 and 1 . The choice of a reflects the different considerations of maximizing optical efficiency of the desired photodetector and minimizing the crosstalk in the others. In some embodiments, a value of a close to 1 is chosen.

[0152] Two specific examples of a spectral router will be presented below for illustrative purposes only and not as a limitation on the spectral routers described herein.

[0153] The first illustrative first spectral router encompasses the visible range of wavelengths (400-700 nm) with 6 spectral channels (centered on 425, 475, 525, 575, 625, and 675 nm). The spectral channels are configured in a 3 by 2 photodetector spatial layout and light is routed to a light-detection layer with photodetectors located in a silicon substrate. The design region of the spectral router is 2 |im tall and has a 990 nm by 660 nm sub-micrometer footprint. That is 15 times smaller than the equivalent repeat unit based on the smallest spectral filters made with lossless materials and 5 times smaller than the individual CMOS image sensor pixels in that filter-based system. The spectral router allows for 330 nm wide sub-wavelength photodetection pixels. This pixel size is a few generations ahead of the current state-of-the-art CMOS image sensors with the smallest pixels. Hence, it shows the ability of a spectral router to scale as future image sensor technology becomes available. The design elements are assumed to be 10 nm by 10 nm wide and 40 nm tall. The dielectric constant of the elements is taken to be er= 2.1 (e.g., silica) and er= 1 (e.g., titanium dioxide), both of which are effectively lossless at the desired wavelengths.

[0154] FIG. 18(a) shows a vertical cross-section through the middle of the spectral router structure, i.e., along the dashed horizontal line indicated in the subpanel below. FIG. 18(b) show the vertical cross-sections taken along thedashed horizontal lines shown in the subpanel below each cross-section. FIG. 18(b) illustrates the optical power flow to the photodetectors of the 6 channels at the operating wavelengths of 425, 475, 525, 575, 625, and 675 nm, respectively. This illustrates the ability of the spectral router to route incident light based on spectral content directly to the photodetectors of the spectral channels. The bottom subpanels in Fig. 18(b) show horizontal cross-sections of the power flow at the output plane of the spectral router, i.e., at the level of the photodetectors, for each of the operating wavelengths. These subpanels illustrate the spectral router’s ability to concentrate light on sub-wavelength size photodetectors. It should be noted that a spectral router achieves this without the need for a propagation layer between the design region and the detector, which results in a more compact structure.

[0155] FIG. 19 shows the detailed layouts of the design elements for individual layers of the structure. In particular, FIG. 19(a) shows a schematic diagram of a vertical cross-section of the optimized spectral router structure and FIG. 19(b) shows the detailed layouts of the design elements for the individual layers of the structure. The dark and light gray regions represent design elements with sr= 7 and sr= 2.1 , respectively, and illustrate the spatially disordered distribution of scatterers in the layers of the spectral router structure.

[0156] FIG. 20 shows the spectral response across the visible wavelength range for an optimized 6-channel spectral router design. The spectra have a full-width at half-maximum (FWHM) of 48 nm to cover the entire visible range while maximizing the light collected in each channel. By specifying a narrower bandwidth for the spectral channels in the design process, a spectral router can be optimized to achieve a higher spectral resolution instead. The performance of the optimized 6-channel spectral router is summarized in Table 3 in terms of optical efficiency, optical crosstalk and reflection. Spectral routing can be essentially ideal for all spectral channels with an optical efficiency greater than 0.99 at the center wavelength of every channel. The optical crosstalk betweenchannels at these wavelengths is effectively zero (less than 0.01 ) and reflected light is also negligible. To compare this with a multispectral filter approach, assume the existence of spectral filters with an ideal unity transmission. Hence, the maximum theoretical efficiency of a filter approach with M = 6 channels would be 1 / 6, indicated by the dashed line in Fig. 20. Thus, the efficiency of the filter approach is 6 times lower than the optical efficiencies of the channels achieved by the spectral router. It should be noted, however, that the scaling of spectral filters to sub-micrometer sizes is currently not possible.TABLE 3

[0157] The second illustrative spectral router covers the short-wave infrared (SWIR) range (1.1 -1.7 pm) with 9 spectral channels (centered on 1.1 , 1.175, 1.25, 1.325, 1.4, 1.475, 1.55, 1.625, and 1.7 pm) in a 3 by 3 photodetector layout. A design region with a 2.97 pm by 2.97 pm footprint that is 6 pm tall is employed. The spectral router therefore enables 990 nm photodetecting pixels, which is 2 times smaller than the smallest SWIR imager pixels demonstrated to date. The design elements are assumed to be 30 nm by 30 nm wide and 120 nm tall. The material properties for the design elements are the same as before, butin this case the photodetectors are located in an InGaAs substrate. In addition to covering a different part of the optical spectrum, this spectral router also serves to illustrate the ability to create spectral routers with larger feature sizes.

[0158] FIG. 21 (a) (left) shows a schematic perspective view of the spectral router structure integrated on a SWIR imager with 990 nm wide photodetectors. FIG. 21 (a) (right) shows the layout of the photodetectors of the 9 spectral channels. FIG. 21 (b) shows the horizontal cross-section of the optical power flow at the output plane of the spectral router, i.e., at the photodetectors of the 9 spectral channels, for the operating wavelengths of 1 .1 , 1.175, 1.25, 1.325, 1.4, 1.475, 1.55, 1.625, and 1.7 pm, respectively. The SWIR spectral router routes all short-wave infrared light based on its spectral content directly o the photodetectors of the intended spectral channels. Table 4 summarizes the optical performance of the optimized 9-channel SWIR spectral router. The routing is essentially ideal with an optical efficiency >0.99 for all spectral channels. The optical crosstalk between channels is effectively zero (<0.01 ). Light reflected by the spectral router is also negligible.TABLE 4

[0159] FIG. 22 shows the spectral response across the SWIR wavelength range for an optimized 9-channel spectral router design. The spectra cover the SWIR range and have a full-width at half-maximum (FWHM) of 60 nm as well as peak efficiencies greater than 0.98 in all spectral channels. The dashed line indicates the maximum theoretical efficiency of a 9-channel filterbased approach, which is 9 times lower than the optical efficiencies of the channels achieved by the SWIR spectral router.

[0160] The two illustrative spectral routers described above demonstrate the ability of spectral routers to break the scaling barrier that exists for multispectral filter arrays. Of course, it is possible to design larger spectral routers matched to larger photodetection pixels as well as the much smaller pixels used in the examples above. While the examples have 6 and 9 spectral channels, spectral routers can be designed with more or fewer channels. Similarly, while identically shaped spectra and equally spaced spectral channels have been chosen for these examples, more generally the channels of the spectral routers described herein can have a wide range of dissimilar spectral shapes with different channel spacings that may even be arbitrarily chosen.[ooi6i] As previously mentioned, the fabrication of the spectral routers can be achieved in a manner similar to that described above for color routers, in which thin dielectric layers are patterned in the design region. It should be noted that existing on-chip Fabry-Perot cavity filters already consist of a stack of thin layers made from dielectric materials deposited on each image sensor pixel. Several tens of layers with thicknesses in the sub-100 nm range are used to create the mirrors and the cavity that form the filters. In a spectral router, each layer would also have to be patterned into design elements. This requires advanced nanolithography methods with feature sizes in the sub-1 OOnm range. While patterning is a challenge, nanolithography with feature sizes down to ~10- 20 nm has already been shown with production extreme ultraviolet lithographysystems. Moreover, the challenge can be alleviated by larger feature sizes (> 100 nm) in spectral routers operating at longer wavelengths (e.g., mid-wave and longwave IR). Regarding integration, the dielectric materials considered herein are compatible with imager technologies and nanolithography tools for multilayer patterning are already in use in state-of-the-art semiconductor manufacturing of solid-state imagers, including CMOS image sensors.

[0162] The spectral routers described herein can be designed or configured for operation in different parts of the optical spectrum, e.g., ultraviolet (UV) range (100-400 nm), visible (VIS) range (400-700 nm), near-infrared (NIR) range (700-1 OOOnm), short-wave infrared (SWIR) range (1-3 pm, InGaAs 1-1.8 m), mid-wave infrared (MWIR) range (3-5 pm), long-wave infrared (LWIR) range (8-12 or 7-14 pm), very long-wave infrared (VLWIR) range (12-30 pm), THz range (30 pm-3 mm), and the like. The human eye is only sensitive to the visible range of the optical spectrum. All other ranges are not visible to humans. A spectral router can be designed for a single range, e.g., for the visible range or the short-wave infrared range, or it may have an operating range that spans multiple ranges, e.g., the visible and the near-infrared range.[ooi63] To design a spectral router that may be sufficiently flexible for use in a wide range of applications, it will generally be desirable for it have more than 4 spectral channels within its operating range. The illustrative spectral routers presented above for the VIS and SWIR ranges have 6 and 9 spectral channels since these correspond to the number of spectral channels in existing snapshot spectral imaging systems based on filter arrays. Of course, the spectral router described herein is not limited to 6 or 9 spectral channels, but may have more or fewer channels, depending on the demands of the application(s) in which it will be employed.[ooi64] In one illustrative embodiment operating in the visible range, the spectral router may have 6 spectral channels and the center wavelengths of the spectral channels may be located at 425, 475, 525, 575, 625, 675 nm (50 nm separation). In another illustrative embodiment operating in the visible range, thespectral router may have 9 spectral channels and the center wavelengths of the spectral channels may be located at 430, 460, 490, 520, 550, 580, 610, 640, 670 nm (30 nm separation). In yet another illustrative embodiment operating in the visible range, the spectral router may have 12 spectral channels and the center wavelengths of the spectral channels may be located at 410, 435, 460, 485, 510, 535, 560, 585, 610, 635, 660, 685 nm (15 nm separation).[ooi65] In one illustrative embodiment operating in the SWIR range, the spectral router may have 9 spectral channels and the center wavelengths of the spectral channels may be located at 1.1 , 1.175, 1.25, 1.325, 1.4, 1.475, 1.55, 1.625, and 1.7 pm (75 nm separation). In another illustrative embodiment operating in the SWIR range, the spectral router may have 16 spectral channels and the center wavelengths of the spectral channels may be located at 1 .025, 1.075, 1.125, 1.175, 1.225, 1.275, 1.325, 1.375, 1.425, 1.475, 1.525, 1.575, 1 .625, 1 .675, 1 .725, 1.775 pm (50 nm separation).

[0166] The spectral shapes of the channels in the illustrative spectral routers presented above were selected to all be identical. In addition, the spectral channels were equally spaced. More generally, however, the channels of a spectral router can be designed with spectral shapes that differ from those shown in the illustrative examples, or they even be designed with dissimilar spectral shapes within a set. In addition, any suitable channel spacing may be employed, including an arbitrary channel spacing.[ooi67] If the spectral channels are equally spaced within the range of operation [2mm,2max] , the center wavelengths in that range will be equally spaced by not more than (max-min) / . For equally-spaced spectral channels, the bandwidth or full-width half maximum (FWHM) of each spectral channel cannot be more than (max- 2mm) / AY to avoid overlapping channels, but they can be smaller than 1 nm. As an example, in some embodiments the bandwidth or FWHM of each spectral channel may range between 1 nm and 50 nm in the visible wavelength range.

[0168] The channels of a spectral router can be designed to have a number of different spectral shapes, e.g., Gaussian shape, Lorentzian shape, flat-top shape, Airy shape, needle-shape, and other spectral shapes. Within the set of spectral channels of the spectral router, the spectral shapes can all be identical, or they may be different for all or some of the spectral channels in the set. Each of the spectral channels may also be designed with a specified optical efficiency and / or a specified spectral crosstalk.[ooi69] The spatial layout of the spectral channels, which refers to the spatial location and size of each spectral channel below the spectral router, corresponds to the location of the photodetectors for each channel. The spatial layout may be determined as a part of the design process, and it can be optimized as part of an iterative design process that optimizes one or more parameters such as the optical efficiency and / or the spectral crosstalk. The spatial layout that is chosen is generally not application-dependent.

[0170] In summary, a spectral router is described herein that is suitable for snapshot multispectral imaging. The spectral router is a device that routes all light incident on the entire device surface directly to a photodetector for each spectral channel based on its spectral content without the need for an additional propagation or spacer layer. The illustrative embodiments presented herein for the visible and SWIR range show that spectral routing can significantly improve photon efficiency for on-chip multispectral imaging compared to multispectral filter arrays, and can be designed to exploit 100% of the incident light. Spectral routers can also have a wavelength size footprint and, hence, may allow the simultaneous collection of both spectral information and all diffractionlimited spatial information in the image.

[0171] In some cases the spectral routers described herein may employ feature selection to detect and identify items or features in a scene being imaged, or to be able to distinguish between one or more items or features in a scene being imaged. Feature selection refers to the process of identifying and choosing for further analysis specific spectral channels from among all theavailable channels. That is, feature selection involves selecting from the available M spectral channels, a smaller number of channels that contain data most relevant for a particular application. As a result, data analysis (e.g., regression or classification) can be performed in this reduced space more accurately than in the original larger M-dimensional space.

[0172] Feature selection may even use the (spatial) image data from only a single spectral channel. This can be accomplished by first specifying the item or feature (e.g., chemical, material, surface, object) to be detected in the imaged scene. Next, the spectral signature(s) of the item or feature is determined. In particular, a narrow spectral signatures (less than 100 nm spectral width) can be used to identify the item or feature in the imaged scene. The spatial image data from the single spectral channel is then selected which overlaps with these spectral signature(s). Finally, the locations (in terms of pixels, regions or segments) in the spatial image data are identified to determine if and where the item or feature is present in the imaged scene.

[0173] A single spatial image may also be obtained by combining the (spatial) data from two or more spectral channels. First, the spectral channels of interest are selected. The spatial data from the selected spectral channels are then combined by using them as arguments of a linear or nonlinear function. In some examples that combine spectral channels according to a function, images may be obtained that represent index information, e.g., the normalized difference red edge index, and the like. The index values are then used to identify and classify locations (pixels, regions, segments) in the spatial image. For example, the normalized difference vegetation index is a widely-used metric for quantifying the health and density of vegetation and is calculated from two specific spectral channels in the red and near-infrared range, respectively. An area with nothing growing in it will have an index of zero. The index will increase in proportion to vegetation growth. An area with dense, healthy vegetation will have an index of one.

[0174] In other cases, the spectral routers described herein may be employed to discover and / or detect items or features in a scene being imaged, or to distinguish between discovered and / or detected items or features in a scene being imaged. The data captured or collected from the spectral channels enabled by the spectral router is sometimes first converted by spectral and / or spatial transformations (either linear or nonlinear) into a transformed space, which is sometimes referred to as “feature” space. If this transformation corresponds to the identity transformation, then the feature space, in fact, directly corresponds to the spectral channel space of the spectral router. This process is also called feature extraction. Note that unlike in a previous embodiment, in this case the spectral signatures are not selected in advance.

[0175] The spectral transformations can comprise the identity transform, multispectral ratios, principle component transformations, and the like. Spatial transformations may consist of the identity transformation, or they may transform only local (spatial) image information, such as convolution and the like, transform global spatial image information, for example, the Fourier transform, and / or comprise spatial transforms between purely local and global.[ooi76] Feature extraction may be used to identify and extract relevant information or features from the (spatial) image data acquired by the spectral imager or from the data in the transformed space. Feature extraction may employ dimensionality reduction techniques to transform the high (M)- dimensional space into a space with fewer dimensions. Feature extraction can be performed using different (non)linear statistical methods (e.g., principle component analysis, neural networks) to create a lower dimensional data set.

[0177] Features can also be learned, this is called feature learning, which can be supervised, unsupervised, or self-supervised. Supervised feature learning is learning features from labeled multispectral data, i.e. , it requires a- priori spectral knowledge of the items or features in the imaged scene to label the data. It can be done, for example, by dictionary learning or with neural networks. Unsupervised feature learning is learning features from unlabeled data and canbe done, for example, with clustering methods (e.g., k-means clustering) or principle component analysis. Feature learning can also be performed with deep learning using multilayer neural networks.

[0178] In some cases the spectral routers described herein may employ clustering, segmentation, and / or classification methods to detect and / or identify items in a scene being imaged, or to distinguish between one or more discovered and / or detected items or features in a scene being imaged, and / or to extract relevant information from the (spatial) image data acquired by the spectral imager or from the data in the transformed space. In the clustering and / or segmentation process, various methods, including clustering methods (e.g., k- means clustering), principle component analysis and the like, can be applied to find similarities in the data, which in turn is used to form regions that have similar spectral signatures and to separate them from regions with dissimilar spectral signatures. This allows to distinguish between regions with different spectral signatures in the imaged scene. These regions with different spectral signatures can then be labeled in a classification process to identify one or more items or features in the imaged scene;

[0179] These methods can be supervised or unsupervised. If classification is supervised, the labels for the regions are known a-priori. For example, in supervised classification, a-priori known spectral signatures are used to search for similar signatures in the (spatial) image data, often this is performed in the transformed space, to identify different classes of items or features in the imaged scene. Clustering, segmentation and / or classification can be performed unsupervised, i.e. , without a-priori knowledge, by determining those spatial locations that have similar spectral signatures and by grouping them together in spatial clusters or segments, and / or by identifying them as members belonging to the same class, while locations with dissimilar spectral signatures will be grouped in different clusters or segments, and be identified as members belonging to different classes. In this case, the labels need to be assigned as part of the classification process.[ooi8o] It is to be understood that the disclosure teaches just some examples of embodiments in accordance with the present disclosure and that many variations can easily be devised by those skilled in the art after reading this disclosure without departing from its scope and, furthermore, that the scope of the present invention is to be determined by the following claims.

Claims

What is claimed is:1 . A method of forming a spectral imager for use in one or more spectral imaging applications, comprising: selecting a spectral operating range of interest and a number of spectral channels to be imaged by the spectral imager within the spectral operating range such that a plurality of different spectral signatures within the spectral operating range are able to be identified by the spectral imager, the different spectral signatures including spectral signatures that are able to distinguish between one or more items or features in a scene being imaged; selecting, for each of the selected spectral channels, a plurality of specified spectral characteristics, the specified spectral characteristics including a center wavelength of the respective channel, a bandwidth or full-width at half maximum (FWHM) of the respective channel, an optical efficiency and optical crosstalk of the respective optical channel, and a spectral response shape of the respective channel to thereby define selected spectral channels having the plurality of specified characteristics; selecting a spatial layout, representing the spatial location and the size, of spatial areas located at an output plane, each of the spatial areas receiving one of the selected spectral channels with the respective plurality of specified spectral characteristics; providing at the output plane a light-detection layer comprising a plurality of photodetectors that are arranged in accordance with the spatial layout that is selected; forming a spectral router on the light-detection layer, the spectral router having a structure comprising a first plurality of scatterers that is arranged within the spectral router in a first arrangement, and wherein the spectral router comprises a first material having a first dielectric constant, and wherein each scatterer of the first plurality thereof comprises at least a second material having a second dielectric constant that is different than the first dielectric constant; anddefining the first arrangement such that the spectral router directly routes each of the selected spectral channels with their respective plurality of specified characteristics to the plurality of photodetectors in the light-detection layer such that each photodetector of the plurality thereof selectively receives one of the spectral channels.

2. The method of claim 1 wherein determining the plurality of spectral signatures that are able to be identified by the spectral imager includes spectral signatures that are able to distinguish between one or more items or features that are indistinguishable and / or invisible to a human visual system and / or to a color imager that captures color images visible to a human vision system.

3. The method of claim 1 further comprising optimizing a design and formation of the spectral router based on the one or more specified spectral characteristics of the spectral router.

4. The method of claim 1 wherein the selected number of spectral channels to be imaged by the spectral router within the spectral operating range is at least 4 spectral channels.

5. The method of claim 1 wherein the spectral router is formed directly on the light-detection layer so that the spectral router is in contact with the lightdetection layer.

6. The method of claim 1 wherein the spectral router further includes a second plurality of scatterers, each scatterer of the second plurality thereof comprising a third material having a third dielectric constant that is different than the first dielectric constant.

7. The method of claim 1 wherein the first arrangement is selected from the group consisting of a linear arrangement and a two-dimensional arrangement.

8. The method of claim 1 wherein the spectral router includes a plurality of sub-layers, each sub-layer including a scatterer pattern comprising scatterers of the first plurality thereof, wherein the scatterer pattern of a first sub-layer of the plurality thereof is different than the scatterer pattern of a second sub-layer of the plurality thereof.

9. The method of claim 1 wherein the spectral router includes at least 10 scatterers.

10. The method of claim 1 wherein the spectral router includes 10-1 ,000,000 scatterers.11 . The method of claim 1 wherein the scatterers are distributed in a 2spatially disordered manner.

12. A spectral imager formed in accordance with the method of claim 1 .

13. A method of performing spectral imaging, comprising: selecting one or more items or features to be identified by a spectral imager; selecting a spectral signature of the one or more items or features to be identified by the spectral imager, the spectral signature being able to identify the one or more items or features or distinguish between the one or more items or features in a scene being imaged; selecting one or more spectral channels of the spectral imager that are collectively able to identify the spectral signature; imaging the scene using the spectral imager to obtain an imaged scene that is represented by a datacube having a set with images for each spectral channel; identifying a location of the spectral signature in the imaged scene; andwherein the spectral imager includes a spectral router having a structure comprising a first plurality of scatterers that is arranged within the spectral router in a first arrangement, and wherein the spectral router comprises a first material having a first dielectric constant, and wherein each scatterer of the first plurality thereof comprises at least a second material having a second dielectric constant that is different than the first dielectric constant, the spectral imager further including a light-detection layer comprising a plurality of photodetectors that are arranged in accordance with a specified spatial layout, the first arrangement of the first plurality of scatters being arranged such that the spectral router directly routes each of the selected spectral channels to the plurality of photodetectors in the light-detection layer such that each photodetector of the plurality thereof selectively receives one of the spectral channels.

14. The method of claim 13 further comprising selecting a spectral operating range for the spectral router sufficient to obtain the spectral signature.

15. The method of claim 14 further comprising, selecting, for each of the selected spectral channels of the spectral router, a plurality of specified spectral characteristics sufficient to obtain the spectral signature, the specified spectral characteristics including a center wavelength of the respective channel, a bandwidth or full-width at half maximum (FWHM) of the respective channel, an optical efficiency and optical crosstalk of the respective optical channel, and a spectral response shape of the respective channel to thereby define selected spectral channels having the plurality of specified characteristics;16. The method of claim 13 wherein at least one of the selected channels is at a wavelength invisible to the human visual system.

17. The method of claim 13wherein identifying the location of the spectral signature in the imaged scene includes performing feature selection and / or feature extraction.

18. The method of claim 13 wherein identifying the location of the spectral signature in the imaged scene includes performing clustering, segmentation and / or classification.

19. A method of performing spectral imaging, comprising: imaging the scene using the spectral imager to obtain an imaged scene that is represented by a datacube having a set with images for each spectral channel; transforming the spectral channel data in the datacube using spatial and / or spectral transforms into a transformed space, applying clustering, segmentation and / or classification methods in the transformed space to discover or identify one or more items or features in the imaged scene; using clustering and / or segmentation based on similarities of and dissimilarities in the data in the transformed space to form regions with similar spectral signatures and to distinguish between regions with different spectral signatures in the imaged scene using a classification process to label the different regions, obtained by clustering and / or segmentation, to identify one or more items or features in the imaged scene; and wherein the spectral imager includes a spectral router having a structure comprising a first plurality of scatterers that is arranged within the spectral router in a first arrangement, and wherein the spectral router comprises a first material having a first dielectric constant, and wherein each scatterer of the first plurality thereof comprises at least a second material having a second dielectric constant that is different than the first dielectric constant, the spectralimager further including a light-detection layer comprising a plurality of photodetectors that are arranged in accordance with a specified spatial layout, the first arrangement of the first plurality of scatters being arranged such that the spectral router directly routes each of the selected spectral channels to the plurality of photodetectors in the light-detection layer such that each photodetector of the plurality thereof selectively receives one of the spectral channels.

Images