Inverse designed nonlocal META-optics for parallel analog image processing

Abstract

A device useful for image processing including a scattering medium patterned for angle dependent scattering of electromagnetic radiation encoding a plurality of different spatial frequency filtering operations on different diffraction orders, polarization states, or wavelengths of electromagnetic radiation incident on the scattering medium from an object, so that an image of the object formed from the electromagnetic radiation encoded by the scattering medium is processed by each of the different spatial frequency filtering operations in parallel.

Description

INVERSE DESIGNED NONLOCAL META-OPTICS FOR PARALLEL ANALOG IMAGE PROCESSING CROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit under 35 U.S.C. Section 119(e) of:

[0002] U.S. Provisional Application No.63 / 686, 254, filed August 23, 2024, by Phillippe Pearson, Gregory Roberts, Andrei Faraon, entitled “INVERSE DESIGNED NONLOCAL META-OPTICS FOR PARALLEL ANALOG IMAGE PROCESSING” (CIT-9057-P2) and

[0003] U.S. Provisional Application No.63 / 801, 977, filed May 8, 2025, by Phillippe Pearson, Gregory Roberts, Andrei Faraon, entitled “INVERSE DESIGNED NONLOCAL META-OPTICS FOR PARALLEL ANALOG IMAGE PROCESSING” (CIT-9057-P3) ;

[0004] both of which applications are incorporated by reference herein. STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH AND DEVELOPMENT

[0005] This invention was made with government support under Grant No. W911NF-22-1-0097 awarded by the US ARMY. The government has certain rights in the invention BACKGROUND OF THE INVENTION

[0006] Field of the Invention.

[0007] The present disclosure relates to scattering media useful for image processing and methods for using and making the same.

[0008] Description of the Related Art

[0009] As computing demand accelerates, there is a growing need for high-speed, low-power data processing. In particular, autonomous vehicles and Augmented  Reality (AR) and Virtual Reality (VR) devices require processing immense amounts of data from a variety of sensors in real time. Cameras constitute a significant part of the sensor suite in both cases, and thus image processing becomes critical. In AR / VR the form factor is extremely constrained, leading to concerns about managing battery life and heating. In response, there has been tremendous progress in the ability to perform computational tasks passively in analog, thus saving valuable computational resources.

[0010] In recent years, metasurfaces have been shown to achieve some basic image processing tasks such as edge detection and blurring. Metasurfaces are planar arrays of subwavelength scatterers that can locally alter the phase, polarization, and amplitude of optical fields incident upon them. By engineering resonances in these structures, it has been demonstrated that analog edge detection can be achieved in a compact form factor. This functionality is due to a quadratic- like increase of the transmission coefficient’s magnitude with increasing in-plane wavevector (i.e. angle of incidence). The device is added into the optical system and the sensor directly captures the processed image.

[0011] Photonic inverse design has emerged over the past decade as a promising approach to design a wide variety of optical devices [21-25]. Specifically, topology optimization expands the metasurface design space by framing the design process as an optimization problem where the metasurface's permittivity is iteratively adjusted to extremize a figure of merit (FoM) that encodes the desired functionality

[0026] . Several applications of topology-optimized free-space devices have been demonstrated, including high-efficiency gratings [27,28], metalenses [29-32], holograms [33,34], active metasurfaces

[0035] , color filter arrays [36-38], and end-to-end design of optical systems [39-44]. In the context of spatial frequency filtering, it has been computationally demonstrated that inverse design can be used to realize complex transfer functions within periodic structures composed of up to 30 patterned layers

[0016] . However, the fabrication challenges  involved with patterning and aligning tens of layers would likely prevent these structures from being realized outside of advanced semiconductor foundries. More recently, a single-layer inverse-designed metasurface was demonstrated for edge detection, albeit with limited multifunctionality, a small numerical aperture (NA), and poor isotropy

[0045] . Edge detection aside, there has been limited work considering low-pass filtering or blurring. Examples include collecting reflected light from a metasurface designed for edge detection in transmission to achieve the complementary transfer function [2] and geometric phase metasurfaces limited to operate with circular polarization

[0046] . SUMMARY OF THE INVENTION

[0012] In contrast to previous work, illustrative embodiments described herein leverage inverse design to simultaneously optimize transmission for multiple wavelengths, angles of incidence, diffraction orders, or polarizations, so as to augment parallel image processing performance compared to rationally designed metasurfaces. For example, a device useful for image processing according to embodiments described herein, comprises a scattering medium patterned for angle dependent scattering of electromagnetic radiation encoding a plurality of different spatial frequency filtering operations on different diffraction orders, polarization states, or wavelengths of electromagnetic radiation incident on the scattering medium from an object, so that an image of the object formed from the electromagnetic radiation encoded by the scattering medium is processed by each of the different spatial frequency filtering operations in parallel. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0014] Referring now to the drawings in which like reference numbers represent corresponding parts throughout:

[0015] Figs.1A-1E: Inverse-designed unit cell (Fig.1A), Scanning Electron Microscope (SEM) image of a fabricated device (Fig.1B), measured (Fig.1C, using the system in Fig.13) and simulated (Fig.1D) angular transmission, and amplitude transfer function (Fig.1E). This device only performs differentiation on the zeroth order. The device was designed using the method of moving asymptotes to optimize permittivity of unit cell, enforcing C4 and x / y reflection symmetries in the unit cell the simulated data shows S- pol is largely reflected near design wavelengths.

[0016] Fig.1F and Fig.1G illustrates simulation of the transmission and transmission phase, respectively, of the material comprising the inverse designed material of Fig.1A.

[0017] Fig.1H illustrates positioning of the device adjacent to a sensor in an imaging system in order to perform the image processing (in this example, edge detection).

[0018] Fig.1I and Fig.1J show the progression of the loss function and the binarization function during the inverse design procedure..

[0019] Fig.2A: Experimental imaging demonstration through the single-function device of Fig.1A illuminated with different polarizations and wavelengths, wherein the imaging is performed using the imaging system of Fig.13.

[0020] Fig.2B is a comparison of the measured and simulated images.

[0021] Fig.3A. Inverse-designed unit cell (Fig.3A) for a device that forms a brightfield and differentiated image on the 0th and +1 orders, respectively.

[0022] Fig.3B-3C. The simulated transfer functions for the device of Fig.3A compared against the targets for both orders (0thorder in Fig.3B, 1storder in Fig. 3C), demonstrating brightfield and edge imaging on different diffraction orders so that both functions can be captured on a sensor in a single shot.

[0023] Fig.3D shows an optimized unit cell permittivity with silicon and air denoted by blue and white regions, respectively.

[0024] Figs.3E-3G shows angle-dependent simulated transmission of the (Fig.  3E) zeroth, (Fig.3F) +1, and (Fig.3G) -1 diffraction orders with p-polarized light, for a metamaterial with the unit cell in Fig.3D.

[0025] Fig.4A illustrates an embodiment designed for increased spectral bandwidth.

[0026] Fig.4B shows the simulated fractional bandwidth for the device in Fig.4A is 8.7% based on design wavelengths and the peak transmission is approximately 0.8 for all the wavelengths, where fractional bandwidth is defined as shows simulated images obtained using the device of Fig.4A.Depiction of the topology optimization process. The grayscale permittivity represents the structure of a periodic metasurface's the unit cell partway through the optimization.

[0029] Fig.5B. Imaging with a multifunctional metasurface imparting distinct transfer functions for ^^^and ^^ଶ.

[0030] Figs.6A- selective high-pass filtering. Fig.6A showsoptimized metasurface unit cell permittivity, where blue regions are silicon and white regions are air; Fig.6B shows an SEM micrograph of the fabricated metasurface (scale bar 3^^ m ); Fig.6C shows a simulated ATF for ^^-polarized light; Fig.6D shows a simulated and measured ^^-polarized transmission coefficient amplitudes for a set of wavelengths and incident angles; Fig.6E shows a measured brightfield image with no metasurface and filtered images with linear vertical and horizontal incident polarizations in the middle and right panels, respectively (scale bar 130^^ m ); Fig.6F shows a line cut comparison of the brightfield and filtered ( ^^-pol) images as well as a comparison of cuts through the measured and simulated images normalized to the intensity of the brightfield square's center.

[0031] Fig.6G. Simulated and measured ^^-polarized angle-resolved transmission for the device presented in Fig.6 for an azimuthal angle of 45 deg, showing the  isotropy of the transfer function.

[0032] Figs.7A-7G. Polarization-selective low-pass filtering. Fig.7A shows final optimized unit cell permittivity for the polarization-selective low-pass filter; Fig. 7B shows an SEM image of the fabricated metasurface (scale bar 5^^ m ); Fig.7C shows a simulated co-polarized transmission coefficient magnitudes for ^^ - and ^^- polarization, which represents the ATF of the metasurface; Fig.7D shows a simulated and Fig.7E shows a measured transmission coefficient magnitudes for^^ - and ^^-polarization across wavelength and incident angle along the ^^ ൌ 0azimuth; Fig.7F shows measured images of a square object filtered through the metasurface with ^^-and ^^-polarized illumination (scale bar 130^^ m ); Fig.7G shows measured contrast for a series of line arrays, as a function of normalized spatial frequency, with no metasurface (green), metasurface with ^^-pol (blue), andmetasurface with ^^ െ pol (red). The error bars correspond to the standarddeviation of contrast calculated from 10 slices through the images of the line arrays.

[0033] Figs.8A-8G. Polarization-independent high-pass filtering. Fig.8A shows Optimized unit cell permittivity; Fig.8B shows SEM of the fabricated metasurface (scale bar 5^^ m ); Fig.8C shows simulated and Fig.8D shows measured transmission coefficient amplitudes for ^^ - and ^^-polarized light across wavelength and incident angle; Fig.8E shows simulated ATFs for ^^ - and ^^- polarized light through the metasurface; Fig.8F shows measured images of a Caltech logo through the metasurface with ^^ - and ^^-polarized light (scale bar 200^^ m ); Fig.8G shows horizontal line cuts through the two images in Fig.8F.

[0034] Figs.9A-9I. Spectrally multiplexed filtering. Fig.9A shows optimized unit cell permittivity of the multifunctional metasurface; Fig.9B shows an SEM image of the fabricated metasurface (scale bar 5^^ m ); Fig.9C shows a simulated and Fig.9D shows measured transmission coefficient amplitude for ^^ - and ^^- polarized rays; Fig.9E shows simulated ATF at a wavelength of 4.3^^ m; Fig.9F shows measured images of resolution target bar groups through the metasurface  with a laser wavelength of 4.37^^ m for ^^-and ^^-polarized light (scale bar 180^^ m ); Fig.9G shows line cuts through the vertical bar group from Fig.9F showing the difference in contrast for the two polarization states; Fig.9H shows simulated ATF for a wavelength of 4.65^^ m; Fig.9I shows measured images of a Caltech logo for ^^ - and ^^-polarized light with a wavelength of 4.7^^ m [same scale bar as in Fig.9F].

[0035] Fig.10. Simulated images of a square object through the metasurface in Fig.6. In the left panel, the object is illuminated with horizontally polarized light, while in the right panel vertically polarized light is used. The intensity of all simulated images are normalized to the input image's intensity, set to unity.

[0036] Fig.11. Simulated images of a Caltech logo through the metasurface in Fig.8. In the left panel, the object is illuminated with vertically polarized light, while in the right panel horizontally polarized light is used. The intensity of all simulated images are normalized to the input image's intensity, set to unity.

[0037] Fig.12. Simulated images of a square through the metasurface in Fig.8 of the main text. In the left panel, the object is illuminated with vertically polarized light, while in the middle panel horizontally polarized light is used. The right panel shows a horizontal line cut through the middle panel. The intensity of all simulated images are normalized to the input image's intensity, set to unity.

[0038] Figs.13A-13B. Characterization Setup. Fig.13A shows experimental setup to measure the transmission amplitude through the metasurfaces as a function of wavelength and incident angle; Fig.13B shows setup used to capture images presented in the example.

[0039] Fig.14 shows simulated and measured ^^-polarized angle-resolved transmission for the device presented in Fig.7 for an azimuthal angle of 45 deg.

[0040] Fig.15A. Optimized unit cell permittivity producing an ^^-polarized high- pass ATF.

[0041] Fig.15B. Simulated angle-resolved transmission spectra for ^^-polarized light; c) simulated ATFs for ^^ - and ^^-polarized light at a wavelength of 4.53^^ m.

[0042] Fig.15C. simulated ATFs for ^^- and ^^-polarized light at a wavelength of 4.53 ^^m.

[0043] Figs.16A-16B. Simulated transmitted power into diffraction orders for the device in Fig.6 at a wavelength of 4.65^^ m for (Fig.16A) ^^-rays and (Fig.16B)^^-rays. The ( ^^, ^^ ) indices label the diffraction orders based on the ^^ and ^^directions.

[0044] Figs.17A-17B. Simulated transmitted power into diffraction orders for the device in Fig.7 at a wavelength of 4.56^^ m for (Fig.17A) ^^-rays and (Fig.17B)^^-rays. The ( ^^, ^^ ) indices label the diffraction orders based on the ^^ and ^^directions.

[0045] Figs.18A-18B. Simulated transmitted power into diffraction orders for the device in Fig.8 at a wavelength of 4.5^^ m for (Fig.18A) ^^-rays and (Fig.18B) ^^-rays. The ( ^^, ^^ ) indices label the diffraction orders based on the ^^ and ^^directions.

[0046] Figs.19A-19D. Simulated transmitted power into diffraction orders for the device in Fig.9 at a wavelength of 4.67^^ m for (Fig.19A) ^^-rays and (Fig.19B) ^^-rays; (Fig.19C) and (Fig.19D) show the same quantities but simulated with awavelength of 4.37^^ m. The ^^^, ^^^ indices label the diffraction orders based onthe ^^ and ^^ directions.

[0047] Figs.20A-20C. S-polarized ATF and transmission results for the polarization-selective high-pass device from Fig.6. (Fig.20A) Simulated ATF for s-polarized light; (Fig.20B) simulated and (Fig.20C) measured transmission amplitude across angle and wavelength.

[0048] Fig. 21. Measured contrast curve at ^^^ ൌ 4.37^^ m for horizontal andvertical polarizations corresponding the device presented in Fig.9.

[0049] Fig.22. Comparison of the simulated versus target transmission amplitudefor the device presented in Fig. 7 using ^^-polarized light and ^^^ ൌ 4.56^^ m.

[0050] Fig.23. Flowchart illustrating a method of performing inverse design.

[0051] Fig.24 is a schematic illustrating a fabrication process for the scattering  medium.

[0052] Fig.25. Example chip packaging for implementing the scattering medium with an image sensor. DETAILED DESCRIPTION OF THE INVENTION

[0053] In the following description of the preferred embodiment, reference is made to the accompanying drawings which form a part hereof, and in which is shown by way of illustration a specific embodiment in which the invention may be practiced. It is to be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention.

[0054] Technical Description

[0055] The present disclosure describes a device useful for performing multiple image processing, or spatial frequency filtering, tasks in parallel. The device comprises a scattering medium patterned for angle dependent scattering (of electromagnetic radiation) encoding a plurality of different spatial frequency filtering operations on different diffraction orders, polarization states, or wavelengths of the electromagnetic radiation incident on scattering medium from an object, so that an image of the object formed from the electromagnetic radiation encoded by the scattering medium is processed by each of the different spatial frequency filtering operations in parallel.

[0056] Example scattering media include, but are not limited to, diffractive optical elements, metasurfaces, metamaterials, periodic scattering elements, and the like.

[0057] Example processing tasks may include, but are not limited to, 2ndorder image differentiation and (e.g., Gaussian) blurring. More generally, any arbitrary spatial frequency filtering operation may be encoded in the transfer function of the device (limited by the optimization algorithm search and fabrication constraints). Multiple operations can be realized in parallel by encoding them on different  diffraction orders, polarization states, or wavelengths. With this approach, a single image can reveal spatial, spectral, and polarization features with no computational overhead.

[0058] The image processing results from the angle-dependent scattering which can depend on a variety of properties of the electromagnetic radiation including but not limited to, polarization, wavelength, or diffraction order. For example, for 1D edge detection along the x-direction, the scattering medium is designed to have a transmission coefficient having a quadratic angular dependence (transmission coefficient is proportional to square of the in plane component of the incident wavevector (approximately the angle of incidence) on the scattering medium). Consider the angular spectrum representation of the electric fields:

[0059] E^^^,^^^ ൌ ^^ ି^    ^^ ି^    ^̂^൫^^௫, ^^௬൯exp ^^^൫^^௫^^ ^ ^^௬^^൯^d^^௫ d^^௬,a , a of -kx2appears in the integrand, implying the transmission coefficient of the metasurface should have a quadratic dependence on incident angle for this type of edge detection.

[0061] The pattern within each unit cell (e.g., determining the angle dependent transmission coefficient for each of the different wavelengths, polarizations, or diffraction orders) is discovered through a gradient-based optimizer seeking to minimize the difference between the desired and actual transfer functions: ℒൌ ^ே∑^^^^    ^^൫^^^ ,^^^ ,^^^,^^^൯^ ^^^ୟ୰^^^൫^^^ ,^^^ ,^^^,^^^൯

[0063] A method of designing the scattering medium using an inverse design process, comprises the following steps:

[0064] (a) selecting an initial pattern for the spatially varying permittivity within the unit cell of the periodic structure;

[0065] (b) solving Maxwell’s equations for the pattern to obtain a field distribution or complex-valued transmission coefficient for all incident angles,  polarizations, wavelengths, and diffraction orders needed to realize the desired filtering operations;

[0066] (c) using the field distribution or transmission coefficient to determine a figure of merit associated with each of the different spatial frequency filtering operations encoded in different wavelengths, polarizations, and diffraction orders, of the electromagnetic field (can be performed all in one step - there is one pattern, that may consist of several layers, that is designed via gradient optimization to produce the desired operations);

[0067] (d)calculating a gradient of the figure of merit with respect to the unit cell permittivity;

[0068] (e) using the gradient to adjust the pattern so as to minimize or maximize the objective function of the figure of merit; and

[0069] (f) iteratively repeating steps (b)-(f) until convergence (minimization or maximization) to a desired level.

[0070] In one embodiment the step (b) uses rigorous coupled wave analysis (RCWA), step (d) uses automatic differentiation, and step (e) uses the method of moving asymptotes (MMA) or any other gradient-based optimizer to minimize the objective function.

[0071] Formulating the device design as an optimization problem provides the ability to readily design devices with parallel functionality. Examples of this functionality may include, but are not limited to the following.

[0072] Different filtering operations on each of the diffraction orders outputted from the scattering medium.

[0073] In one embodiment, the 0thorder performs brightfield imaging and +1 orderperforms edge detection. In another embodiment, the -1 order performs ( e . g . ,G a u s s i a n ) blurring, t h e 0th order performs brightfield imaging, a n d t h e+1 order performs edge imaging for a wavelength of 3.5 um (see Figure 3). In yet another embodiment, a horizontal edge image is formed on the 0thorder and a  vertical edge image is formed on the +1 order.

[0074] In another embodiment, high- or low-pass filters of different spatial scales are encoded on each diffraction order for parallel multiscale feature extraction.

[0075] Different filtering operations on different polarizations of the electromagnetic radiation.

[0076] For example:

[0077] High-pass (or edge detection) filter for p-polarized light while rejecting s- polarized light for all angles of incidence (see e.g., Fig.6, Fig.10, Fig.20). This produces directional edge detection when illuminating the object with a linear polarization.

[0078] High-pass (or edge detection) filter for s-polarized light while rejecting p- polarized light for all angles of incidence (see e.g., Fig.15). This produces directional edge detection when illuminating the object with a linear polarization, but the edge direction is orthogonal to that in the previous case since the s and p responses are reversed.

[0079] Low-pass (or blurring) filter for p-polarized light while maximally transmitting s-polarized light for all angles of incidence (see e.g., Fig.7, Fig.14). This produces directional blurring when illuminating the object with a linear polarization, which has not been demonstrated before.

[0080] High-pass filter for both s- and p-polarizations (see e.g., Fig.8, Fig.11, Fig. 12). This produces isotropic edge detection for any incident polarization of light.

[0081] Analogous filtering procedures could be implemented using left and right circularly polarized electromagnetic radiation.

[0082] For two (or more) distinct wavelengths, a different transfer function is encoded on each wavelength.

[0083] In one embodiment, the scattering medium is patterned as a high-pass filter for p-polarization while rejecting s-polarization for all angles near wavelength λ  =4.65um; and as a low- pass filter for p-polarization while transmitting s- polarization for all angles near wavelength λ =4.35um (Fig.9). This is a multifunctional metasurface that performs edge detection at one wavelength, while blurring at another wavelength.

[0084] These multifunctionalities are novel and have not been demonstrated before. It is possible to extend these multifunctionalities by designing distinct filters for more than two wavelengths while also changing the polarization behavior.

[0085] In the most basic embodiment, the device consists of a single patterned layer, but in general a device may consist of multiple patterned layers separated by un-patterned spacers layers. Multi- layered devices provide improved performance and enable the design of increasingly complex transfer functions. The stacked layers are designed as a whole to produce the desired operations, typically more complex than what can be achieved with a single patterned layer. More layers provides more degrees of freedom to better minimize the figure of merit.

[0086] While the metasurfaces in the working embodiments were designed to operate in the mid-infrared wavelength range, near 4.5 micrometers, the devices can be designed to operate at different wavelengths. However, there are a variety of applications for mid infrared wavelengths in sensing, imaging, and metrology. Furthermore, the longer wavelength relative to visible or near infrared light makes it easier to fabricate devices since the feature sizes can be larger, making them more robust to fabrication errors.

[0087] First Example

[0088] In one embodiment, the scattering medium consists of a thin, patterned layer of silicon on a fused silica substrate. The silicon layer is patterned with a unit cell tiled periodically in a 2D array and the period and thickness of the silicon layer are design parameters.

[0089] Fig.1B shows an SEM image of an example fabricated inverse-designed device performing edge detection only on the zeroth diffraction order. The device  consists of an approximately 1 micron thick film of silicon (Si) on a sapphire substrate (although other substrates, such as silicon, could be used). In other embodiments, other substrates such as, but not limited to, silicon or fused silica maybe used.An SiO2 etch stop layer may be added when using a silicon substrate tobetter control the thickness of the patterned layers. The Si film was patterned with a periodic structure, determined by the optimization algorithm, using electron beam lithography and plasma etching.

[0090] The device permittivity was selected using an inverse design method wherein Autograd (https: / / github.com / HIPS / autograd, https: / / github.com / weiliangjinca / grcw

[0017] ) efficiently computed the gradient of the loss function L reproduced above

[0091] to determine how to adjust permittivity. The method of moving asymptotes was used to optimize permittivity of unit cell, and C4 and x / y reflection symmetries were enforced in the unit cell. Fig.1M shows the progression of the loss function and the binarization function.

[0092] Polarization-dependent imaging results from this device are shown in Fig. 2, clearly showing that edges are highlighted. The algorithm used to simulate the imaging is in section 7.

[0093] Second Example (

[0016] )

[0016] : P. Pearson, G. Roberts, and A. Faraon, "Inverse-designed metasurfaces for multifunctional spatial frequency filtering," Optica 12, 1090-1099 (2025).

[0094] Design of a Metasurface as Spatial Frequency Filter

[0095] In a traditional 4^^ system, filtering can be realized by placing a mask in the Fourier plane to modify the spatial frequency content of the incident optical field. However, this configuration is simplified by designing a metasurface whose scattering properties have an angular dependence to modulate the corresponding angular spectrum components. Inserting such a metasurface into the optical  system imparts an identical transformation to that of a 4^^ system. Consider the angular spectrum representation of the electric field over a plane in two dimensions for a single polarization [1]:

[0096] ^^^^^,^^^ ൌ ^^ ^ ି^    ^ ି^    ^̂^൫^^௫ ,^^௬൯exp ^^^൫^^௫^^ ^ ^^௬^^൯^d^^௫ d^^௬, ^1^, its effect is to add a factor of െ^^௫ଶin the integrand, implying that the transmissioncoefficient or amplitude transfer function (ATF) must be of the form ห^^൫^^௫ , ^^௬൯ห ∝^^௫ଶ.

[0098] More generally, a matrix-valued transfer function ^^, whose elements are the ATFs corresponding to transformations applied to co- and cross-polarized rays impinging on the metasurface: ^^^^, out ൫^^௫, ^^௬൯ ^^^ ൌ ^ ^^൫^^௫, ^^௬൯ ^^^^൫^^௫, ^^௬൯ ^^ ൫^^ , ^^ ൯^ ^^, in ௫ ௬ , ^2^^^ ^^^ ^|^^^. In this example, the aim is to design metasurfaces that encode isotropic ATFs to modify the spatial frequency content of the fields incident upon them. The final image thus depends on the combination of the object's polarization state and the ATFs associated with each matrix element in Eq. (2). The provides a more detailed description of the image formation model and examples of simulated images. We will demonstrate how topology optimization can be used to target a variety of transfer functions with different polarization behaviors that enable blurring, edge detection, and multifunctional processing.

[0101] Inverse Design

[0102] Topology optimization has emerged as a powerful technique for designing a variety of optical devices. It provides a flexible platform for realizing functionality that is difficult to achieve with rational design approaches based on  simple scatterer geometries. To design metasurfaces for spatial frequency filtering, we seek to minimize the figure of merit ^FoM^ℒ : ℒൌ^∑^^^^ ^^൫^^^ ,^^^ ,^^^ ,^^^൯^ ^^^ୟ୰^^^൫^^^ ,^^^ ,^^^ ,^^ ൯

[0103] ே^ଶazimuthal angles^^^^^, and polarization states^^^^^.^^ is the total number of theseparameter combinations included in the optimization, and ^^ serves as a fixed hyperparameter specifying the relative weight of each combination. In all optimizations, more weight is given to transmission at the smallest and largest polar angles to increase the contrast between the angular stop and pass bands, as well as to improve the overall transmission efficiency. A full breakdown of the optimization parameters can be found in Table 1. The design process is summarized in Fig.5A. To minimize the FoM, the method of moving asymptotes (MMA) implemented in the open-source NLOpt package and the gradient of the FoM is obtained via automatic differentiation [47,48]. To ensure uniform filtering across the device's aperture, we limit ourselves to periodic metasurfaces and use differentiable rigorous coupled-wave analysis (RCWA) to simulate the metasurfaces, allowing gradients to be computed without an explicit adjointsimulation [26,49]. The unit cell is discretized into a 100 ൈ 100 grid, and at eachiteration, its permittivity is mapped to a density variable ^^ ∈ ^0,1^, followed bythe application of a Gaussian blur and a sigmoidal projection filter to control the minimum feature size and ensure binarization, respectively ^16,36,50^. The projection filter takes the following form:

[0105] ^^^^^, ^^^ ൌ^ୟ୬୦ ^ఉఎ^ା^ୟ୬୦ ^ఉ^ఘ^௫,௬^ିఎ^^ ^4^dilation of the pattern. The width of the Gaussian blur kernel is kept fixed, while the sigmoid filter becomes sharper throughout the optimization (by gradually  increasing the parameter ^^ ) to binarize the structure. Furthermore, the unit cell is forced to have ^^ସ,^^௫, and ^^௬symmetries to increase the isotropy of the ATFs and reduce the number of simulations required. With this symmetry, we only need tosimulate azimuthal angles in the range ^0∘, 45∘^, and the transmission coefficientsof ^^ - and ^^-rays are restricted to be identical at normal incidence. All of these constraints are differentiable, which makes it straightforward to obtain gradients of the FoM with respect to the unit cell's permittivity. More generally, other symmetries can be imposed on the unit cell's structure, including the absence of any symmetry whereby the unit cell is fully freeform. This could be useful for implementing dichroic responses and polarization conversion, for example.

[0107] 3. Results

[0108] Polarization-Selective Filtering

[0109] The first aim was to design metasurfaces that have different transfer functions for ^^ - and ^^-polarizations. Edge detection is targeted only for ^^- polarized rays with no cross-polarized components. The transfer function takes the form:

[0110] ^^ ൌ ^^^^^൫^^௫ , ^^௬൯ 0൨ ^5^while all other polarizations are reflected. Similar transfer functions have been demonstrated in [5-7], where it was shown that, when illuminating with linearly polarized light, the edge orientation is determined by the orientation of the incident polarization. The ^^-polarized light was chosen arbitrarily, but the same procedure can be applied to perform the filtering operation only on ^^-polarized light, as illustrated in Fig.14. Furthermore, the design approach can be extended to target not only ^^^^ states, but also circularly polarized and ^^^^ states oriented inthe lab frame. The symmetry of the unit cell forces ^^^^ ൌ ^^^^ at normal incidence,but by lifting the symmetry, it is possible to design a device with distinct  transmission coefficients at normal incidence for orthogonal polarizations.

[0112] The metasurface's lattice constant is 5^^ m, slightly larger than the targetwavelength of ^^^ ൌ 4.65^^ m. While it is possible for diffraction orders topropagate, we find that they have minimal transmission within the NA of our imaging system based on RCWA calculations (Figs.16-19). It is also possible to explicitly suppress transmission into higher diffraction orders during the optimization as in

[0016] , but we find that this is not strictly necessary in practice.The metasurface consists of a 1^^ m (or ∼ ^^^ / 5 ) layer of crystalline silicon (c-Si)on a 500^^ m sapphire substrate (MTI Inc). The thickness has been chosen based on commercial availability and empirical experimentation to produce good optimization results while remaining thin enough to allow plasma etching of the smallest features. We assume the refractive index of silicon and sapphire to be 3.48 and 1.65, respectively [51,52]. The unit cell permittivity is optimized to produce the following transmission amplitude target for ^^-polarized light:

[0113] ห^^^^^^^,^^^ห ൌ 1 െ exp ൫െ^^ଶ / ^^ଶఏ൯ ^6^(6), ^^ఏsets the width of the angular rejection band, and the transformation^^^௫, ^^௬൧ ൌ ^^^sin ^^^^^cos ^^^^, sin ^^^^^ has been applied to express theoptimization problem in terms of polar and azimuthal angles. We choose this form of the transfer function to realize a general high-pass filter, not necessarily the Laplacian operator that was described above. It is possible to modify the optimization target to be a Laplacian if desired, such that all light within the NA is filtered accordingly. Equation (6) also naturally lends itself to enforcing high transmission at angles beyond ^^ఏto maximize the edge signal. A larger ^^ఏwill transmit higher spatial frequency components to produce an image with finer edge features, albeit with lower intensity. With this consideration, we choose ^^ఏto be relatively small to produce prominent edge features. Optimization details for all devices presented in this work can be found in section 9.

[0115] The optimized unit cell permittivity and a scanning electron microscope  (SEM) image of the fabricated structure are shown in Figs.6A and 6B. Fig.6C shows the simulated transmission coefficient amplitude obtained via RCWA for^^-polarized light with a free-space wavelength of ^^^ ൌ 4.5^^ m up to a NA of 0.65. Evidently, ห^^^^ห is highly isotropic and agrees well with the targeted transmission function. incidence, the transmission amplitude is 0.03 , and it peaks at 0.88 nearkey feature of this design is that the average transmissionamplitude for ^^-rays is approximately 0.8 for 0.2 ^ ^^௫,௬ ^ 0.6, which means thathigh spatial frequencies are transmitted efficiently and contribute to a stronger edge signal. Furthermore,|^^^^|remains low for all ^^௫,௬points (see Fig.20), ensuring significantly weaker transmission of the ^^-polarized signal. Indeed, the average transmitted power for ^^-rays within an NA of 0.65 is 16%.

[0116] To measure the transmission coefficient, polarized, collimated light wasweakly focused onto the device with a ^^ ൌ 150 mm Si lens. The light transmittedthrough the device was collected by a ^^ ൌ 100 mm Si lens and directed to athermopile photodetector (ThorLabs). The sample was mounted on an automated rotation stage to set the incident angle while a tunable mid-IR quantum cascade laser (MIRcat, Daylight Solutions) scans the incident wavelength. Absorption in the sapphire substrate was accounted for by taking reference measurementsthrough it at all ^^^, ^^^ pairs for each polarization. Fig. 13A is a diagram of themeasurement setup. Figure 6D shows the simulated and experimentally measuredtransmission coefficient amplitudes along the ^^ ൌ 0∘ azimuth as a function ofincident polar angle and wavelength. The measured and simulated results are in agreement, aside from an overall approximate 2% blue shift of the measured spectra, attributed to pattern erosion during the fabrication process. Fig.6G showsan additional simulation and measurement of angle-resolved ห^^^^ห along the ^^ ൌ45∘azimuthal direction for further confirmation that the metasurface is working as intended. The ^^-polarized transmission coefficient amplitude is also measured and shown in Fig.20, confirming that this polarization component is mostly rejected. In the measured spectra, there is a weak ripple due to leaky Fabry-Perot  modes of the sapphire substrate. Its amplitude is reduced by adding an SiOଶanti- reflective coating to the sapphire's backside following Si patterning (see Section5). The refractive index of SiOଶ at ^^^ ൌ 4.5^^ m is around 1.36

[0053] .

[0117] The functionality was verified by inserting the metasurface into an imaging system, shown in Fig.13B and imaging a 260^^ m-wide square onto anInSb sensor (FLIR A6700) with a ^^ ൌ 12.7 mmZnSe aspheric lens. In order toensure that all scattered rays were filtered correctly, the metasurface was fabricated over a 1 mmଶarea and placed within 500^^ m of the imaging target. As discussed in

[0018] , the distance between the target and the metasurface does not affect the filtering operation, but as the distance increases, the metasurface's area must scale such that all scattered rays within the NA go through the device. In the left panel of Fig.6E, a brightfield image with no device present is shown captured with anincident wavelength of ^^^ ൌ 4.64^^ m. With the device in place between theimaging target and lens, sharp vertical and horizontal edges were observed when illuminating with ^^ - and ^^-polarized light, respectively. This behavior is expected because the ^^-rays are rejected by the metasurface for all angles, while only ^^-rays scattered away from normal incidence are transmitted. Fig.6F illustrate line cuts through the vertically polarized and brightfield panels of Fig. 6E showing the relative intensity of the filtered image. Although the edges seem faint in comparison to the brightfield image, this is the expected behavior and is demonstrated by comparing the filtered line cut to one from a simulated image using the simulated ATFs of the metasurface [Fig.6F right panel; simulated image shown in Fig.10]. To compare the simulated and experimentally measured line cuts directly, the measured image was normalized to the average intensity in the center of the brightfield image scaled by the incident power of the laser. The simulated image is normalized to an input power of unity. There is good agreement between the simulated and measured images, which quantitatively confirms that the metasurface is operating near its theoretical efficiency.  Additional noise in the measured images stems from stray light in the optical system due to reflections from the custom imaging target made of chromium (Cr). The efficiency of the devices was compared to that of the polarization-selective metasurface presented in [7] in terms of the ^^avgmetric described therein, which is the edge intensity normalized to the input image intensity. This is a useful metric because it accounts for the overall edge transmission efficiency agnostic to the exact form of the ATF. The peak intensity of the line cuts in Fig.6F corresponds to ^^avgfor our metasurface and is near 7% (both in simulation and experiment), while in [7], it is less than 2%. Our device achieves such high efficiency due to its narrow angular stop band and large transmission up to an NA of at least 0.65, enabling more spatial frequency content to be collected by the imaging system.

[0118] As a complement to edge detection, a polarizations elective low-pass, or blurring, filter was designed. In this case, higher spatial frequency components of the ^^-polarized field were rejected, while the ^^-polarized field is fully transmitted for all angles of incidence, such that the transfer function is

[0119] ^^ ൌ ^^^^^൫^^௫ ,^^௬൯ 0൨. ^7^polarization's orientation. The target transmission is a Gaussian function defined by

[0121] ห^^ ଶ ଶ^^^^^,^^^ห ൌ exp ൫െ^^ / ^^ఏ൯ ^8^band, such that reducing ^^ఏleads to a stronger blurring effect. As with our definition of the transmission target for the high-pass filter, Eq. (8) is suitable to realize devices that retain their target transmission at large angles of incidence. Compared to the work presented in [2] where the low-pass functionality was achieved in reflection, the metasurface in this example was designed to operate in transmission to enable easier integration into optical systems. Gaussian filters can  apodize coherent imaging systems to reduce ringing artifacts introduced by the sharp spatial frequency cutoff at the imaging system's NA

[0054] . It has also been shown that apodization can reduce the sensitivity of coherent imaging systems to aberrations

[0055] . Owing to the ability of metasurfaces to manipulate the phase and amplitude of light, it may be possible for a metasurface to simultaneously apodize and correct aberrations in an optical system

[0056] .

[0123] The device was designed to operate near ^^^ ൌ 4.5^^ m. By updating thetarget transmission function, the optimization procedure generates the unit cell permittivity shown in Fig.7A, and an SEM image of the fabricated device is shown in Fig.7B. The simulated transmission amplitude for ^^ - and ^^-polarized light is shown in Fig.7C as a function of ^^௫and ^^௬at 4.56^^ m, demonstrating an isotropic low-pass filtering behavior for ^^-rays. A comparison to the target transmission amplitude is presented in Fig.22 and shows good agreement. The simulated transmission coefficients are also shown in Fig.7D across wavelength and incident angle. Te fabricated device operates as expected as evidenced by the measurement of its transmission coefficients ห^^^^ห and|^^^^|, as shown in Fig.7E.Fig. 14 shows measured and simulated transmission spectra along the ^^ ൌ 45∘azimuth, showing very good agreement between the simulated and experimentally measured transmission spectra, indicating high-fidelity fabrication. To further characterize the low-pass functionality, Fig.7F shows captured images of a square imaging target when illuminated with ^^ - and ^^-polarized light in the left and right panels, respectively, with a wavelength of 4.56^^ m. With ^^-polarized illumination, the vertical edges are preserved while the horizontal edges are strongly blurred, a consequence of the ATF's polarization selectivity. Conversely, with ^^-polarized light, the blurring direction is horizontal. More generally, the blurring direction can be tuned continuously by varying the orientation of the incident linear polarization.

[0124] The polarization-dependent blurring was quantified by experimental measurement of the contrast from a series of line array by custom fabrication of an  imaging target with sets of line arrays of various periods. Images of horizontal lines were captured with and without the metasurface for orthogonal linearpolarizations. The contrast, defined as ^^ ൌ ^^^max െ ^^min ^ / ^^^max ^ ^^min ^, iscomputed from background-subtracted images and plotted in Fig.7(g) as a function of line array spatial frequencyspace wavevector's magnitude). With ^^-polarized light, the line contrast is comparable to that of the ^^-polarized and brightfield cases for small spatial frequencies but drops off dramatically for line arrays with smaller pitch. The contrast trend qualitatively agrees with the simulated ATFs and quantifies the effect of the metasurface within the imaging system. In theory, these curves should match a cut through the simulated ATFs, but in practice, they differ for a few reasons. First, there are aberrations in the optical system from using a single aspheric lens for imaging. Second, stray light propagates because of reflections from the Cr imaging target and from the metasurface. Coherent illumination leads to interference effects that magnify the detrimental effects of stray light, such as ghost images. Finally, the lines consist of sharp edges, so the measured contrast does not only correspond to a single spatial frequency. These additional components add coherently and produce artifacts that cause deviations from the simulated ATFs. To excite only a single spatial frequency, a continuously varying amplitude grating would be necessary, which is difficult to fabricate. Despite these factors, the curves in Fig. 7(g) clearly demonstrate the directional contrast reduction introduced by the metasurface.

[0125] b. Polarization-Independent Filtering

[0126] The above sections considered polarization-selective devices that filter ^^ - or ^^-polarized light; in many cases, polarization independent filtering is desired,such that ^^^^൫^^௫, ^^௬൯ ൌ ^^^^൫^^௫ ,^^௬൯. It is difficult to achieve this performancewith rationallybecause it generally requires precise dispersion engineering [6]. To demonstrate this behavior, a metasurface was  optimized such that it encoded high-pass filtering for ^^ - and ^^ polarizations using the same target function as the device in Fig.6. The transfer function in this case is ^^^^൫^^௫, ^^ ൯ 0

[0127] ^^ ൌ ^ ௬ ^ . ^9^0 ^^^^൫^^ , ^^. thickness of 1^^ m was used as for the devices of the above section, targeting awavelength of ^^^ ൌ 4.5^^ m. The resulting unit cell permittivity and correspondingSEM of the fabricated device are shown in Figs.8A and 8B, respectively. The simulated ATFs for ^^ - and ^^-polarization both exhibit highpass behavior, in contrast to the polarization-selective devices [Fig.8C]. The high spatial frequency content is transmitted efficiently, and the ^^-polarized transfer function is almost perfectly isotropic. For the ^^-polarization, there is some variation in the azimuthal direction, meaning that the edge contrast along certain azimuths will be slightly reduced. In both cases, the transmission efficiency remains high up to a large NA. Simulated and experimentally measured angle-resolved transmission spectra are in excellent agreement for both polarizations [Figs.8D and 8E].

[0129] Images of a Caltech logo captured at ^^^ ൌ 4.55^^ m (slightly red-shiftedcompared to the design wavelength due to fabrication errors) are shown in Fig.8F and confirm that the metasurface extracts high-contrast edge information for vertically and horizontally polarized illumination. These results highlight that, despite azimuthal variation in the ^^-polarized ATF, the image is not noticeably affected, an effect that has been observed elsewhere [5,6]. Brighter spots appear in the measured images where two edges meet, which is expected based on image simulations shown in Fig.11. Horizontal line cuts through the images in Fig. 8G show the similarity for both polarization states. Assuming no cross-polarized contribution, we expect the efficiency to be similar to that of the polarization- selective device because ^^ - and ^^-rays contribute to orthogonal edges rather than  doubling the efficiency of any individual edge. Indeed, Fig.12 shows simulated images of a square object along with a line cut through it, demonstrating efficient transmission of edges with comparable intensity to those from Fig.6. Compared to the polarization-independent metasurface in [6] with simulated edge efficiencies below 3% for vertical edges, this device transmits edges with an efficiency of above 10%. Our inverse design approach enables the discovery of structures with a narrow angular pass band and high efficiency transmission, such that strong edge features are produced.

[0130] c. Multifunctional Filtering

[0131] The above sections demonstrate filtering functionality whose polarization response can be readily altered by reformulating the desired transfer function elements embedded within Eq. (3). In this section, the ability to extend the functionality of these devices by targeting unique, isotropic ATFs for differentwavelengths of light is explored. Near ^^^ ൌ 4.65^^ m, the metasurface performsedge detection for ^^-rays while suppressing transmission of ^^-rays. Conversely,near ^^^ ൌ 4.35^^ m, a low-pass ATF is targeted for ^^-rays with ^^-rays fullytransmitted. Essentially, the two functionalities of the devices shown in Figs.6 and 7 were combined into a single metasurface targeting those ATFs at distinct wavelengths. Specifically, the transfer function now takes the form: ^^ ൫^^ ,^^ , ^^൯ 0

[0132] ^^ ^ ^^ ௫ ௬ ^ , ^10^optimized simultaneously: three centered around 4.375^^ m following Eq. (8) and three centered around 4.675^^ m following Eq. (6). The optimized unit cell and an SEM image of the fabricated device are shown in Figs.9A and 9B, and RCWA calculations of the transmission amplitudes are shown in Fig.9C. The measured transmission of the fabricated metasurface is shown in Fig.9D and agrees closely  with the simulations. The optimization algorithm converges on a structure hosting two resonances that cross near the Γ-point near 4.5^^ m to achieve the desired spectrally multiplexed transfer functions. Simulations also show that the ATFs at these wavelengths are almost perfectly isotropic, meaning that the filtering action is virtually identical for all azimuths [Figs.9(e) and 9(h)].

[0134] With 4.37^^ m light, a polarization-selective blur was observed, as shown in Fig.9F, which shows images of a Caltech logo. With ^^-polarized light, the contrast is reduced in the horizontal direction, as demonstrated in the line cuts presented in Fig.9G. Illuminating with orthogonal polarization decreases the contrast along the vertical direction instead. Thus, in Fig.9(g), the contrast for the ^^-polarization and the brightfield case are similar. The ^^ polarized pass band of the optimized metasurface is wider than was desired, leading to a weaker blurring effect, yet it still approximates a Gaussian function and attenuates higher spatial frequencies. Because of this, a smaller object was used for a clearer demonstration of the blurring effect. The measured contrast curve is shown in Fig.21, showing that the contrast is reduced for higher spatial frequencies for ^^-polarized light, as expected. In Fig.9(i), images of the Caltech logo are captured with both ^^ - and ^^-polarizedillumination at ^^^ ൌ 4.7^^ m. With ^^-polarized light, horizontal edges arehighlighted, while with ^^-polarized light, vertical edges are highlighted. In the experiment, we use 4.7^^ m light because of the slight red shift of the transmission spectra relative to the RCWA calculations.

[0135] 4. Possible Modifications and Variations

[0136] The results presented for this example demonstrate multifunctional inverse-designed metasurfaces for spatial frequency filtering at mid-IR wavelengths. These devices control transmission for several angles of incidence, wavelengths, and polarizations to realize high-efficiency and isotropic ATFs that preserve their target transmission for large NAs. In the experiments, the  metasurface was inserted immediately behind the object for convenience. This configuration limits the filtered field of view to the size of the metasurface and potentially restricts the object to be back-illuminated. However, a principal feature of the metasurfaces illustrated herein is their potential to be placed immediately above an image sensor to realize compact form factors

[0018] . As light passes through the imaging lens, the object is magnified and the spatial frequencies are scaled accordingly. Thus, to design an image-processing metasurface placed directly above an image sensor, the spatial frequencies of the targeted ATF in the optimization must be scaled to be commensurate with those of the magnified image. A potential challenge in this regard is increasing the metasurface aperture to cover the entire image sensor, which can be on the order of 1 cmଶ. In principle, electron beam lithography (EBL) can be used, but the writing times become long and field stitching errors must be mitigated, which limits the practicality of mass production (although in principle it can be practically achieved). Alternatively, techniques such as deep UV projection lithography or nano-imprint lithography (NIL) can be utilized, both of which have been used to fabricate large-area metasurfaces [57-59]. To fabricate inverse-designed structures in a commercial semiconductor foundry, additional design rules must be strictly enforced [60,61].

[0137] Even though, in principle, the effect of a 4^^ system with any linear Fourier plane mask can be realized with a single metasurface filter to reduce bulkiness, there are practical limitations to this approach. Importantly, the filtering is determined by the dispersion relation of the resonant modes, such that it may be difficult to implement extremely narrow pass or stop bands corresponding to sharp angular responses. Furthermore, because low-Q modes typically have broad angular features, realizing filters with narrow spatial frequency stop or pass bands over a wide spectral range is particularly challenging for metasurfaces and may be better suited to a Fourier plane mask in a 4^^ system. Metasurfaces will excel in situations where wavelength- and polarization-dependent filtering is required. Here, we have shown a multiwavelength metasurface with two targeted ATFs, but  it is possible to increase the number of multiplexed ATFs in the optimization. Having said that, different transfer functions cannot be realized within an arbitrarily narrow spectral band and the spectral band must be selected as a function of the transfer function being implemented.

[0138] Targeting the mid-IR spectral band benefits the designs in this example, as the longer wavelength enables inverse-designed structures to have deeply sub- wavelength features that can be readily fabricated via EBL (see Section 5). Furthermore, there are many applications in the mid-IR wavelength band for sensing, imaging, and metrology [63-65]. Accessing this feature size regime is helpful for realizing high-performance metasurfaces. In principle, however, these structures can be scaled to any target wavelength, and with EBL systems capable of writing features on the order of tens of nanometers, there is potential for demonstrating inverse-designed metasurface filters at near-IR and visible wavelengths. Because the refractive index of silicon is similar in the telecom range as in the mid-IR, the current designs could be directly scaled down to operate at these wavelengths. In the visible range, transparent materials such as titania, gallium nitride, or silicon carbide could be used while accounting for their respective indices of refraction in the optimizations.

[0139] The capacity of this platform to realize multifunctional devices may be useful for next-generation imaging systems, where different spatial-frequency bands of the object can be accessed by modulating the wavelength or polarization of light. When combined with spectral color splitters, the decomposed spatial frequencies could be extracted in a single shot

[0036] . In the context of accelerating machine vision tasks with analog processing, the metasurfaces presented in this example can replace computationally intensive 2D convolutions, and they may become particularly useful in low-power, resource-constrained settings.

[0140] As described above, the multifunctional metasurfaces can be designed for filtering spatial frequency content not only based on wavelength, but also based  on diffraction order, which is used for multi-channel imaging. Furthermore, while this example targeted transfer functions that modify the amplitude of light, it is also possible to realize metasurfaces with tailored angle-dependent phase for other functionalities such as aberration correction and compression of free-space propagation [66-68]. Multilayer patterned metasurfaces could enable these complex functions, potentially leading to the miniaturization of optical systems and efficient information processing.

[0141] 5. Fabrication method

[0142] The devices in this example were fabricated on commercially bought crystalline Si epitaxially grown on a sapphire substrate (MTI Inc). The Si and sapphire thicknesses were approximately 1^^ m and 500^^ m thick, respectively. The sample was cleaned by sonicating in acetone and rinsing with isopropyl alcohol, followed by oxygen plasma. After baking the sample on a hotplate at 180∘C, ZEP520A (Zeon Corporation) was spin-coated on the silicon surface. The sample was then baked again to remove the solvent from the resist. Next, aquaSAVE (Mitsubishi Chemical Group) was spin-coated on top of the resist to act as a conductive charge dissipation layer. The pattern was written with an EBPG 5200 (Raith) EBL system. Following exposure, the aquaSAVE was removed by immersing the sample in DI water. The resist was developed in ZED developer (Zeon Corporation) and rinsed with MIBK and IPA. Subsequently, a 20 nm thick alumina hard mask was evaporated onto the sample and lifted off with Remover PG (Kayaku). The pattern was transferred to Si with a pseudo-Bosch etch using a mixture of SF^and CସF଼gases. To remove the hard mask, the samplewas immersed in a 1: 1 mixture of NHସOH and HଶOଶ. An additional 840 nm layerof SiOଶwas deposited via PECVD on the backside of the sample, which acted as an anti-reflection coating.

[0143] 6. Calculation methods

[0144] All optimizations and RCWA calculations were performed on a high- performance computing cluster. At each iteration, the transmission for every combination of wavelength, angle, and polarization is simulated in parallel with eight Intel CPU cores (Skylake, Cascadelake, and Icelake processors).

[0145] 7. Image simulation

[0146] A similar procedure to that in ^1,2^ was used to simulate the images captured with the spatial frequency filtering metasurfaces in the imaging system.First, consider an object of the form ^^^^^^^,^^^ ൌ ^^^^,௫^^^,^^^^̂^ ^ ^^^^,௬^^^,^^^^̂^. Theobject's scattered light is a bundle of rays represented by:

[0147] ^̂^^^൫^^௫ ,^^௬൯ ൌ ℱ^^^^^^^^,^^^^ ൌ ℱ^^^^^,௫^^^,^^^^^̂^ ^ ℱ^^^^^,௬^^^,^^^^^̂^ ^^^1^wasa in the ^^^^ frame, the scattered field components were converted from the latter frame to the former. The transformation is encoded inthe following 3 ൈ 3 matrix ^^ :^^^ sin ^^^^cos ^^^^ sin ^^^^sin ^^^^ cos ^^^^^^௫^^ ^ ^^^^ ^^^^ ^^^^^ ^^azimuthal components of the scattered field in spherical coordinate, taking the ^^ - and ^^-polarized waves to be ^^థand ^^ఏ, respectively. There is no ^^-component in the input image and the longitudinal components of the scattered field in the ^^^^ frame are ignored such that: ^̂^cos ^^^^cos ^^^^ cos ^^^^^̂^ ^^^,^ ^ ^ sin ^^^^^ ^^,௫ ^is done through the transfer function matrix ^^ (defined above) whose elements are the individual transfer functions corresponding to co- and cross-polarized  transmission through the metasurface: ^̂^out ,൫^^ ,^^൯^^ ൫^^ ,^^ ൯ ^^ ൫^^ ,^^ ൯ ^̂^ ൫^^ , ^^ ൯

[0153] ^^ ௫ ௬ ^ ൌ ^^^ ௫ ௬ ^^ ௫ ௬ ^ in ,^ ௫ ௬ ^^^4^^̂^^ ^out൫^^,^^൯^^^^൫^^ , ^^ ൯ ^^^^൫^^ , ^^ ൯ ^̂^in ൫^^ , ^^ ൯function. Finally, the filtered scattered fields is transformed back to the ^^^^ frame and the inverse Fourier transform is taken to recover the real space image filtered by the metasurface: cos ^^^^cos ^^^^ െ^̂^ ൫^^,^^൯

[0155] ^^out ^^^,^^^ ൌ ℱି^ ^^ sin ^^^^ out ,^ ௫ ௬cos ^^^^sin ^^^^ cos ^^^^ ൨ ^ ^^^5^^̂^ ^^^^ห^^out ,௫^^^, ^^^ห ^ ห^^out ,௬^^^, ^^^หthe device sections 3.1 and 3.2 o are shown infor ^^ - and ^^-polarized illumination. To simulate these images, the complex transmission coefficients from the simulated ATF is used and the image simulation procedure described above is applied.

[0158] 8. S-polarization selective high-pass

[0159] Fig.15 illustrates a polarization-selective device similar to the ones characterized Section 3.1, but with the high-pass ATF applied to ^^-polarized rays rather than ^^-polarized. The simulated transmission shows an isotropic transfer function that reflects ^^-polarized light at normal incidence and transmits at larger angles of incidence.

[0160] 9. Optimization details

[0161] The unit cell permittivity is represented by a density variable ^^ ∈ ^0,1^obtained by rescaling the permittivity according to ^^^^^,^^^ ൌ ^^^^^^, ^^^ െ ^^min ^ / ^^^max െ ^^min ^, which serves as the optimization variable. The unit cell is discretized into a ^^ ൈ ^^ where ^^ ൌ 100 and density values are initialized bydrawing from a normal distribution with a mean of 0.5. The initial density is then convolved with a Gaussian kernel to remove high-frequency variations. At each iteration, the figure of merit (FoM) is derived from the transmission coefficients at each desired wavelength, polarization, and polar / azimuthal angle calculated via rigorous coupled-wave analysis (RCWA). To obtain the gradient of the figure of merit with respect to the densities, automatic differentiation is used such that no adjoint simulation is required. These gradients are passed to an optimizer (Method of Moving Asymptotes or MMA) that minimizes the FoM over several hundred iterations. The optimization is broken into epochs each consisting of tens of iterations. Across different epochs, the binarization strength and artificial loss (discussed below) are increased but held constant within an epoch. The weighting scheme mentioned in the main text is detailed in the following sections for each device. These weights are empirically determined and adjusted to guide the optimizer towards high-performance devices.

[0162] It has been shown previously that adding artificial loss in the simulation can prevent an optimizer from stuck in local due to many high-Q resonances in the initial random structure [3,4]. This additional loss is incorporated via the parameter ^^^^^which adds an imaginary part to the frequency of light in theRCWA simulation as ^^ ൌ ^^^ ^^1 ^ ^^ଶொೌ್ೞ ^ where ^^^ ൌ ^^ / ^^^ and ^^ is the speed oflight in vacuum.increases over the course of the optimization until it reaches its final value (see Table S1). At each epoch, the value of ^^^^^is increased exponentially. In the finalepoch, ^^^^^ ൌ ∞ such that the true physics of the device are captured.

[0163] In order to ensure a binary final device, a projection filter is applied to the permittivity at every iteration of the optimization [5]:

[0164] ^^^^^, ^^^ ൌ^ୟ୬୦ ^ఉఎ^ା^ୟ୬୦ ^ఉ^ఘ^௫,௬^ିఎ^^ ^^^6^  strength, respectively. During the optimization, ^^ is fixed at 0.5 while ^^ is increased at each epoch to produce a fully binarized structure. The minimum feature size is controlled by applying a Gaussian filter to the unit cell permittivity. The width of the kernel determines the approximate smallest feature size, but does not enforce this condition strictly.

[0166] Table 1. Optimization parameters for all devices

[0167] Table 1 summarizes the parameters used to optimize the various devices described in this example. The top row specifies the labels for each device.

[0168] The minimum feature size (MFS) specifies the width of the Gaussian kernel convolved with the unit cell permittivity at each iteration - it does not necessarily mean that there are no features with critical dimensions less than this. The ^^ parameter controls the strength of the binarization filter in Equation S6. The above table specifies ^^ in the first and final epochs. The static weighting is defined in Eq. (4). Typically more weight is attributed to the transmission values  at normal incidence and at the largest angle of incidence. Particularly for the high- pass ATFs, it is critical to reject as much light as possible near normal incidence and we empirically find that this weighting scheme is effective for finding high- performance devices. In general, it promotes structures that have high efficiency and strong contrast between the spatial frequency pass and stop bands. The remainder of Table 1 shows the exact combinations of polar angle, azimuthal angle, and wavelength used for optimizing each device as well as the specific form of their respective transmission goals.

[0169] 10. Simulation of non-zeroth diffraction order transmission (first four diffraction orders)

[0170] In principle, since the wavelength is comparable to the metasurface's period, light scattered from the object can couple to diffraction orders as it goes through the metasurface, which in turn can redirect it to a different angle and interfere with image formation. Of most concern is the possibility of light outside the NA of the optical system being scattered back into it through a diffraction order. Fig.6 shows there is good quantitative agreement between the measured and simulated images, which is a confirmation that the filtered image is not noticeably affected by light scattered through diffraction orders. Figs.16-19 show the calculated power transmitted into the first four diffraction orders for incident plane waves with different ( ^^௫,^^௬) components. In these plots, ^^௫,௬denotes the^^-vectors of the transmitted light. Thus, any light satisfying ^^^௫ଶ ^ ^^௬ଶ / ^^^ ^ ^^^^will fall within the NA of the optical system, and any light with ^^^௫ଶ ^ ^^௬ଶ / ^^^ ^ 1will be evanescent. These results confirm that there is minimal power being transmitted into the higher diffraction orders that could affect the imaging process. Moreover, the objects considered in this example primarily scatter light to small angles such that a negligible amount will couple to the diffraction orders of the metasurface in the first place.

[0171] 11. Vision applications for optical analog processing

[0172] When the image size grows [6,7], performing convolutions as an analog process becomes advantageous over digital processing. This is fundamentally because the fast Fourier transform (FFT) has a complexity of 2^^ଶlog ^^^^, where the image is ^^^^^^ pixels [8]. Thus, convolution, which requires at least two FFTs, can quickly become the most time-consuming part of the overall computational workload. Modern embedded processors can perform hundreds of billions of FLOPs per second (FLOPS) such as the NVIDIA Jetson series, but their power consumption is typically greater than 10 W , which precludes their use in low- power embedded vision systems aiming to operate well below the single-watt level [9, 10].

[0173] Considering a one-megapixel image sensor, each FFT may take on the order of tens of microseconds with a processor operating at 500 GFLOPS, much faster than the frame rate of a typical image sensor (tens of milliseconds). In this case, accelerating the FFTs with a metasurface does not bring much benefit, and the image sensor remains the latency bottleneck. However, when limited to low- power processors, such as those based on ARM Cortex M architectures, the processing speed is orders of magnitude slower, and thus the time to compute FFTs can be much greater than the sensor's frame rate. Significant effort has been put towards developing inference models that run on microcontrollers [11-14]. These systems aim to consume power on the order of a few milliwatts so that they can operate continuously via battery power. This can enable several tasks including gesture detection, object localization, object counting, and wake words, finding applications in industrial monitoring, mobile devices, robotics, and agriculture [11, 14]. In these cases, analog processing with metasurfaces can potentially enable real-time vision tasks with higher resolution than previously achievable. Furthermore, metasurfaces are compatible with compact form factors, an important consideration in embedded systems, and can use polarization, wavelength, and phase information to augment sensing capabilities. These  qualitative estimates provide an order-of-magnitude estimation of a potential application space. Even with a fast processor, analog processing can be advantageous if an event camera is used since they have equivalent frame rates orders of magnitude greater than most image sensors

[0015] .

[0174] Process Steps

[0175] In general, a method for making a device comprises using an inverse method to design a pattern (or otherwise obtaining the pattern) of spatially varying permittivity varying with sub-wavelength precision and configured to scatter the electromagnetic radiation so as to encode the operations in the electromagnetic radiation. Fig.23 illustrates the method of designing the scattering medium using an inverse design process, comprises the following steps:

[0176] Block 2300 represents (a) selecting an initial pattern for the spatially varying permittivity within the unit cell of the periodic structure;

[0177] Block 2302 represents (b) solving Maxwell’s equations for the pattern to obtain a field distribution or complex-valued transmission coefficient for all incident angles, polarizations, wavelengths, and diffraction orders needed to realize the desired filtering operations.

[0178] Block 2304 represents (c) using the field distribution or transmission coefficient to determine a figure of merit associated with each of the different spatial frequency filtering operations encoded in different wavelengths, polarizations, and diffraction orders, of the electromagnetic field (can be performed all in one step - there is one pattern, that may consist of several layers, that is designed via gradient optimization to produce the desired operations);

[0179] Block 2306 represents (d) calculating a gradient of the figure of merit with respect to the unit cell permittivity;

[0180] Block 2308 represents (e) using the gradient to adjust the pattern so as to minimize or maximize the objective function of the figure of merit; and

[0181] Block 2310 represents (f) iteratively repeating steps (b)-(f) until  convergence (minimization or maximization) to a desired level.

[0182] The fabrication can further comprise lithographically defining the pattern on a substrate (using a mask or otherwise).

[0183] Fig.24 illustrates a fabrication process for the scattering medium, comprising the following steps.

[0184] Block 2400 represents depositing photoresist on a substrate. As disclosed herein, example substrates include, but are not limited to silicon, silicon on sapphire, or silicon on quartz, or TiO2 (especially for visible wavelengths).

[0185] In general larger index contrast is better for the inverse design algorithm, but in principle other materials with lower contrast could be used, particularly in the case of a multi-layer structure where each layer needs to be encapsulated in another material. Combinations could be SiO2 / Si, TiO2 / air (useful for visible wavelengths where Si is not transparent), SiO2 / TiO2, or SU-8 / Si (SU-8 is a photoresist that can encapsulate Si). There are also mid-IR specific materials that could be used, although they are rather specialized like CaF2.

[0186] Block 2402 represents performing lithography to transfer the pattern. The lithography step can be performed with any suitable patterning technique such as e-beam, optical lithography, imprint lithography, UV projection lithography etc.

[0187] Block 2404 represents depositing the etch mask.

[0188] Block 2406 represents liftoff.

[0189] Block 2408 represents etching and removing mask leaving the scattering medium comprising a pattern of varying permittivity.

[0190] Fig.1H, Fig.5, 3A, Fig.3D, Fig.9, and Fig.25 illustrate examples of the resulting device. Examples include, but are not limited to, the following.

[0191] 1. A device useful for image processing, comprising:

[0192] a scattering medium 300, 900 patterned for angle dependent scattering of electromagnetic radiation encoding a plurality of different spatial frequency filtering operations on different diffraction orders (e.g., 0, + / - 1), polarization states (e.g., vertical or horizontal polarization, left or right circular polarization),  or wavelengths of electromagnetic radiation incident on the scattering medium from an object 100, 500 (see Fig.1H and Fig.5), so that an image (e.g., 112 in Fig.1H, or 2504, 2506 in Fig.25)) of the object formed from the electromagnetic radiation encoded by the scattering medium is processed by each of the different spatial frequency filtering operations in parallel.

[0193] 2. The device of clause 1, wherein the operations comprise convolutions.

[0194] 3. The device of clause 2, wherein the convolutions comprise at least two of blurring, edge detection, or two dimensional (2D) image differentiation.

[0195] 4. The device of any of the clauses 1-3, wherein the scattering medium is patterned such that the blurring and / or an edge of the object detected by edge detection are identifiable as variations in intensity in the image.

[0196] 5. The device of any of the clauses 1-4, wherein the scattering medium is patterned to scatter the electromagnetic radiation into the different diffraction orders such that each of the diffraction orders are encoded with a different one of the operations.

[0197] 6. The device of clause 5, wherein the operations further comprise brightfield imaging.

[0198] 7. The device of any of the clauses 1-6, wherein the operations comprise vertical edge detection encoded on one of the diffraction orders and horizontal edge detection on another of the diffraction orders.

[0199] 8. The device of any of the clauses 1-7, wherein the operations comprise a high pass filter and a low pass filter on different spatial scales and encoded on different ones of the diffraction orders.

[0200] 9. The device of any of the clauses 1-8, wherein the operations comprising different transfer functions are encoded on different ones of the wavelengths.

[0201] 10. The device of any of the clauses 1-9, wherein the image reveals spatial, spectral, and polarization features of the object processed by the operations in parallel without requiring computation in a computer.

[0202] 11. The device of any of the clauses 1-10, wherein the operations comprise a high pass filter performing edge detection for p -polarization (s-polarization) of the electromagnetic radiation while rejecting s-polarization (p-polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium, thereby performing directional edge detection when illuminating the object with a linear polarization of the electromagnetic radiation (see e.g., Fig.6, Fig.10, Fig.20, Fig.15).

[0203] 12. The device of any of the clauses 1-10, wherein the operations comprise a low pass filter performing blurring for p -polarization (s-polarization) of the electromagnetic radiation while maximally transmitting s-polarization (p- polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium, thereby performing directional blurring when illuminating the object with linearly polarized electromagnetic radiation (see e.g., Fig.7, Fig.14).

[0204] 13. The device of any of the clauses 1-10, wherein the operations comprise a high pass filter performing edge detection for p -polarization and s- polarization of the electromagnetic radiation, thereby performing isotropic edge detection when illuminating the object with any polarization of the electromagnetic radiation (see e.g., Fig.8, 11, 12).

[0205] 14. The device of any of the clauses 1-10, wherein the operations comprise:

[0206] a high pass filter performing edge detection for p -polarization (s- polarization) of the electromagnetic radiation while rejecting s-polarization (p- polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium at a first wavelength, thereby performing directional edge detection when illuminating the object with a linear polarization of the electromagnetic radiation having a first one of the wavelengths; and

[0207] a low pass filter performing blurring for p -polarization (s-polarization) of  the electromagnetic radiation while maximally transmitting s-polarization (p- polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium at a second one of the wavelengths different from the first wavelength, thereby performing directional blurring when illuminating the object with a linear polarization of the electromagnetic radiation at the second one of the wavelengths (see e.g., Fig.9).

[0208] 15. The device of any of the clauses 1-10, wherein the operations comprise a high pass filter performing edge detection for left circular polarization (right circular polarization) of the electromagnetic radiation while rejecting right circular polarization (left circular polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium, thereby performing directional edge detection when illuminating the object with a circular polarization of the electromagnetic radiation.

[0209] 16. The device of any of the clauses 1-10, wherein the operations comprise a low pass filter performing blurring for left circular polarization (right circular polarization) of the electromagnetic radiation while maximally transmitting right circular polarization (left circular polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium, thereby performing directional blurring when illuminating the object with circularly polarized electromagnetic radiation.

[0210] 17. The device of clause 1, wherein the operations comprise a high pass filter performing edge detection for left circular polarization and right circular polarization of the electromagnetic radiation, thereby performing isotropic edge detection when illuminating the object with any polarization of the electromagnetic radiation.

[0211] 18. The device of any of the clauses 1-10, wherein the operations comprise:

[0212] a high pass filter performing edge detection for left circular polarization (right circular polarization) of the electromagnetic radiation while rejecting right circular polarization (left circular polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium at a first wavelength, thereby performing directional edge detection when illuminating the object with a circular polarization of the electromagnetic radiation having a first one of the wavelengths; and

[0213] a low pass filter performing blurring for left circular polarization (right circular polarization) of the electromagnetic radiation while maximally transmitting right circular polarization (left circular polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium at a second one of the wavelengths different from the first wavelength, thereby performing directional blurring when illuminating the object with a circular polarization of the electromagnetic radiation at the second one of the wavelengths.

[0214] 19. The device of any of the clauses 1-18, wherein the scattering medium comprises one or more layers of the scattering medium patterned with a spatially varying (e.g., dielectric) permittivity 302, 902 varying with sub- wavelength precision and configured to scatter the electromagnetic radiation so as to encode the operations in the electromagnetic radiation.

[0215] 20. The device of clause 19, wherein the layers each comprise a silicon layer patterned with a unit cell 904 tiled periodically in a 2D array having a period and thickness defining the spatially varying permittivity.

[0216] 21. The device of any of the clauses 1-20, wherein the scattering medium comprises multiple layers patterned with subwavelength precision for encoding different ones of the operations.

[0217] 22. The device of any of the clauses 1-21, wherein the electromagnetic radiation comprises mid infrared wavelengths.

[0218] 23. A system 110 for performing image processing and comprising the  device of clauses 1-22, further comprising:

[0219] a camera or sensor comprising pixels 102;

[0220] a lens 104 or system of lenses positioned between the object 100 and the camera or sensor for imaging the electromagnetic radiation encoded by the scattering medium 106, 300, 900 onto the camera or sensor.

[0221] 24. The device of any of the clauses 1-24, wherein the scattering medium is an inverse designed structure optimized for encoding the plurality of the different operations.

[0222] 25. Fig.23 illustrates a method of designing the scattering medium of any of the clauses 1-24 using an inverse design process, comprising:

[0223] selecting an initial pattern for the spatially varying permittivity;

[0224] solving Maxwell’s equations for the pattern to obtain a field distribution or complex-valued transmission coefficient for all incident angles, polarizations, wavelengths, and diffraction orders needed to realize the desired filtering operations;

[0225] (c) using the field distribution or transmission coefficient to determine a figure of merit associated with each of the different spatial frequency filtering operations encoded in different wavelengths, polarizations, and diffraction orders, of the electromagnetic field;

[0226] (d)calculating a gradient of the figure of merit (or objective function of the figure of merit) as a function of the permittivity;

[0227] (e) using the gradient to adjust the pattern so as to minimize the figure of merit (or the objective function of the figure or merit); and

[0228] (f) iteratively repeating steps (b)-(f) until convergence (minimization) to a desired level.

[0229] 26. The method of clause 25, wherein the step (b) uses rigorous coupled wave analysis (RCWA), step (d) uses automatic differentiation, and step (e) uses the method of moving asymptotes (MMA) or any other gradient-based optimizer to minimize the

[0230] 27. A device, e.g., of any of the clauses 1-26, configured to get multiple transfer functions with tailored angular response in one device via multiplexing wavelengths, diffraction orders, polarization is novel here compared to other works

[0231] 28. The device of any of the clauses 1-27, wherein the exact angle dependence is what determines the filtering.

[0232] 29. The device of any of the clauses 1-28, wherein the surface area of the scattering medium (aperture) is scaled to greater than 1cm2, or has an area of the order 1mm2, or an area of at least 1 mm2

[0233] 30. The device of any of the clauses 1-29, wherein the exact definition of numerical aperture (NA) range is very application-dependent or is selected for the application.

[0234] 31. The device of any of the clauses 1-30, wherein the NA is in a range over which the metasurface approximates the target response well, i.e, retains either high or low transmission.

[0235] 32. The device of any of the clauses 1-31, wherein the NA having a different form of the angle response up to some NA.

[0236] 33. The device of any of the clauses 1-32, having high transmission up to large angles to maximize the edge signal and thus signal to noise ration (SNR).

[0237] 34. The device of any of the clauses 1-33, wherein the scattering medium is patterned to scatter the electromagnetic radiation into different polarizations (e.g., vertical or horizontal linear, circular) such that each of the polarizations are encoded with a different one of the operations.

[0238] 35. The device of any of the clauses 1-33, wherein the scattering medium is patterned to scatter the electromagnetic radiation into different wavelengths such that each of the different wavelengths are encoded with a different one of the operations.

[0239] 36. The device of any of the clauses 1-33, wherein the scattering medium is patterned to scatter the electromagnetic radiation such that each of  different polarizations (e.g., one or more of vertical, horizontal, left and right circular) are encoded with a different one of the operations.

[0240] 37. The device of any of the clauses 1-33, wherein the scattering medium is patterned to scatter the electromagnetic radiation such that each of the different wavelengths are encoded with a different one of the operations.

[0241] 38. The device of any of the clauses 1-37, wherein the scattering medium is patterned to scatter the electromagnetic radiation such that at least two of a wavelength, a polarization, or a diffraction order are encoded with a different one of the operations.

[0242] 39. The device of any of the clauses 1-38, wherein the scattering medium is patterned to scatter the electromagnetic radiation such that at least two of (a), (b), or (c), are encoded with a different one of the operations.

[0243] where

[0244] (a) is One or more wavelengths,

[0245] (b) is One or more polarizations,

[0246] (c) is one or more diffraction orders

[0247] Example Applications

[0248] The scattering medium described herein can be coupled to a camera (e.g., directly on or adjacent the pixels sensor, e.g., as shown in FIG.1H) in a variety of imaging systems:

[0249] An imaging system using the edge detection and / or brightfield imaging and / or polarization dependent imaging and / or blurring for gesture detection, object localization, object counting, or other computer vision application e.g.,. in industrial monitoring, mobile devices, robotics, agriculture, AR or VR system.

[0250] An imaging system using the edge detection and / or brightfield imaging and / or polarization dependent imaging and / or blurring for traffic sign recognition, lane departure warning, park assist, or surround view or other computer vision application in an autonomous vehicle navigation system.

[0251] An infrared imaging system using the edge detection and / or brightfield imaging and / or blurring for tracking eye movement (e.g., by identifying the boundary of the eye pupil or iris).

[0252] In some embodiments, the device may be most useful in resource- constrained environments, where there is no access to a fast processor. The large 2D image convolutions are typically the most intensive operations.

[0253] Advantages and Improvements

[0254] The rapid development of artificial intelligence models has spurred a dramatic increase in demand for data centers and high-performance computing clusters. However, far from these massive and power- hungry resources, there is a need to deploy inference models on the edge of the network where computing is limited for applications including robotics, mobile devices, industrial monitoring, and agriculture. Here, developing low-power, high-speed information processing is critical to enabling advanced sensing. In machine vision applications, 2D convolutions are often the first and most computationally intensive steps in processing pipelines. We have designed optical metasurfaces with tailored angular, spectral, and polarization scattering characteristics that, when inserted in an optical system, perform convolutions entirely in the optical domain such that they no longer consume computing resources, saving power and reducing latency. Compared to other work using metasurfaces for this purpose, we leveraged inverse design, a technique that frames the design problem as an optimization and allows us to realize devices with enhanced performance. We designed metasurfaces where different convolution filters were encoded on different wavelengths, polarizations, and diffraction orders to realize parallel processing operations that can reveal an object's spatial, spectral, and polarization features.

[0255] Definitions and Terminology

[0256] Blurring is a processing technique that smooths an image by reducing  high-frequency details (like sharp edges and noise) using an function as the weighting filter. In the case of Gaussian blurring, the function for the weighting filter is a Gaussian function. In one or embodiments, blurring is used to reduce noise prior to edge detection.

[0257] Image differentiation (e.g., edge detection) is an image processing technique used to highlight changes in intensity—in other words, it emphasizes edges, lines, and other sharp transitions in the image. In some embodiments, it is based on taking the derivative of the image’s intensity function with respect to spatial coordinates In some embodiments, it is the gradient of the derivative of the intensity function.

[0258] Edge detection is an image processing technique for finding boundaries (e.g., perimeter) between regions in an image where the intensity (brightness) changes sharply. It’s one of the most fundamental steps in computer vision and image analysis because edges often correspond to important structural features.

[0259] Brightfield imaging produces images with intensity-based contrast, so processing techniques like Gaussian blurring, image differentiation, and edge detection can be applied directly to analyze features, measure structures, or segment regions.

[0260] Polarization-dependent imaging is an imaging technique where the contrast or information in the image depends on the polarization state of the light used for illumination and / or detection. Instead of just recording light intensity, the system measures how the sample interacts differently with light of various polarizations — revealing structural, chemical, or stress-related properties that are invisible in ordinary brightfield imaging.

[0261] In image processing, high-pass and low-pass filters controlling which spatial frequencies in an image are kept and which are suppressed — essentially deciding whether you want to emphasize fine details or smooth areas. A Low- pass filter passes low spatial frequencies (slow changes in intensity) and  attenuates high spatial frequencies (rapid changes, fine details, noise). This may smooth or blur an image, or reduce noise, and remove sharp edges. A high-pass filter passes high spatial frequencies (rapid changes in intensity) and attenuates low spatial frequencies (smooth areas), to enhances edges and fine details - makes the image appear sharper (e.g., for edge detection).

[0262] Edge detection can be more general than just a quadratic dependence, can be anything that blocks transmission for small angles and has high transmission for large angles of incidence. The quadratic dependence is the special case of a Laplacian operator i.e., second order derivative.

[0263] Example Chip Package

[0264] Fig.25 illustrates an example packaged chip 2501 comprising the scattering medium 2500 formed on top of (e.g., on or above, in contact with, or coating) an array of pixels (e.g., CCD, image pixel array wherein each of the pixels comprise a photodiode). An integrated circuit IC 2502 (e.g., application specific integrated circuit or field programmable gate array) is used to process the electronic signals generated at each of the pixels in response to the electromagnetic radiation received from the target / object through the scattering medium, e.g., to output image processing (or image processed image) wherein operations are identifiable as variations in intensity in the image 2504, 2506 of the object. The image 2504, 2506 reveals spatial, spectral, or polarization features of an object or target processed by the operations in parallel without requiring further image processing computation in a computer (e.g., other than converting the electronic signals into intensity).

[0265] The image sensor array can be engineered to detect different polarizations by incorporating polarization-sensitive elements at the pixel level. For example, micro-polarizer filters can be patterned directly over individual pixels or subpixels so that some pixels only respond to horizontally polarized light, others to vertical, and still others to diagonal or circular states. Another approach is to exploit  anisotropic nanostructures, such as wire-grid polarizers or metasurfaces, that selectively transmit or scatter particular polarization components into designated pixels.

[0266] Similarly, sensitivity to different wavelengths or diffraction orders can be achieved by introducing wavelength-selective or angle-selective structures in front of the pixels or subpixels. Microscopic diffraction gratings, photonic crystals, or multilayer interference filters can be aligned with individual pixels or pixel groups to ensure that only a specific spectral band or diffraction order reaches them.

[0267] In one example, as described herein, the scattering medium can be inversely designed as a high pass filter (e.g., passing only high spatial frequencies for edge detection) on a first wavelength, polarization, or diffraction order, and as a low pass filter (e.g., passing only low spatial frequencies for blurring) on a second wavelength, polarization, or diffraction order. In this way, the integrated circuit can collect the electrical signals from the pixels to form data representing an edge detection of the target, and a blurred image of the target formed completely in the optical domain without computational filtering algorithms.

[0268] References

[0269] The following references are incorporated by reference herein. 1. H. Kwon, A. Cordaro, D. Sounas, et al., "Dual-Polarization Analog 2D Image Processing with Nonlocal Metasurfaces," ACS Photonics 7, 1799-1805 (2020). 2. M. Cotrufo, S. Singh, A. Arora, et al., "Polarization imaging and edge detection with image-processing metasurfaces," Optica 10, 1331 (2023). 3. W. Jin, W. Li, M. Orenstein, and S. Fan, "Inverse Design of Lightweight Broadband Reflector for Relativistic Lightsail Propulsion," ACS Photonics 7, 2350-2355 (2020).  4. X. Liang and S. G. Johnson, "Formulation for scalable optimization of microcavities via the frequency-averaged local density of states," Opt. Express 21, 30812 (2013). 5. F. Wang, B. S. Lazarov, and O. Sigmund, "On projection methods, convergence and robust formulations in topology optimization," Struct Multidisc Optim 43, 767-784 (2011). 6. S. Colburn, Y. Chu, E. Shilzerman, and A. Majumdar, "Optical frontend for a convolutional neural network," Appl. Opt.58, 3179 (2019). 7. L. Huang, Q. A. A. Tanguy, J. E. Fröch, et al., "Photonic advantage of optical encoders," Nanophotonics 13, 1191-1196 (2024). 8. M. Mathieu, M. Henaff, and Y. LeCun, "Fast Training of Convolutional Networks through FFTs," (2014). 9. "NVIDIA Jetson for Next-Generation Robotics," . 10. M. Shafique, T. Theocharides, V. J. Reddy, and B. Murmann, "TinyML: Current Progress, Research Challenges, and Future Roadmap," in 2021 58th ACM / IEEE Design Automation Conference (DAC), (IEEE, San Francisco, CA, USA, 2021), pp.1303-1306. 11. Y. Liang, Z. Wang, X. Xu, et al., "MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memory," in 37th Conference on Neural Information Processing Systems, (2023). 12. J. Lin, W.-M. Chen, Y. Lin, et al., "MCUNet: Tiny Deep Learning on IoT Devices," (2020). ArXiv:2007.10319 [cs]. 13. A. Chowdhery, P. Warden, J. Shlens, et al., "Visual Wake Words Dataset," (2019). ArXiv:1906.05721 [cs]. 14. R. David, J. Duke, A. Jain, et al., "TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems," (2021). ArXiv:2010.08678 [cs]. 15. D. Gehrig and D. Scaramuzza, "Low-latency automotive vision with event cameras," Nature 629, 1034-1040 (2024). 16. https: / / opg.optica.org / optica / fulltext.cfm?uri=optica-12-7- 1090&id=574181 / P. Pearson, G. Roberts, and A. Faraon, "Inverse-  designed metasurfaces for multifunctional spatial frequency filtering," Optica 12, 1090-1099 (2025), and supplementary information. Contains further information on embodiments disclosed herein. 17. Autograd (https: / / github.com / HIPS / autograd, https: / / github.com / weiliangjinca / grcw) efficiently computed the gradient of the loss function L reproduced above.

[0270] References for sections 6-11 The following references are incorporated by reference herein. 1. J. Goodman, Introduction to Fourier Optics, 3rd ed. (Roberts & Company, 2005). 2. C. Guo, M. Xiao, M. Minkov, et al., "Isotropic wavevector domain image filters by a photonic crystal slab device," J. Opt. Soc. Am. A 35, 16851691 (2018). 3. H. Kwon, A. Cordaro, D. Sounas, et al., "Dual-polarization analog 2D image processing with nonlocal metasurfaces," ACS Photonics 7, 1799-1805 (2020). 4. H. Kwon, D. Sounas, A. Cordaro, et al., "Nonlocal metasurfaces for optical signal processing," Phys. Rev. Lett.121, 173004 (2018). 5. Y. Zhou, H. Zheng, I. I. Kravchenko, et al., "Flat optics for image differentiation," Nat. Photonics 14, 316-323 (2020). 6. M. Cotrufo, A. Arora, S. Singh, et al., "Dispersion engineered metasurfaces for broadband, high-NA, high-efficiency, dual-polarization analog image processing," Nat. Commun.14, 7078 (2023). 7. M. Cotrufo, S. Singh, A. Arora, et al., "Polarization imaging and edge detection with image-processing metasurfaces," Optica 10, 1331-1338 (2023). 8. A. Cordaro, H. Kwon, D. Sounas, et al., "High-index dielectric metasurfaces performing mathematical operations," Nano Lett.19, 8418-8423 (2019).  9. F. Zangeneh-Nejad and R. Fleury, "Topological analog signal processing," Nat. Commun.10, 2058 (2019). 10. T. Zhu, Y. Lou, Y. Zhou, et al., "Generalized spatial differentiation from the spin Hall effect of light and its application in image processing of edge detection," Phys. Rev. Appl.11, 034043 (2019). 11. T. Zhu, Y. Zhou, Y. Lou, et al., "Plasmonic computing of spatial differentiation," Nat. Commun.8, 15391 (2017). 12. W. Xue and O. D. Miller, "High-NA optical edge detection via optimized multilayer films," J. Opt.23, 125004 (2021). 13. L. Wesemann, E. Panchenko, K. Singh, et al., "Selective near-perfect absorbing mirror as a spatial frequency filter for optical image processing," APL Photonics 4, 100801 (2019). 14. J. Zhou, H. Qian, C.-F. Chen, et al., "Optical edge detection based on high-efficiency dielectric metasurface," Proc. Natl. Acad. Sci. USA 116, 11137- 11140 (2019). 15. B. T. Swartz, H. Zheng, G. T. Forcherio, et al., "Broadband and largeaperture metasurface edge encoders for incoherent infrared radiation," Sci. Adv.10, eadk0024 (2024). 16. H. Wang, W. Jin, C. Guo, et al., "Design of compact meta-crystal slab for general optical convolution," ACS Photonics 9, 1358-1365 (2022). 17. P. Huo, L. Tan, Y. Jin, et al., "Broadband and parallel multiple-order optical spatial differentiation enabled by Bessel vortex modulated metalens," Nat. Commun.15, 9045 (2024). 18. H. Wang, C. Guo, Z. Zhao, et al., "Compact incoherent image differentiation with nanophotonic structures," ACS Photonics 7, 338-343 (2020). 19. A. Cordaro, B. Edwards, V. Nikkhah, et al., "Solving integral equations in free space with inverse-designed ultrathin optical metagratings," Nat. Nanotechnol.18, 365-372 (2023).  20. A. Silva, F. Monticone, G. Castaldi, et al., "Performing mathematical operations with metamaterials," Science 343, 160-163 (2014). 21. S. Molesky, Z. Lin, A. Y. Piggott, et al., "Inverse design in nanophotonics," Nat. Photonics 12, 659-670 (2018). 22. J. Jensen and O. Sigmund, "Topology optimization for nano- photonics," Laser Photonics Rev.5, 308-321 (2011). 23. T. W. Hughes, M. Minkov, I. A. D. Williamson, et al., "Adjoint method and inverse design for nonlinear nanophotonic devices," ACS Photonics 5, 4781-4787 (2018). 24. L. Su, A. Y. Piggott, N. V. Sapra, et al., "Inverse design and demonstration of a compact on-chip narrowband three-channel wavelength demultiplexer," ACS Photonics 5, 301-305 (2018). 25. Z. Li, R. Pestourie, Z. Lin, et al., "Empowering metasurfaces with inverse design: principles and applications," ACS Photonics 9, 2178-2192 (2022). 26. O. D. Miller, "Photonic design: from fundamental solar cell physics to computational inverse design," Ph.D. thesis (University of California, 2012). 27. D. Sell, J. Yang, S. Doshay, et al., "Large-angle, multifunctional metagratings based on freeform multimode geometries," Nano Lett.17, 3752- 3757 (2017). 28. S. Ans, F. Zamkotsian, and G. Demésy, "Topology optimization of blazed gratings under conical incidence," J. Opt. Soc. Am. A 41, 1531-1543 (2024). 29. M. Mansouree, H. Kwon, E. Arbabi, et al., "Multifunctional 2.5D metastructures enabled by adjoint optimization," Optica 7, 77-84 (2020). 30. Z. Li, R. Pestourie, J.-S. Park, et al., "Inverse design enables large- scale high-performance meta-optics reshaping virtual reality," Nat. Commun.13, 2409 (2022). 31. E. Bayati, R. Pestourie, S. Colburn, et al., "Inverse designed metalenses with extended depth of focus," ACS Photonics 7, 873-878 (2020).  32. H. Chung, F. Zhang, H. Li, et al., "Inverse design of high-NA metalens for maskless lithography," Nanophotonics 12, 2371-2381 (2023). 33. S. So, J. Kim, T. Badloe, et al., "Multicolor and 3D holography generated by inverse-designed single-cell metasurfaces," Adv. Mater.35, 2208520 (2023). 34. Y. Yin, Q. Jiang, H. Wang, et al., "Multi-dimensional multiplexed metasurface holography by inverse design," Adv. Mater.36, 2312303 (2024). 35. H. Chung and O. D. Miller, "Tunable metasurface inverse design for 80% switching efficiencies and 144∘angular deflection," ACS Photonics 7, 2236- 2243 (2020). 36. G. Roberts, C. Ballew, T. Zheng, et al., "3D-patterned inverse- designed mid-infrared metaoptics," Nat. Commun.14, 2768 (2023). 37. C. Kim, J. Hong, J. Jang, et al., "Freeform metasurface color router for deep submicron pixel image sensors," Sci. Adv.10, eadn9000 (2024). 38. P. B. Catrysse, N. Zhao, W. Jin, et al., "Subwavelength Bayer RGB color routers with perfect optical efficiency," Nanophotonics 11, 2381-2387 (2022). 39. Z. Lin, C. Roques-Carmes, R. Pestourie, et al., "End-to-end nanophotonic inverse design for imaging and polarimetry," Nanophotonics 10, 1177-1187 (2021). 40. G. Arya, W. F. Li, C. Roques-Carmes, et al., "End-to-end optimization of metasurfaces for imaging with compressed sensing," ACS Photonics 11, 2077- 2087 (2024). 41. K. Wei, X. Li, J. Froech, et al., "Spatially varying nanophotonic neural networks," Sci. Adv.10, eadp0391 (2024). 42. L. Huang, Q. A. A. Tanguy, J. E. Fröch, et al., "Photonic advantage of optical encoders," Nanophotonics 13, 1191-1196 (2024). 43. H. Zheng, Q. Liu, Y. Zhou, et al., "Meta-optic accelerators for object classifiers," Sci. Adv.8, eabo6410 (2022).  44. H. Zheng, Q. Liu, I. I. Kravchenko, et al., "Multichannel meta-imagers for accelerating machine vision," Nat. Nanotechnol.19, 471-478 (2024). 45. Z.-X. Zhou, X.-Y. Zhang, H.-R. Qin, et al., "Inverse design of multiplexable meta-devices for imaging and processing," ACS Photonics 12, 246- 252 (2024). 46. W. Fu, D. Zhao, Z. Li, et al., "Ultracompact meta-imagers for arbitrary alloptical convolution," Light Sci. Appl.11, 62 (2022). 47. S. G. Johnson, "The NLopt nonlinear-optimization package". 48. K. Svanberg, "A class of globally convergent optimization methods based on conservative convex separable approximations," SIAM J. Optim.12, 555-573 (2002). 49. W. Jin, W. Li, M. Orenstein, et al., "Inverse design of lightweight broadband reflector for relativistic lightsail propulsion," ACS Photonics 7, 2350- 2355 (2020). 50. F. Wang, B. S. Lazarov, and O. Sigmund, "On projection methods, convergence and robust formulations in topology optimization," Struct. Multidisc. Optim.43, 767-784 (2011). 51. D. Chandler-Horowitz and P. M. Amirtharaj, "High-accuracy,midinfrared ( 450 cm1 ^ ^^ ^ 4000 cm1 ) refractive index values of silicon," J.Appl. Phys.97, 123526 (2005). 52. I. H. Malitson, F. V. Murphy, and W. S. Rodney, "Refractive index of synthetic sapphire," J. Opt. Soc. Am.48, 72-73 (1958). 53. I. H. Malitson, "Interspecimen comparison of the refractive index of fused silica," J. Opt. Soc. Am.55, 1205-1209 (1965). 54. M. E. Krisl, "Aposidation of coherent optical systems," Ph.D. thesis (University of Rochester, 1978). 55. J. P. Mills and B. J. Thompson, "Effect of aberrations and apodization on the performance of coherent optical systems," J. Opt. Soc. Am. A 3, 704-716 (1986).  56. Y. Shao, R. Lupoiu, J. Jiang, et al., "Multifunctional spaceplates for optical aberration correction," ACS Photonics 11, 1753-1760 (2024). 57. J.-S. Park, S. W. D. Lim, A. Amirzhan, et al., "All-glass 100 mm diameter visible metalens for imaging the cosmos," ACS Nano 18, 3187-3198 (2024). 58. J. Kim, J. Seong, W. Kim, et al., "Scalable manufacturing of high- index atomic layer-polymer hybrid metasurfaces for metaphotonics in the visible," Nat. Mater.22, 474-481 (2023). 59. V. J. Einck, M. Torfeh, A. McClung, et al., "Scalable nanoimprint lithography process for manufacturing visible metasurfaces composed of high aspect ratio TiOଶmeta-atoms," ACS Photonics 8, 2400-2409 (2021). 60. M. F. Schubert, A. K. C. Cheung, I. A. D. Williamson, et al., "Inverse design of photonic devices with strict foundry fabrication constraints," ACS Photonics 9, 2327-2336 (2022). 61. A. Y. Piggott, E. Y. Ma, L. Su, et al., "Inverse-designed photonics for semiconductor foundries," ACS Photonics 7, 569-575 (2020). 62. S. Fan and J. D. Joannopoulos, "Analysis of guided resonances in photonic crystal slabs," Phys. Rev. B 65, 235112 (2002). 63. A. Tittl, A. Leitis, M. Liu, et al., "Imaging-based molecular barcoding with pixelated dielectric metasurfaces," Science 360, 1105-1109 (2018). 64. B. H. Stuart, Infrared Spectroscopy: Fundamentals and Applications (Wiley, 2004). 65. L. Dong, X. Yang, C. Zhang, et al., "Nanogapped Au antennas for ultrasensitive surface-enhanced infrared absorption spectroscopy," Nano Lett.17, 5768-5774 (2017). 66. K. Shastri and F. Monticone, "Nonlocal flat optics," Nat. Photonics 17, 36-47 (2023). 67. M. Pahlevaninezhad and F. Monticone, "Multi-color spaceplates in the visible," ACS Nano 18, 28585-28595 (2024).  68. K. Shastri, O. Reshef, R. W. Boyd, et al., "To what extent can space be compressed? Bandwidth limits of spaceplates," Optica 9, 738-754 (2022).

[0271] Conclusion

[0272] This concludes the description of the preferred embodiment of the present invention. The foregoing description of one or more embodiments of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto.

Claims

WHAT IS CLAIMED IS:

1. A device useful for image processing, comprising: a scattering medium patterned for angle dependent scattering of electromagnetic radiation encoding a plurality of different spatial frequency filtering operations on different diffraction orders, polarization states, or wavelengths of electromagnetic radiation incident on the scattering medium from an object, so that an image of the object formed from the electromagnetic radiation encoded by the scattering medium is processed by each of the different spatial frequency filtering operations in parallel.

2. The device of claim 1, wherein the operations comprise convolutions.

3. The device of claim 2, wherein the convolutions comprise at least two of blurring, edge detection, or 2D image differentiation.

4. The device of claim 3, wherein the scattering medium is patterned such that the blurring and / or an edge of the object detected by edge detection are identifiable as variations in intensity in the image.

5. The device of claim 3, wherein the scattering medium is patterned to scatter the electromagnetic radiation into the different diffraction orders such that each of the diffraction orders are encoded with a different one of the operations.

6. The device of claim 5, wherein the operations further comprise brightfield imaging.

7. The device of claim 1, wherein the operations comprise vertical edge detection encoded on one of the diffraction orders and horizontal edge detection on another of the diffraction orders.

8. The device of claim 1, wherein the operations comprise a high pass filter and a low pass filter on different spatial scales and encoded on different ones of the diffraction orders.

9. The device of claim 1, wherein the operations comprising different transfer functions are encoded on different ones of the wavelengths.

10. The device of claim 1, wherein the operations comprise a high pass filter performing edge detection for p -polarization (s-polarization) of the electromagnetic radiation while rejecting s-polarization (p-polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium, thereby performing directional edge detection when illuminating the object with a linear polarization of the electromagnetic radiation.

11. The device of claim 1, wherein the operations comprise a low pass filter performing blurring for p -polarization (s-polarization) of the electromagnetic radiation while maximally transmitting s-polarization (p- polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium, thereby performing directional blurring when illuminating the object with linearly polarized electromagnetic radiation.

12. The device of claim 1, wherein the operations comprise a high pass filter performing edge detection for p -polarization and s-polarization of the  electromagnetic radiation, thereby performing isotropic edge detection when illuminating the object with any polarization of the electromagnetic radiation.

13. The device of claim 1, wherein the operations comprise: a high pass filter performing edge detection for p -polarization (s- polarization) of the electromagnetic radiation while rejecting s-polarization (p- polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium at a first wavelength, thereby performing directional edge detection when illuminating the object with a linear polarization of the electromagnetic radiation having a first one of the wavelengths; and a low pass filter performing blurring for p -polarization (s-polarization) of the electromagnetic radiation while maximally transmitting s-polarization (p- polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium at a second one of the wavelengths different from the first wavelength, thereby performing directional blurring when illuminating the object with a linear polarization of the electromagnetic radiation at the second one of the wavelengths.

14. The device of claim 1, wherein the operations comprise a high pass filter performing edge detection for left circular polarization (right circular polarization) of the electromagnetic radiation while rejecting right circular polarization (left circular polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium, thereby performing directional edge detection when illuminating the object with a circular polarization of the electromagnetic radiation.

15. The device of claim 1, wherein the operations comprise a low pass filter performing blurring for left circular polarization (right circular polarization) of the electromagnetic radiation while maximally transmitting right circular polarization (left circular polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium, thereby performing directional blurring when illuminating the object with circularly polarized electromagnetic radiation.

16. The device of claim 1, wherein the operations comprise a high pass filter performing edge detection for left circular polarization and right circular polarization of the electromagnetic radiation, thereby performing isotropic edge detection when illuminating the object with any polarization of the electromagnetic radiation.

17. The device of claim 1, wherein the operations comprise: a high pass filter performing edge detection for left circular polarization (right circular polarization) of the electromagnetic radiation while rejecting right circular polarization (left circular polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium at a first wavelength, thereby performing directional edge detection when illuminating the object with a cicular polarization of the electromagnetic radiation having a first one of the wavelengths; and a low pass filter performing blurring for left circular polarization (right circular polarization) of the electromagnetic radiation while maximally transmitting right circular polarization (left circular polarization) of the electromagnetic radiation for a majority of (or all) angles of incidence of the electromagnetic radiation on the scattering medium at a second one of the wavelengths different from the first wavelength, thereby performing directional blurring when illuminating the object with a circular polarization of the  electromagnetic radiation at the second one of the wavelengths.

18. The device of claim 1, wherein the scattering medium comprises one or more layers of the scattering medium patterned with a spatially varying permittivity varying with sub-wavelength precision and configured to scatter the electromagnetic radiation so as to encode the operations in the electromagnetic radiation.

19. The device of claim 18, wherein the layers each comprise a silicon layer patterned with a unit cell tiled periodically in a 2D array having a period and thickness defining the spatially varying permittivity.

20. The device of claim 1, wherein the scattering medium comprises multiple layers patterned with subwavelength precision for encoding different ones of the operations.

Images