Neural network for metasurface configuration optimization

Abstract

To create a high performance metasurface device (110) in an inverse design process over a large design space (100), a deep neural-like network can be used as a substitute model instead of full physical simulation to more effectively predict a merit figure for a given input metasurface configuration during iterative configuration optimization. The neural network may also be used to efficiently calculate a gradient with respect to the merit of the design parameter by back propagation, such as to update the configuration in each iteration.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority under the patent laws of U.S. Provisional Application No. 63 / 355,671, filed on June 27, 2022, the contents of which are relied upon and incorporated herein by reference in their entirety. Technical Field

[0003] The present application is directed to optical devices. More specifically, the present application is directed to neural networks for metasurface configuration optimization. Background Art

[0004] Metasurfaces have recently attracted much research interest in optical device design because their unique subwavelength characteristics make it possible to very precisely engineer the way light interacts with the surface. As a result, metasurfaces have been used to implement a wide range of optical devices, including metalenses, compact spectrometers, orbital angular momentum lasers, and ultrafast optical pulse shapers, to name a few. The thin film properties of metasurfaces combined with the choice of high-index materials can achieve significant improvements in form factor, making metasurfaces particularly attractive for wearable or augmented-reality (AR) and / or virtual-reality (VR) applications. Furthermore, the large design space available for metasurfaces enables simultaneous engineering of multiple functionalities, leading to devices and functions that are simply not possible with traditional bulk optics, such as compact full-Stokes polarization cameras, incident angle independent focusing, etc.

[0005] Traditional metasurface designs are usually based on simple geometric shapes that can be characterized by only a few parameters. Due to the limited size of the design library, the device efficiency is usually also limited. As an alternative method, reverse design with accompanying configuration optimization has been used to design metasurfaces with significantly larger degrees of freedom. Instead of limiting the design to certain shapes, such as multiple elliptical columns or cubes, reverse design presents the overall configuration of structural variables, resulting in a significantly larger search space and a possible better solution. However, overall reverse design optimization usually involves hundreds of iterations, each of which requires a complete physical simulation of the complete structure, which is time-consuming. Depending on the size of the device, it can take hours or even days to optimize a structure. In addition, if the purpose is to create an overall library of such structures with all different goals, the process quickly becomes difficult to handle. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] This article describes a method for inverse design of metasurfaces that uses neural networks as a substitute for full physical simulation, thereby rendering configuration optimization over large design spaces computationally tractable. Various embodiments and examples are described with reference to the accompanying drawings, in which:

[0007] Figure 1 A flowchart illustrating an exemplary process for creating a metasurface device;

[0008] Figure 2 A conceptual diagram illustrating the use of a neural network model for metasurface configuration optimization at a high order;

[0009] Figure 3 is a flow chart of an exemplary reverse design process for metasurface configuration optimization;

[0010] Figure 4 For use Figure 3 Schematic cross section of an exemplary supergrating designed by the process of;

[0011] Figure 5 To create Figure 3 The reverse engineering process is used to design Figure 4 2D scatter plot of the scattering efficiency within the training data set of the neural network of the supergrating;

[0012] Figure 6 Shown to create as in Figure 3 The reverse engineering process is used to design Figure 4 An exemplary metasurface configuration for a training data set for a neural network of a hypergrating, and a similar version of the metasurface configuration for augmenting the training data set;

[0013] Figure 7 For Figure 3 The reverse engineering process is used to design Figure 4 A schematic diagram of an exemplary architecture of a neural network of a supergrating;

[0014] Fig. 8A and Figure 8B For use in Figure 5 trained on the training dataset exemplified in Figure 7 Scatter plot of the neural network's predicted and actual scattering efficiencies.

[0015] Fig. 9 For comparison purposes Figure 7 Graph of the gradient of the scattering efficiency calculated by the neural network versus the numerically calculated design parameters.

[0016] Fig.10 Show use Figure 3 The process and Figure 7 The neural network for Figure 4Example metasurface configurations produced by supergratings.

[0017] Fig.11 is a block diagram of an exemplary computing machine on which the reverse engineering method may be performed. DETAILED DESCRIPTION

[0018] The present disclosure relates to methods for creating high-performance metasurface devices, and in particular, computer-implemented metasurface design methods that integrate configuration optimization with surrogate modeling using deep neural networks (DNNs). The methods can find application, for example, in the optimization of optical metasurfaces, which are thin-film optical components that consist of sub-wavelength microstructures that modulate the amplitude, phase, and / or polarization of light. The microstructures are typically defined by thickness variations of a thin layer of material (herein, a "metasurface layer") disposed on an underlying substrate. These thickness variations are typically binary (meaning that the thickness varies between two discrete values, one of which may be zero), although metasurfaces with stepped microstructures involving three or more discrete thicknesses are also possible. The thickness variations define the "configuration" of the metasurface that the metasurface design method seeks to optimize for certain optical device performance metrics, which are referred to herein as "figures of merit."

[0019] In contrast to previous neural network-based metasurface design methods that typically strictly restrict the design space, such as by limiting the surface configuration to an array of simple geometric shapes (e.g., pillars) with a limited number of free design parameters, the current method allows for substantially free-form design of the shape of the metasurface layer. It represents the configuration of a metasurface generically with a pixel-type image (e.g., with 400 or more pixels), thereby covering a large range of possible configurations (limited only by the image resolution). The pixel-type image serves as an input to a DNN, which produces a corresponding figure of merit as output. Once trained on a known metasurface design and a corresponding figure of merit (e.g., as calculated with a fully numerical electromagnetic or similar physics simulation), the DNN can predict the performance of a metasurface device not only for any arbitrary pattern on the device with high accuracy, but also very efficiently calculate the sensitivity of the performance with respect to the design through back propagation. Incorporating the trained DNN as a surrogate model for numerical physics simulation into iterative configuration optimization can significantly reduce the computational time to achieve high-performance metasurface design. In some examples, inverse configuration optimization using DNNs according to the present disclosure is more than two orders of magnitude faster than conventional inverse configuration optimization.

[0020] The foregoing high-level overview will become apparent from the following description of various exemplary embodiments, with reference to the accompanying drawings.

[0021] Figure 11 is a flow chart illustrating an exemplary process 100 for creating a metasurface device. Process 100 involves reversely designing the configuration of a metasurface layer in a freeform design domain, for example, as described below with reference to Figure 3 The description is detailed (act 102), resulting in a pixel-wise image 104 describing the configuration.

[0022] The physical fabrication of a metasurface according to the optimized design begins with the deposition of a substantially thin layer, or film, of a metasurface material on a substrate (action 106). Various deposition techniques suitable for this purpose are known in the art and include, but are not limited to, physical vapor deposition (PVD) techniques such as sputtering, thermal evaporation, or pulsed laser deposition, and chemical vapor deposition (CVD) such as plasma enhanced chemical vapor deposition (CVD), low pressure CVD, or atomic layer deposition. The resulting metasurface layer thickness may be in the range of from about 100 nm to about 2 μm. The metasurface layer material may be a semiconductor, a dielectric (e.g., silicon nitride, gallium nitride, silicon oxide, titanium oxide, amorphous silicon, high index glass, high index polymer), or a metal. Materials that may be suitable for the substrate include, for example, glass, silicon, sapphire, and polymers. Typically, the selection of materials and material properties for the metasurface layer and substrate may be informed by the type of device or end application. For example, for an optical device to operate in transmission, the selected substrate will be transparent to light in the operating wavelength band, while a substrate for a reflective device may be (but need not necessarily be) opaque. The metasurface layer material may be selected based on, among other things, its refractive index and absorption coefficient. For example, amorphous silicon is a good material choice for near-infrared telecommunication applications due to its high refractive index, but a poor choice for applications in the visible energy spectrum due to its high absorption.

[0023] The deposited layer of the metasurface material is patterned in action 108 according to the surface configuration design as represented by the pixel-type image 104. Various microlithographic techniques suitable for this purpose are known in the art. For example and without limitation, a photoresist layer can be deposited on top of the metasurface layer and patterned by photolithography to create an etching mask covering what will become the raised structure in the metasurface, and the mask pattern can then be transferred to the metasurface layer by etching away the exposed portion of the layer. The etching mask itself is then removed. Optionally, a coating material of sufficiently different properties than the metasurface material (e.g., providing a high refractive index contrast) can be applied to fill the recesses in the resulting metasurface layer, and the layer can be planarized (action 110). However, in many cases, air can act as a coating, leaving a rough metasurface.

[0024] Although in the exemplary process 100, patterning is preceded by deposition of a uniform layer of metasurface material on a substrate, the reverse order of steps is possible. For example, in a lift-off patterning process, a layer of sacrificial material is deposited on a substrate and patterned according to a negative of a pixel-type image, exposing areas of the substrate where the raised structures will be. The metasurface material is then deposited on the sacrificial material and substrate, and the sacrificial material is thereafter washed away along with any metasurface material deposited on top, leaving behind a final patterned metasurface layer.

[0025] Metasurface devices are typically planar devices formed by a metasurface layer and an underlying flat substrate of uniform thickness. For example, metalenses are typically planar components that are patterned to achieve the refractive properties of conventional bulk optical lenses without the need for a curved surface. However, it is also possible to apply metasurfaces to non-planar or bulk components. For example, refractive optics can be created from a wedge-shaped substrate that itself constitutes a bulk optical component augmented by a metasurface layer deposited on one or both sides of the wedge. In principle, it is also possible to create a curved metasurface device by forming a metasurface layer directly on top of a curved surface of a device, or by forming a metasurface layer on a planar but flexible substrate and then conforming the substrate to the curved surface.

[0026] Now turning to the design of the metasurface configuration (action 102), Figure 2 A conceptual diagram illustrating the use of a neural network model 200 for metasurface configuration optimization at a high order. The neural network 200 is used as a surrogate model to replace the computationally expensive numerical simulations conventionally used in inverse configuration design. The neural network 200 may be a DNN, meaning a network having multiple layers of artificial neurons between an input layer and an output layer (e.g., between a vector input representing a pixel-type image and a scalar or vector output quantizing one or more merits). The specific architecture of the DNN may be defined by a person skilled in the art, for example, based on observed neural network performance or based on a priori knowledge of the task at hand (e.g., the type or merit of the metasurface device to be designed), or selected from known neural network architectures, but in general, any multi-layer network architecture may be used. For example, the DNN may be fully connected, or may be a deep convolutional network (DCN), optionally including layers configured as a residual neural network (ResNet).

[0027] The neural network 200 takes as input a pixel-type image 202 representing the metasurface configuration of the metasurface device and generates as output one or more merits 204 characterizing the resulting performance of the device. The relevant merits generally depend on the type of metasurface device. For example, merits that may be of interest in characterizing the performance of an optical metagrating include scattering efficiency, scattering power, or similar scattering metrics associated with, for example, a specific grating order, polarization, and / or wavelength or wavelength range, or overall transmission or reflection efficiency. For a metalens, the merit may be or include focusing efficiency or intensity at the focus. The merit for a polarization determination device may include a phase difference or scattering efficiency difference between two orthogonal polarizations, while the merit for a wavelength determination device may include a phase difference between two specified wavelengths or a phase gradient relative to the wavelength. Additional merits will occur to those skilled in the art.

[0028] The neural network 200 is trained on a large, mutually exclusive training data set 206 that includes data for metasurface devices having both high and low values ​​of merit (e.g., high scattering efficiency and low scattering efficiency). For each of the devices in the training data set, the data includes a pixel-type image describing the metasurface configuration of the device, the pixel-type image paired with a value(s) of the merit(s). The merit(s) are calculated from the pixel-type image using rigorous physics (e.g., electromagnetic) simulations. Such simulations can be performed, for example, using commercially available electromagnetic field simulation tools that implement algorithms such as finite-difference time domain (FDTD), finite-difference frequency domain (FDFD), finite element (FE), or rigorous coupled-wave analysis (RCWA) algorithms. In some instances, the training data set is augmented with modified versions of the pixel-type images and associated merit(s) from the original data set. Using certain heuristics, this augmentation can increase the variety of training data without incurring the cost of having to simulate the added configurations. For example, using symmetry considerations, such as invariance to a certain figure of merit for a translation or mirror image of a pixel-based image, the mirror and shifted images can be added to the training data paired with the figure of merit of the original image.

[0029] The training data set is used to train the neural network 202 in supervised training 208. In general, training involves iteratively adjusting the free parameters of the neural network 202, such as the weights associated with the nodes in each neural network layer, to minimize an error function (also commonly referred to as a cost function), which conceptually measures the difference between the value(s) of the figure of merit(s) output by the neural network 202 for a given pixel-type image input and the corresponding simulated value(s) provided as part of the training data pair. The neural network can be trained by back-propagation of the error using gradient descent, which is a learning algorithm well known to those of ordinary skill in the art. In this iterative method, the gradient of the error with respect to the weights of the network is calculated, working backward through the network to determine the adjustments to the weights during each training iteration. The training process ends when the error converges and / or falls below a specified critical value corresponding to the desired accuracy of the predictions made by the neural network.

[0030] After training, the neural network 200 is then able to predict with high accuracy the performance of any arbitrary new metasurface configuration that the neural network has not seen before. This capability is used during iterative configuration optimization 210 to design new metasurface devices based on certain requirements on the figure of merit, a method generally referred to as "reverse design".

[0031] Figure 3 Flowchart of an exemplary inverse design process for metasurface configuration optimization (210). The process begins with a set of initial design variables ρ0 (300) (e.g., in many cases, random numbers), which are used to initialize (at 302) current design variables ρi (304) to be optimized in an iterative process (where ρ indicates an array of multiple scalar design variables, and i is an iteration index). The design variables ρi (304) correspond to a pixel-based image of lower spatial resolution but greater bit depth than a pixel-based image ∈(r) (306) representing the configuration of the actual metasurface. (In the pixel-based image ∈(r), r indicates a two-component vector specifying the pixel coordinates (x, y) within the image, and ∈ is the dielectric constant of the material at those coordinates, and is binary, for example, ∈ 低 or ∈ 高 The spatial resolution specifies the number of pixels per column and row across the image region that covers the extent of the metasurface being designed. The bit depth specifies the binary logarithm of the number of (color or grayscale) values ​​that each pixel can take. In the case of a binary metasurface design, the bit depth of the pixel-based image ∈(r) (306) is 1, corresponding to 2 1The bit depth of the design variables ρi (304) can be much larger, for example, in the range from 8 to 16 (corresponding to between 256 values ​​and over 65,000 color or grayscale values). On the other hand, the number of design variables ρi is typically much smaller than the number of pixels in the image ∈(r), which can range from hundreds to hundreds of thousands, for example, on the order of tens or hundreds.

[0032] Typically, the number of pixels in the image ∈(r) depends on the lateral dimensions of the metasurface and on the spatial resolution of each pixel, which in turn depends on the operating wavelength range. For example, the pixel size may be chosen to be less than 1 / 20 of the wavelength within the metasurface material to achieve the desired optical function; for a wavelength in vacuum of 1.55 μm and a refractive index in the metasurface material of 3.5, this imposes an upper limit of about 22 nm on the pixel size. The spatial resolution of the design variable ρi may correspond to a critical dimension of the microstructures that make up the metasurface, e.g., the diameter of a pillar-like microstructure. This critical dimension may be limited by the manufacturing method. For example, in electron beam lithography, a minimum feature size of 100 nm may be implemented, and thus the spatial resolution of ρi will be ~100 nm. This sets the spatial frequency of the design pattern to be no higher than the manufacturing limit. As will be appreciated, to accurately describe, for example, a circular pillar, the spatial resolution of the image representing the actual structure needs to be much larger than the size of the pillar.

[0033] The design variables ρi (304) are mapped to a pixel-based image ∈(r) (306) that fully describes the actual metasurface configuration in a feature mapping process (308) that involves several image processing functions, such as image magnification, filtering, and thresholding. Image magnification converts the low-resolution image of ρi into a high-resolution image with the same resolution as ∈(r). Image filtering can be used, for example, to smooth the edges of microstructures using convolution of the design variables ρi and a suitable filter kernel. Exemplary kernels for edge smoothing include cone kernels, Gaussian kernels, and disk kernels:

[0034]

[0035] h 高斯 =exp(-(x 2 +y 2 ) / 2σ 2 )

[0036]

[0037] After filtering, a threshold function may convert the high bit depth image to a lower bit depth, eg, a binary image.

[0038] A surrogate model (310) with a trained neural network 200 (e.g., DNN) is used instead of numerical simulation to predict the performance of the device as characterized in one or more figures of merit (FOM) (312), given a complete description of the physical structure (i.e., in this case, configuration) of the metasurface via pixel-wise images ∈(r) (306). A sensitivity analysis (314) is then used to calculate the derivative(s) of the FOM(s) with respect to the device configuration. Instead of adjoint analysis, as is conventionally used when metasurface devices are numerically simulated, the neural network 200 allows derivatives to be obtained very efficiently by back-propagating through the network 200. The time taken for the sensitivity analysis (316) is a function of the design variable ρ i The derivative of the device configuration is independent of the number of Derivative with respect to the design variable This can be calculated using the chain law: These derivatives (318) are then used to update (at 322) the design variables (304) in a direction such that the updated design variables correspond to a metasurface device with higher performance. (Although in principle it is also possible to ∈(r) is directly optimized, but the resulting solution may include features that are too small to be reliably manufacturable; this problem is addressed by optimizing the design variables that are then mapped onto the pixel-like image ∈(r). The process is iterated until the configuration design converges (at 324), meaning that the derivative is small (e.g., compared to a specified convergence threshold) so that the figure of merit no longer changes appreciably.

[0039] Figures 4 to 10 The metasurface configuration optimization method 210 is illustrated with the example of its application to the inverse design of a supergrating. The goal is to design a 1.6 μm x 0.5 μm supergrating made of amorphous silicon (a-Si) as the metasurface material with air as the cladding. Figure 4 The xz cross section of the device is marked, where x is in the plane of the metasurface and z is perpendicular to the metasurface. The middle layer 400 depicted is the metasurface layer for which a two-dimensional pattern (in the xy plane) will be designed. For the purpose of simplifying computational design optimization, both the top and bottom layers are assumed to be air, such as the space in the metasurface layer from which the metasurface material has been removed. (In actual physical implementation, the metasurface will be placed on some type of substrate such as, for example, fused silica.) The purpose will be to scatter the vertically incident light (0th order) to the first diffraction (+1 order) in transmission (thick darker blue arrow) by minimizing both reflection and scattering to other orders. This device can be manufactured using thin film deposition and lithography tools.

[0040] A good neural network model for predicting the performance of a metasurface device depends on a training dataset that spans a large range of potential metasurface configurations, in other words, a large portion of the potential search space. In the illustrated example, the dataset was acquired using Rigorous Coupled-Wave Analysis (RCWA) to model approximately 30,000 different metasurface configurations. Each data point in the dataset consists of a 128x40 pixel grayscale image representing the metasurface configuration in the xy plane, as well as two figures of merit, power s and power p The two merits correspond to the scattering efficiency from 0th order to 1st order for s-polarized light and p-polarized light, respectively. The data is divided into three sets: a training set (~23,000 data points), a validation set (used in developing the model itself), and a test set (for final evaluation of the model). Figure 5 is a two-dimensional scatter plot of the efficiencies calculated for this training set. As can be seen, the training set covers a range from zero to metasurface performance well over 90% efficiency.

[0041] To further increase the variety of training data, the training dataset was augmented with similar versions of existing configuration designs. Randomly laterally shifted versions of the original pixel-type images paired with the scattering efficiencies of those original images were added to the dataset, thereby exploiting the periodic nature of the periodicity of the grating, which implies that shifts of the grating in the plane (x and y) do not change the scattering efficiency. In addition, mirrored versions of the original pixel-type images paired with the scattering efficiencies of those original images were also added, based on the fact that the figure of merit measures scattering to the +1st order in the x direction, so it is not affected by mirrored images in the y direction. Figure 6 An example of this data augmentation is provided, showing an example of the original image (top left) as well as a shifted version (bottom left) and two mirrored images (top right and bottom right).

[0042] Figure 7 Schematically illustrates how to design Figure 4General architecture of an exemplary DNN for supergratings. This particular DNN is fully connected, meaning that every node in each layer is connected to every node in the immediately following layer. The input layer of the DNN is a one-dimensional array of size (5120,1) of pixels corresponding to the pixel-type image ∈(r), and the output layer has two nodes representing the scattering efficiency for s-polarization (TE) and p-polarization (TM), resulting in an output output size (2,1). The DNN has six hidden layers with 128, 256, 512, 512, 256, and 128 nodes, respectively. Overall, the DNN has approximately 1.25 million parameters. All internal layers have rectified linear units (relu) as activation functions. The last layer has a sigmoid activation function so that the output is always mapped to a number in the range (0,1). The DNN is fit to Figure 5 training data with L1 regularization to avoid overfitting and Adamex as the optimizer.

[0043] For example, Figure 5 trained on the training dataset exemplified in Figure 7 The predictive performance of DNNs, Fig. 8A and Figure 8B Scatter plots showing the predicted scattering efficiency and the actual (numerically simulated) efficiency (corresponding to the ground truth in the training data) for both s-polarization and p-polarization. 2 ), a statistic summarizing the amount of variability in the response explained by the model, has a value of about 0.96. The average error is about 5%. Once the DNN model is trained, it can very efficiently compute the output efficiency for arbitrary input designs. DNNs achieve an estimated ~200x speedup in evaluating new designs.

[0044] Trained DNN( Figure 7 The trained DNN has Fig. 8A and Figure 8B The predictive performance exemplified in Figure 3 The Tensorflow interface tf.GradientTape is used to calculate the model output (power) relative to the model input (5120 design pixel values). s and power p ). This interface records the values ​​in intermediate stages during the forward pass and then uses the stored values ​​to compute the derivatives throughout the backward pass, making the computation very efficient. Additional derivatives related to image preprocessing (i.e., the mapping between design variables and pixel-based images) are computed according to the chain law. For a comparison between neural network-based gradient computation and conventional numerical gradient computation, Fig. 9Plotting the gradients computed with the brute force finite difference approximation and the gradients computed using autodiff implemented by tf.GradientTape in Tensorflow shows good agreement between the two. However, using autodiff to compute the gradients is much more computationally efficient.

[0045] Considering the efficient gradient computation, the limited-memory Broyden-Fletcher-Goldfarb-Shannon (LM-BFGS) algorithm implemented in SciPy is used to iteratively search for the optimal design of the supergrating starting from a random initial configuration. Fig.10 Shown are exemplary results produced by the optimizer after fifty iterations in six separate runs of the optimizer. As can be seen by the skilled artisan, the generated configurations show a large diversity, indicating that the optimizer is able to search over a large design space.

[0046] The disclosed methods for inverse metasurface configuration design using a neural network as described herein can generally be implemented in software stored on and executed on general-purpose computing hardware (e.g., one or more central processing units (CPUs) accessing associated memory), with special-purpose hardware (e.g., a graphics processing unit (GPU), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC), or using a combination of both. For example, software implementing a general iterative optimization loop can be interfaced with a hardware accelerator that implements a neural network for predicting a figure of merit for a given metasurface configuration.

[0047] Fig.111 is a block diagram of an exemplary machine 1100, on which any one or more of the techniques discussed herein may be performed. In alternative embodiments, the machine 1100 may operate as a standalone device or may be connected (e.g., a network link) to other machines. In a network-linked deployment, the machine 1100 may operate as a server machine, a client machine, or both in a server-client network environment. In one example, the machine 1100 may act as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. The machine 1100 may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a smart phone, a network appliance, a network router, a switch or a bridge, a server computer, a database, a conference room device, or any machine capable of executing instructions (sequentially or otherwise) that specify actions to be taken by that machine. In addition, although only a single machine is illustrated, the term "machine" shall also be construed to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (SaaS), and other computer aggregation configurations. In various embodiments, the machine (s) 1100 may perform the above-mentioned Figure 2 and above Figure 3 One or more of the processes described.

[0048] The machine (e.g., computer system) 1100 may include a hardware processor 1102 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory 1104, and a static memory 1106, some or all of which may communicate with each other via an interconnect (e.g., a bus) 1108. The machine 1100 may further include a display unit 1110, an alphanumeric input device 1112 (e.g., a keyboard), and a user interface (UI) navigation device 1114 (e.g., a mouse). In one example, the display unit 1110, the input device 1112, and the UI navigation device 1114 may be a touch screen display. The machine 1100 may additionally include a storage device (e.g., a drive unit) 1116, a signal generating device 1118 (e.g., a speaker), a network interface device 1120, and one or more sensors 1121. The machine 1100 may include an output controller 1128, such as a serial (e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.)) connection for communicating or controlling one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0049] The storage device 1116 may include a machine-readable medium 1122 on which one or more sets of data structures or instructions 1124 (e.g., software) are stored that embody or are utilized by any one or more of the techniques or functions described herein. The instructions 1124 may also reside, in whole or in part, within the main memory 1104, within the static memory 1106, or within the hardware processor 1102 during execution thereof by the machine 1100. In one example, one or any combination of the hardware processor 1102, the main memory 1104, the static memory 1106, or the storage device 1116 may constitute a machine-readable medium. Although the machine-readable medium 1122 is illustrated as a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store one or more instructions 1124.

[0050] The term "machine-readable medium" may include any medium capable of storing, encoding, or carrying instructions for execution by the machine 1100 and causing the machine 1100 to perform any one or more of the techniques of the present disclosure, or capable of storing, encoding, or carrying data structures used by or associated with such instructions. Machine-readable media may include computer storage media, such as solid-state memory, optical or magnetic media, or other hardware storage devices, volatile or non-volatile, removable or non-removable, which store computer-readable instructions, data structures, program modules, or the like in a tangible form. Conversely, machine-readable media may also include transmission media that embody computer-readable instructions, data structures, program modules, or the like in a modulated data signal, such as a carrier wave, or other transmission mechanism. As defined herein, computer storage media does not include communication media; therefore, computer storage media should not be construed as transitory propagating signals themselves. Specific examples of computer storage media may include: non-volatile memory, such as semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; random access memory (RAM); solid state drives (SSD); and CD-ROM and DVD-ROM disks.

[0051] The instructions 1124 may further be transmitted or received over a communication network 1126 through the network interface device 1120 using a transmission medium. The machine 1100 may communicate with one or more other machines using any of a number of transmission protocols (e.g., frame relay, internet protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Exemplary communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), a mobile phone network (e.g., a cellular network), a Plain Old Telephone (POTS) network, and a wireless data network (e.g., known as a wireless network). The Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards, known as The network interface device 1120 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas to connect to the communication network 1126. In one example, the network interface device 1120 may include multiple antennas to communicate wirelessly using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technology. In some examples, the network interface device 1120 may communicate wirelessly using multi-user MIMO technology.

[0052] Examples as described herein may include or may operate on logic or a number of components, modules, or mechanisms (hereinafter all referred to as "modules"). A module is a tangible entity (e.g., hardware) that is capable of performing a specified operation and can be assembled or configured in a certain manner. In one example, a circuit can be configured in a specified manner (e.g., internally or relative to an external entity such as other circuits) as a module. In one example, one or more computer systems (e.g., independent client or server computer systems) or one or more hardware processors, in whole or in part, can be assembled into a module that operates to perform a specified operation through firmware or software (e.g., instructions, application portions, or applications). In one example, the software may exist on a machine-readable medium. In one example, the software, when executed by the underlying hardware of the module, causes the hardware to perform a specified operation.

[0053] Thus, the term "module" is understood to encompass a tangible entity, an entity that is physically constructed, specifically assembled (e.g., hardwired), or temporarily (e.g., transiently) assembled (e.g., programmed) to operate in a specified manner or to perform part or all of any of the operations described herein. Considering instances where modules are temporarily assembled, each of the modules need not be instantiated at any one time. For example, where the modules include a general-purpose hardware processor assembled using software, the general-purpose hardware processor may be assembled into respective different modules at different times. Software may thus assemble a hardware processor, for example, to constitute a particular module at one instance in time and to constitute a different module at a different instance in time.

[0054] Examples of the disclosed subject matter include the following:

[0055] 1. A method for manufacturing an optical device having a metasurface layer, the metasurface layer being designed to optimize one or more figures of merit. The method involves inversely designing the configuration of the metasurface layer in a free-form design domain using a deep neural network trained on training data, the training data comprising pixel-type images representing the configuration of the metasurface layer and pairs of figures of merit calculated based on the pixel-type images by numerical electromagnetic simulation. Inverse design involves iteratively operating the deep neural network on the pixel-type images representing the configuration of the metasurface layer to calculate one or more figures of merit, operating the deep neural network backward to calculate the derivatives of the figure of merit (multiple) relative to the pixel values ​​of the pixel-type images representing the configuration, and updating the pixel-type images representing the configuration based on the derivatives. The thin film can then be deposited on a substrate of the optical device, and the thin film can be patterned according to the updated pixel-type images to manufacture an optical device optimized for the figure of merit (multiple).

[0056] 2. The method of Example 1, wherein a pixel-type image is created from a set of design variables, and wherein updating the pixel-type image based on derivatives comprises: calculating derivatives of one or more merits relative to design variables from derivatives of one or more merits of pixel values ​​relative to the pixel-type image; updating the design variables based on the derivatives of the one or more merits relative to the design variables; and creating an updated pixel-type image from the pixel-type design variables.

[0057] 3. The method of example 2, wherein the design variables include an array of values ​​having a lower spatial resolution and a higher bit depth than the pixel-type image, and wherein creating the pixel-type image from the design variables includes image filtering and thresholding.

[0058] 4. The method of any of examples 1-3, further comprising, before creating and iteratively updating the pixel-based image, randomly initializing the set of design variables.

[0059] 5. The method of any of examples 1-4, wherein the pixel-based imagery has binary pixel values.

[0060] 6. The method of any of examples 1-5, wherein the training data further comprises pairs of pixel-type images and figures of merit calculated from shifted or mirrored versions of the pixel-type images by electromagnetic simulation.

[0061] 7. The method of any of examples 1-6, wherein the one or more figures of merit include at least one scattering metric selected from scattering efficiency or scattering power.

[0062] 8. The method of any of examples 1-7, wherein the one or more figures of merit include scattering efficiency and the training data covers a range of scattering efficiency from 0 to at least 80%.

[0063] 9. The method of any of Examples 1-8, wherein the optical device is a supergrating and the one or more figures of merit include a scattering metric associated with a particular grating order, a scattering metric associated with a particular polarization, a scattering metric associated with a particular wavelength or wavelength range, a transmission efficiency, and / or a reflection efficiency.

[0064] 10. The method of any of examples 1-8, wherein the optical device is a polarization-sensitive device, and the one or more figures of merit include a phase difference and / or a scattering efficiency difference at a specific angle between two orthogonal polarizations.

[0065] 11. The method of any of examples 1-8, wherein the optical device is a metalens, and the one or more figures of merit include light intensity at focus and / or focusing efficiency.

[0066] 12. The method of any of Examples 1-8, wherein the optical device is a wavelength-sensing device, and the one or more figures of merit include a phase difference between two specified wavelengths, a phase gradient with respect to wavelength, and / or scattering efficiency for different wavelengths.

[0067] 13. The method of any of examples 1-12, wherein the optical device is an augmented reality or virtual reality device.

[0068] 14. One or more non-transitory computer-readable media store instructions that, when executed by one or more computer processors, cause the processors to iteratively perform operations for designing a metasurface layer of an optical device to optimize one or more figures of merit. The operations include: operating a deep neural network on a pixel-type image representing a configuration of the metasurface layer to calculate one or more figures of merit, the deep neural network having been trained on training data comprising pairs of pixel-type images and figures of merit calculated based on the pixel-type images by numerical electromagnetic simulation; operating the deep neural network backward to calculate derivatives of one or more figures of merit relative to pixel values ​​of the pixel-type image representing the configuration; and updating the pixel-type image representing the configuration based on the derivatives.

[0069] 15. One or more non-transitory computer-readable media of Example 14, wherein a pixel-type image is created from a set of design variables, and wherein updating the pixel-type image based on derivatives comprises: calculating derivatives of one or more figures of merit relative to the design variables from derivatives of one or more figures of merit relative to pixel values ​​of the pixel-type image; updating the design variables based on the derivatives of the one or more figures of merit relative to the design variables; and creating an updated pixel-type image from the pixel-type design variables.

[0070] 16. One or more non-transitory computer-readable media of example 14 or example 15, wherein the design variables include an array of values ​​having a lower spatial resolution and a higher bit depth than the pixel-type image, and wherein creating the pixel-type image from the design variables includes image filtering and thresholding.

[0071] 17. The one or more non-transitory computer-readable media of any of examples 14-16, wherein the training data further comprises pairs of pixel-type images and figures of merit calculated from shifted or mirrored versions of the pixel-type images by electromagnetic simulation.

[0072] 18. The one or more non-transitory computer-readable media of any of examples 14-17, wherein the one or more figures of merit include at least one scattering metric selected from scattering efficiency or scattering power.

[0073] 19. The one or more non-transitory computer-readable media of any of examples 14-18, wherein the one or more figures of merit include scattering efficiency and the training data covers a range of scattering efficiency from 0 to at least 80%.

[0074] 20. A system for designing a metasurface layer of an optical device to optimize one or more figures of merit. The system comprises one or more computer processors and one or more computer-readable media storing instructions that, when executed by the one or more computer processors, cause the one or more computer processors to iteratively perform the following operations: operating a deep neural network on a pixel-type image representing a configuration of a metasurface layer to calculate one or more figures of merit, the deep neural network having been trained on training data comprising pairs of pixel-type images and figures of merit calculated based on the pixel-type images by numerical electromagnetic simulation; calculating derivatives of one or more figures of merit relative to pixel values ​​of the pixel-type image representing the configuration; and updating the pixel-type image representing the configuration based on the derivatives.

[0075] 21. A method of designing a metasurface layer of an optical device to optimize one or more figures of merit by performing the computational operations of any of Examples 1-20.

[0076] Although specific embodiments have been illustrated and described herein, it should be understood that any configuration calculated to achieve the same purpose may be substituted for the specific embodiments shown. The present disclosure is intended to cover any and all adaptations or variations of the various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent to those skilled in the art after reviewing the above description.

Claims

1. A method of manufacturing an optical device having a metasurface layer, wherein the metasurface layer is designed to optimize one or more figures of merit, the method comprising: The configuration of the metasurface layer is reversely designed in a free-form design domain by iteratively performing the following actions: operating a deep neural network on a pixel-wise image representing the configuration of the metasurface layer to compute the one or more figures of merit, the deep neural network having been trained on training data comprising pairs of pixel-wise images and figures of merit computed based on the pixel-wise images by numerical electromagnetic simulations; operating the deep neural network backward to compute derivatives of the one or more figures of merit with respect to pixel values ​​of the pixel-based imagery representing the configuration; and updating the pixel-based image representing the configuration based on the derivative; A thin film is deposited on a substrate of the optical device and patterned according to the updated pixel-type image.

2. The method of claim 1, wherein the pixel-based image is created from a set of design variables, and wherein updating the pixel-based image based on the derivative comprises: calculating derivatives of the one or more figures of merit with respect to the design variables from the derivatives of the one or more figures of merit with respect to the pixel values ​​of the pixel-type image; updating the design variables based on the derivatives of the one or more figures of merit with respect to the design variables; as well as An updated pixel-type image is created from the pixel-type design variables.

3. The method of claim 2, wherein the design variables include an array of values ​​having a lower spatial resolution and a higher bit depth than the pixel-based image, and wherein creating the pixel-based image from the design variables includes: image filtering and thresholding.

4. The method of claim 1, further comprising: randomly initializing the set of design variables before creating and iteratively updating the pixel-based image. The method of claim 1 , wherein the pixel-based image has binary pixel values.

6. The method of claim 1, wherein the training data further comprises pairs of pixel-wise images and figures of merit computed from shifted or mirrored versions of the pixel-wise images by electromagnetic simulation.

7. The method of claim 1, wherein the one or more figures of merit include at least one scattering metric selected from scattering efficiency or scattering power.

8. The method of claim 1, wherein the one or more figures of merit include scattering efficiency and the training data covers a range of the scattering efficiency from 0 to at least 80%.

9. The method of claim 1, wherein the optical device is a supergrating and the one or more figures of merit include at least one of a scattering metric associated with a specific grating order, a scattering metric associated with a specific polarization, a scattering metric associated with a specific wavelength or wavelength range, a transmission efficiency, or a reflection efficiency.

10. The method of claim 1, wherein the optical device is a polarization-sensitive device, and the one or more figures of merit include a phase difference or a scattering efficiency difference at a specific angle between two orthogonal polarizations.

11. The method of claim 1 , wherein the optical device is a metalens and the one or more figures of merit include light intensity at a focus or focusing efficiency.

12. The method of claim 1, wherein the optical device is a wavelength-sensing device and the one or more figures of merit include a phase difference between two specified wavelengths, a phase gradient with respect to wavelength, or scattering efficiency for different wavelengths.

13. The method of claim 1, wherein the optical device is an augmented reality or virtual reality device.

14. A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors, cause the processors to perform operations for designing a metasurface layer of an optical device to optimize one or more figures of merit, the operations comprising iteratively performing the following actions: operating a deep neural network on a pixel-wise image representing the configuration of the metasurface layer to compute the one or more figures of merit, the deep neural network having been trained on training data comprising pairs of pixel-wise images and figures of merit computed based on the pixel-wise images by numerical electromagnetic simulations; operating the deep neural network backward to compute derivatives of the one or more figures of merit with respect to pixel values ​​of the pixel-based imagery representing the configuration; and The pixel-based image representing the configuration is updated based on the derivative.

15. The one or more non-transitory computer-readable media of claim 14, wherein the pixel-based image is created from a set of design variables, and wherein updating the pixel-based image based on the derivative comprises: calculating derivatives of the one or more figures of merit with respect to the design variables from the derivatives of the one or more figures of merit with respect to the pixel values ​​of the pixel-type image; updating the design variables based on the derivatives of the one or more figures of merit with respect to the design variables; as well as An updated pixel-type image is created from the pixel-type design variables.

16. The one or more non-transitory computer-readable media of claim 14, wherein the design variables include an array of values ​​having a lower spatial resolution and a higher bit depth than the pixel-based image, and wherein creating the pixel-based image from the design variables includes: image filtering and thresholding.

17. The one or more non-transitory computer-readable media of claim 14, wherein the training data further comprises pairs of pixel-wise images and figures of merit calculated from shifted or mirrored versions of the pixel-wise images by electromagnetic simulation.

18. The one or more non-transitory computer-readable media of claim 14, wherein the one or more figures of merit include at least one scattering metric selected from scattering efficiency or scattering power.

19. The one or more non-transitory computer-readable media of claim 14, wherein the one or more figures of merit include scattering efficiency and the training data covers a range from 0 to at least 80% of the scattering efficiency.

20. A system for designing a metasurface layer of an optical device to optimize one or more figures of merit, the system comprising: one or more computer processors; and One or more computer-readable media storing instructions, which, when executed by the one or more computer processors, cause the one or more computer processors to iteratively perform operations comprising: operating a deep neural network on the pixel-wise image representing the configuration of the metasurface layer to compute the one or more figures of merit, the deep neural network having been trained on training data comprising pairs of pixel-wise images and figures of merit computed based on the pixel-wise images by numerical electromagnetic simulations; calculating derivatives of the one or more figures of merit with respect to pixel values ​​of the pixelated image representing the configuration; and The pixel-based image representing the configuration is updated based on the derivative.

Images