Dielectric splitter for use in imaging - Patent Application 20070123633

Abstract

Described herein is an image sensor comprising: a splitter comprising a block of structured dielectric material having a plurality of nanophotonic geometric structures formed therein, where incident electromagnetic radiation is split into a plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges by transmission through the plurality of nanophotonic geometric structures, the block having an input surface for receiving the incident electromagnetic radiation and an output surface for transmitting the plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges, the input surface and the output surface being independently communicable with the plurality of nanophotonic geometric structures; and a pixelated photodetector array providing an active layer surface for receiving the plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges, the active layer surface being oriented opposite the output surface of the block, the separation of the active layer surface and the output surface being less than 500 nanometers.

Description

[Technical field]

[0001] The present invention relates to digital imaging, and more particularly to splitting incident electromagnetic radiation to match the pixel pattern of an image sensor. [Background technology]

[0002] Digital imaging and digital image sensors are becoming more and more common. The two main types of digital image sensors are charge-coupled devices (CCDs) and active pixel sensors (CMOS sensors), which are fabricated in complementary metal-oxide semiconductor (CMOS) or negative-type metal-oxide semiconductor (NMOS or Live MOS) technology. Both CCD and CMOS sensors are based on metal-oxide semiconductor (MOS) technology, with the MOS capacitor being the building block of the CCD and the MOSFET amplifier being the building block of the CMOS sensor.

[0003] Digital image sensors are used in electronic imaging devices that benefit everyday consumer implementations as well as specialized industrial workflows, such devices include, for example, digital cameras, camera modules, camera phones, optical mouse devices, medical imaging equipment, night vision equipment such as thermal imaging devices, microwave imaging devices, among others. An increasing reliance on high-resolution cameras that contain digital image sensors can be seen in mobile, automotive, and medical and life science applications. For example, Sony optronics offers CMOS HD camera image sensors for digital mammography. In other examples, CMOS cameras can reproduce color variations and details for endoscopy and surgical imaging systems, digital pathology cameras integrated into microscopes and whole slide scanners to capture human tissue slices, biopsy tests, and cell samples.

[0004] Image sensors typically incorporate components, such as filters, splitters, collimating lenses, focusing lenses, etc., that modify, manipulate, or process the incident electromagnetic radiation.

[0005] Many image sensors benefit from spectral separation of incident electromagnetic radiation and typically incorporate filters or splitters to transmit spectrally distinct beams onto a pixel configured photo-detector array. For example, in color imaging, the image sensor typically must incorporate a color selection or color separation component because the image sensor's base photo-sensor detects light intensity with little or no wavelength specificity and therefore cannot separate out color information. For color imaging, digital image sensors that incorporate color filter arrays or color splitters are common.

[0006] Color imaging is most commonly achieved through an absorptive Bayer color filter array, where each of the three color pixel filters allows approximately 1 / 3 of the incident light to be transmitted to the active material layer of the image sensor's photosensor, and thus the selectivity of individual colors is penalized by loss of transmitted intensity.

[0007] An alternative approach to filtering is to split the incoming light by color and redirect the appropriate wavelength range to the corresponding pixel. In theory, this can increase the light intensity to the active material of the image sensor by a factor of three on average. Color splitting for imaging applications has been previously explored through a fairly complex process, often involving high-index (e.g., refractive index greater than 2) material processing steps and computationally expensive image reconstruction. Many designs furthermore only work with a specific polarization of light, which limits their efficiency for general imaging applications.

[0008] Thus, there remains a need for alternative components for directing incident electromagnetic radiation to the active material of an image sensor. Summary of the Invention

[0009] In one aspect, a splitter comprising a block of structured dielectric material having a plurality of nanophotonic geometric structures formed therein, wherein incident electromagnetic radiation is split by transmission through the plurality of nanophotonic geometric structures into a plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges; a splitter, the block having an input surface for receiving incident electromagnetic radiation and an output surface for transmitting a plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges, the input surface and the output surface being independently communicative with a plurality of nanophotonic geometric structures; 1. A pixelated photodetector array providing an active layer surface for receiving a plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges, comprising: a pixelated photodetector array, the active layer surface of which is oriented opposite the output surface of the block, the separation between the active layer surface and the output surface being less than 500 nanometers; An image sensor is provided, comprising:

[0010] In another aspect, a splitter having an input surface for receiving incident electromagnetic radiation and an output surface for transmitting a plurality of beams of spectrally distinct electromagnetic wavelength ranges, the splitter having a minimum focal length to the output surface of less than 200 nm; 1. A pixelated photodetector array providing an active layer surface for receiving a plurality of beams of spectrally distinct electromagnetic wavelength ranges, comprising: a pixelated photodetector array, the active layer surface of which is oriented opposite an output surface of the block, the separation between the active layer surface and the output surface being less than 400 nanometers; An image sensor is provided, comprising: [Brief description of the drawings]

[0011] [Figure 1] FIG. 2 is an exploded perspective view of an image sensor including a splitter and a pixelated photodetector array. [Diagram 2] 2A-2C show several exemplary comparisons of the image sensor shown in FIG. 1 with a prior art image sensor having an angle splitter. [Diagram 3] FIG. 2 illustrates an example of an adjoint-based reverse design process for constructing the splitter shown in FIG. 1. [Figure 4] FIG. 4a shows an example of a prior art Bayer filter and FIG. 4b shows an example of a prior art angle splitter. [Diagram 5] Diagram of the color splitter design. Fig. 5a is an isometric rendering of the four-layer topology-optimized splitter structure, and Fig. 5b is a plan view of the color-coded layers for the four-layer topology-optimized splitter structure. Fig. 5c is an isometric view of the full 3D splitter structure, and Fig. 5d is a plan view of the average density projection of the full 3D splitter structure. The dashed lines in Fig. 5a and Fig. 5c represent the actively simulated regions. [Figure 6] Diagram of the splitter operation. Figures 6a and 6b show the Poynting vector z-component (towards the sensor) in three color bands (red, green, blue) visualized at the sensor surface. The pixel active areas are shown as dashed lines. Figures 6c and 6d plot the light transmission from the full 2x2 pixel area to the sensor active area for three different pixel colors. The 3D splitter (solid lines) shows lower selectivity than the Bayer filter (dashed lines), but the much higher transmission still results in a doubling of the color correction efficiency (34% vs. 17%, respectively). [Figure 7]Simulated imaging responses. Figure 7a shows pixel transmission density functions for the color splitter and Bayer filter in color-corrected (solid) and grayscale (dashed) imaging for a given test image. Figure 7b shows a reference image assuming perfect splitting at 50% efficiency. Figures 7c and 7d show color-corrected simulated images for comparing filter and splitter performance, respectively. Figures 7e and 7f show grayscale simulated images for the filter and splitter, respectively. [Figure 8] Performance vs. complexity tradeoff. Figure 8a illustrates splitter performance in terms of relative color-corrected imaging efficiency, shown for both color (blue) and grayscale (red) imaging. Performance is normalized to a Bayer filter with an assumed perfect microlens structure (solid black line), and the performance without such a lens (dashed black line) is shown as well. Single-layer and two-layer geometries are shown in Figures 8b and 8c, respectively. [Figure 9] Color splitter performance. Poynting vector intensity to the sensor surface integrated over three 100 nm color bands (blue - 400-500 nm, green - 500-600 nm, red - 600-700 nm) for single and double layered structures (Figures 9a and 9b, respectively). Integrated transmission to the corresponding color pixel active area for single and double layered structures (Figures 9c and 9d, respectively). [Figure 10] 10A and 10B are diagrams of the spatial distribution of power transmission. Three color band path slices (blue, green and red lines) are shown in Fig. 10a, 10b, 10c and 10d for 1-layer, 2-layer and 4-layer structures and a full 3D structure, respectively. The path taken by the line trace is shown inset in Fig. 10a. [Figure 11]Angular response of the splitter structure. The relative sorting efficiency for the full 3D and 4-layer 2D structures versus the incoming light angle θ (shown in the inset) is shown in Fig. 11a and 11b, respectively. The absolute sorting efficiency is shown in Fig. 11c as horizontal grey dashed and solid lines, respectively, along with the corresponding limits in the absence of light focusing and the limiting performance of the color Bayer filter. The line colors (red, green, blue) correspond to the respective color pixels. [Figure 12] Color transmission diagram for off-normal incidence light. The transmission spectra into three pixel color active areas for angles of 4, 8, and 16 degrees, corresponding to the angle θ shown in the inset in Fig. 11, are shown in Figs. 12a-c for the full 3D splitter and 12d-f for the four-layer 2D structure. The line colors (red, green, blue) correspond to the respective color pixels, and the black line represents the overall grayscale efficiency (the sum of the three color components). [Figure 13] The specific transmission for two orthogonal linear polarizations of light is shown in Fig. 13a and 13b for a full 3D splitter and in Fig. 13c and 13d for a four-layer 2D splitter structure. The line colors (red, green, blue) correspond to the respective color pixels and the black line represents the overall grayscale efficiency (sum of the three color components). The electric field polarization for the splitter structure is shown inset in each plot. [Figure 14] Energy probability density function of the Poynting vector angle (θ) relative to the sensor surface normal. The full 3D splitter response is shown in Fig. 14a, and the 4-layer 2D structure response is shown in Fig. 14b. The probability densities are normalized so that they merge into one. The line colors (red, green, blue) correspond to the respective color pixel areas and color bands included in the calculation. [Figure 15]FIG. 14 shows the sorting efficiency of 1-, 2-, 3-, 4-, and 40-layer splitters compared to filter and lens combinations. [Figure 16] FIG. 13 is a probability density function plot showing the distribution of angles at the output surface of the splitter for a test case of normal incidence light received by a three-layer splitter, showing the distribution centered around 0 degrees. [Figure 17] FIG. 13 is a plot of the cumulative distribution function for normal incidence light for a three-layer splitter, showing that 80% of the power leaving the splitter is contained within 10 degrees of the surface normal. [Figure 18] FIG. 13 is a probability density function plot showing the distribution of angles at the output surface of the splitter for a test case of 10 degree incidence angle for a three-layer splitter, showing the distribution centered around 10 degrees. [Figure 19] FIG. 13 is a cumulative distribution function for a 10 degree incidence angle for a three-layer splitter showing that >80% of the power is contained within 10 degrees of the 10 degree incidence angle (i.e., from 0 to 20 degrees). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Described herein is a dielectric splitter for spectrally separating incident electromagnetic radiation into multiple spectrally distinct wavelength ranges or bands in a directional pattern that matches an associated pixelated photodetector array. The splitter may be made from one or more dielectric materials having a low refractive index (e.g., a refractive index less than 2), and may be readily constructed of a single dielectric material having a low refractive index.

[0013] An image sensor 10, incorporating a dielectric splitter 12 and a matching associated pixelated photodetector array 14, can provide digital imaging in a number of implementations, including consumer electronics, medical imaging, and thermal imaging, among others.

[0014] The pixelated photodetector array is sometimes conventionally referred to as a sensor, and thus photodetector, photosensor, and sensor may be used interchangeably. However, an image sensor is distinguished from a sensor in that an image sensor includes both a splitter and a photodetector array, and a sensor may refer only to a photodetector array.

[0015] The dielectric splitter 12 comprises a block 16 of structured dielectric material having a plurality of nanophotonic geometric structures 18 formed therein. In some examples, the block 16 is formed from a plurality of stacked layers, more particularly, the plurality of stacked layers are integrated to form the block and the plurality of nanophotonic geometric structures. The geometric structures define the topology of the structured dielectric material and thus define regions within the block that are characterized by the material properties of the dielectric material, including properties such as refractive index and density. For example, the geometric structures can define boundaries for a transition from a first refractive index to a second refractive index, and the geometric structures can form a pattern of differentiated refractive index that splits broadband light into wavelength ranges with near-field focus compared to existing splitters and with much less attenuation than existing filters.

[0016] Incident electromagnetic radiation projected onto or received by the block is split by transmission through the plurality of nanophotonic geometric structures into a plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges.

[0017] The block has an input surface 20 for receiving incident electromagnetic radiation and an output surface 22 for transmitting a plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges. Because the geometric structure topology at the input surface may provide regions of solid and void depending on the particular implementation, the input surface may be considered a plane defined by the sides of the block configured for receiving incident electromagnetic radiation, and thus the terms input surface and input plane may be used interchangeably. Because the geometric structure topology at the output surface may provide regions of solid and void depending on the particular implementation, the output surface may be considered a plane defined by the sides of the block configured for transmitting a plurality of beams of spectrally distinct electromagnetic wavelength ranges to an opposing pixelated photodetector array, and thus the terms output surface and output plane may be used interchangeably.

[0018] It is intended that the evaluation of the substantially parallel characteristic of the beams is performed at the output surface, not at a distance from or far from the output surface of the block. Substantially parallel means that the angular deviation of the beams at the output surface can be constrained typically within a 20 degree angular range, and more typically within a 15 degree angular range, and often within a 10 degree angular range. The substantially parallel characteristic can be demonstrated in a test case of incident electromagnetic radiation received at a normal (i.e., perpendicular) angle to the input surface, which then corresponds to multiple substantially parallel beams of spectrally distinct electromagnetic wavelength ranges transmitted at the splitter output surface at an angular range of less than 10 degrees from the normal angle to the output surface. In the test case of normal incident radiation, an insignificant portion of the beams at the output surface may occur as angular outliers outside the intended angular range of less than 10 degrees from the normal to the output surface (i.e., a cone-shaped profile defined by a 10 degree projection from the normal, where the normal is the central axis of the cone), but even a conservative estimate (i.e., a three-layer splitter that provides better performance than a filter but is not expected to perform to the level of splitters with more layers, such as 10-layer, 20-layer, 30-layer, or 40-layer splitters) would have at least 80% of the power within an angular range of less than 10 degrees from the normal to the output surface and at least 90% of the power within an angular range of less than 14 degrees from the normal to the output surface, so that the majority of the beams fall within the intended angular range. It should be understood that the angle relative to the normal to the output surface may be projected in any direction such that the entire angular distribution profile is contained within the cone of the specified angle. The test case can be extended to non-normal incident radiation, where incident radiation received in an angular range of + / - 10 degrees normal to the input surface will correspond to a conservative estimate of at least 80% of the power for multiple beams at the output surface within an angular range of less than 10 degrees from the same angle of incidence transferred to the output surface (i.e., the angle of incidence maintained at the output surface as the reference angle for the angular range).

[0019] The plurality of nanophotonic geometric structures are distributed throughout the block from the input surface to the output surface, and the input surface and the output surface are independently communicable with the plurality of nanophotonic geometric structures. Independently communicable means that the input surface and the output surface may be in communication with the same continuous geometric structure or different non-continuous geometric structures or a combination thereof, and the communication of the input surface with the geometric structure may be independent from the communication of the output surface with the geometric structure. Thus, in some examples, the input surface may be in communication with a first group of the plurality of nanophotonic geometric structures, and the output surface may be in communication with a second group of the geometric structures, where the first group and the second group are structurally separate and non-continuous, and in further examples, both the input surface and the output surface may be in communication with a third group of nanophotonic geometric structures that extend continuously from the input surface to the output surface.

[0020] The image sensor operates in conjunction with a dielectric splitter disposed adjacent a pixelated photodetector array that presents an active layer surface 24 that receives a plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges transmitted from an output surface 22 of the block.

[0021] The active layer surface 24 is oriented parallel to and opposite the output surface of the block, without the need for intervening artificial or synthetic components, although the insertion of intervening artificial or synthetic components is not prohibited. The separation between the active layer surface 24 and the output surface 22 of the block may be minimized, and the separation between the active layer surface 24 and the output surface 22 will typically be less than 500 nanometers (nm). For example, the separation between the active layer surface and the output surface may be less than 400 nanometers.

[0022] The active layer surface may be considered to span all of the individual photodiode active areas in the photodetector array. Thus, the active layer and active area may be used interchangeably, since the active layer is simply the accumulation of all the active areas in the photodetector array that lie on a common plane. The term active area is well recognized in photodiode implementations and is understood to be the defined area within a photodiode that can receive photons and convert them into electrical current.

[0023] Each beam of the multiple beams of spectrally distinct electromagnetic wavelength ranges need not be separated without overlap, and overlap is often not avoided since most imaging sensors work with three or four distinct wavelength ranges, and splitters providing three or more wavelength ranges will likely result in some overlap. Thus, the multiple beams of spectrally distinct electromagnetic wavelength ranges will provide at least a first set of beams and a second set of beams, where the peak intensity of the wavelength range of the first set of beams occurs at a different wavelength than the peak intensity of the wavelength range of the second set of beams. Not all beams in the first set of beams need to be of the same wavelength range, variations are allowed, and similarly not all beams in the second set of beams need to be of the same wavelength range, variations are allowed. When considering all beams in the first set of beams and all beams in the second set of beams, the combined wavelength range of the first set of beams is different from the combined wavelength range of the second set of beams, but overlap is allowed. Generally, the overlap is minimized, and generally the intensity of the wavelengths of the first set or second set in the overlap considers less than 40% (in some instances less than 35%, or in other instances less than 30%) of the total intensity of the entire wavelength range of the first set or second set, respectively. Similar considerations apply when the multiple beams of spectrally distinct electromagnetic wavelength ranges include three sets of beams or four sets of beams or even more beam sets. Spectrally distinct includes spectral separation with or without overlap, and thus spectrally distinct encompasses the possibility of spectrally discriminated, providing separated wavelength ranges without overlap.

[0024] Image sensors have been found to have various beneficial characteristics, for example, image sensors have been found to provide superior performance in accommodating variations in non-normal angles of incident electromagnetic radiation.

[0025] In another example of a beneficial feature, the transmission efficiency of the incident electromagnetic radiation is improved in that the total intensity of the multiple substantially parallel beams of spectrally distinct electromagnetic wavelength ranges transmitted from the splitter output surface is greater than 50% compared to the corresponding total intensity of the incident electromagnetic radiation received into and through the splitter input surface (specified to exclude electromagnetic radiation reflected from the splitter input surface). In other examples, a transmission efficiency of greater than 60% may be achieved. In a further example, a transmission efficiency of 70% may be achieved. The improved transmission efficiency may also be expressed as a reduced attenuation of the incident electromagnetic radiation as compared to its corresponding multiple substantially parallel beams of spectrally distinct electromagnetic wavelength ranges, where the attenuation may be less than 50%, or less than 40%, or less than 30%.

[0026] In yet another example of a beneficial feature, the splitter provides near-field focus in that the focus of the multiple substantially parallel beams of spectrally distinct electromagnetic wavelength ranges transmitted from the splitter output surface is characterized by a working distance of less than 500 nm relative to the output surface and a minimum focal distance of less than 300 nm relative to the output surface. The working distance refers to the normal (i.e., vertical) distance between the splitter output surface and the active surface layer of the pixelated photodetector array. The minimum focal distance refers to the minimum vertical distance required from the splitter output surface to achieve beam focus. The working distance can be greater than the minimum focal distance, but the same is also achievable. In some examples, the working distance is less than 400 nm and the minimum focal distance is less than 200 nm, and in other examples, the working distance is less than 300 nm and the minimum focal distance is less than 100 nm. In a further example, the working distance is less than 200 nm and the minimum focal distance is at or near 0 (one example of near 0 is less than 20 nm and another example of near 0 is less than 10 nm).

[0027] In yet another example of a beneficial feature, the splitter provides angle preservation by maintaining the angle of incident radiation within a constrained distribution of beam angles at the output surface in that test cases of incident electromagnetic radiation received at a normal (i.e., perpendicular) angle to the input surface correspond to a plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges transmitted from the splitter output surface in an angular range of less than 10 degrees from the normal angle to the output surface (i.e., a cone-shaped profile defined by a 10 degree projection from the normal, where the normal is the central axis of the cone). It is intended that the evaluation of the angular range be performed at the output surface, not at a distance from or far from the output surface of the block. In another example, the corresponding plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges transmitted from the splitter output surface, tested with incident electromagnetic radiation at a normal angle, are within an angular range of less than 7 degrees from the normal angle to the output surface, and in another example, the angle of the beams relative to the output surface are within an angular range of less than 5 degrees from the normal angle to the output surface. In the test case of normal incident radiation, an insignificant portion of the beams may occur as angular outliers outside the intended angular range of less than 10 degrees from the normal to the output surface (i.e., a cone-shaped profile defined by a 10 degree projection from the normal, where the normal is the central axis of the cone), but even a conservative estimate (i.e., a three-layer splitter that provides better performance than a filter but is not expected to perform to the level of a comparable splitter with more layers, such as a 10-layer, 20-layer, 30-layer, or 40-layer splitter) would have at least 80% of the power within an angular range of less than 10 degrees from the normal to the output surface and at least 90% of the power within an angular range of less than 14 degrees from the normal to the output surface, so that the majority of the beams will fall within the intended angular range. It should be understood that the angle relative to the normal to the output surface may be projected in any direction such that the entire angular distribution profile is contained within the cone of the specified angle.The test case can be extended to non-normal incident radiation, where incident radiation received in an angular range of + / - 10 degrees normal to the input surface will correspond to a conservative estimate of at least 80% of the power for multiple beams at the output surface within an angular range of less than 10 degrees from the same angle of incidence transferred to the output surface (i.e., the angle of incidence maintained at the output surface as the reference angle for the angular range).

[0028] FIG. 2 is a schematic representation of the near-field focus and angle preservation benefits of an image sensor in one implementation of a color image sensor, with the left column showing examples and data for a comparator image sensor incorporating an angle splitter, and the right column showing examples and data for an image sensor 10 of the present disclosure that transmits a substantially parallel beam from the output surface 22 of the splitter block 12. For ease of illustration, the diagram is shown with red wavelength incident light and only red (R) and blue (B) pixels. The first row shows a diagram of nominal operation, where the comparator requires a relatively larger spacer 26 midway between the output surface 22 and the active layer surface 24. In the first row, both the comparator and the present image sensor 10 efficiently direct the incident light to the red pixels in the pixelated photodetector array. In the second row, the spacing is changed while all other elements remain the same as shown in the first row. In the second row, the spacing of the comparator is reduced and the spacing of the present image sensor 10 is increased. As a result, red light is not focused onto the red pixels in the comparative benchmark because the spacer is less than the specified distance requirement for focusing the beam from the output surface of the angle splitter, but due to the substantially parallel beam output of the present splitter, an increased size of the spacer (relative to the diagram in the first row) is easily accommodated without changing the focused transmission of the red light to the red pixels. Thus, the near-field focus and angle preservation advantages of the image sensor 10 of the present disclosure underlie the further advantage of accommodating variations in the spacing between the output surface 22 and the active layer surface 24. In the third row, the angle of the incident red light is varied while all other elements remain the same as shown in the first row. The comparative benchmark with the angle splitter suffers from poor angle preservation, whereas the present image sensor is able to accommodate changes in the angle of incidence due to the angle preservation benefits of the present splitter. The fourth row shows that while the minimum focal length for the angle splitter is approximately 600 nm, the present splitter has a minimum focal length of nearly zero and does not significantly reduce the efficient focused transmission until approximately 200 nm, and from approximately 200 nm to approximately 800 nm, the efficiency steadily reduces but still remains higher than conventional color absorptive filters.The fifth row graphically presents data showing that the splitter of the present disclosure is better able to accommodate changes in incidence angle than the comparative reference angle splitter.

[0029] 3 illustrates an illustrative reverse design flow process 100 that may be used to construct splitter 12. An illustrative example of a free-form topology of four laminations of dielectric material integrated to form splitter 12 and its geometric structure for a color (red / green / blue) imaging implementation will be described with reference to the reverse design process steps illustrated in FIG.

[0030] Step 101 - Setup of initial conditions. The optimization starts by selecting the initial density and number of layers. Without loss of generality, the parameters assumed for this illustrative example are 4 stacks for this structure, a structural material with refractive index n=1.5 (glass-like photoresist) and a background refractive index of n=1 (air), as well as a working distance of 0 (the output surface of the splitter structure is directly above and abuts against the photodetector array active layer surface). For a structure with splitting behavior governed by spatial displacement (as opposed to angular redistribution) capable of outputting a substantially parallel beam, a working distance less than the average wavelength of light in the background medium should be selected.

[0031] In any optimization, it is useful to run multiple values ​​for the initial relative density of the structures, which can be varied between 0 (background index, here equal to n=1) and 1 (structure index, here equal to n=1.5). Since the optimization is local in nature, different initial conditions lead to different performance levels at the end of the optimization. More complex figures of merit (including non-differentiable functions, such as taking the minimum of the three color performances) can also be used at the end of the optimization to similarly compare the results of varying initial conditions. For example, specifications on minimum radius of curvature, binarization threshold, and maximum step size are set similarly, but generally have less impact on the final result than the initial structure guess.

[0032] Step 102 - Topology optimization loop. The topology optimization operates to perturb the refractive index of the initial guess material to increase the figure of merit. This is done in three sub-steps.

[0033] Step 102-1-a / b - Polarization. Each of the two cardinal polarizations (102-1a, 102-1b) is run separately to allow results for randomly polarized light to be obtained. These can be run concurrently since they are independent. The calculation consists of two simulations, a forward simulation and an adjoint simulation, shown in step 102-2.

[0034] Step 102-2-a / b / c / d - Forward and adjoint simulations. Two simulations are run for each polarization, the forward simulation (102-2a and 102-2c) corresponding to a plane wave launched towards the structure, and the adjoint simulation (102-2b and 102-2d) driving three sources from the sensor pixel locations. Here, three separate dipole sources are utilized, one at the center of each pixel area, with the dipole spectrum corresponding to the target spectrum for the pixel where the dipole source is located. A single dipole may be utilized that matches the polarization of the forward simulation. The dipole sources assume an ideal light distribution centered on the center of each pixel, but multiple dipole sources may be utilized to allow for a more distributed light intensity profile (or target in detail). This approach allows all three colors to be run in a single simulation since they are spectrally distinct. The use of a single dipole for each pixel also benefits from optimization to maintain the incoming angle of the light on exit from the splitter structure, a benefit made possible by the spatial displacement.

[0035] Step 102-3-a / b - Gradient calculation. The electric field values ​​in the splitter regions from the forward and adjoint simulations are combined to determine the field gradient at each spatial point and for each color band (102-3a, 102-3b), and the performance within each spectral band is averaged. This provides a four-dimensional (three spatial and one spectral) array of field gradients. For layered structures, such as the four-layer structure described here, the performance within each layer region is averaged to reduce the number of points in the layer direction to the number of layers (here, four). Here, equal layer thicknesses are utilized, but any other thickness combinations, or thicknesses that are allowed to adapt to minimize the divergence of the gradient within each layer, are possible to further optimize the performance of the layered structure or to better suit fabrication requirements (e.g., thinner upper layers for sequential nanoimprint lithography).

[0036] Step 103 - Figure of merit calculation. The figure of merit determines the extent to which the optimization has successfully progressed (how much the adjustments from the previous change in the dielectric constant have improved performance). It is used by the optimizer to determine the magnitude of subsequent adjustments. Here, the figure of merit is calculated based on the electric field strength at the center point of each pixel in the forward simulation (the same point where the dipole source is located in the adjoint simulation). The field strength is averaged for the two polarizations. The three color values ​​are then averaged by a weighting factor based on the performance of the previous iteration, which increases the weight for under-performing colors and decreases the weight for better performing colors. For the working distance = 0 used here in this example, this calculation corresponds to determining the spatial displacement of the three color bands to the appropriate pixel. This is importantly in contrast to previous studies, which do not consider the spatial distribution of light at the output of the splitter structure, but the focusing behavior at significant distances away from the splitter structure (greater than the mean wavelength of light, about 500 nm).

[0037] Step 104 - Update the refractive index. The gradient of the refractive index versus the figure of merit is then calculated using the gradient of the field from step 102-3-a / b and the figure of merit from step 103. The optimizer determines the overall scaling of the refractive index change, and the relative change is determined by the field gradient and the color band weightings from the figure of merit. These result in the density of material being increased or decreased at each point in the splitter design space, within the allowed range of [0:1].

[0038] Step 105 - Linestep convergence check. The optimizer utilized here takes multiple steps before varying the step size (which controls how much the calculated gradient is translated into a change in refractive index) so that it is determined whether either the gradient falls below a minimum threshold or the change in the figure of merit is less than a given threshold. If neither, the next iteration proceeds as normal, if either, the step size is reduced and the topology optimization loop continues. This time is utilized to update the color normalization values ​​to allow for a reweighting of the coefficients, increasing the weighting for poorly performing colors and decreasing the weighting for colors that are already performing relatively well. This allows the optimization to ensure that all colors perform sufficiently well without adding a non-differentiable function to the figure of merit itself. These coefficients modify both the figure of merit and the gradient to ensure self-consistency between iterations.

[0039] Step 106 - Update structural parameters: When line step convergence is complete, the parameters controlling the enforced minimum feature size and the strictness of the binarization are updated by the optimizer. The optimization loop then starts again, resetting the line step parameters in the meantime (also allowing a reduction in the figure of merit to be accepted due to increased requirements on binarization and feature size).

[0040] Step 107 - Global convergence: Once the optimization loop has converged to the binarization threshold at the target value, the optimization is considered to be fully converged and proceeds to the next step.

[0041] Step 108 - Imaging performance calculation. To calculate the color imaging efficiency, a matrix inversion process is employed and the overall performance is limited by the minimum performance of the three color bands. This allows a single value to be calculated for the efficiency of the splitter. This is calculated for the final optimized structure.

[0042] Step 109 - Comparison of Initial Condition Ensembles. The imaging performance of the ensembles of initial conditions optimized through the above process may then be compared. Since the initial conditions may result in significantly different performance in the splitter, a comparison of the minimum and average performance of the color bands may be useful. Additional constraints may also be tested, such as further limitations on feature size, fabrication robustness (invariance to vertical displacement, expansion / contraction, or misalignment), operation at additional angles of incidence, etc. This allows a single structural choice to be utilized for the color splitter design. In particular, determining performance over a range of angles of incident light is crucial for practical imaging applications. It has been found that a small working distance (i.e. = 0) allows for superior performance in this important metric. Without wishing to be bound by theory, this is likely due to a change in the fundamental principles by which light is separated for small or large working distances, where small distances require a spatial displacement of the focused field distribution with little change in the angular distribution, while larger distances may rely almost entirely on changes in the angular distribution that exits the color splitter and is later focused onto the imaging plane. Thus, the lack of reliance on a particular angular distribution for small working distances increases the tolerance of performance to non-normal angles of incoming light.

[0043] The splitter and its use in an image sensor have been validated by experimental testing on a three color (red / green / blue) image sensor implementation. The experimental test results demonstrate the ability of the splitter to efficiently create and direct beams of spectrally distinct wavelength ranges onto a pixelated photodetector array. The following experimental examples are for illustrative purposes only and are not intended to be limiting.

[0044] Experimental example: Background. Color imaging is most commonly achieved through an absorptive Bayer filter array (shown in Figure 4a), where each of the three color pixel filters allows one-third of the incident light to be transmitted to the active material below (Bayer, 1976). This allows high selectivity of the individual colors and therefore simple image processing, but significant losses are becoming an increasing problem, especially as the trend toward smaller and higher resolution sensors further reduces the average number of incoming photons per pixel.

[0045] An alternative approach to filtering is to split the incoming light by color and redirect the appropriate wavelength range to the corresponding pixel (Fig. 4b). In theory, this can increase photon collection by a factor of three on average, and built-in lensing / funneling (Sounas and Alu, 2016; Johlin et al., 2018) allows further improvements over front-illuminated CMOS configurations (although often only 50% of the illuminated surface is photosensitive) (Zhang et al., 2010). Color splitting for imaging applications has been previously explored (Chen et al., 2016) through a rather complicated process that often involves high-index material processing steps (Sounas and Alu, 2016; Miyata et al., 2019; Chen et al., 2017; Nishiwaki et al., 2013; Tamang et al., 2019; Zhao et al., 2020), new elements in the far-field of the sensor (Miyata et al., 2019; Chen et al., 2017; Nishiwaki et al., 2013; Xiao et al., 2016; Camayd-Munoz et al., 2020), and computationally expensive image reconstruction (Wang and Menon, 2015; Sahoo et al., 2017). Many designs furthermore only work with a particular polarization of light, which limits their efficiency for typical imaging applications (Xu et al., 2010; Nguyen-Huu et al., 2011; Kanamori et al., 2006). Furthermore, since sensors currently typically employ pixel sizes with pitches up to 800 nm (Sony Corporation, 2018), the color splitter must be capable of handling this high pixel density as well, which, to our knowledge, has not yet been achieved previously, with many existing designs being 10-100 times larger (Davis et al., 2017; Wang and Menon, 2015; Chen et al., 2017).

[0046] Essentially, there are three competing objectives for either color splitters or filters: selectivity, transmittance, and simplicity. Filters excel in selectivity and simplicity but have low transmittance, while splitters typically sacrifice simplicity and selectivity for increased transmittance. Selectivity can be compensated to some extent through processing (Wang and Menon, 2015; Hauser et al., 2019), but thus far the need for multiple additional materials, as well as additional layers precisely aligned away from the sensor surface, has prevented the widespread adoption of color splitters for compact sensing.

[0047] In this work, we computationally explore the design process and performance of fully dielectric, low-index (<2) nanophotonic color splitters for imaging applications, where the splitter elements are printed directly on the surface of (and therefore in the near field of) the active material layer. This will allow these novel splitters to act as simple drop-in replacements for current absorptive filters, while significantly increasing the sensor transmittance. In detail, we investigate components that include a range of geometrical complexities, from single-layer 2D patterns (which can be generated through simple single-step photolithography) to fully 3D structures (which can be fabricated through multiphoton lithography (Deubel et al., 2004) or holography (Yuan and Herman, 2016)). In all cases, we consider only a single low-index (n=1.5) dielectric material printed directly on top of a CMOS sensor, which to our knowledge has not been investigated before. Even after rigorous (but computationally trivial) color correction, we observe up to a 2.0x improvement in photon collection efficiency, with the option to increase collection by a factor of 2.4x in grayscale imaging (e.g. for low light imaging, night vision) for the same sensor.

[0048] Experimental example: Results. Splitter Design. The general philosophy of this work is to utilize adjoint-based inverse design to determine structures that successfully split broadband, randomly polarized light into pixel arrays based on three distinct wavelength bands: 400-500 nm (blue), 500-600 nm (green), and 600-700 nm (red). We investigate structures ranging in complexity from single-layer (fully 2D, metasurface) coatings, to four-layer (4 x 2D) coatings (Figures 5a, b), to fully 3D photonic crystal-like structures (Figures 5c, d), which can be produced through one to four standard photolithography steps, respectively.

[0049] A two-step process is utilized for the overall design of the splitter geometry. First, a topology optimization using a simple figure of merit is performed for a range of initial conditions, returning a locally optimal structure for each provided starting point. This is followed by a more precise analysis of the final design, using a color correction efficiency metric to determine the single best structure produced by the various initial conditions. The process for each of the steps is described below, with additional details in the Supporting Information.

[0050] Topology Optimization. Adjoint-based inverse design is an efficient method for nanophotonic topology optimization (Rodriguez et al., 2018). It works by utilizing appropriately configured forward and reverse (adjoint) simulations to calculate the gradient of the figure of merit for discrete changes in structural permittivity. By iterating this process, one can converge to a locally optimal solution for the figure of merit given initial conditions. This technique has been widely used in nanoscale optics to design building blocks ranging from optical resonators (Lu, Boyd, and Vuckovic, 2011) to integrated photonic circuits (Lalau-Keraly et al., 2013; Elesin et al., 2014; Piggott et al., 2015) to metasurfaces capable of performing computational tasks (Estakhri, Edwards, and Engheta, 2019; Liu et al., 2018).

[0051] Here, an inverse design is utilized to maximize the concentration of on-band (correctly colored) light in the center of the corresponding pixel. The pixel size is fixed at 800 nm to accommodate the highest resolution CMOS detectors publicly available today (Sony Corporation, 2018), with an assumed active area fill factor of 0.5, corresponding to a 565 nm wide square active area (Zhang et al., 2010). The total optimization area thickness was limited to 1000 nm, similar to the thickness of a color filter (FUJIFILM Holdings, no date). The material to be structured is taken to be a pure dielectric with a refractive index of n=1.5, comparable to that of a transparent photoresist, on a glass substrate of equivalent refractive index, with an air background (n=1.0) in front of the structured area and on any material-free areas.

[0052] Each optimization iteration step consists of four broadband finite-difference time-domain (FDTD) simulations, each corresponding to two orthogonal light polarizations, with one forward simulation and one adjoint simulation. By orienting the simulation coordinate system diagonally with respect to the sensor layout, system symmetry can be exploited, which requires that only ¼ of the splitter area is actively simulated. For an 800 nm pixel size, the active simulation area therefore has dimensions of 565 × 1130 nm.

[0053] The optimization figure of merit (FoM) is a function whose gradient the optimization takes to determine the evolution of the structure permittivity and hence the geometry. Ideally, the final color correction response (described below) would be used for the optimization, but such an expression is not differentiable, making the computation of the gradient impossible. To address this, the intensity of the on-band light electric field (|E| 2 ), which in turn corresponds to a simple adjoint simulation, corresponding to a single dipole emitter of the appropriate emission frequency at the center of each pixel, scaled by the electric field from the forward simulation. This follows the general method of field-localized optimization described in (Miller, 2012) and, coincidentally, is similar to the method used by a parallel work in (Camayd-Muñoz et al., 2020).

[0054] As adjoint-based topology optimization is a local optimization technique, the choice of initial conditions has a significant impact on the resulting design and performance. For each layer complexity, a series of optimizations was performed, ranging from fully filled to fully empty, with nine equally spaced uniform index levels in between as starting points for the optimization. This allows us to compare the eleven resulting structures for each configuration, utilizing a full color correction efficiency calculation, and to select the one with the best overall performance. Additional details and flow charts of the optimization and simulation process can be found in the Supporting Information.

[0055] Color Correction Efficiency. At the most basic level, a color splitter can be operated similarly to a color filter, which attenuates the incoming light to a point where the off-band color response (e.g., red light into a blue pixel region) is sufficiently low, and then determines the transmission efficiency of the on-band color response. However, since the signals from four single-color pixels are combined to create one RGB image pixel, correlation between pixels can be used to more accurately determine the colors in the image. This allows color selectivity to be improved, at the expense of some additional transmission. Here we describe a general method for evaluating the overall color imaging efficiency using a simple matrix inversion process, applicable to any color isolating component (splitter or filter).

[0056] We start by creating a 3x3 matrix, R, of color responses, where the rows correspond to the red, green, and blue pixels, and the columns are the average red, green, and blue color transmissions to the respective pixels. In a theoretically perfect splitter, R would be an identity matrix, but in practical designs there will be some degree of off-color transmission represented by the non-zero off-diagonal elements. The inverse of the response matrix represents a linear combination of the pixel responses, which will restore the identity matrix (I) and thus the ideal response, by using a combination of color pixels to effectively eliminate the off-band response. However, to avoid amplification of noise, the value of any element in the inverse matrix must be limited to 1 (i.e., allowing for signal attenuation, not gain). Furthermore, the total performance of the system will ultimately be limited by the response of the worst performing color correction element, and therefore the response must be scaled uniformly. Therefore, we normalize the inverse matrix by the largest element in the matrix, which will result in:

number

[0057] Thus, the color correction response to a uniform input is RQ, which is equal to ηI, where η is the overall color correction efficiency.

[0058] Note that while matrix inversion is often a computationally expensive process, the small 3×3 size of these response matrices, and the need for this to be performed only once during the splitter design stage, makes the color correction process computationally trivial; during actual imaging, the correction is simply a uniform linear combination of the sensor's red, green, and blue signals.

[0059] Splitter performance. The best splitter performance was achieved with a full 3D structure. The structure is depicted in Figures 5c and 5d, the color response is shown in Figure 6, and the Poynting vector components (representing the flux of energy) normal to the substrate at the surface are shown in Figure 6b. It can be seen that all three colors are not only spatially isolated but also well focused at the center of the active area of ​​the corresponding pixel, even without the need for a microlens array. The four-layer structure (shown in Figures 5a and 5b) looks similar in spatial power transmission (Figure 6a) and also shows a strong spectrally selective focusing of light onto the sensor surface. However, the selectivity is slightly lower, especially for the green pixel, as can be seen in Figure 6c.

[0060] The simulated transmission spectra are shown in Figure 6c and Figure 6d for the four-layer splitter and the 3D splitter, respectively, and are compared to those of a representative commercial Bayer filter absorptive layer (FUJIFILM Holdings, no date). It is observed that the collection efficiency of each color component is higher than that of the absorptive filter, indicating that the light is successfully rerouted to the correct filter region. This is likely done through a combination of refractive and interference effects, which allows an efficient broadband response to be created in the near field of the sensor (Johlin et al., 2018). It is also noted that while the collection efficiency is much higher, the selectivity is lower. However, as explained above, this can be compensated for using a linear combination of the color pixel readings via Equation 1 to produce the final fully color-corrected image, at the expense of some sensitivity. The estimated efficiency of the 3D color splitter is 34% for fully color-corrected imaging and 81% for grayscale imaging. This compares to the Bayer filter, which has a color correction estimation efficiency of 17% and a grayscale efficiency of 33%.

[0061] Interestingly, the color separation mechanism of the splitters investigated here appears to be distinct from those previously well studied, in that the close proximity of the color splitting structures precludes the use of angular redistribution to spectrally separate the light (Nishiwaki et al., 2013), relying instead on perfect spatial separation at the base of the splitter with little redistribution in angular space of the transmitted light. This also provides an additional benefit in that the splitters designed here appear to perform better at off-angle illumination than previous designs (Camayd-Muñoz et al., 2020), which relied more on scattering or diffraction to reconstruct the angular distribution from the splitter. See the Supporting Information for further characterization of these effects.

[0062] Simulated full-color imaging. To verify the accuracy of the color correction as well as to visualize the increased sensitivity, it is instructive to directly compare the performance of the two sensor designs (Bayer filter vs. splitter) on realistic images. To do this, we use a freely available 2000 × 2000 pixel hyperspectral image (Brainard, DH 2004) to simulate the operation of both the 3D color splitter as well as the Bayer array for comparison. Through a pixel-by-pixel spectral convolution, first with the light source spectrum (AM1.5 solar spectrum is used here), followed by our color response, and finally applying the linear color correction operation from Eq. 1, we obtain the expected full-color image response for each design. A reference image with 50% efficiency and perfect color separation is shown in Fig. 7b, along with the filter and splitter responses after color correction as well as in grayscale imaging.

[0063] By comparing the color-corrected response of the filter (Fig. 4c) and the splitter (Fig. 4d) to the reference image, the splitter reproduces the true color accurately and with a higher brightness than the standard Bayer filter. This is increasingly visible in the grayscale images (Fig. 4e, 4f), where the splitter exceeds even the brightness of the 50%-efficient reference image. From the imaging efficiency distribution in Fig. 4a, it can be seen that, indeed, for color imaging, the collection efficiency is increased by a factor of 2.0, from 15.9% for the Bayer filter to 31.3% for the color splitter, and the grayscale efficiency is increased by a factor of 2.4, from 33% to 80.0%. The difference in performance from the estimated efficiency occurs due to the non-uniform color distribution of the reference image, which is also responsible for the extended distribution in the splitter color distribution.

[0064] Furthermore, all of the above comparisons conservatively assume a perfect microlens structure above the Bayer filter array that is lossless and focuses all transmitted light onto the active area. Without the microlens array, the Bayer filter efficiency would be further reduced by a factor of 2 if there was an imperfect lens somewhere in between. Splitters do not require such a microlens array since they, inherently, already focus the light onto the active area.

[0065] Note that the color correction used here is rigorous in that all cross-color responses must be eliminated. Because of this, even Bayer filters that have some color contamination, especially in the green pixel response, will show reduced efficiency when similarly corrected. By limiting the rigor of this correction, any position on the trade-off continuum from no color correction to full color correction can be selected, and thus increased efficiency can be realized for lower color contrasts.

[0066] Complexity and robustness. Reducing the layer complexity to only four individual layers (the structures shown in Fig. 5a and 5b) by an order of magnitude still maintains most of the performance of the color splitter. For full color-corrected imaging, an efficiency of 21% is obtained, still significantly higher than the filter, especially when compared to the filter without the microlens array. For grayscale imaging, the performance is even slightly higher than the full 3D splitter, at 83%. With two layers (Fig. 8c) and even just a single layer (Fig. 8b), the splitters are still able to generate full color images, albeit at a much lower efficiency. As expected, they still maintain high grayscale performance without the need for a microlens array.

[0067] This trend in performance versus complexity is made even clearer in Figure 8a, which shows the simulated color correction imaging efficiency versus the number of splitter layers. Note that the full 3D structure here is equivalent to a 40-layer structure with a finite optimized voxel size. In particular, the large gain in performance between the 2-layer and 4-layer configurations is clearly discernible, with the imaging efficiency moving from below the filter performance to well above it. This jump in performance indicates that the distinct layers spectrally separate the light as it travels through the splitter, and demonstrates that the change in structure topology progressively separates and focuses the three color bands (as the splitter thickness is constant for ease of comparison of the results in this experimental example). Further characterization of the 1-layer and 2-layer splitter structures is shown in the Supporting Information.

[0068] Finally, for such a color splitter to be usable in a practical device, robustness against both imperfections and non-ideal operating conditions should be guaranteed. Due to the broadband response of the splitter, the performance should be generally invariant to small (uniform) expansion or contraction of the structure. Furthermore, the strong focusing within the active area of ​​the pixel allows some built-in robustness against misalignment. Although all the analyses here assume randomly polarized light, we further ensure that such a splitter works well for any linear polarization as well as for non-normal incident light, as is common in conventional imaging systems. For further discussion of these effects, see the Supporting Information.

[0069] Experimental examples; proven benefits. Herein, we show that efficient color splitters can be designed using low-index dielectric layered 2D or fully 3D structures directly on the surface of a CMOS sensor. The performance significantly exceeds that of conventional filter arrays for splitter designs, even with only four layers, and the efficiency gain is as much as four times higher than CMOS sensors without a microlens array. The splitter structures exhibit added functionality in the ability to choose any point in the color accuracy vs. sensitivity tradeoff, allowing accurate color imaging, nearly lossless grayscale imaging, and anything in between, all on the same sensor. This can confer particular advantages in color biological imaging, where high efficiency, small pixel size, and color sensitivity are all desirable (Wu et al., 2008; Farsiu et al., 2004). Furthermore, these splitter structures do not require a focusing microlens array, even when used with an 800 nm pixel CMOS sensor configuration with a 50% active area fraction, further simplifying the use of such components.

[0070] The placement of the splitter near-field of the sensor surface offers inherent advantages over previous color splitter configurations in both design and performance: removing the need for a precise but thick separation layer can simplify fabrication, and operating without angular redistribution of the light transmitted to the sensor surface appears to offer increased invariance to non-normal illumination conditions, as well as reducing losses and crosstalk at the sensor (Zhang et al., 2010).

[0071] Finally, the color correction efficiency metric developed here for the splitter design process should be general and useful in comparing the performance of new splitter or filter structures as well. Similar design and analysis techniques can be further utilized for other applications, such as infrared imaging or photovoltaics, particularly where splitting and focusing are both advantageous (Xiao et al., 2016).

[0072] Experimental example: Supplementary information. Inverse design. Inverse design is performed through topology optimization using a modified version of the Lumopt package (Lalau-Keraly et al., 2013). Broadband (400-700 nm wavelength) simulations are utilized to leverage the efficiency of the FDTD technique with a large spectral range and to prevent the need for additional simulations for each wavelength range. Ideally, under broadband, plane wave illumination, all light in each wavelength range would be perfectly focused at the center of the corresponding pixel with no spectral overlap between pixels. This is represented in the optimization by forward (400-700 nm plane wave illumination) and adjoint (3x dipoles placed at pixel centers, each with an 80 nm wavelength range centered in a 100 nm color band to reduce spectral overlap) simulations.

[0073] The full 3D optimization allows each volume element of a predefined structural region to be modified independently. For 2D and layered structures, the spatial gradients are averaged in the direction orthogonal to the 2D design plane, which allows the average performance in the optimization region to determine the direction of optimization. The optimization is performed using the limited-memory Broyden-Fletcher-Goldfarb-Shanno algorithm with bound constraints (L-BFGS-B) (Van Der Walt, Colbert, and Varoquaux, 2011), implemented in Scipy. The software utilizes built-in hyperparameters to converge both the binarization of the dielectric constant (starting from a continuous value of the dielectric constant and moving to either the presence or absence of material) as well as the maximum radius of curvature of the structure (set to 100 nm to be compatible with 3D nanolithography) (Johlin et al., 2018; Deubel et al., 2004). The optimization is performed using a binarization threshold of 99%, and a threshold of 10 -4 We end up with a gradient in the relative magnitude of

[0074] Each color pixel contributes its own value of the Figure of Merit (FoM), the sum of which simply corresponds to the sum of the three color components. The minimum value is not used in this case because, again, the FoM operation must be differentiable in order for the gradient to be calculated correctly for the optimization. However, it is still important to bias the optimization towards improving the worst performing color pixels. To do so, between line search steps in the optimization (at the same time that penalty functions for discreteness and curvature are implemented), a uniform linear weighting is applied to each of the three color components, equal to the inverse of the individual color performance from the previous optimization round. This allows for increased weighting of the less well performing color components without introducing errors into the gradient calculation.

[0075] The layer complexities of 1-, 2-, and 4-layer structures, as well as full 3D structures (equivalent to 40 layers with a 25 nm pixel size), were individually optimized. The layered structures do not enforce minimum feature size restrictions between layers.

[0076] Simulation. Simulations are performed using an FDTD solution (Lumerical Inc.). The optimization is performed using a mesh size of 25 nm in all directions and 31 equally spaced wavelength points in the optimization region. The optimization region is located just above the glass substrate and the figure of merit is measured at one grid point below the surface.

[0077] The forward simulation uses the magnitude of the electric field at the center of the pixel as a figure of merit. After optimization, the final structure is simulated on a higher resolution grid of 10 nm and fully binarized to ensure that no gradient index material remains from the optimization.

[0078] Design Process Flowchart. A flow chart of the complete reverse design process is shown in Figure 3.

[0079] Lower complexity structures. Lower complexity 1-layer and 2-layer structures are shown in Figures 8b and 8c. The total thickness of all splitter configurations is fixed at 1000 nm, and the layers are equally spaced within this domain. We will now further characterize these lower complexity structures.

[0080] The optical responses of the lowest complexity single and two-layer structures are shown in Figure 9. The Poynting Vector images in Figures 9a and 9b show improved color separation as layer complexity increases, visible through increasingly pure color focal spots in the center of the pixel active area. The transmission responses in Figures 9c and 9d similarly show improved color separation with increasing number of layers, especially when moving to the four-layer design (Figure 6a).

[0081] Although single-layer structures offer very limited selectivity and contrast, the responses of the different color pixels are still distinct enough to allow color correction methods to be employed, making color imaging possible (albeit at lower sensitivity) even with these very simple structures. Moving to two layers improves this performance significantly, and the trend in imaging performance is shown in Figure 8a.

[0082] Design robustness. The broadband response of the structures designed here inherently provides some robustness to fabrication imperfections. Due to the scale invariance of Maxwell's equations, global expansion or contraction of the structure should only result in a proportional shift in the color response. The broad response, spanning 100 nm wavelengths per color band, suggests that such problems are minimal.

[0083] Moreover, the design is also fairly robust to alignment problems; plots of the magnitude of the Poynting vector (its integral gives the power transmission; Figs. 9 and 6) demonstrate that slight misalignments of the array will still result in most of the power being centrally located within the corresponding pixel. This power concentration is made even clearer in Fig. 10, where we observe that the transmission at each pixel is highly confined to the central region, with a full width at half maximum between 200 nm and 250 nm for the transmission using the 3D structure.

[0084] In terms of sensitivity to misalignment, for example with a four-layer splitter, the red pixel (with the widest distribution of the three colors) still contains 97% of the collected light in the 400 nm diameter central area of ​​the pixel, and 90% in the central 275 nm diameter. This suggests that even for a misalignment of 145 nm, the collection efficiency should be reduced by less than 10%. This sensitivity is even more robust if the fill factor of the CMOS detector is greater than 0.5, which allows for larger misalignments while still maintaining high efficiency.

[0085] Angular Response. Because the light incident on the photodetector in any practical imaging system is not perfectly columnar, it is important to consider the response of the system to non-normal incoming light. Here we calculate the expected angular response for incoming light with non-normal angles ranging from 0 to 20 degrees.

[0086] For a particular angular response, the metric of color imaging efficiency is not particularly meaningful, as any real image pixel is subject to a distribution of angles, and thus the need to image perfectly with a single angular response is not feasible. This would therefore result in meaningless color correction inversion matrices and imaging efficiencies. Instead, we calculate the sorting efficiency, defined here as the fraction of on-band incoming light that falls into the correct active area. This allows relative performance under increasingly non-normal incident light to be compared, as well as comparisons to previous work (Camayd-Muñoz et al., 2020) that employs a similar metric (albeit without the limited active sensor area).

[0087] From Fig. 11, the performance of the three color band responses (corresponding to red, blue, and two green pixels) can be seen relative to their normal responses. From Fig. 11a, it can be seen that although the performance will likely be limited by the red pixel response (for angles < 12 degrees), the relative sorting efficiency will still be greater than 0.5. This compares favorably with the relative angular sorting efficiency of previous color splitters that utilized a significant gap between the splitter and the sensor surface (Camayd-Munoz et al., 2020). This is likely indeed due to the increased proximity of the splitter to the sensor limiting the displacement of the focal spot, thereby reducing the detrimental effect of non-normal incident light on the splitter performance.

[0088] Interestingly, the four-layer splitter structure in Fig. 11b shows even better relative angle performance, not falling below a relative sorting efficiency of 0.5 even for incoming light angles as high as 20 degrees, indicating that the performance degradation between the full 3D structure and the layered 2D structure is reduced in practical imaging applications.

[0089] This is further elucidated in Fig. 11c, which shows the absolute sorting efficiency of the two configurations. The limiting color response (red or blue, depending on the angle and configuration) still exceeds even the normal incidence limiting response of a perfectly focused color Bayer filter (sorting efficiency of 0.18 for blue pixels, from Fig. 6c, 6d), shown for reference as a solid grey line. Note that in practice the assumption of a perfectly ideal microlens array for the color filters would be even more overly optimistic, which means that this response would in fact be significantly lower (and therefore the splitter would perform better by an even larger margin). Furthermore, especially for the red and blue pixels, the active area represents only 12.5% ​​of the sensor surface, so even at significant angles of incoming light, the emission is still substantially focused in the correct area, with the defocused limit (which assumes all unreflected light is uniformly spread over the sensor) shown as a dashed grey line.

[0090] The transmission into the active area for three off-normal angles is shown in Figure 12 for both the 3D structure (Figures 12a-c) and the four-layer 2D structure (Figures 12d-f). It can be seen that as the angle of the incoming light increases, the selectivity decreases, but even at 16 degrees incidence, some selectivity is still present. The total light capture (grayscale transmission) is interestingly superior in the four-layer structure for high angle incident light, indicating that although the spectral selectivity is reduced, light focusing by the structure is still maintained. Even at a 16 degree incoming angle, over 75% of the light is transmitted to the sensor area for a fill factor of 0.5. The ability of all structures to maintain high overall transmission even at significant angles of incoming light is likely due to the spatial separation of light at the base of these structures, as opposed to the angular separation (and reliance on multi-micrometer gaps) of those previously studied.

[0091] Polarization Response. All responses shown in the text assume randomly polarized incoming light (i.e., a uniform distribution of polarization angles), but in some cases there may be a significant degree of linear polarization of the incoming light, in which case the polarization response of the splitter becomes important. In Figure 13 we investigate the specific responses of two main splitter designs to two distinct, orthogonal linear polarizations of the incoming light.

[0092] From Fig. 13a and Fig. 13b, the green response is slightly higher in Fig. 13a, and the red and blue are slightly higher in Fig. 13b, but the overall efficiency and contrast are almost similar between the two orthogonal conditions for the 3D splitter design. In detail, the average sorting efficiency is 44% for polarization a and 48% for polarization b. Similarly, in Fig. 13c and Fig. 13d, there is a slight difference in the efficiency of the red and blue responses, but again, the overall efficiency and contrast are similar, with average sorting efficiencies of 41% and 37% for polarization a and polarization b, respectively. This shows that efficient color imaging can be performed using the splitter structure even in situations where imaging is performed with linearly polarized incoming light (even for any unknown polarization angle).

[0093] Direction of energy transmission. Although the color splitters considered here are designed to be printed directly on the sensor surface (and thus in the near field of the sensor), the transmitted energy can have significant non-normal angular components, especially when light is collected from a large area and transmitted to a small focusing area. When analyzing the transmission of the color splitter, we only consider the contribution of the Poynting vector component in the surface normal direction, but significant components in other directions can contribute to energy losses and thereby reduce the efficiency of the system.

[0094] In Figure 14, we quantify these off-normal components, which show the energy probability density of the three light color bands for the two splitter structures versus their angle from the surface normal. Each line shows the probability density for a different Poynting vector angle of the correct color band at the sensor surface and the energy within the pixel active area. Thus, the delta function at 0 corresponds to all energy propagating perfectly normal to the surface (equivalent to a plane wave input), and the more extended the distribution, the more dominant the off-angle contributions to the energy at the sensor's surface. It can be seen in both Figures 14a and 14b (3D splitter and 4-layer splitter, respectively) that we have extremely narrow distributions for all three colors, with over 90% of the energy contained at angles less than 0.5 degrees. This fits well with the high performance observed in the transmission of the splitter structures, as the higher angle contributions would be seen as losses in our calculations. This also suggests that due to the increased difficulty in focusing and maintaining a narrow angular transmission distribution of longer wavelength light, in some cases it will be more efficient for these designs to include two red pixels (as opposed to two green pixels).

[0095] The narrow angular distribution of these devices is also in stark contrast to the performance of previous splitter structures that rely heavily on diffraction (Nishiwaki et al., 2013) or scattering (Camayd-Muñoz et al., 2020) to purposefully redistribute different spectral bands of light to different angles to facilitate splitting. This indicates that the design of splitters directly on the sensor surface not only modifies the configuration, but also favors designs for different mechanisms of operation.

[0096] Experimental example: Reference list. Bayer, Bryce E., 1976, "Color Imaging Array" (U.S. Patent No. 3,971,065(A)). Brainard,DH, 2004. Hyperspectral Image Data. Camayd-Munoz,Philip、Conner Ballew、Gregory Roberts、およびAndrei Faraon、2020、「Multifunctional Volumetric Meta-Optics for Color and Polarization Image Sensors」, Optica7(4):280~83. Chen, Bo Han, Pin Chieh Wu, Vin-Cent Su, Yi-Chieh Lai, Cheng Hung Chu, I Chen Lee, Jia-Wern Chen, 2017, Nano Letters17(10):6345~5 Chen, Qin, Xin Hu, Long Wen, Yan Yu, David RS Cumming, 2016, Nanophotonic Image Sensors, Small12(36):4922~35. Davis,Matthew S, Wenqi Zhu, Ting Xu, Jay K Lee, Henry J Lezec, and Amit Agrawal, 2017. Nature Communications8(1):1347. Deubel, Markus, Georg Von Freymann, Martin Wegener, Suresh Pereira, Kurt Busch, Costas M Soukoulis, 2004, Nature Materials3(7):444~47. Elesin,Y.、BSLazarov、JSJensen、およびO.Sigmund、2014、Photonics Nanostructures-Fundamentals and Applications12(1):23~33. Estakhri,Nasim Mohammadi, Brian Edwards, Nader Engheta, 2019, Inverse-designed metastructures that solve equations Science363(6433):1333~38 https: / / doi.org / 10.1126 / science.aaw2498. Farsiu,Sina, Dirk Robinson, Michael Elad, Peyman Milanfar, 2004, International Journal of Imaging Systems and Technology14(2):47~57. FUJIFILM Holdings, undated, "RGB Primary Color Filters for Image Sensor Applications: 5th Generation COLOR MOSAIC", https: / / www.fujifilm.com / us / en / business / semiconductor-materials / image-sensor-color-mosaic / rgb / applications. Hauser, Jonathan, Michael A Golub, Amir Averbuch, Menachem Nathan, Valery A Zheludev, Omer Inbar, and Shay Gurevitch, 2019, "High-Photon-Throughput Snapshot Colour Imaging Using a Monochromatic Digital Camera and a Pupil-Domain Diffuser", Journal of Modern Optics 66(7): 710 - 25. Johlin, Eric, Sander A Mann, Sachin Kasture, A Femius Koenderink, and Erik C Garnett, 2018, "Broadband Highly Directive 3d Nanophotonic Lenses", Nature Communications 9(1): 1 - 8. Kanamori, Yoshiaki, Masaya Shimono, and Kazuhiro Hane, 2006, "Fabrication of Transmission Color Filters Using Silicon Subwavelength Gratings on Quartz Substrates", IEEE Photonics Technology Letters 18(20): 2126 - 28. Lalau-Keraly、Christopher M.、Samarth Bhargava、Owen D.Miller、およびEli Yablonovitch、2013、「Adjoint shape optimization applied to electromagnetic design」、Optics Express21(18):21693、https: / / doi.org / 10.1364 / oe.21.021693。 Liu,Dianjing、Yixuan Tan、Erfan Khoram、およびZongfu Yu、2018、「Training Deep Neural Networks for the Inverse Design of Nanophotonic Structures」、ACS Photonics5(4):1365~69、https: / / doi.org / 10.1021 / acsphotonics.7b01377。 Lu,Jesse、Stephen Boyd、およびJelena Vuckovic、2011、「Inverse design of a three-dimensional nanophotonic resonator」、Optics Express19(11):10563、https: / / doi.org / 10.1364 / OE.19.010563。 Miyata,Masashi、Mitsumasa Nakajima、およびToshikazu Hashimoto、2019、「High-Sensitivity Color Imaging Using Pixel-Scale Color Splitters Based on Dielectric Metasurfaces」、ACS Photonics6(6):1442~50。 Nguyen-Huu、Nghia,Yu-Lung Lo、およびYu-Bin Chen、2011、Color Filters Featuring High Transmission Efficiency and Broad Bandwidth Based on Resonant Waveguide-Metallic Grating」、Optics Communications284(10~11):2473~7 Nishiwaki, Seiji, Tatsuya Nakamura, Masao Hiramoto, Toshiya Fujii, Masaaki Suzuki, 2013, Nature Photonics7(3):240. Piggott,Alexander Y.、Jesse Lu、Konstantinos G.Lagoudakis、Jan Petykiewicz、Thomas M.Babinec、およびJelena Vuckovic、2015、Inverse Design and Demonstration of a Compact and Broadband On-Chip Wavelength demultiplexer Nature Photonics9(6M):374~77 https: / / doi.org / 10.1038 / nphoton.2015.69. Rodriguez, Alejandro W., Zin Lin, Jelena Vuckovic, Weiliang Jin, Sean Molesky, Alexander Y.Piggott, 2018, Nature Photonics12(11):659~70; Sahoo,Sujit Kumar、Dongliang Tang、およびCuong Dang、2017、「Single-Shot Multispectral Imaging with a Monochromatic Camera」、Optica4(10):1209~13。 Sony Corporation、2018、「Sony Releases Stacked CMOS Image Sensor for Smartphones with Industry’s Highest 48 Effective Megapixels」、https: / / www.sony.net / SonyInfo / News / Press / 201807 / 18-060E / 。 Sounas,Dimitrios L、およびAndrea Alu、2016、「Color Separation Through Spectrally-Selective Optical Funneling」、ACS Photonics 3(4):620~26。 Tamang,Asman、Rion Parsons、Ujwol Palanchoke、Helmut Stiebig、Veit Wagner、Alberto Salleo、およびDietmar Knipp、2019、「Color Sensing by Optical Antennas: Approaching the Quantum Efficiency Limit」、ACS Photonics6(8):2041~48。 Van Der Walt,S.、Colbert,S.C.、Varoquaux,G.、2011、The numpy array: a structure for efficient numerical computation、Computing in Science&Engineering13、22。 Wang,Peng、およびRajesh Menon、2015、「Ultra-High-Sensitivity Color Imaging via a Transparent Diffractive-Filter Array and Computational Optics」、Optica2(11):933~39。 Wu,Changfeng、Barbara Bull、Craig Szymanski、Kenneth Christensen、およびJason McNeill、2008、「Multicolor Conjugated Polymer Dots for Biological Fluorescence Imaging」、ACS Nano2(11):2415~23。 Xiao,T.Patrick、Osman S.Cifci、Samarth Bhargava、Hao Chen、Timo Gissibl、Weijun Zhou、Harald Giessen、Kimani C.Toussaint、Eli Yablonovitch、およびPaul V.Braun、2016、「Diffractive Spectral-Splitting Optical Element Designed by Adjoint-Based Electromagnetic Optimization and Fabricated by Femtosecond 3D Direct Laser Writing」、ACS Photonics3(5):886~94、https: / / doi.org / 10.1021 / acsphotonics.6b00066。 Xu,Ting、Yi-Kuei Wu、Xiangang Luo、およびL Jay Guo、2010、「Plasmonic Nanoresonators for High-Resolution Colour Filtering and Spectral Imaging」、Nature Communications1:59。 Yuan, Liang Leon, and Peter R Herman, 2016, "Laser Scanning Holographic Lithography for Flexible 3d Fabrication of Multi-Scale Integrated Nano-Structures and Optical Biosensors", Scientific Reports6:22294. Zhang,Fudi, Jianqi Zhang, Cui Yang, and Xiang Zhang, 2010, "Performance Simulation and Architecture Optimization for CMOS Image Sensor Pixels Scaling down to 1.0μm", IEEE Transactions on Electron Devices57(4):788~94. Zhao, Nathan, Peter B Catrysse, and Shanhui Fan, undated, "Perfect RGB-IR Color Routers for Sub-Wavelength Size CMOS Image Sensor Pixels," Advanced Photonics Research, doi:10.1002 / adpr.202000048.

[0097] Exemplary versions and several variations of the dielectric splitter and its use in image sensors have been described above without any intended loss of generality. Further examples of modifications and variations are now provided. Still further variations, modifications and combinations thereof are contemplated and will be apparent to those skilled in the art. It should be understood that the exemplary variations or modifications are provided for the purpose of improving the understanding of those skilled in the art and are not intended as a limiting description.

[0098] For example, splitter blocks may accommodate a variety of sizes and shapes that will generally accommodate the type of photodetector array and other features of the operating environment. A splitter will require a minimum thickness to split the incident radiation into multiple beams and to focus the multiple beams at or near the output surface of the block. The thickness is defined as the perpendicular distance between the input and output surfaces of the block. A useful estimate for the minimum thickness (t_min) for a splitter block is:

number

[0099] In another example of the variation, the splitter is designed to split the incident radiation into multiple beams with a near-field focus relative to the output surface of the splitter block. The near-field focus feature advantageously allows for variation in the positioning of the output surface of the splitter block near the active layer surface of the pixelated photodetector array. The extent of the variation is that the separation of the active layer surface and the output surface, oriented parallel and opposite to each other, can be any vertical distance between the active layer surface and the output surface that is less than 600 nm. Generally, the separation of the active layer surface and the output surface is less than 500 nm. In one example, the separation of the active layer surface and the output surface is less than 450 nm. In another example, the separation of the active layer surface and the output surface is less than 400 nm. In another example, the separation of the active layer surface and the output surface is less than 350 nm. In another example, the separation of the active layer surface and the output surface is less than 300 nm. In another example, the separation of the active layer surface and the output surface is less than 250 nm. In another example, the separation between the active layer surface and the output surface is less than 200 nm. In another example, the separation between the active layer surface and the output surface is less than 150 nm. In another example, the separation between the active layer surface and the output surface is less than 100 nm. In another example, the separation between the active layer surface and the output surface is less than 50 nm.

[0100] The splitter output surface can accommodate variations, as well as variations in the topology of the overall splitter block, so long as the splitter features that split the incident radiation into multiple beams and focus the multiple beams at or near the output surface of the block are not eliminated. The output surface can often be configured and / or selected to enhance the splitter feature that focuses the multiple beams at or near the output surface. For example, the output surface of the block can be configured with regions of lower intensity transmission to separate the multiple beams so that each individual beam is localized at the output surface and separated from neighboring beams.

[0101] The splitter may be used in combination with many different pixelated photodetector array types, including, for example, passive pixel sensors or active pixel sensors. In other examples, the pixelated photodetector array types may include CMOS sensors, NMOS sensors, or CCD sensors.

[0102] The splitters can be manufactured by multiple techniques, for example by combining multiple laminations together or without stacking or layering, and it is feasible to manufacture in sub-blocks that are oriented differently than the layers (i.e., columnar or vertical orientation), or in whole blocks. It may also be feasible to manufacture from a central point outwards.

[0103] Layers can be distinguished from blocks in that within an individual layer, the xy dimensions of the geometric structures remain unchanged at any point in the z dimension. In a layer structure, a first face (first xy plane) is oriented toward the input surface of the block, a second face (second xy plane) is oriented toward the output surface of the block, and the thickness (z dimension) is defined as the perpendicular distance between the first and second faces. Within each layer, the xy shape of each geometric structure remains unchanged throughout any point in the thickness (z dimension) of the layer. In contrast, in a block integrated from multiple layers, in going through the thickness (z dimension) from the input surface to the output surface, the xy shape is expected to change stepwise in the transition from the first layer to the second layer, or from the second layer to the third layer, etc., since many layers make up the block. Due to this difference in the terms layer and block, a single layer that meets the sufficient minimum thickness set in Equation 2 is indistinguishable from a block.

[0104] In some examples, the splitter may be made as a single formed block, or may be integral, or may be multiple laminates integrated to form a block and multiple geometric structures. When made from multiple laminates, the number of laminates is often greater than 2 laminates, greater than 3 laminates, greater than 4 laminates, greater than 5 laminates, greater than 10 laminates, greater than 15 laminates, greater than 20 laminates, greater than 25 laminates, greater than 30 laminates, greater than 35 laminates, or any number of laminates in between. A complete 3D splitter is considered to be 40 laminates for purposes of the experimental example only. However, in other implementations, a complete 3D splitter may be fewer or greater than 40 laminates. In some examples, a complete 3D splitter may be any number of laminates greater than 10 laminates. In other examples, a complete 3D splitter may be any number of laminates greater than 20 laminates. Experimental observations have confirmed that splitter blocks made from three or more layers perform reliably better than filters in comparable performance metrics (i.e., even when metrics related to attenuation of incident radiation intensity are ignored or not compared, as the filters are comparatively poor in such metrics). Figure 15 shows a plot of average sorting efficiency as an example of such a comparable metric. The average sorting efficiency for a color imaging implementation is determined according to the method described in the experimental example and plotted with data points for a 1-layer splitter, a 2-layer splitter, a 3-layer splitter, a 4-layer splitter, and a 40-layer splitter, and the plot is compared to the average sorting efficiency value (dashed line) for a conventionally available filter with added microlenses. The sorting efficiency is the fraction of light collected by the correct color pixel (e.g., blue light that travels to the active area of ​​a blue pixel), where we take the arithmetic average of the three colors (red / green / blue). In this performance metric, the two-layer splitter is similar to the filter and lens combination, but the three-layer splitter clearly outperforms the filter and lens combination.

[0105] When the splitter is made from multiple laminations, the size and shape of the laminations may vary from block to block and even within a particular block. Furthermore, even within an individual layer, the length, width and thickness dimensions need not be constant and considerable variation may occur and may be intended. In a layered structure, a first face (first xy plane) is oriented toward the input surface of the block, a second face (second xy plane) is oriented toward the output surface of the block, and the thickness (z dimension) is defined as the perpendicular distance between the first and second faces. This thickness may be uniform or may vary as desired for a particular implementation. In some examples, the layers are of uniform thickness throughout the block. In other examples, the layers are of different thicknesses throughout the block. In still other examples, at least one layer in a block has a varying thickness (i.e., the perpendicular distance between the first and second faces of the layer) along the circumference or diameter of the layer, or a varying thickness when considering two or more points of the layer. In a further example, at least one layer in a block has a uniform thickness at all points.

[0106] Dielectric splitters can be incorporated into many types of image sensors to benefit multiple imaging implementations, including, for example, digital cameras, camera modules, camera phones, optical mouse devices, medical imaging equipment, night vision equipment such as thermal imaging devices, microwave imaging devices, ultraviolet imaging devices, biological imaging, machine vision, smart sensing materials, real-time motion detection, high resolution imaging, and aerospace imaging, among others. The adjoint-based inverse design shows an approach that can be easily adapted to splitter configurations for multiple imaging implementations. Design and analysis approaches similar to those demonstrated in the experimental examples can be easily adapted to other applications, such as infrared imaging or multijunction photovoltaics, and may be suitable for applications where splitting and near-field focusing are both advantageous.

[0107] The splitter may be made from a single dielectric material, but may also be readily adapted to be made from a combination of two dielectric materials, or a combination of three dielectric materials, or a combination of four or more dielectric materials. While ease of manufacture and cost of production may benefit from a single dielectric material, particularly in high volume applications such as consumer electronics, in more specific and higher margin applications, two or more dielectric materials may be used as needed. While dielectric materials with low refractive indices (n<2) may offer cost and other benefits at manufacturing scale, the use of dielectric materials with high refractive indices (n>2) may be accommodated or even desired in certain implementations, for example, in implementations where two or more dielectric materials are combined to form the splitter. Common dielectric materials are magnesium fluoride (n=1.4), silicon dioxide (n=1.5), tantalum pentoxide (n=2.3), zinc sulfide (n=2.3), titanium dioxide (n=2.4), and silicon nitride (n=2.1), as well as polymeric materials such as poly(methyl methacrylate) (PMMA: n=1.5), polydimethylsiloxane (PDM: n=1.4), and polyvinyl chloride (PVC: n=1.5). Another dielectric material option is a photoresist (also known simply as resist), a light-sensitive material, typically a polymer, used in processes such as photolithography and photoengraving. In the case of a positive photoresist, the light-sensitive material is degraded by light, and the developer dissolves away the areas exposed to light. In the case of a negative photoresist, the light-sensitive material is strengthened (either polymerized or crosslinked) by light, and the developer dissolves away only the areas not exposed to light.

[0108] The dielectric splitter can be fabricated by several available techniques. Exemplary options for the fabrication of a single layer structure include conventional UV lithography, direct-write lithography, or electron-beam lithography.

[0109] Option 1.1 - Conventional UV Lithography. The single layer structure should be relatively trivial to produce using standard patterning of photoresist using deep UV or extreme UV photolithography, depending on the particular resolution desired of the structure. The patterned photoresist may either act as an etch mask to enable the creation of high index structures, or directly act as the splitter layer itself. This will allow for rapid fabrication of single layer structures utilizing the same CMOS processing techniques already employed in sensor fabrication.

[0110] Benefits – Fast processing (time per chip), fast development cycles (using ubiquitous processes), this allows the possibility of high index materials when used as etch masks.

[0111] Disadvantages - Limited to single layer structures which exhibit limited efficiency, which requires high resolution exposure masks which can be relatively expensive if only a small number of pixels (e.g. a sub-area in one sensor) are used for testing.

[0112] Option 1.2 - Direct-write lithography. Single-photon or multi-photon direct-write lithography will be possible to replace older UV lithography, using a focused laser beam to write into photoresist without the need for a photomask, and possibly utilizing a two-photon absorption process.

[0113] Benefits – Fast development cycles (common process), this allows the possibility of high refractive index materials when used as etch masks.

[0114] Disadvantages - Limited to single layer structures which exhibit limited efficiency, point-by-point writing makes it slow to be used for full 3D splitters, single-photon lithography may have too low resolution for sufficient performance.

[0115] Option 1.3 - Electron beam lithography. Similar to direct-write lithography, but instead of a focused laser beam, a focused electron beam is used, thereby allowing much higher resolution.

[0116] Benefits - Fast development cycles (ubiquitous process), this allows the possibility of high refractive index materials when used as etch masks, extremely high resolution can be achieved.

[0117] Disadvantages – Limited to single layer structures exhibiting limited efficiency, point-by-point writing makes it slow to use for full 3D splitters, higher resolution generally trades off for longer writing times.

[0118] Exemplary options for the fabrication of structures of several laminates (e.g., 2-5 layers) include multi-step UV photolithography, sequential nanoimprint lithography, or micro-transfer printing.

[0119] Option 2.1 - Multi-step UV photolithography. Repeated steps of single-layer lithography can build up more complex 2.5D structures. Two layers have been demonstrated previously (C. Greiner, E. Arzt, and A. del Campo, "Hierarchical gecko-like adhesives," Advanced Materials, vol. 21, no. 4, pp. 479-482, 2009).

[0120] Benefits - Fast processing (time per chip), fast development cycles (using ubiquitous processes), this allows the possibility of high refractive index materials if the etch mask thickness is precisely tuned, also compatible with soft lithography (using structures created via this method as a mold).

[0121] Disadvantages - Layer must be fully supported by the layer below (overhanging structures not possible), requires multiple high resolution exposure masks, difficulty increases with increasing number of layers.

[0122] Option 2.2 - Sequential nanoimprint lithography. Nanoimprint lithography uses a mold that is pressed into an uncured polymer material to transfer a pattern through mechanical deformation. After imprinting, the polymer is cured. Repeated step nanoimprinting before curing can also produce 2.5D structures. Two layers have been demonstrated previously (F. Zhang and H.Y. Low, "Ordered three-dimensional hierarchical nanostructures by nanoimprint lithography", Nanotechnology, vol.17, no.8, pp.1884-1890, 2006).

[0123] Benefit – Fast processing (time per chip), which allows the possibility of high index materials if the etch mask thickness is precisely adjusted.

[0124] Disadvantages - Layer must be fully supported by the layer below (overhanging structures not possible), may require smaller and thinner top layers vs. base layers, requires multiple imprint molds, second imprint step may complicate design of first step to prevent degradation, alignment may be more difficult than in photolithography, difficulty increases with increasing number of layers.

[0125] Option 2.3 - Microtransfer printing. A mold can be filled with a polymer material and then transferred onto a substrate, similar to that in nanoimprint lithography. This can then be repeated to build up layered structures. This has been demonstrated for multiple layers with overhangs (Zhao, Xiao-Mei, Younan Xia, and George M. Whitesides, "Fabrication of three-dimensional micro-structures: Microtransfer molding," Advanced Materials vol. 8, no. 10, pp. 837-840, 1996).

[0126] Benefits - relatively fast processing (time per chip), which can allow multiple (3+) layers to be deposited with less difficulty than other methods for several stacks, not limited to 2.5D structures (overhangs are possible).

[0127] Disadvantages - There can be challenges to ensure complete transfer of the pattern may require more complex processing (e.g., addition of a sacrificial release layer that is dissolved), there are some further design limitations as some types of structures (e.g., cantilevers) are probably not possible, alignment can be more difficult than in photolithography.

[0128] Exemplary options for fabrication of fully 3D structures (eg, 10 or more layers) include multiphoton nanolithography or focused electron beam-induced deposition (FEBID).

[0129] Option 3.1 - Multiphoton nanolithography. Complex 3D structures can be written directly into photoresist using two-photon absorption processes. (Johlin, Eric, Sander A. Mann, Sachin Kasture, A. Femius Koenderink, and Erik C. Garnett, "Broadband highly directive 3D nanophotonic lenses," Nature communications 9, no. 1, p. 4742, 2018). Dip-in liquid lithography (DILL), in which a liquid photoresist acts as an immersion liquid in direct contact with the focused target, can be used to print onto non-transparent substrates.

[0130] Benefits - High resolution full 3D structures that exhibit the best computational performance can be created, no masks or molds are required, relatively fast development time.

[0131] Disadvantages – Relatively slow print times (writing one point at a time for 3D structures), which can be troublesome issues from reflections from the substrate in DILL.

[0132] Option 3.2 - Focused electron beam induced deposition (FEBID). Precursor gases are injected into an electron beam lithography (or microscopy) setup, the gases adsorb onto the sample surface, and the electron beam causes decomposition of the precursor molecules resulting in solid deposition at exposed locations. This has been shown to allow the fabrication of arbitrary 3D geometries (Skoric, Luka, Dedalo Sanz-Hernandez, Fanfan Meng, Claire Donnelly, Sara Merino-Aceituno, and Amalio Fernandez-Pacheco, "Layer-by-layer growth of complex-shaped three-dimensional nanostructures with focused electron beams," Nano Letters 20, no. 1, pp. 184-191, 2019).

[0133] Benefits - High resolution full 3D structures can be created that exhibit the best quantitative performance, no masks or molds are required, extremely high resolution (tens of nanometers) is possible, many materials possible through the choice of precursor gases.

[0134] Disadvantages – relatively slow printing times (writing one point at a time for 3D structures), stochastic adsorption processes can limit the writing speed, angle of overhanging structures may be limited or require tilting of the substrate.

[0135] The embodiments described herein are for illustrative purposes, without loss of generality intended. Further variations, modifications, and combinations thereof will be contemplated and recognized by those skilled in the art. Thus, the above detailed description is not intended to limit the scope, applicability, or configuration of the claimed subject matter.

Claims

1. a splitter comprising a block of structured dielectric material having a plurality of nanophotonic geometric structures formed therein, wherein incident electromagnetic radiation is split by transmission through the plurality of nanophotonic geometric structures into a plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges; a splitter, the block having an input surface for receiving the incident electromagnetic radiation and an output surface for transmitting the plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges, the input surface and the output surface being independently in communication with the plurality of nanophotonic geometric structures; a pixelated photodetector array providing an active layer surface for receiving said plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges, a pixelated photodetector array, the active layer surface being oriented opposite the output surface of the block, the separation between the active layer surface and the output surface being less than 500 nanometers; An image sensor comprising:

2. 10. The image sensor of claim 1, wherein the splitter provides a near-field focus in that a focus of the multiple substantially parallel beams of spectrally distinct electromagnetic wavelength ranges transmitted from the splitter output surface is characterized by a working distance relative to the output surface of less than 500 nm and a minimum focal distance relative to the output surface of less than 300 nm.

3. 3. The image sensor of claim 1, wherein the splitter provides angular conservation in that the incident electromagnetic radiation received at a normal angle to the input surface corresponds to the multiple substantially parallel beams of spectrally distinct electromagnetic wavelength ranges transmitted from the splitter output surface at an angular range of less than 20 degrees from the normal angle to the output surface.

4. 4. An image sensor as described in any one of claims 1 to 3, wherein a total intensity of the multiple substantially parallel beams of spectrally distinct electromagnetic wavelength ranges transmitted from the splitter output surface is greater than 50% compared to a corresponding total intensity of the incident electromagnetic radiation received into and through the splitter input surface.

5. 5. The image sensor of claim 1 , wherein the block is constructed according to an adjoint-based inverse design simulation specifying parameters including the separation between the active layer surface and an output surface being less than 500 nm, and a single dipole centrally located within each pixel of the pixelated photodetector array.

6. 6. The image sensor of claim 1, wherein the separation between the active layer surface and the output surface is less than 400 nm.

7. 7. The image sensor of claim 1, wherein the separation between the active layer surface and the output surface is less than 300 nm, less than 200 nm, less than 100 nm, less than 50 nm, approximately zero, or zero.

8. 8. An image sensor according to claim 1, wherein the output surface of the block is configured with areas of lower intensity transmission to separate the plurality of substantially parallel beams.

9. The image sensor of claim 1 , wherein the block is formed from a single dielectric material.

10. The image sensor of claim 9 , wherein the refractive index of the single dielectric material is less than 2.

11. 11. The image sensor of claim 1, wherein the pixelated photodetector array is a passive pixel sensor or an active pixel sensor.

12. The image sensor of claim 11 , wherein the active pixel sensor is a CMOS sensor.

13. 13. The image sensor of claim 1, wherein the plurality of substantially parallel beams of spectrally distinct electromagnetic wavelength ranges are in the blue, green and red wavelength ranges (400-500 nm blue, 500-600 nm green and 600-700 nm red).

14. The image sensor of claim 1 , wherein a plurality of laminations are integrated to form the block and the plurality of geometric structures.

15. The image sensor of claim 14 , wherein the plurality of stacked layers includes at least three layers.

16. The image sensor of claim 14 , wherein the plurality of stacked layers includes at least four layers.

17. 17. An image sensor according to any one of claims 14 to 16, wherein the laminate is of uniform thickness.

18. 17. An image sensor according to any one of claims 14 to 16, wherein the laminates are of different thicknesses.

19. 17. The image sensor of claim 14, wherein at least one lamination has a varying thickness along its circumference or diameter.

20. 17. An image sensor according to any one of claims 14 to 16, wherein at least one lamination has a uniform thickness at all points.

21. 21. The image sensor of claim 1, wherein the image sensor is a visible color image sensor, an ultraviolet image sensor, an infrared image sensor, or a microwave image sensor.

22. a splitter having an input surface for receiving incident electromagnetic radiation and an output surface for transmitting a plurality of beams of spectrally distinct electromagnetic wavelength ranges, the output surface having a minimum focal length of less than 200 nm; a pixelated photodetector array providing an active layer surface for receiving said plurality of beams of spectrally distinct electromagnetic wavelength ranges, a pixelated photodetector array, the active layer surface being oriented opposite the output surface of the block, the separation between the active layer surface and the output surface being less than 400 nanometers; An image sensor comprising:

23. 23. The image sensor of claim 22, wherein the splitter transmits the multiple beams of spectrally distinct electromagnetic wavelength ranges within an angular range at the output surface, and when the angle of the incident electromagnetic radiation is normal to the input surface, at least 80% of the power of the multiple beams is within an angular range of less than 10 degrees from the normal angle to the output surface.

24. 24. The image sensor of claim 22 or 23, wherein the splitter was constructed according to an adjoint-based inverse design simulation specifying parameters including the separation between the active layer surface and an output surface being less than 500 nm, and a single dipole centrally located within each pixel of the pixelated photodetector array.

25. 25. The image sensor of claim 22, wherein the output surface of the splitter is configured with areas of lower intensity transmission to separate the multiple beams separately.

Images