Metasurface Neural Network Co-Optimization for Image Sensor Photon Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional RGGB color filters in imaging sensors are inefficient, as they discard approximately ⅔rd of available photons, leading to incomplete light capture and suboptimal image reconstruction.
Innovation Solution
Designing optical metasurfaces that transmit and scatter light based on wavelength, combined with a neural network system to optimize image reconstruction, using a computational inverse design tool that co-optimizes the metasurface structure and neural network algorithm to minimize loss and preserve photons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If conventional RGGB color filters are used in imaging sensors, then color information can be captured, but approximately 2/3rd of available photons are discarded, leading to inefficient light capture
Solution Approach 1:
The patent removes the conventional color filter array from the imaging sensor and replaces it with a computational approach. By extracting the optical color separation function and replacing it with a learned computational model, the system eliminates the physical barriers that block 2/3rd of photons, allowing all photons to reach the photo-sensors while color information is recovered through post-processing algorithms
Solution Approach 2:
The patent substitutes the mechanical/optical color filter system with a computational algorithm. Instead of using physical filters to separate colors at the optical level, the system uses machine learning algorithms to reconstruct color information from grayscale or reduced-color data, replacing the mechanical filtering approach with intelligent software processing
2Measurement precision
If conventional color filter arrays are used, then image capture is simplified, but image reconstruction quality deteriorates due to incomplete color samples
Solution Approach 1:
The patent applies preliminary computational processing to the captured image data using trained neural networks or demosaicing algorithms. By pre-training computational models on large datasets and applying them to reconstruct color information, the system achieves high-quality image reconstruction without requiring complex hardware modifications, improving measurement precision while managing complexity through software optimization
3Loss of energy
If metasurface is used to transmit and scatter light based on wavelength, then light capture efficiency is improved, but manufacturing complexity increases
Solution Approach 1:
The patent optimizes metasurface parameters such as pillar height, width, spacing, and material composition to control light scattering and wavelength separation. By systematically varying these geometric and material parameters, the system achieves high light capture efficiency and precise spectral control while maintaining compatibility with existing semiconductor manufacturing processes, thus managing fabrication complexity
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution significantly enhances image capture efficiency by preserving more light information, improving image reconstruction quality while accounting for manufacturing constraints and costs.
Implementation Method 1
The color filter array can be replaced by a metasurface that substantially transmits all incident light, or at least considerably more light than conventional RGGB filters, while scattering the transmitted light to one or more photo-sensors based on the light's wavelength.
Implementation Method 2
The metasurface can be co-optimized with a neural network system that transforms the response of the imaging sensor's photo-sensor array into a full color image.
Implementation Method 3
The interaction between the metasurface and a full spectrum illumination can be simulated by numerically solving Maxwell's equations at each voxel and interactions between each voxel and its neighboring voxels.
Data Source
AI summary
A computer-implemented method for designing an image processing device includes defining a loss function within a simulation space composed of a plurality of voxels; defining an initial structure for one or more physical features of a metasurface and one or more architectural features of a neural network in the simulation space; determining, using a computer system, values for at least one structural parameter, and/or at least one functional parameter for the one or more physical features and at least one architectural parameter for the one or more architectural features, using a numerical solver to solve Maxwell's equations so that a loss determined according to the loss function is within a threshold loss; defining a final structure of the metasurface based on the values for the one or more structural parameters; and defining a final structure of the neural network based on the values for the at least one architectural parameter.