Deep Photometric Learning for Microscopic Surface 3D Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional photometric stereo techniques struggle with 3D reconstruction of microscopic surfaces due to challenges posed by small features and uneven lighting conditions, limiting their effectiveness in applications such as semiconductor wafers.
Innovation Solution
Employing deep photometric learning models, specifically convolutional neural networks, to generate 3D-reconstructions of microscopic surfaces by incorporating synthetic and experimental data sets that simulate non-ideal conditions, reducing noise susceptibility and improving reconstruction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional photometric stereo techniques are used for 3D reconstruction, then the method is simple and fast, but the reconstruction accuracy deteriorates on microscopic surfaces with small features and uneven lighting
Solution Approach 1:
The patent introduces deep learning models as an intermediary between the imaging system and 3D reconstruction process. The neural network processes the images captured by the photometric stereo system, learning to correct for non-ideal lighting conditions and small feature detection issues, thereby improving reconstruction accuracy without requiring changes to the physical imaging hardware
Solution Approach 2:
The patent transforms the reconstruction problem from direct geometric calculation to a learned parameter estimation problem. By training the deep learning model on synthetic and experimental data with varied lighting conditions and surface properties, the system adapts its parameters to handle microscopic surfaces effectively, improving accuracy while maintaining computational efficiency
2Measurement precision
If deep photometric learning models are used to improve reconstruction accuracy on microscopic surfaces, then measurement precision improves, but computational complexity and processing time increase
Solution Approach 1:
The patent performs the computationally intensive deep learning model training in advance using synthetic and experimental data. Once trained, the model can be deployed for rapid inference on new samples. This preliminary action separates the heavy computational burden from the actual measurement process, allowing fast processing during actual 3D reconstruction tasks
Solution Approach 2:
The patent uses synthetic data copies to train the deep learning model, creating virtual training samples that mimic real microscopic surfaces under various lighting conditions. This allows the model to learn from extensive diverse data without requiring equally extensive physical sample preparation and imaging, reducing the time needed for actual measurement campaigns
3Adaptability or versatility
If conventional photometric stereo assumes Lambertian surfaces, then the theoretical model remains simple, but applicability to real-world specimens deteriorates due to non-Lambertian reflections
Solution Approach 1:
The deep learning model acts as an intermediary that compensates for violations of the Lambertian assumption. Instead of modifying the physical imaging system or requiring perfect Lambertian surfaces, the neural network learns to correct for non-Lambertian reflections in the image processing stage, extending the method's applicability to real-world specimens
Solution Approach 2:
The patent moves from fixed theoretical parameters of the Lambertian model to learned parameters in the deep learning model. The network adapts its internal parameters during training on real or simulated non-Lambertian data, allowing it to handle various reflection types without requiring changes to the underlying imaging hardware or fundamental measurement approach
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The deep learning approach enhances the robustness of 3D reconstruction on complex microscopic surfaces, enabling rapid and precise inspection without destructive methods, and reducing reliance on Lambertian surfaces.
Implementation Method 1
Surface normal and reflectance map of an object may be computed from input images taken using a fixed viewing angle with different illumination directions
Data Source
AI summary
An imaging system is disclosed herein. The imaging system includes an imaging apparatus and a computing system. The imaging apparatus includes a plurality of light sources positioned at a plurality of positions and a plurality of angles relative to a stage configured to support a specimen. The imaging apparatus is configured to capture a plurality of images of a surface of the specimen. The computing system in communication with the imaging apparatus. The computing system configured to generate a 3D-reconstruction of the surface of the specimen by receiving, from the imaging apparatus, the plurality of images of the surface of the specimen, generating, by the imaging apparatus via a deep learning model, a height map of the surface of the specimen based on the plurality of images, and outputting a 3D-reconstruction of the surface of the specimen based on the height map generated by the deep learning model.