3D Sample Imaging With Multimodal Focal Stacks and ML Depth Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D imaging techniques for electronic circuits suffer from limited spatial accuracy and precision, and alternative methods that provide higher accuracy are costly or computationally intensive, complicating systems and reducing throughput.
Innovation Solution
A system and method utilizing an optical assembly that captures multi-modality focal stacks with distinct illumination angles and employs a machine learning algorithm trained on these stacks, generating depth maps with increased spatial resolution and precision without relying on material reflection models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If current focus-based 3D imaging techniques are used, then cost efficiency is improved, but spatial accuracy and precision deteriorate
Solution Approach 1:
The patent introduces an intermediate computational processing stage using machine learning algorithms that acts as a mediator between the captured image data and the final 3D reconstruction. This intermediary processing layer extracts depth information more accurately by learning from training data, thereby improving measurement precision without requiring expensive hardware modifications
Solution Approach 2:
The patent transforms the problem from direct geometric measurement to a parameter optimization problem by training machine learning models on synthetic and real data. The system changes the approach from using fixed geometric relationships to using learned parameters that adapt to different imaging conditions, achieving higher precision while maintaining cost efficiency
2Measurement precision
If interferometry-based 3D imaging techniques are used, then spatial accuracy and precision are improved, but device complexity and cost increase
Solution Approach 1:
The patent replaces complex mechanical/optical interferometry systems with a computational approach using machine learning algorithms. Instead of using physical interference patterns and complex optical path differences, the system uses trained neural networks to infer depth information from standard images, thereby reducing device complexity while maintaining high measurement precision
Solution Approach 2:
The patent uses synthetic training data that copies and simulates real-world imaging scenarios to train the machine learning model. By creating virtual training datasets that replicate various lighting conditions, materials, and geometries, the system achieves high precision without requiring complex physical measurement setups during operation
3Measurement precision
If interferometry-based 3D imaging techniques are used, then spatial accuracy and precision are improved, but productivity deteriorates
Solution Approach 1:
The patent performs preliminary training of machine learning models using synthetic data before actual measurement. This preliminary action creates a pre-trained system that can rapidly process real images without requiring complex real-time calculations during measurement, thereby maintaining high precision while improving throughput and productivity
Data Source
AI summary
A method for 3D imaging of samples using a machine learning algorithm is disclosed. The method uses multimodality focal stacks, which consist of a plurality of images acquired at two or more distances between the sample and a front focal plane, with at least one image acquired using the first modality and additional images acquired using additional modalities. The modalities may have different illumination angles and optionally different spectral distributions. The method may include receiving training images of samples, which include a plurality of training focal stacks, and ground truth 3D data (depth maps) for each training focal stack, and training a machine learning algorithm based on the training images and the ground truth data. The method subsequently receives product images of a sample, the product images including a focal stack from an optical assembly, and generates a 3D depth map.