3D Shape Reconstruction Using Neural Network and Multiple Light Sources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D shape reconstruction technologies face challenges in achieving high-resolution images of wide areas without compromising resolution, and existing methods require complex phase measurement setups or large numbers of captured images.
Innovation Solution
A method and device using multiple light sources, specifically LEDs, to reconstruct 3D shapes by obtaining measured images under different lighting conditions, defining 3D grids, and employing an artificial neural network model to calculate scattering potential values, predict images, and determine refractive index distributions, thereby simplifying the reconstruction process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If traditional imaging systems are used to secure a large area, then the observation area is increased, but resolution is lost
Solution Approach 1:
The patent divides the imaging task into multiple segments by using multiple light sources at different angles. Each light source captures a specific portion of the frequency spectrum, and these segments are combined through computational methods to achieve both wide area coverage and high resolution, resolving the trade-off between observation area and resolution
Solution Approach 2:
The patent transitions from traditional 2D imaging to 3D reconstruction by adding the angular dimension of light incidence. By capturing images from multiple angles and using Fourier ptychography to reconstruct the full frequency spectrum, the system achieves high resolution across a wide area that cannot be obtained in conventional 2D imaging
2Measurement precision
If Fourier ptychography technology is used to obtain high-resolution images of wide area, then resolution and observation area are improved, but the phase reconstruction process becomes complex
Solution Approach 1:
The patent replaces the complex mechanical phase measurement setup with a computational approach. Instead of using complex optical components to measure phase directly, the system uses multiple light sources at different angles and processes the images through Fourier transforms and neural networks to reconstruct the 3D shape, simplifying the overall system while maintaining high resolution
Solution Approach 2:
The patent introduces an artificial neural network as an intermediary to simplify the reconstruction process. The neural network takes the captured images and intermediate frequency data as input and outputs the final 3D shape or refractive index distribution, automating the complex phase reconstruction calculations and reducing the need for manual intervention and complex optical phase measurement
3Measurement precision
If multiple light sources are used to reconstruct 3D shape, then reconstruction quality is improved, but the number of captured images increases
Solution Approach 1:
The patent applies partial action by using a limited number of strategically positioned light sources and images. Instead of requiring complete angular coverage, the system uses a subset of images at specific angles to capture sufficient frequency information, and the neural network fills in the remaining data, reducing the total number of images needed while maintaining high reconstruction quality
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables high-quality 3D shape reconstruction with a reduced number of images and without explicit complex light wave measurement, achieving high-resolution 3D refractive index distributions by aligning and training the neural network model using gradient descent and Fourier transforms.
Implementation Method 1
obtaining a set of predicted images corresponding to the different lighting conditions based on the inference result related to the scattering potential
Implementation Method 2
the inference result may include scattering potential data or scattering potential spectrum data obtained by applying a three-dimensional Fourier transform to the scattering potential data
Implementation Method 3
the acquiring the set of predicted image sets may include applying a two-dimensional inverse Fourier transform to the scattering potential spectrum data
Implementation Method 4
multiple light sources may include a plurality of light emitting diodes (LEDs) configuring the different lighting conditions, the set of measured images may be obtained through light emitted through each of the plurality of LEDs
Data Source
AI summary
A device, method, and system for restoring a 3D shape based on multiple light sources are disclosed. A method performed by a device may include obtaining a set of measured images of a specimen by photographing the specimen under different lighting conditions; defining a first three-dimensional grid within a 3D reconstruction area including all or part of the specimen; obtaining a first set of coordinate values for calculating a scattering potential value within a first 3D grid or a frequency grid region corresponding to the first 3D grid; and obtaining a 3D refractive index distribution of the specimen based on the output data obtained through an artificial neural network model, and the artificial neural network model is trained based on differences between the set of predicted images and the set of the measured images.