Light Field Camera 3D Reconstruction via Ray Space Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional structure from motion methods for 3D model reconstruction from light field images fail to effectively utilize geometric constraints, resulting in inaccurate and robustness issues, especially with complex objects like flowers, and are not suitable for applications in virtual and augmented reality.
Innovation Solution
A new structure from motion method that exploits ray geometry transformations under light field pose variations, using a system comprising a light field camera and a data processing unit for capturing and processing light field images, which includes camera calibration, structure from motion estimation, stereo matching, and depth fusion to generate accurate 3D point clouds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional structure from motion methods are used for 3D model reconstruction, then the process is simpler, but the accuracy and robustness deteriorate due to failure to utilize geometric constraints in light fields
Solution Approach 1:
The patent transforms the light field data representation by changing parameters from traditional image coordinates to ray space coordinates (u, v, s, t), enabling geometric constraints to be explicitly utilized in the structure from motion process. This parameter transformation allows for more accurate 3D reconstruction by leveraging the inherent geometric properties of light fields.
Solution Approach 2:
The patent introduces ray space as an intermediary representation between traditional image space and 3D reconstruction. By working in ray space, the method mediates between the complex light field data and the structure from motion algorithm, enabling effective utilization of geometric constraints while maintaining computational feasibility.
2Reliability
If traditional structure from motion methods are used, then the implementation is faster, but the reliability deteriorates especially with topologically complex objects
Solution Approach 1:
By changing the parameterization from traditional image coordinates to ray space coordinates, the method improves reliability for complex objects. The ray space representation (u, v, s, t) explicitly encodes geometric constraints that make the reconstruction process more robust to topological complexity, while the optimization framework efficiently handles the computation.
Solution Approach 2:
The patent employs an optimization framework that uses feedback from the geometric constraints to iteratively improve the 3D reconstruction. The method computes reprojection errors and uses gradient descent to minimize these errors, providing continuous feedback that enhances reliability for complex objects while managing computation time through efficient optimization.
3Measurement precision
If light field geometric constraints are utilized, then the 3D model accuracy improves, but the processing complexity increases
Solution Approach 1:
The patent changes the parameterization to ray space coordinates, which naturally incorporates geometric constraints. This parameter change simplifies the utilization of light field properties while improving disparity map accuracy, as the ray space representation makes geometric relationships explicit and easier to process.
Data Source
AI summary
A method of generating a three-dimensional model of an object is disclosed. The method may use a light field camera to capture a plurality of light field images at a plurality of viewpoints. The method may include capturing a first light field image at a first viewpoint; capturing a second light field image at the second viewpoint; estimating a rotation and a translation of a light field from the first viewpoint to the second viewpoint; obtaining a disparity map from each of the plurality of light field image; and computing a three-dimensional point cloud by optimizing the rotation and translation of the light field and the disparity map. The first light field image may include a first plurality of subaperture images and the second light field image may include a second plurality of subaperture images.