3D Point Cloud Reconstruction via SFM and DFD Fusion
Solution Overview
Problem
Current machine vision systems face limitations in creating accurate 3D models from 2D images, particularly when camera movement is involved, due to errors in structure from motion (SFM) and depth from defocus (DFD) algorithms, which require substantial computational resources and are prone to failure in degenerate conditions such as small camera distances and image shifts.
Innovation Solution
The integration of SFM and DFD algorithms with machine learning, allowing for live training using output from SFM and DFD measurements, and the use of a robust relative blur estimation algorithm to mitigate errors and adapt to varying environments, enabling 3D reconstruction even with cameras in motion and minimal computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If SFM algorithms are used for 3D reconstruction, then depth information can be obtained from multiple images, but substantial computational resources are required and errors occur in degenerate conditions
Solution Approach 1:
The patent segments the 3D reconstruction process into distinct modules: feature detection, feature matching, fundamental matrix computation, and triangulation. This modular approach allows each component to be optimized independently and enables parallel processing, reducing overall computational complexity while maintaining depth measurement accuracy.
Solution Approach 2:
The patent performs preliminary actions by pre-computing and storing the fundamental matrix from matched feature points before actual depth calculation. This pre-processing step organizes the data in a way that accelerates subsequent triangulation operations and reduces real-time computational burden.
2Length of moving object
If SFM algorithms are used with small camera distances, then compact system configuration is achieved, but measurement errors increase significantly
Solution Approach 1:
The patent compensates for small baseline distances by utilizing temporal dimension - processing multiple images captured at different time points. By accumulating observations across time and applying iterative refinement algorithms, the system achieves accurate depth measurement even when spatial separation between cameras is minimal.
Solution Approach 2:
The patent implements feedback mechanisms where depth estimates from initial triangulation are used to refine feature matching in subsequent iterations. This iterative feedback loop corrects errors introduced by small baseline distances and improves overall measurement precision through continuous refinement.
3Ease of operation
If traditional feature matching is used to find corresponding points, then 3D reconstruction can be performed, but errors occur when image point locations are similar
Solution Approach 1:
The patent applies local quality by using distinctive local features (such as corners, edges, and textured regions) for matching instead of relying on global image characteristics. This approach ensures that even when overall images are similar, unique local patterns can be reliably identified and matched across views.
Solution Approach 2:
The patent changes matching parameters dynamically based on image content and camera geometry. By adjusting feature detection thresholds, matching criteria, and outlier rejection parameters according to the specific imaging conditions, the system maintains high matching accuracy even when images are visually similar.
4Productivity
If real-time 3D estimation is implemented, then rapid depth calculation is achieved, but substantial computational power and parallel processing resources are required
Solution Approach 1:
The patent replaces complex iterative optimization mechanisms with closed-form mathematical solutions where possible. By using direct triangulation formulas and analytical solutions for fundamental matrix computation, the system achieves real-time performance without requiring extensive parallel processing hardware.
Data Source
AI summary
Machine vision systems/methods and related application systems/methods are provided that includes steps/control sections including capturing pairs of multiple images from at least two cameras having overlapping fields of views and camera settings, first and second category depth estimation (DE) modules (DEM) that generates a first and second depth estimate (z), DE neural network trainer (NN) trigger system, a camera setting module, and an application that uses outputs from the first or second category DEM. The first category DEM includes featuring matching, structure from motion (SFM), depth from defocus (DFD), ratios of depth (RoD) and relative blur estimates (RBE) generators, systems of equations (SoEs) based on camera model projective geometry equations and thin lens equations module, and multiple SoE variable elimination modules using the RoDs and RBEs to reduce variables in the SoEs. The second DEM includes a NN DE trainer/use system. Also uses a reinforcement learning camera setting selection system.