3D Structure Construction Using Image Stream Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Constructing a 3D structure from 2D images is complex and time-consuming due to the limited processing capabilities of image capturing devices, which require transmitting and storing multiple images before processing, thereby increasing the overall time for generating a 3D structure.
Innovation Solution
A system comprising a processor and memory with modules for imaging, correspondence generation, 3D reconstruction, optimization, registration, and dense reconstruction, which processes a stream of images with timestamps, extracts features, generates point correspondences, filters outliers, optimizes 3D structures, and performs dense reconstruction using techniques like CMVS, enabling near real-time 3D structure construction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple images are captured and stored before processing, then the completeness of data for 3D reconstruction is improved, but the time required for constructing the 3D structure increases
Solution Approach 1:
The patent applies preliminary action by performing feature extraction and correspondence generation on images as they arrive, before all images are fully stored. The system prepares data structures and pre-processes images in advance, so that when all images are received, the complex 3D reconstruction can begin immediately without waiting for complete storage, thus reducing total processing time while maintaining data completeness
Solution Approach 2:
The patent segments the 3D reconstruction process into multiple independent modules: feature extraction, correspondence generation, triangulation, and structure optimization. Each module processes data independently and can operate in parallel or pipeline fashion, allowing the system to process images continuously as they arrive rather than waiting for complete image sets, thereby reducing overall construction time while ensuring all necessary data is captured
2Manufacturing precision
If complex data processing is performed to construct accurate 3D structure, then the precision of 3D reconstruction is improved, but the processing capability requirement increases
Solution Approach 1:
The patent divides the complex 3D reconstruction process into distinct modular stages: feature extraction module, correspondence generation module, triangulation module, and structure optimization module. Each module handles a specific aspect of the processing, allowing for specialized algorithms in each stage while keeping individual module complexity manageable. This modular approach enables high overall precision without requiring any single device component to handle excessive complexity
Solution Approach 2:
The patent introduces intermediate data structures and processing steps that facilitate accurate 3D reconstruction. Correspondence maps between image pairs serve as intermediaries that bridge 2D image data and 3D spatial relationships. Triangulation of correspondence points creates intermediate 3D point clouds that are then refined through optimization. These intermediary representations enable precise reconstruction while distributing computational complexity across multiple processing stages rather than requiring single-step complex processing
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present subject matter discloses a system and a method for constructing a 3D structure. A stream of images having timestamps associated is received at first. A plurality of image sub-groups is created using the stream of images. The plurality of image sub-groups are used for extracting features of the images present in the image sub-groups. The features are used for generating point correspondences between the images. Epipolar geometries for the image sub-groups are calculated for filtering of the point correspondences, and generating filtered point correspondences. Further, 3D points representing one or more 3D point structures are generated using the filtered point correspondences. A registered 3D structure is constructed using the one or more 3D structures upon considering a scale correction and transformation between the one or more 3D structures. Further, the registered 3D structure is dense reconstructed for improving the pixel density of the 3D structure.