Camera Scene Reconstruction for Static and Dynamic Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Scene reconstruction using lidar is expensive, leading to high implementation costs.
Innovation Solution
Implement a scene reconstruction method and apparatus that utilizes a camera-based approach, including background segmentation, point-cloud sparse and dense reconstruction, and dynamic-static segmentation to separately process static and dynamic scenes, reducing reliance on costly lidar.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If lidar is used for scene reconstruction, then measurement precision and reliability are improved, but implementation cost increases significantly
Solution Approach 1:
The patent replaces expensive lidar with cheap cameras that can be mass-produced. Multiple standard cameras are used instead of a single costly lidar sensor, achieving comparable reconstruction precision through software algorithms rather than expensive hardware
Solution Approach 2:
The patent substitutes the mechanical laser ranging system (lidar) with an optical imaging system (camera) combined with computational algorithms. The reconstruction is achieved through image processing and geometric computation rather than direct mechanical measurement
2Ease of manufacture
If camera-based reconstruction is used, then implementation cost is reduced, but measurement precision and reliability deteriorate
Solution Approach 1:
The patent divides the scene into static and dynamic segments, processing each with appropriate algorithms. Static scenes use structure-from-motion (SfM) while dynamic scenes use optical flow methods, improving overall precision by tailoring processing to scene characteristics
Solution Approach 2:
The patent combines multiple camera views and multiple processing algorithms (SfM, optical flow, depth estimation) to compensate for individual limitations. By merging information from multiple sources, the system achieves precision comparable to lidar
3Measurement precision
If dynamic-static segmentation is performed, then reconstruction accuracy for moving objects is improved, but processing time and complexity increase
Solution Approach 1:
The patent performs dynamic-static segmentation early in the processing pipeline, before detailed reconstruction. This preliminary classification allows subsequent processing to be optimized for each scene type, reducing overall processing time while maintaining accuracy
Data Source
Figure 1-1~1-2
Figure 2~3
Figure 4~5-1
AI summary
Disclosed in the embodiments of the present disclosure are a scene reconstruction method and apparatus, a storage medium, and an electronic device. The method includes: performing background segmentation on a first image sequence captured by a camera provided on a mobile device, to obtain a second image sequence; performing point-cloud sparse reconstruction by using the second image sequence, to obtain a first reconstructed point cloud and a first camera-pose sequence of the camera; performing dynamic-static segmentation on the first image sequence to obtain a third image sequence corresponding to a static scene and a fourth image sequence corresponding to a dynamic scene; performing point-cloud dense reconstruction by using the first reconstructed point cloud, the first camera-pose sequence, and the third image sequence, to obtain a second reconstructed point cloud; sequentially performing point-cloud sparse reconstruction and dense reconstruction by using the first camera-pose sequence and the fourth image sequence, to obtain a third reconstructed point cloud; and determining, based on the second reconstructed point cloud and the third reconstructed point cloud, a scene reconstruction result corresponding to the first image sequence.