3D Reconstruction from Multi-View Videos with Moving Cameras
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional three-dimensional reconstruction techniques face challenges in accurately reconstructing time-series models from moving images captured by multiple fixed or non-fixed cameras, especially when the subject is in motion, due to difficulties in correspondence between two-dimensional images and limited camera positions.
Innovation Solution
A three-dimensional reconstruction method that estimates camera parameters and reconstructs models using multi-view videos from multiple cameras, allowing for synchronous or asynchronous capturing, and calculates corresponding keypoints to create time-series models with consistent coordinate axes, regardless of camera motion or subject movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional keypoint matching is performed on time-series images captured by a monocular camera, then three-dimensional model reconstruction is achieved, but accurate correspondence between two-dimensional images becomes difficult when the subject is moving
Solution Approach 1:
The patent transitions from monocular to multi-view imaging by introducing multiple cameras with different viewpoints. This dimensional change from single-view to multi-view geometry enables accurate three-dimensional position estimation through triangulation, resolving the correspondence difficulty that arises with moving subjects in monocular systems.
Solution Approach 2:
The patent introduces a multi-view correspondence establishment unit that acts as an intermediary to match keypoints across multiple camera views. This intermediary process establishes reliable correspondences by finding matching keypoints in multiple viewpoints, which then serve as the basis for accurate three-dimensional reconstruction even when subjects are in motion.
2Ease of manufacture
If synchronous capturing is performed only at calibration time as in PTL 3, then coordinate conversion to virtual camera viewpoint is achieved, but precise estimation of three-dimensional positions of moving subjects becomes difficult
Solution Approach 1:
The patent implements continuous synchronous capturing at multiple time points rather than a single calibration snapshot. This dynamic approach allows the system to track moving subjects across different timestamps, maintaining measurement precision while adapting to subject motion through repeated synchronized measurements throughout the observation period.
3Measurement precision
If a stereo camera with fixed positional relationship is used as in PTL 2, then keypoint matching between key frames is achieved, but camera positions become restricted
Solution Approach 1:
The patent creates a universal multi-camera system that can handle both fixed and moving camera configurations. By establishing correspondence relationships in a unified three-dimensional coordinate system that accommodates arbitrary camera positions and orientations, the system achieves versatility while maintaining accurate keypoint matching capabilities across different deployment scenarios.
4Duration of action of moving object
If multiple fixed or non-fixed cameras are used to capture moving images, then time-series three-dimensional model reconstruction becomes possible, but difficulty in making correspondence between two-dimensional images increases
Solution Approach 1:
The patent segments the correspondence establishment process into distinct functional units: a multi-view correspondence establishment unit that handles keypoint matching across views, and a three-dimensional reconstruction unit that processes the correspondences. This segmentation reduces complexity by dividing the overall task into manageable, specialized components that can operate independently and efficiently.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided is a three-dimensional reconstruction method of reconstructing a three-dimensional model from multi-view images. The method includes: selecting two frames from the multi-view images; calculating image information of each of the two frames; selecting a method of calculating corresponding keypoints in the two frames, according to the image information; and calculating the corresponding keypoints using the method of calculating corresponding keypoints selected in the selecting of the method of calculating corresponding keypoints.