Image Fusion via 3D Parallax Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image fusion techniques fail to seamlessly integrate high-detail images with panoramic images due to perspective misalignment and image distortions, especially when objects of interest are at arbitrary viewpoints, leading to artifacts and misregistrations.
Innovation Solution
Employing stereo or plenoptic cameras to capture 3D structure and correct parallax errors, allowing for seamless integration of high-detail views into panoramic images by retrieving 3D structure, estimating camera pose, and performing photo-realistic warping to match the panoramic perspective, followed by smooth blending of borders.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution separate camera captures objects of interest, then detail level of objects improves, but perspective misalignment and image distortion occur
Solution Approach 1:
The patent transitions from 2D image space to 3D scene space by retrieving depth information and camera pose, then synthesizing a parallax-corrected view that matches the panoramic perspective. This dimensional transformation enables accurate fusion by operating in the 3D scene coordinate system rather than attempting to align 2D images directly.
Solution Approach 2:
The patent introduces an intermediary representation - the parallax-corrected detailed view synthesized from the separate camera image using retrieved 3D structure and pose information. This intermediary serves as a bridge between the separate camera view and the panoramic view, enabling seamless fusion by eliminating perspective mismatches.
2Measurement precision
If 3D structure retrieval and parallax correction are performed, then fusion accuracy improves, but computational complexity increases
Solution Approach 1:
The patent performs preliminary retrieval of 3D structure and camera pose information before the actual image fusion process. By preparing these geometric parameters in advance, the subsequent parallax correction and view synthesis operations can proceed more efficiently with pre-computed transformation parameters rather than performing complex optimization during fusion.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
Method and apparatus for fusion of images A method and an apparatus (30) for improving a main image (1) by fusing the richer information contained in a secondary image (2) are described. A 3D structure of objects contained in the secondary image (2) is retrieved (10) and a parallax-corrected version of the secondary image (2) is generated using the 3D structure. For this purpose a camera pose for which a projection of the 3D structure of the objects contained in the secondary image (2) best resembles the perspective in the main image (1) is determined and the parallax-corrected version of the secondary image (2) is synthesized based on the determined camera pose. The parallax-corrected version of the secondary image (2) is then fused with the main image (1).