3D Panoramic Mosaic Generation via Rectified Texture Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D reconstruction techniques face limitations in achieving accurate and omnidirectional reconstruction of large scenes due to camera calibration issues, limited spatial and angular extent, noise, and difficulties in reconstructing complex shapes, especially with stereoscopic systems that require precise camera geometry and limited field of view.
Innovation Solution
A panoramic system capable of covering a wide angular sector, allowing 3D reconstruction and mosaicking over large spatial and angular extents by using non-overlapping sensors that move within the scene, with temporal stereoscopy and direct rectification onto freely chosen planes, enabling robust and precise textured 3D reconstruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If stereoscopic systems with fixed or moving cameras are used for 3D reconstruction, then 3D spatial information can be obtained, but camera calibration becomes difficult and reconstruction accuracy deteriorates due to vibration and limited baseline
Solution Approach 1:
The patent replaces the mechanical stereoscopic camera system with a panoramic camera system that captures images in a panoramic format. This substitution eliminates the need for complex calibration between multiple cameras while maintaining 3D reconstruction capability through the panoramic image's inherent angular information.
Solution Approach 2:
The panoramic camera system serves multiple functions: it captures omnidirectional images for 3D reconstruction, provides texture information for mosaicking, and eliminates the need for separate calibration processes. This multi-functional approach resolves the contradiction by simplifying the system while improving accuracy.
2Area of stationary object
If stereoscopic systems with limited field of view cameras are used, then 3D reconstruction can be performed, but the spatial and angular extent of the reconstructed scene is limited
Solution Approach 1:
The patent employs a panoramic camera that captures images in a spherical or near-spherical field of view, replacing the limited planar field of view of conventional cameras. This enables omnidirectional 3D reconstruction and mosaicking over large spatial and angular extents, resolving the contradiction between scene coverage area and angular adaptability.
3Manufacturing precision
If conventional 2D mosaicking by projecting images onto a main plane is used, then mosaicking can be performed, but geometric deformations occur on non-flat surfaces and 3D elements
Solution Approach 1:
The patent transitions from 2D planar mosaicking to 3D mosaicking by utilizing the panoramic image's angular information to reconstruct 3D scene geometry. This enables accurate projection of textures onto 3D surfaces and handling of non-flat surfaces and concave objects, eliminating geometric deformations while maintaining versatility across different surface types.
4Reliability
If stereoscopic systems with limited overlap between images are used, then 3D reconstruction can be performed, but the reconstruction becomes noisy and contains errors due to low overlap
Solution Approach 1:
The panoramic camera system provides continuous angular coverage with significant overlap between adjacent panoramic images. This continuous and overlapping coverage ensures consistent feature matching and 3D point reconstruction across the entire scene, eliminating noise and errors associated with low overlap while maintaining high reliability.
Data Source
Figure 1~4
Figure 2
Figure 3
AI summary
A method for mosaicking a scene into a 3D mosaic, without prior information about the scene, characterized in that at least one 3D reconstruction has been obtained during the following steps: - Acquisition of successive images by a panoramic sensor moving along an unconstrained 3D reconstruction path, such that the image of at least one point of the scene is in at least 3 successive 2D panoramic images obtained from different panoramic system-point directions of the scene, - Rectification of the acquired images on rectification planes and matching of the rectified images, - 3D reconstruction on several reconstruction planes from the matched rectified images, it further comprises the following steps: A) Acquisition of successive images by the panoramic sensor, the last acquired image being designated the current 2D panoramic image,B) Selection of several projection surfaces on which the mosaic will be built, C) Selection of sectors of the current 2D panoramic image, and selection of textures in the selected sectors by selecting visible parts, i.e., those not hidden by a scene surface, with the help of 3D reconstruction, D) Projection of the selected textures onto each projection surface and merging of the textures on each projection surface to obtain a consistent mosaic on each projection surface.