Portable Terminal Panoramic Image Stitching via Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Accurate panoramic image pickup is hindered by camera shake and displacement of the rotational axis, leading to errors in synthesized images.
Innovation Solution
A portable terminal equipped with three-axis gyro, acceleration, and geomagnetic sensors, along with GPS, detects inclination and orientation to stitch images accurately, even without a pan-tilt camera, by using posture and orientation information to associate pan and elevation angles with each image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If panoramic image pickup is carried out using a digital camera or portable equipment, then image capture becomes accessible and convenient, but accurate panoramic image pickup cannot be achieved due to camera shake and displacement of rotational axis
Solution Approach 1:
The patent replaces the mechanical pan-tilt camera system with a portable equipment (such as a smartphone) equipped with sensors (three-axis gyro, acceleration sensor, geomagnetic sensor) to detect posture and orientation. This substitution allows accurate panoramic image pickup without requiring a complex mechanical pan-tilt mechanism, thereby maintaining ease of operation while improving synthesis accuracy.
Solution Approach 2:
The patent introduces sensors (three-axis gyro, acceleration sensor, geomagnetic sensor) as intermediaries to detect camera shake and displacement of the rotational axis. These sensors provide movement information that is used to correct image alignment during synthesis, enabling accurate panoramic image generation even when using simple portable equipment without mechanical stabilization.
2Extent of automation
If image matching is used to synthesize panoramic images, then automatic synthesis is achieved, but errors occur in the synthesized panoramic image due to camera movement
Solution Approach 1:
The patent performs preliminary detection of camera posture and orientation using sensors before image synthesis. By acquiring movement information in advance through three-axis gyro, acceleration, and geomagnetic sensors, the system can pre-correct for camera shake and rotational axis displacement, ensuring accurate image matching and synthesis without errors.
Solution Approach 2:
The patent implements a feedback mechanism where sensor data (three-axis gyro, acceleration sensor, geomagnetic sensor) continuously provides movement information about camera shake and rotational axis displacement. This feedback is used to adjust and correct image alignment during synthesis, maintaining high reliability and accuracy in the synthesized panoramic image.
Data Source
Figure 1(a)~1(b)
Figure 2
Figure 3(a)~3(b)
AI summary
A characteristic point extraction unit 82 extracts a characteristic point in an overlapping region of a plurality of component images picked up using a camera built in portable equipment and having pan angles or elevation angles different from each other. A self-position estimation unit 84 estimates three-dimensional coordinate values of the characteristic point and three-dimensional coordinate values of a camera position, which are associated with each other, at the same time in two component images in which the camera positions are displaced from each other. A panoramic image generation unit 30 adjusts, based on the estimated three-dimensional coordinate values of the camera position, the three-dimensional coordinate values of the characteristic point of the component image to correct the component image and combine a plurality of component images having pan angles or elevation angles different from each other to generate a synthesis image.