Drone Image Alignment Using GPS and IMU Sensor Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for creating composite images using a camera mounted on a mobile object require significant computational power, lead to increased data storage costs, and restrict the area that can be imaged due to the need for extensive overlap between images, which is challenging with limited battery capacity.
Innovation Solution
An image processing system that associates position information with images captured by an imaging device on a mobile object, allowing for efficient image processing and composite image creation without the need for extensive feature point extraction, using a drone equipped with sensors and a cloud-based processing system to align images based on GPS and IMU data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If feature points are extracted from every single image and images are superimposed while cross-checking feature points, then composite images can be created with high precision, but enormous computational processing power is required
Solution Approach 1:
The patent extracts only the necessary information (position information from GPS and attitude information from IMU) rather than processing the entire image data to find feature points. This selective extraction approach maintains alignment precision while dramatically reducing computational requirements.
Solution Approach 2:
Position information and attitude information are acquired and associated with images during the imaging process itself, before composite image creation. This preliminary association eliminates the need for computationally intensive feature point extraction and cross-checking during later processing.
2Reliability
If more overlap between adjacent images is achieved to extract feature points consistently, then feature extraction becomes more reliable, but a larger number of images is required leading to increased data storage costs
Solution Approach 1:
The patent replaces the mechanical/image-based feature point extraction system with a sensor-based positioning system (GPS and IMU). Instead of relying on image overlap and feature matching, the system uses independent position and attitude data to accurately locate and orient each image, eliminating the need for extensive overlap and reducing storage requirements.
3Area of stationary object
If back-and-forth motions on a narrow interval are performed to ensure more image overlap between adjacent paths, then image coverage completeness improves, but it becomes difficult to expand the area that can be imaged given battery capacity constraints
Solution Approach 1:
The patent changes the fundamental parameters used for image alignment from image-based feature points to sensor-based position and attitude data. This allows for increased flight intervals and larger imaging areas per flight because the system no longer requires narrow back-and-forth motions to ensure sufficient image overlap for feature matching.
4Power
If position information is associated with images, then computational requirements for image processing are reduced, but additional sensors and data management systems are required
Solution Approach 1:
The GPS receiver and IMU serve multiple functions: they provide position and attitude information for image alignment, and this same data can be used for flight navigation, positioning, and other autonomous operations. This multi-functionality justifies the added sensors by providing broad system benefits beyond just image processing.
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
Provided is an information processing device configured to: acquire an image captured according to an imaging instruction by an imaging device mounted to a mobile object; acquire sensor information including position information about the mobile object from a sensor device according to a signal transmitted from the imaging device in response to the imaging instruction; and associate the acquired sensor information with the acquired image.