Camera Localization via Ground-Aerial Feature Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for determining the pose of vehicles, robots, and drones using satellite images often lack the necessary resolution for precise navigation, typically providing +/−3 meters in location and +/−3 degrees in orientation, which is insufficient for tasks like navigating traffic lanes.
Innovation Solution
The method employs satellite image guided geo-localization that uses two neural networks to extract features and confidence maps from both ground and aerial images, iteratively determining geometric correspondence between 3D feature points to achieve a high-definition three degree-of-freedom pose without requiring high-definition maps, utilizing convolutional neural networks and the Levenberg-Marquardt algorithm for refinement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If satellite images are used to determine vehicle pose, then location and orientation data can be obtained, but the resolution is insufficient (±3 meters location, ±3 degrees orientation) for precise navigation
Solution Approach 1:
The patent combines satellite image data with ground view images captured by the vehicle's camera system. By merging these two different perspectives and using neural networks to extract and match features between them, the system achieves high-definition pose estimation (±1 meter location, ±1 degree orientation) that exceeds the capability of satellite images alone.
2Measurement precision
If high-definition maps are used to improve pose determination accuracy, then navigation precision increases, but mapping efforts and computational resources required increase significantly
Solution Approach 1:
Instead of requiring pre-existing high-definition maps, the system creates a virtual copy of the environment by capturing ground view images with the vehicle's camera. Neural networks process these images to extract features and determine pose, effectively creating a lightweight, on-demand representation of the environment that avoids the complexity of maintaining comprehensive HD maps.
Solution Approach 2:
The vehicle's own camera system serves as the mapping tool. By using the vehicle-mounted camera to capture ground view images and process them through neural networks, the system performs its own environmental understanding and pose determination without relying on external mapping infrastructure or heavy computational resources.
Data Source
AI summary
A computer that includes a processor and a memory, the memory including instructions executable by the processor to determine a first feature map and a first confidence map from a ground view image with a first neural network. First feature points can be determined based on the first feature map and the confidence map. First three-dimensional (3D) feature locations of the first features can be determined based on the first features and the first confidence map. A second feature map and a second confidence map can be determined from an aerial-view image with a second neural network. Second 3D feature locations can be based on the first 3D features, the second feature map and the second confidence map. A three degree-of-freedom (DoF) pose of a ground view camera in global coordinates can be determined by iteratively determining geometric correspondence between the first and second 3D feature locations.