Visual Localization Using Synthetic LIDAR Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high cost of 3D LIDAR scanners limits their use in consumer-grade autonomous vehicles, necessitating a cost-effective alternative for precise localization, as existing methods rely on expensive sensor suites and are susceptible to environmental changes.
Innovation Solution
A visual localization system utilizing a graphics processing unit (GPU) to generate synthetic camera images, which are compared against real-time vehicle imagery to maximize normalized mutual information, allowing for accurate localization with a monocular camera and leveraging LIDAR-based ground maps for metric and surface reflectivity, enabling real-time localization at approximately 10 Hz.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D LIDAR scanners are used for localization, then measurement precision is improved, but device cost increases
Solution Approach 1:
The patent creates synthetic LIDAR images from 3D maps and uses these synthetic images as a reference for comparing against real camera images. This copying approach allows the system to achieve LIDAR-level localization accuracy using only a monocular camera, eliminating the need for expensive 3D LIDAR scanners while maintaining centimeter-level precision
Solution Approach 2:
The patent replaces expensive 3D LIDAR scanners with inexpensive monocular cameras. The camera is a low-cost sensor that can be mass-produced, and by using synthetic image generation and normalization techniques, the system achieves comparable localization performance without the high device cost of LIDAR
2Device complexity
If feature-point based visual localization is used, then device complexity is reduced, but measurement precision deteriorates
Solution Approach 1:
The patent replaces traditional feature-point extraction and matching algorithms with a direct image normalization and comparison approach. Instead of detecting and matching discrete features, the system normalizes the camera image and synthetic LIDAR images to a common representation and compares them directly, achieving higher precision with simpler mechanics
Solution Approach 2:
The patent transforms images into a normalized representation space where lighting, color, and other varying parameters are standardized. By changing the parameter space from raw pixel values to normalized features, the system achieves robust and precise localization without complex feature-point algorithms
3Productivity
If real-time localization is implemented, then productivity is improved, but use of energy increases
Solution Approach 1:
The patent pre-generates synthetic LIDAR images from the 3D map and pre-computes normalization parameters before real-time localization. This preliminary action allows the real-time system to simply compare normalized images without performing computationally intensive processing, achieving real-time performance with reduced energy consumption
Solution Approach 2:
The patent extracts only the essential normalization parameters and synthetic image data needed for localization, separating these from the full 3D map data. By taking out only the necessary information for comparison, the system reduces computational load and energy consumption while maintaining real-time localization capability
Data Source
AI summary
An apparatus and method for visual localization of a visual camera system outputting real-time visual camera data and a graphics processing unit receiving the real-time visual camera data. The graphics processing unit accesses a database of prior map information and generates a synthetic image that is then compared to the real-time visual camera data to determine corrected position data. The graphics processing unit determines a camera position based on the corrected position data. A corrective system for applying navigation of the vehicle based on the determined camera position can be used in some embodiments.