Semantic Domain Vehicle Localization via Aerial Map Registration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for vehicle localization, such as GNSS, cellular towers, and 3D visual feature matching, face limitations in accuracy, precision, and computational efficiency, especially in urban areas and adverse weather conditions, due to signal attenuation and high computational overhead.
Innovation Solution
The use of 2D aerial-view semantic maps and real-time semantic images for vehicle localization, where the map and images are semantically segmented and registered in a semantic domain, enabling precise determination of vehicle position and orientation using semantic features and inverse perspective mapping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If GNSS-based triangulation methods are used for vehicle localization, then the system can determine position using satellite signals, but the accuracy and precision are limited to several meters and performance degrades in urban areas and adverse weather conditions
Solution Approach 1:
The patent introduces aerial imagery and semantic feature extraction as an intermediary system between the vehicle and satellite signals. By capturing images from above (aerial perspective) and extracting semantic features (drivable surfaces, buildings, vegetation), the system creates a map-based reference framework that works independently of satellite signal quality, thereby mediating the localization problem in urban canyons and adverse weather where GNSS fails
Solution Approach 2:
The patent replaces the physical satellite signal transmission mechanism with an image-based semantic mapping mechanism. Instead of relying on electromagnetic satellite signals that are blocked by buildings and weather, the system uses captured aerial images processed through semantic segmentation to create localization references, substituting one physical mechanism (satellite signals) with another (image processing and semantic feature matching) that is not subject to the same environmental limitations
2Measurement precision
If 3D visual feature matching methods are used for vehicle localization, then the system can determine position and orientation using map and image correlation, but the computational overhead is high and processing complexity increases
Solution Approach 1:
The patent extracts only the essential semantic features (drivable surfaces, buildings, vegetation) from the full 3D visual data through semantic segmentation. By taking out only the relevant semantic information needed for localization rather than processing all 3D visual features, the system reduces computational overhead while maintaining the ability to determine position and orientation through map-image correlation
Solution Approach 2:
The patent segments the visual data into distinct semantic categories (drivable surfaces, buildings, vegetation) rather than processing raw 3D visual features as a whole. This segmentation allows the system to work with classified semantic maps and images, reducing the complexity of feature matching while preserving the geometric and spatial information needed for accurate localization
3Measurement precision
If conventional 3D visual maps and drive-time images are used for localization, then the system can correlate visual features to determine vehicle location, but the storage requirements and computational processing are significant
Solution Approach 1:
The patent changes the representation parameters of visual data from raw 3D pixel values and geometric features to semantic class labels through semantic segmentation. By transforming the data from continuous visual parameters to discrete semantic categories, the system reduces storage requirements while maintaining the spatial and geometric information needed for location determination through semantic map-image correlation
Data Source
AI summary
Localizing a vehicle on the Earth's surface, via the registration of a map and real-time images of the vehicle's environment, is discussed. Both the map and real-time images are 2D representations of the surface, both are from an aerial-view perspective of the surface, and both are represented in a semantic-domain, rather than a visual-domain. The map is an aerial-view semantic map that includes 2D semantic representations of objects located on the surface. The semantic representations of the map indicate semantic labels and absolute positions of the objects. The real-time images are real-time aerial-view semantic images that include additional 2D semantic representations of the objects. The additional semantic representations of the real-time images indicate semantic labels and relative positions of the objects. Via image registration, the absolute position and orientation of the vehicle is determined based on a spatial and rotational correspondence between the absolute and relative positions of the objects.