Map Deviation Detection via Camera-Map Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated and autonomous vehicles rely on high-definition map data, but deviations between stored maps and real-world environments can lead to safety risks due to outdated or incorrect map data, especially in dynamic environments like construction sites, and existing detection methods perform poorly in adverse weather conditions.
Innovation Solution
A deep learning-based map deviation detection system that uses a neural network to fuse camera image data with map projections, performing instance segmentation and classification to detect and verify map elements, with optional 3D positioning and artefact compensation, allowing for reliable map updates even in challenging conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing detection methods are used for HD map elements, then map deviation detection is performed, but performance decreases in bad weather conditions and when elements are occluded
Solution Approach 1:
The system performs preliminary actions by projecting map elements into the camera image before deviation detection. This pre-processing step creates hypothesis about element locations and features, allowing the neural network to compare expected versus actual sensor data and detect deviations more reliably even in challenging conditions
Solution Approach 2:
The patent introduces an intermediary approach by fusing projected map data with camera image data. This fusion creates a processed map that serves as a mediator between the stored map and real-world observations, enabling the neural network to leverage both map prior knowledge and sensor data for more robust deviation detection
2Productivity
If map data is used without deviation detection, then driving functions can operate, but safety risks increase due to incorrect map data
Solution Approach 1:
The system implements feedback by continuously comparing projected map elements with actual camera sensor data and using the neural network to detect deviations. This feedback loop allows the system to identify when map data does not match the real environment, enabling safety-critical alerts while maintaining continuous driving function operation
Solution Approach 2:
The patent replaces traditional mechanical verification methods with a neural network-based computational approach. The neural network processes the fused map and image data to automatically detect deviations, providing a more reliable and scalable solution compared to conventional methods
3Device complexity
If map deviation detection is performed without neural networks, then processing is simpler, but detection performance is insufficient
Solution Approach 1:
The patent applies dimensionality change by projecting three-dimensional map elements onto the two-dimensional camera image plane. This projection creates a direct comparison between map hypotheses and 2D sensor observations, enabling the neural network to perform precise deviation detection using well-established 2D image processing techniques
Data Source
Figure 1
Figure 2
AI summary
It is provided a method for detection of map deviations, system and vehicle. The method for a detection of map deviations comprising receiving sensor data from a camera and receiving data from a map; projecting elements from the map into the camera image; providing a processed map which is fused with image data from the camera; performing an instance segmentation using a neural network detecting verifications and deviations; and providing a classification of map elements.