Autonomous Vehicle Road Marking Detection via Multimodal AI
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in maintaining accurate maps due to frequent changes in road markings, such as new constructions or repainting, which can lead to inaccurate navigation, especially when using data from different LiDAR sensors and varying environmental conditions like wet or snowy roads, or seasonal changes.
Innovation Solution
A multimodal approach using artificial intelligence (AI) models and sensor data from multiple modalities, including LiDAR and cameras, to identify and update road markings in real-time by comparing on-board and off-board data, ensuring all components of the autonomous vehicle system receive accurate and timely map updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If maps are updated frequently to reflect road marking changes, then map accuracy is improved, but system complexity and data processing requirements increase
Solution Approach 1:
The system segments the map update process into distinct modules: road marking detection using AI models, change detection by comparing on-board and off-board data, and selective map updates. This segmentation allows each module to be optimized independently, improving map accuracy without proportionally increasing overall system complexity.
Solution Approach 2:
The system performs preliminary actions by pre-processing sensor data through AI models to identify road markings before comparison. Off-board data is pre-processed and stored for later comparison with on-board data, enabling efficient change detection and reducing real-time processing complexity.
2Reliability
If multiple sensor modalities are used to detect road markings, then detection reliability is improved, but data processing complexity increases
Solution Approach 1:
The system merges data from multiple sensor modalities (LiDAR, cameras) into a unified representation of road markings. By combining these data sources through AI models, the system achieves improved detection reliability while managing processing complexity through integrated analysis rather than separate processing of each modality.
Solution Approach 2:
AI models serve as intermediaries that process and interpret raw sensor data from multiple modalities. These models transform complex multi-sensor inputs into standardized road marking detections, reducing the complexity of directly processing and integrating raw data from diverse sensor sources.
3Loss of time
If real-time comparison of on-board and off-board data is performed, then map update timeliness is improved, but computational load increases
Solution Approach 1:
The system performs partial comparison by focusing only on relevant road segments where changes are detected rather than processing entire maps. This selective approach maintains timely map updates while reducing overall computational load by avoiding unnecessary processing of unchanged areas.
Solution Approach 2:
The system uses feedback from preliminary AI-based road marking detection to guide the comparison process. By first identifying potential changes through on-board sensor processing and then verifying against off-board data only where needed, the system achieves timely updates with reduced computational requirements compared to exhaustive full-map comparisons.
Data Source
AI summary
Disclosed herein are system, method, and computer program product embodiments for handling changes in road markings. For example, the method includes identifying a road marking in a road trajectory of a vehicle using an artificial intelligence (AI) model and sensor data from a sensor of the vehicle and performing a map update based on a determination that there is a change in road markings in the road trajectory. The determination is based on at least on-board data generated when the vehicle is traversing the road trajectory and off-board data generated using stored data. The sensor data includes two or more sensor modalities.