HD Map Localization Using Object Detection Instead of Pixel Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional autonomous vehicle systems require computationally expensive comparisons between camera images and high-definition maps for object and ego vehicle localization, which are sensitive to camera calibration and can lead to latency issues, potentially hazardous in autonomous driving scenarios.
Innovation Solution
A machine-learning model is trained to identify object locations on a high-definition map using object detection data from images, allowing for efficient localization by predicting ego vehicle and object positions without the need for extensive pixel-by-pixel matching, thereby reducing computational load and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional pixel-by-pixel matching between camera images and HD map attributes is used for localization, then measurement precision of object and ego vehicle locations is achieved, but computational complexity and processing time increase significantly
Solution Approach 1:
The patent replaces the mechanical/computational image matching system with a machine learning-based prediction system. Instead of performing pixel-by-pixel comparisons between camera images and HD map attributes, a neural network model predicts ego vehicle location and object locations directly from image features and HD map data, significantly reducing computational complexity while maintaining localization accuracy
Solution Approach 2:
The patent introduces a machine learning model as an intermediary between the raw sensor inputs (camera images, HD map data) and the localization output. This model processes the input data through learned features and predictions to produce location estimates, avoiding the need for direct pixel-by-pixel matching while preserving measurement precision
2Measurement precision
If conventional image-to-HD map comparisons are performed for localization, then object and ego vehicle locations are identified, but processing latency increases which can be hazardous in autonomous driving
Solution Approach 1:
The patent performs preliminary processing by extracting relevant features from camera images and HD map data before localization. The machine learning model is pre-trained with synthetic data to learn the mapping between inputs and locations, enabling fast inference during actual operation without performing computationally intensive pixel-by-pixel matching in real-time
Solution Approach 2:
The patent substitutes the time-consuming mechanical image matching process with a trained neural network that performs rapid prediction. The model directly outputs location estimates from processed inputs, eliminating the iterative pixel-by-pixel comparison process and significantly reducing processing latency
3Productivity
If machine learning models are used for localization, then computational load and processing time are reduced, but the system requires training data and model setup infrastructure
Solution Approach 1:
The patent performs the complex model training and data preparation work in advance, before deployment to the autonomous vehicle. Synthetic training data is generated offline, and the machine learning model is trained beforehand, so that during actual operation only inference is required. This shifts the computational burden from runtime to setup time, improving real-time productivity
Solution Approach 2:
The patent uses synthetic training data that copies and simulates real-world scenarios to train the model without requiring extensive real annotated data. This approach simplifies the data collection and preparation infrastructure while still enabling effective model training for the localization task
Data Source
AI summary
Embodiments provide improved techniques for identifying object and vehicle locations on an HD map using camera images. Synthetic data may be generated by populating a high-definition (HD) map with an object. The populated HD map can be projected to a two-dimensional camera view image depicting the object. The HD map and object detection data generated from image (e.g., identifying lane/object locations within the image) can be used to train the model to identify HD map locations of an autonomous vehicle capturing images as it travels, as well as various objects detected from those captured images. Subsequently, a new image may be processed using object detection techniques to detect lane/object locations within the image. An HD map and the detected lane/object data may be provided to the model, which in turn identifies, on the HD map, the locations of the vehicle and the various objects.