Automatic Image Labeling for Autonomous Driving via Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Supervised machine learning requires large amounts of labeled data, which is often generated manually, making it labor-intensive and prone to errors, especially for applications like autonomous vehicle systems that need thousands to millions of input-output pairs for effective training.
Innovation Solution
A method and system that automatically label images of roadside objects using a camera mounted on a vehicle, combined with a DGPS/IMU system for accurate position and orientation data, and a high-definition map for correlating and synchronizing the data to generate input-output pairs, leveraging HD map positional information and optical corrections for precise labeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling is used to generate training data for supervised machine learning, then labeling accuracy can be maintained, but labor intensity and time consumption increase significantly
Solution Approach 1:
The system uses the vehicle's own sensing capabilities (camera, DGPS/IMU) and pre-existing HD map data to automatically generate labels for training data, eliminating the need for external manual annotation services. The vehicle essentially labels its own training data through self-observation and correlation with map information.
Solution Approach 2:
The patent replaces the mechanical process of manual labeling with an automated computational system that uses sensor data fusion, coordinate transformation, and algorithmic correlation between image data and HD map positional information to generate labels automatically.
2Reliability
If manual labeling is used to generate training data, then error rates can be controlled, but labor costs and resource requirements increase
Solution Approach 1:
The system incorporates feedback loops where the machine learning model's predictions are compared against ground truth from HD maps, and the labeling process uses feedback from successful correlations to continuously improve accuracy. The system validates labels by cross-referencing multiple data sources and correcting discrepancies through iterative refinement.
Solution Approach 2:
The automated labeling system serves multiple functions simultaneously: it captures images, determines vehicle position and orientation, correlates image positions with map data, generates labels, and creates training datasets - all within a single integrated system that replaces multiple separate manual processes.
3Reliability
If large amounts of training data are collected for machine learning, then model performance improves, but data processing and storage requirements increase
Solution Approach 1:
The system performs preliminary actions by pre-processing images during capture, immediately correlating them with vehicle sensor data and HD map information to generate labels on-the-fly. This preliminary labeling during data collection reduces the burden of post-processing and enables more efficient storage of already-processed training data.
Solution Approach 2:
The patent merges multiple data streams (camera images, DGPS position data, IMU orientation data, HD map information) into a single integrated labeling process, reducing the complexity of handling separate data processing pipelines and enabling more efficient data management through unified processing architecture.
Data Source
AI summary
A method of automatic labeling of images for supervised machine learning includes obtaining images of roadside objects with a camera mounted to a vehicle, recording a position and orientation of the vehicle within a defined coordinate system while obtaining the images recording position information for each roadside object with the same defined coordinates system as used while recording the position and orientation of the vehicle, and correlating a position of each of the obtained images of the roadside objects with the position information of each roadside object in view of the recorded position and orientation of the vehicle. The images are labeled to identify the roadside objects in view of the correlated position of each of the obtained images of the roadside objects.