Cross-Sensor Ground Truth Transfer for Recognizer Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The manual annotation of ground truth data for machine learning in autonomous driving vehicles is time-consuming and costly, especially when sensor configurations are updated, requiring re-collection and re-annotation of large data sets.
Innovation Solution
A method and system that automatically generates ground truth data by using the recognition results from a first sensor to annotate and project onto the data of a second sensor with a different modality, leveraging coordinate transformations and calibration to align sensor systems, thereby facilitating machine learning for the second sensor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation is performed to create ground truth data for machine learning, then the accuracy and quality of training data is improved, but the time consumption and cost increase significantly
Solution Approach 1:
The system performs preliminary annotation using the first recognizer on the first sensor data, and stores these recognition results. When ground truth data is needed for the second sensor, the pre-computed recognition results are transformed and projected onto the second sensor data, eliminating the need for real-time manual annotation and significantly reducing time consumption.
Solution Approach 2:
The recognition results obtained from the first sensor data are copied and transformed to serve as ground truth annotations for the second sensor data. Instead of creating annotations from scratch for each sensor, the system copies the semantic information from the first recognizer's output, transforms it to the second sensor's coordinate system, and uses it as ground truth, thereby reducing repetitive manual work.
2Adaptability or versatility
If sensor configuration is updated with replacement or new installation, then the system can utilize improved or additional sensing capabilities, but the ground truth data must be re-collected and re-annotated
Solution Approach 1:
The recognition results from the first sensor are designed to be universally applicable to multiple sensor types through coordinate transformation and projection. The system transforms the recognition results to match the coordinate systems of different sensor configurations (camera, LiDAR, radar), allowing the same annotated data to serve multiple sensor modalities without requiring separate annotation processes for each sensor type.
Solution Approach 2:
The system changes the coordinate system parameters and projection parameters to adapt the recognition results from the first sensor to the second sensor's configuration. By adjusting transformation matrices, coordinate references, and projection geometries, the system makes the ground truth data compatible with different sensor configurations without re-annotation, maintaining productivity during sensor updates.
3Measurement precision
If ground truth data is manually annotated for each sensor individually, then the accuracy for each specific sensor is ensured, but the overall process complexity and resource requirements increase
Solution Approach 1:
The system merges the annotation process for multiple sensors into a single unified process. Instead of performing separate manual annotations for each sensor, the recognition results from one sensor are transformed and projected onto multiple sensors simultaneously. This combining of annotation efforts reduces system complexity while maintaining the accuracy needed for each sensor type through proper coordinate transformation.
Data Source
AI summary
According to the present method, first, first surrounding environment data is acquired by a first sensor. Next, second surrounding environment data is acquired by a second sensor at a same time as an acquisition time of the first surrounding environment data in a same region as an acquisition region of the first surrounding environment data. Next, first recognition result is acquired by inputting the first surrounding environment data into a first recognizer machine-learned using surrounding environment data acquired by the first sensor. Next, the first recognition result is projected onto the second surrounding environment data by transformation from a coordinate system of the first sensor to a coordinate system of the second sensor. Then, ground truth data for machine-learning a second recognizer configured to recognize surrounding environment data acquired by the second sensor is generated based on the second surrounding environment data onto which the first recognition result is projected.