Vehicle Sensor Data Labeling via 2D-3D Track Aggregation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current approaches for labeling sensor data from vehicles are labor-intensive, time-consuming, and prone to human error, particularly when dealing with large datasets required for autonomous navigation and HD map creation, as they rely on manual annotation of 2D and 3D data from multiple sources.
Innovation Solution
A computer-based method that processes and aggregates 2D and 3D sensor data to generate time-aggregated 3D visualizations, allowing for automatic labeling of objects and reducing the need for capture-by-capture annotation, using techniques like semantic segmentation and motion modeling to improve data representation and labeling efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation of 2D and 3D sensor data is used, then labeling accuracy can be maintained, but labor intensity and time consumption increase significantly
Solution Approach 1:
The system performs preliminary automated labeling using machine learning models to generate initial 2D and 3D labels before final review. This preliminary action creates a draft labeling that can be quickly reviewed and corrected manually, significantly reducing the time required compared to starting from scratch with manual annotation.
Solution Approach 2:
The patent introduces an automated labeling system as an intermediary between raw sensor data and final labeled datasets. This intermediary uses machine learning algorithms to generate preliminary labels that serve as a foundation for manual verification, reducing the burden on human annotators while maintaining accuracy.
2Manufacturing precision
If capture-by-capture annotation is performed, then detailed labeling can be achieved, but the process becomes excessively lengthy for large datasets
Solution Approach 1:
The system merges multiple captures containing the same object into a single aggregated representation for labeling purposes. By combining data from multiple captures, the system creates a comprehensive view of each object that can be labeled once rather than repeatedly across captures, significantly improving productivity while maintaining detailed labeling quality.
Solution Approach 2:
The patent implements a universal labeling approach where a single label application can serve multiple captures simultaneously. The system identifies objects that appear across multiple captures and allows annotators to apply labels that propagate across all relevant captures, making the labeling process multi-functional and highly efficient.
3Reliability
If 2D and 3D sensor data from multiple sources are processed separately, then data integrity is maintained, but the labeling process becomes more complex
Solution Approach 1:
The system merges 2D image data and 3D point cloud data into a unified labeling framework. By integrating these different data types and allowing labels to be applied across both modalities simultaneously, the system maintains the integrity of each data source while simplifying the overall processing workflow through unified label propagation.
Data Source
AI summary
Examples disclosed herein may involve (i) based on an analysis of 2D data captured by a vehicle while operating in a real-world environment during a window of time, generating a 2D track for at least one object detected in the environment comprising one or more 2D labels representative of the object, (ii) for the object detected in the environment: (a) using the 2D track to identify, within a 3D point cloud representative of the environment, 3D data points associated with the object, and (b) based on the 3D data points, generating a 3D track for the object that comprises one or more 3D labels representative of the object, and (iii) based on the 3D point cloud and the 3D track, generating a time-aggregated, 3D visualization of the environment in which the vehicle was operating during the window of time that includes at least one 3D label for the object.