Automatic Object Labeling via Spatio-Temporal Traversal for Autonomous Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current autonomous driving systems require tedious manual processes for object labeling, which is inefficient and lacks effective methods for labeling objects in autonomous vehicles, especially for objects that are too small, blurred, or occluded.
Innovation Solution
An automatic object labeling system that traverses past images to identify and label objects based on the moving trail of detected objects, allowing for offline annotation without user intervention, using sensor data from cameras, LIDAR, and RADAR devices to connect lane lines and detect obstacles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation processes are used for object labeling, then labeling accuracy can be maintained, but the process becomes tedious and time-consuming
Solution Approach 1:
The system performs preliminary object detection and labeling on current frames where objects are clearly visible, then uses this pre-processing information to automatically label corresponding objects in past frames. This preliminary action on easily detectable objects enables subsequent automatic labeling of difficult objects in historical data without manual intervention.
Solution Approach 2:
The system copies labeling information from current frames to corresponding past frames by matching objects based on spatial-temporal relationships. Objects detected in current frames are copied to their historical positions in past frames, creating training labels automatically without repeating the detection process.
2Speed
If automatic object detection is used in real-time, then processing speed is maintained, but objects that are too small, blurred, or occluded cannot be detected
Solution Approach 1:
The system performs preliminary detection on current frames where objects are in optimal detection conditions (larger size, less blur, less occlusion). This preliminary detection on favorable conditions enables subsequent labeling of the same objects in past frames where detection would be difficult, effectively preprocessing easy cases to solve hard cases.
Solution Approach 2:
The system transitions from spatial-only detection to spatio-temporal detection by utilizing information across multiple time frames. Objects that are undetectable in individual past frames become detectable when considering their temporal trajectory and position across the video sequence, adding the time dimension to overcome detection limitations.
3Reliability
If more training data with annotated objects is collected, then model training quality improves, but manual labeling effort increases proportionally
Solution Approach 1:
The system copies detection results and labeling information from current frames to corresponding past frames, automatically generating training annotations without manual labeling. This copying approach enables rapid expansion of training datasets while maintaining consistent labeling quality across all frames.
Solution Approach 2:
The system performs self-labeling by automatically detecting objects in current frames and generating corresponding labels for past frames without human intervention. This self-service capability enables the system to autonomously create training data, eliminating the need for manual annotation processes.
Data Source
AI summary
A list of images is received. The images were captured by a sensor of an ADV chronologically while driving through a driving environment. A first image of the images is identified that includes a first object in a first dimension (e.g., larger size) detected by an object detector using an object detection algorithm. In response to the detection of the first object, the images in the list are traversed backwardly in time from the first image to identify a second image that includes a second object in a second dimension (e.g., smaller size) based on a moving trail of the ADV represented by the list of images. The second object is then labeled or annotated in the second image equivalent to the first object in the first image. The list of images having the labeled second image can be utilized for subsequent object detection during autonomous driving.