3D Object Detection With Learned Similarity for Occluded Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-learning-based solutions for scene understanding in robotics, particularly for autonomous robots, suffer from limited performance in object detection and tracking, especially in handling occluded objects and requiring separate training of detectors and similarity models.
Innovation Solution
A system that extracts features from time-sequential sensor data using a three-dimensional (3D) detection backbone, incorporating bidirectional temporal aggregation to generate bird's-eye-view (BEV) feature images, and trains a neural-network 3D detection head with a similarity objective to improve object tracking and detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine-learning-based solutions are used for scene understanding, then object detection capability is improved, but performance in handling occluded objects and temporal consistency deteriorates
Solution Approach 1:
The system performs preliminary feature extraction and temporal aggregation before final object detection and tracking. By pre-processing sensor data through a 3D detection backbone and aggregating temporal information bidirectionally, the system prepares enhanced feature representations that improve subsequent tracking performance, especially for occluded objects
Solution Approach 2:
The system implements feedback mechanisms where detection results from multiple time steps are fed back into the tracking process. The object tracker uses historical detection data and temporal feature aggregations to refine current object states, creating a closed-loop system that continuously improves detection and tracking accuracy
2Adaptability or versatility
If separate training of detectors and similarity models is performed, then model specialization is improved, but system complexity and training time deteriorate
Solution Approach 1:
The system merges the detection and similarity modeling functions into a unified neural network architecture. The 3D detection backbone and detection head work as an integrated system that simultaneously performs feature extraction, object detection, and similarity computation, eliminating the need for separate training processes while maintaining specialized capabilities
Solution Approach 2:
The unified detection model serves multiple functions: it extracts features from sensor data, detects objects in the current frame, computes similarity with historical detections, and tracks objects across time. This multi-functional approach reduces system complexity while preserving the adaptability needed for specialized detection tasks
3Stability of the object's composition
If bidirectional temporal aggregation is used, then temporal consistency is improved, but computational complexity deteriorates
Solution Approach 1:
The system segments the temporal aggregation process into distinct forward and backward passes through the detection backbone. By dividing the bidirectional processing into manageable segments that can be computed separately and then combined, the system reduces computational complexity while maintaining temporal consistency across all time steps
Data Source
AI summary
Systems and methods described herein relate to detecting and tracking objects. In one embodiment, a system extracts first features from time-sequential perceptual sensor data to generate a first set of bird's-eye-view (BEV) feature images. The system also extracts second features from the first set of BEV feature images using a three-dimensional (3D) detection backbone to generate a second set of BEV feature images. The system also consumes the second set of BEV feature images using a neural-network 3D detection head that is trained with a similarity objective to support an object tracker for use in one of (1) controlling an autonomous robot and (2) generating automatically labeled perception data to train one or more of an online perception model, an online prediction model, and an online planning model used to control an autonomous robot.