Cross-Sensor Object Tracking Using Learned Feature Descriptor Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object tracking systems, particularly in autonomous driving and ADAS, face challenges with motion models that rely on key points, which fail in scenarios with changes in camera viewpoint, partial observations, and complex object movements. Additionally, convolutional DNN-based models are not optimized for object tracking, leading to identification switches and high computational costs.
Innovation Solution
The system employs learned feature vectors for object tracking across single and multiple sensors, using vector triplets to train a DNN directly for object tracking. This approach leverages pixel-level feature descriptors to track objects even when partially observed or occluded, across different sensors and time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If motion models with key points are used for object tracking, then tracking can be achieved using simple feature descriptors, but tracking fails when camera viewpoint changes, objects are partially observed, or objects move in complex ways
Solution Approach 1:
The patent transforms the tracking approach by changing the feature representation parameters from simple key points to deep learned feature descriptors. This parameter change enables the system to capture high-level semantic features that remain stable under viewpoint changes, occlusions, and complex movements, thereby improving tracking reliability while maintaining computational efficiency through the use of pre-extracted features.
Solution Approach 2:
The patent replaces the traditional motion model-based tracking mechanism with a deep learning-based feature matching mechanism. Instead of relying on pre-defined motion models and hand-tuned thresholds, the system uses learned feature descriptors from convolutional neural networks to identify and track objects, eliminating the need for manual parameter tuning and improving robustness to various tracking challenges.
2Measurement precision
If convolutional DNN-based models are used for object tracking, then high-level semantic features can be captured, but computational cost increases and object identification switches occur
Solution Approach 1:
The patent applies preliminary action by pre-extracting and caching deep feature descriptors from object detection outputs before the tracking phase. This allows the tracking algorithm to work with pre-processed high-quality features rather than performing expensive DNN inference during tracking, thereby maintaining high measurement precision while significantly improving tracking speed and reducing computational burden.
Solution Approach 2:
The patent extracts only the necessary feature descriptors from the object detection pipeline and separates them for dedicated tracking processing. By extracting and caching these features independently from the main detection flow, the system avoids redundant computations and enables efficient matching during tracking, thus resolving the contradiction between accuracy and speed.
3Adaptability or versatility
If hand-tuned threshold values are used in motion models, then tracking can be adapted to different scenarios, but the thresholds need to be re-tuned for different road conditions and scenarios
Solution Approach 1:
The patent implements self-service by using deep learning models to automatically learn optimal feature representations and matching criteria from data, eliminating the need for manual threshold tuning. The system adapts to different road conditions and scenarios automatically through the learned features, removing the burden of hand-tuned parameter adjustment while maintaining high adaptability across diverse environments.
Data Source
AI summary
In various examples, live perception from sensors of a vehicle may be leveraged to generate object tracking paths for the vehicle to facilitate navigational controls in real-time or near real-time. For example, a deep neural network (DNN) may be trained to compute various outputs—such as feature descriptor maps including feature descriptor vectors corresponding to objects included in a sensor(s) field of view. The outputs may be decoded and/or otherwise post-processed to reconstruct object tracking and to determine proposed or potential paths for navigating the vehicle.