Scalable Cross-Modal Multi-Camera Tracking With Transformer Memory Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object tracking methods fail to leverage correlations between point trajectories from the same objects, leading to reduced accuracy and increased computational complexity, especially in scenarios with occlusions and complex background motion, and are often trained on synthetic data unsuitable for 3D object detection.
Innovation Solution
An object motion predictor that samples key points, combines short-term and long-term features using locality sensitive hashing and transformer layers to model interactions, predicting 3D boxes with tracklets, and trains using regression loss to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing object tracking methods are used, then computational complexity is reduced, but object tracking accuracy deteriorates due to failure to leverage correlations between point trajectories
Solution Approach 1:
The patent segments the tracking problem into key point detection and trajectory correlation components. By identifying and tracking specific key points on objects rather than processing entire object masks or frames, the system reduces computational complexity while maintaining accuracy through focused correlation analysis of point trajectories across frames
Solution Approach 2:
The patent introduces point trajectory correlations as an intermediary mechanism to connect object detection across frames. By modeling the temporal correlations of key point trajectories, the system achieves accurate tracking without requiring complex full-object processing, thus resolving the contradiction between accuracy and computational complexity
2Reliability
If existing object tracking methods are used, then processing speed is maintained, but tracking reliability deteriorates in scenarios with occlusions and complex background motion
Solution Approach 1:
The patent performs preliminary key point detection and feature extraction on individual objects before conducting correlation analysis. By pre-identifying stable key points and their features in advance, the system prepares correlation-ready data structures that enable reliable tracking through occlusions without requiring re-processing of entire frames, thus maintaining processing speed while improving reliability
Solution Approach 2:
The patent implements feedback through temporal correlation modeling of point trajectories. By continuously tracking how key points move and correlate across frames, the system adapts to occlusions and background motion patterns, maintaining reliable tracking performance without sacrificing processing efficiency through intelligent reuse of historical trajectory information
3Productivity
If synthetic data training is used, then training efficiency is improved, but detection precision deteriorates for 3D object detection in real-world scenarios
Solution Approach 1:
The patent changes the training data parameter from synthetic to real-world captured images. By training on actual camera footage with real lighting conditions, occlusions, and motion patterns, the system achieves accurate 3D object detection performance while maintaining training efficiency through the use of standard supervised learning frameworks adapted to real data
Data Source
AI summary
A method of object tracking includes detecting a dynamic object in a scene, sampling key points of the dynamic object, extracting short term features of the key points, combining long-term key point features read from a key point features database into combined key point features, applying attention processing to hash the combined key point features, applying a plurality of transformer layers on top of attention processing to update the combined key point features using the interactions of the key points to form the long-term key point features and storing the long-term key point features in the key point features database, and predicting an updated 3D box for the dynamic object from the updated combined key point features, the updated 3D box including a tracklet representing motion of the dynamic object in the scene over time.