Bayesian Multiple Object Tracking via Neural Network Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object tracking technologies in video applications face challenges in tracking multiple objects robustly, especially in online analysis, due to reliance on bounding boxes and specific object types, and are limited by their offline nature and sensitivity to noise and motion.
Innovation Solution
A multiple object tracking method that combines neural networks within a Bayesian framework, using category and instance likelihoods to track objects probabilistically, allowing for online analysis and robustness against noise and motion, without relying on future frames or specific object embeddings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple object trackers operate offline on recorded video using references to future and past frames, then tracking paths can be determined accurately, but the system cannot be used for online analysis of live video feeds
Solution Approach 1:
The system pre-computes and stores appearance embeddings for each object instance in a database before tracking begins. These pre-computed embeddings serve as reference templates that enable the tracker to make accurate predictions in real-time without needing to access future frames, thus enabling online analysis while maintaining tracking accuracy
2Adaptability or versatility
If generalized single-object trackers use bounding boxes as input, then they can track objects in subsequent frames, but they cannot handle multiple objects simultaneously
Solution Approach 1:
The system segments the tracking problem by maintaining separate state representations and appearance embeddings for each object instance. Each object is tracked independently with its own probability distribution and embedding vector, allowing the system to handle multiple objects simultaneously while retaining the simplicity of single-object tracking approaches
3Measurement precision
If multiple object trackers specialize in tracking people using facial recognition and skeletal modeling, then tracking accuracy for people is improved, but the system becomes unsuitable for tracking other object types
Solution Approach 1:
The system uses a universal object embedding representation that can represent any object type, not just people. The appearance embeddings are computed from generic object features rather than person-specific features like facial recognition or skeletal modeling, making the tracker adaptable to any object type while maintaining accuracy through the probabilistic framework
4Device complexity
If object tracking relies on deterministic state estimates, then computation is simpler, but the system becomes sensitive to noise and motion causing tracking failures
Solution Approach 1:
The system changes the parameter representation from deterministic state estimates to probabilistic state distributions. Each object's state is represented by a probability distribution over possible locations and appearances, allowing the system to naturally handle noise and motion uncertainty while maintaining computational tractability through efficient sampling and embedding comparison methods
Data Source
AI summary
Techniques for multiple object tracking in video are described in which the outputs of neural networks are combined within a Bayesian framework. A motion model is applied to a probability distribution representing the estimated current state of a target object being tracked to predict the state of the target object in the next frame. A state of an object can include one or more features, such as the location of the object in the frame, a velocity and/or acceleration of the object across frames, a classification of the object, etc. The prediction of the state of the target object in the next frame is adjusted by a score based on the combined outputs of neural networks that process the next frame.