Multi-Object Multi-View Association via 3D Bounding Box Graph Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-camera tracking systems face delays and inaccuracies due to high computational loads when attempting real-time multi-object multi-view association, particularly in scenarios with occlusions and similar object appearances, such as sports analytics.
Innovation Solution
A spatially-assisted method using a multi-view association graph formulation that focuses on 3D bounding boxes instead of feature points, with graph pruning to reduce computational load and improve accuracy, especially in environments with known camera setups like athletic stadiums.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional multi-camera tracking systems use traditional feature point-based multi-view association methods, then they can track multiple objects across different camera views, but they suffer from high computational loads causing delays and cannot achieve sufficiently accurate real-time tracking
Solution Approach 1:
The patent segments the tracking problem by focusing only on bounding box centers rather than all feature points, and by processing each camera view independently before association. This segmentation reduces the computational complexity from O(N*M) feature point comparisons to O(N*M) bounding box center comparisons with simpler distance metrics, enabling real-time performance while maintaining accuracy.
Solution Approach 2:
The patent changes the representation parameters from detailed feature points to simplified bounding box centers (x, y coordinates). This parameter reduction transforms the complex feature matching problem into a simpler geometric proximity problem, significantly reducing computational load while preserving the essential spatial information needed for accurate tracking association.
2Reliability
If conventional systems process all feature points for multi-object association, then they can handle complex scenarios with occlusions and similar appearances, but the computational load increases causing real-time performance degradation
Solution Approach 1:
The patent extracts only the essential information (bounding box centers) from the full feature point data, discarding redundant details. This extraction approach maintains the critical spatial relationships needed for reliable association while eliminating computational overhead, achieving both reliability and real-time productivity.
Solution Approach 2:
Instead of matching features from multiple views and then deriving bounding boxes, the patent inverts the approach by detecting bounding boxes independently in each view and then matching them based on center proximity. This inversion simplifies the computational pipeline and enables parallel processing, improving both speed and reliability.
3Measurement precision
If the system uses detailed feature point data for association, then it can achieve accurate object identification, but the computational complexity prevents real-time processing of multiple objects
Solution Approach 1:
The patent uses inexpensive bounding box center coordinates instead of computationally expensive feature point descriptors. These simplified representations are sufficient for the task and can be processed rapidly, trading detailed information for computational efficiency while maintaining adequate identification accuracy for real-time applications.
Data Source
AI summary
Example image processing methods, apparatus, systems and articles of manufacture (e.g., computer readable media) to implement multi-object multi-view association are disclosed. Examples disclosed herein obtain initial two-dimensional (2D) bounding boxes of detected objects in multiple images of a scene from multiple cameras, generate a plurality of three-dimensional (3D) bounding boxes potentially corresponding to one or more objects to be tracked, and generate a weight that represents a strength of association between two detected objects respectively in two images of the multiple images. The weight is based on projection of one of the 3D bounding boxes back to at least one of the two images to generate a re-projection 2D bounding box.