Graph-Based Appearance Embeddings for Occlusion-Robust Multi-Object Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object-tracking technologies struggle with accurately tracking multiple moving objects in a scene due to occlusions and manual initialization errors, leading to misidentifications and increased computational complexity.
Innovation Solution
A system using a convolutional neural network (CNN) to analyze image features and a graph network to refine appearance embeddings based on spatial and temporal relationships between detections, enabling accurate clustering and tracking of multiple objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing object-tracking technologies are used to track multiple moving objects, then tracking capability is provided, but accuracy deteriorates due to occlusions and manual initialization errors
Solution Approach 1:
The patent replaces manual initialization methods with an automated appearance embedding system. Instead of manually initializing tracklets for each detected object, the system automatically computes appearance embeddings from image data and uses graph networks to associate detections across frames, eliminating manual intervention and its associated errors.
Solution Approach 2:
The patent introduces appearance embeddings as an intermediary representation between raw image detections and tracklet associations. These embeddings serve as a mediator that captures distinguishing visual characteristics, enabling more accurate matching and reducing misidentifications caused by occlusions or manual errors.
2Reliability
If multiple objects are tracked simultaneously, then comprehensive tracking coverage is achieved, but computational complexity increases
Solution Approach 1:
The patent segments the multi-object tracking problem into distinct processing stages: detection, appearance embedding computation, graph construction, and association. By dividing the complex task of tracking multiple objects into these manageable segments, the system reduces overall computational complexity while maintaining comprehensive tracking coverage.
Solution Approach 2:
The patent performs preliminary computation of appearance embeddings for each detection before attempting associations. This preliminary action pre-processes the data by extracting distinguishing features in advance, which simplifies subsequent matching operations and reduces the computational burden during the actual tracking process.
3Ease of operation
If manual initialization is used for each object, then individual tracking can be established, but errors increase and automation decreases
Solution Approach 1:
The patent replaces manual initialization operations with automated appearance-based association. Instead of requiring manual creation of tracklets for each detected object, the system automatically computes appearance embeddings and uses graph networks to establish associations, achieving both ease of operation and high automation.
Solution Approach 2:
The system enables self-service tracking initialization by automatically processing detections and creating tracklets based on appearance embeddings without human intervention. The graph network autonomously associates detections across frames, allowing the system to initialize and maintain tracking independently.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A multiple-object tracking system includes a convolutional neural network that receives a set of images of a scene that have each been extracted from a frame of a scene. Each of the images corresponds to a detected instance of one of multiple objects that appears in the scene. The convolutional neural network computes, for each image of the set, an appearance embedding vector defining a set of distinguishing characteristics for the image, and a graph network then modifies the appearance embedding vector for each image based on determined relationships between the image and a subset of the images corresponding to detection times temporally separated from a detection time. The modified appearance embedding vectors are then used to identify subsets of the images corresponding to identical targets.