Optical-Flow Detector-Tracker for Occlusion-Robust Object Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer vision tracking systems often lose track of objects due to changes in appearance, occlusion, or rotation, making it difficult to accurately identify and follow objects across sequential images.
Innovation Solution
A machine-learning architecture that uses a combination of a backbone model, detection portions, and an optical flow model to generate feature maps and optical flow vectors, allowing for the deformation and concatenation of feature maps across images, enabling accurate tracking of objects even when they rotate, translate, or change appearance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a detector correctly detects an object in two sequential images, then detection accuracy is improved, but the tracking system cannot disambiguate whether the two detections correspond to the same object or two different objects of the same classification
Solution Approach 1:
The patent introduces an optical flow model as an intermediary component that computes optical flow vectors between sequential images. These vectors serve as a mediator to bridge the detector outputs across frames, providing motion context that enables the tracker to disambiguate whether detections correspond to the same object. The optical flow model processes the relationship between consecutive image frames and provides this intermediate information to the tracking system.
Solution Approach 2:
The patent merges multiple data sources including detector outputs, optical flow vectors, and feature maps into a unified tracking framework. By combining detection results with optical flow information and deformed feature maps from multiple frames, the system creates a comprehensive representation that enables reliable object identity disambiguation while maintaining high detection accuracy.
2Adaptability or versatility
If the object rotates or is occluded making it appear different, then appearance variability increases, but the tracking system loses track of the object
Solution Approach 1:
The patent employs dynamic feature map deformation based on optical flow vectors to adapt the tracking representation to object appearance changes. The feature maps are dynamically warped to align with the current object appearance while preserving temporal context from previous frames. This dynamic adaptation allows the tracker to maintain reliability despite rotations, occlusions, and other appearance variations.
Solution Approach 2:
The system performs preliminary deformation of feature maps using optical flow vectors before combining them with current frame detections. By pre-aligning the feature maps to account for expected object motion and appearance changes, the system prepares the tracking data structure in advance, making it more robust to appearance variability when the actual comparison and tracking decision are made.
3Measurement precision
If the system processes multiple sequential images to maintain tracking, then tracking accuracy is improved, but computational redundancy increases
Solution Approach 1:
The patent performs preliminary processing of feature maps by deforming them with optical flow vectors before the main tracking comparison. This preliminary action prepares the data in a form that reduces computational complexity during the actual tracking decision phase, as the feature maps are already aligned and ready for efficient comparison with current detections.
Solution Approach 2:
The system extracts and utilizes only the essential optical flow vectors and deformed feature map regions that are most relevant for tracking decisions. By extracting only the necessary information from the full image sequences and focusing computation on salient regions, the system maintains high tracking accuracy while reducing redundant computational operations across multiple sequential images.
Data Source
AI summary
A machine-learning (ML) architecture may comprise a first ML model and/or an optical flow model that receive, as input, a first image and a second image. The first ML model may output a first feature map corresponding to the first image and a second feature map corresponding to the second image. The optical flow model may output an estimated optical flow. A deformation component may modify the second feature map, as a deformed feature map, based at least in part on the estimated optical flow. The deformed feature map and the first feature map may be concatenated together as a concatenated feature map, which may be provided to a second ML model. The second ML model may be trained to output an output ROI and/or a track in association with an object represented in the first image.