Object Tracking in Low Frame-Rate Video Using Pyramid Feature Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional object tracking methods face challenges in accurately tracking objects in low frame-rate videos due to long time intervals between frames, leading to difficulties in predicting motion changes and extracting similar features, especially when object shapes or poses change significantly.
Innovation Solution
The method involves generating conversion feature maps by converting features from consecutive frames into feature descriptors with location information, allowing for matching between cropped and pyramid feature maps across frames, enabling accurate tracking regardless of frame rate and object transformations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional object tracking methods are used in low frame-rate videos, then the tracking process can be performed with simple algorithms, but the tracking accuracy deteriorates due to long time intervals between frames causing significant object shape or pose changes
Solution Approach 1:
The patent segments the feature matching process into multiple scales using pyramid feature maps. Instead of attempting to match features across the entire low frame-rate interval directly, the method divides the feature space into multiple resolution levels, allowing accurate matching at each scale while handling large displacements through coarse-to-fine progression.
Solution Approach 2:
The patent introduces a scale dimension by constructing pyramid feature maps with multiple resolutions. This transforms the single-scale matching problem into a multi-scale problem, enabling the system to handle large temporal displacements in low frame-rate videos by matching features at appropriate scales rather than being constrained to a single resolution level.
2Productivity
If the time interval between frames is long in low frame-rate videos, then the frame rate can be reduced for lower data transmission and processing requirements, but the motion prediction becomes inaccurate due to significant object movement and shape changes
Solution Approach 1:
The patent performs preliminary feature extraction and pyramid map construction from the current frame before processing the next frame. By pre-computing multi-scale feature representations and preparing cropped feature maps for predicted object regions, the system reduces the computational burden during the actual tracking step, enabling efficient handling of long time intervals.
Solution Approach 2:
The patent introduces cropped feature maps as an intermediary representation between the full pyramid feature maps and the final matching result. These cropped maps focus computational attention on relevant object regions, serving as a bridge that connects coarse global feature matches with fine local feature comparisons, thereby improving motion prediction accuracy without proportionally increasing overall computation.
3Reliability
If feature matching is performed over the entire frame region, then all possible object locations can be searched, but the searching time increases significantly especially when objects move large distances in low frame-rate videos
Solution Approach 1:
The patent applies local quality by creating cropped feature maps that concentrate computational resources on regions containing objects of interest. Instead of uniformly processing the entire frame, the method generates cropped representations focused on predicted object locations, thereby reducing the search space while maintaining tracking robustness through multi-scale pyramid structures that preserve global context.
4Measurement precision
If multiple feature descriptors are generated for each feature point, then the feature information becomes more comprehensive for accurate matching, but the computational complexity increases due to the need to process and match numerous descriptors
Solution Approach 1:
The patent segments the computational workload by organizing feature descriptors into a hierarchical pyramid structure. Instead of processing all descriptors uniformly, the method divides them into multiple scales and regions, processing coarser scales first to establish rough correspondences, then refining matches at finer scales only in relevant regions, thereby reducing overall computational complexity while maintaining matching accuracy.
Data Source
AI summary
A method for tracking an object in a low frame rate video is provided. Matching processes are performed between consecutive frames by using conversion feature maps acquired by converting each of features on feature maps of the consecutive frames into feature descriptors including each corresponding feature information and each corresponding location information, thereby allowing object tracking regardless of whether time interval per frame is long or short. The object tracking is performed by matching feature descriptors on a plurality of pyramid feature maps on an entire area of a next frame and feature descriptors on a plurality of cropped feature maps generated by cropping object areas extracted on a current frame, thereby allowing not only quick matching between the cropped areas and the entire area but also the increased accuracy due to no limitation of the feature searching area.