Pixel Tracking via Anchor Frame Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for tracking pixels in videos face challenges such as error accumulation, sensitivity to occlusion, and non-robustness, particularly in complex scenes with large motions and illumination changes, leading to inconsistent trajectories and difficulty in recovering from temporary occlusions.
Innovation Solution
A computer-implemented method that estimates multiple position candidates for a pixel in a current frame by chaining its position in anchor frames with optical flow, allowing for selection or combination of these candidates to determine the pixel's position, thereby reducing drift and improving robustness through the use of non-visibility and uncertainty scores.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If optical flow is concatenated between consecutive frames to form trajectories, then the method is simple to implement, but error accumulation leads to drift and trajectories become inconsistent
Solution Approach 1:
The patent segments the trajectory formation process by introducing multiple anchor frames at different time points. Instead of concatenating optical flow continuously between consecutive frames, the method divides the trajectory into multiple segments, each anchored to a specific anchor frame. This segmentation reduces error accumulation within each segment while maintaining implementation simplicity.
Solution Approach 2:
The patent applies preliminary action by pre-identifying multiple anchor frames throughout the video sequence before trajectory formation. These anchor frames serve as predetermined reference points that stabilize the trajectory estimation process. By having these reference points established in advance, the method prevents drift accumulation without requiring complex real-time corrections.
2Device complexity
If optical flow concatenation is used for long-term tracking, then computational complexity is reduced, but the method becomes sensitive to occlusion and non-robust in complex scenes
Solution Approach 1:
The patent changes the parameter of temporal spacing between anchor frames. By strategically positioning anchor frames at varying intervals throughout the video sequence, the method adapts to different scene conditions. In complex scenes with occlusions, anchor frames are positioned to provide better coverage, while in simpler scenes, the spacing can be reduced to maintain low computational complexity.
Solution Approach 2:
The patent introduces anchor frames as intermediary reference points between the start and end of trajectories. These intermediary frames act as mediators that break the direct long-term tracking into shorter, more reliable segments. Each anchor frame serves as a local reference that is less susceptible to occlusion effects, thereby improving robustness without significantly increasing computational complexity.
3Reliability
If multiple anchor frames are used to estimate position candidates, then robustness and accuracy are improved, but the number of position candidates increases
Solution Approach 1:
The patent applies partial action by selectively using only the necessary number of anchor frames and position candidates required for reliable tracking. Instead of using all possible anchor frames to generate all possible position candidates, the method selects a subset that provides sufficient robustness. This partial approach maintains tracking reliability while controlling the quantity of position candidates to avoid excessive computational burden.
Data Source
Figure 1~3

AI summary
A method for tracking at least one pixel from a reference frame (#0) to a current frame (#7) in a video comprising the reference frame (#0), the current frame (#7) and at least one intermediate frame (#1-#6) in-between, the pixel having a known position (30) in the reference frame (#0), the method comprising: - estimating a plurality of position candidates (36) for the pixel in the current frame (#7), wherein each of the position candidates (36) is obtained by chaining a position (32) of the at least one pixel in a respective anchor frame with an optical flow (34) from the respective anchor frame to the current frame (#7), wherein the anchor frame (#0, #3, #5, #6) is chosen among the reference frame (#0) and the at least one intermediate frame (#1-#6), and; - determining the position of the at least one pixel in the current frame (#7) by selecting one of the position candidates (36) or combining two or more of the position candidates (36).