Pixel Tracking via Anchor Frame Segmentation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing methods for tracking pixels in videos face challenges such as error accumulation, sensitivity to occlusion, and non-robustness, particularly in complex scenes with large motions and illumination changes, leading to inconsistent trajectories and difficulty in recovering from temporary occlusions.

Innovation Solution

A computer-implemented method that estimates multiple position candidates for a pixel in a current frame by chaining its position in anchor frames with optical flow, allowing for selection or combination of these candidates to determine the pixel's position, thereby reducing drift and improving robustness through the use of non-visibility and uncertainty scores.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If optical flow is concatenated between consecutive frames to form trajectories, then the method is simple to implement, but error accumulation leads to drift and trajectories become inconsistent

Engineering Contradiction:
ImproveEase of implementationVSAvoidTrajectory consistency
Core Design Contradiction:
Ease of manufactureVSReliability

Solution Approach 1:

The patent segments the trajectory formation process by introducing multiple anchor frames at different time points. Instead of concatenating optical flow continuously between consecutive frames, the method divides the trajectory into multiple segments, each anchored to a specific anchor frame. This segmentation reduces error accumulation within each segment while maintaining implementation simplicity.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies preliminary action by pre-identifying multiple anchor frames throughout the video sequence before trajectory formation. These anchor frames serve as predetermined reference points that stabilize the trajectory estimation process. By having these reference points established in advance, the method prevents drift accumulation without requiring complex real-time corrections.

Inventive Principle:
Principle #10Preliminary action

2Device complexity

If optical flow concatenation is used for long-term tracking, then computational complexity is reduced, but the method becomes sensitive to occlusion and non-robust in complex scenes

Engineering Contradiction:
ImproveComputational complexityVSAvoidRobustness to occlusion
Core Design Contradiction:
Device complexityVSReliability

Solution Approach 1:

The patent changes the parameter of temporal spacing between anchor frames. By strategically positioning anchor frames at varying intervals throughout the video sequence, the method adapts to different scene conditions. In complex scenes with occlusions, anchor frames are positioned to provide better coverage, while in simpler scenes, the spacing can be reduced to maintain low computational complexity.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent introduces anchor frames as intermediary reference points between the start and end of trajectories. These intermediary frames act as mediators that break the direct long-term tracking into shorter, more reliable segments. Each anchor frame serves as a local reference that is less susceptible to occlusion effects, thereby improving robustness without significantly increasing computational complexity.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Reliability

If multiple anchor frames are used to estimate position candidates, then robustness and accuracy are improved, but the number of position candidates increases

Engineering Contradiction:
ImproveTracking robustnessVSAvoidNumber of position candidates
Core Design Contradiction:
ReliabilityVSQuantity of substance

Solution Approach 1:

The patent applies partial action by selectively using only the necessary number of anchor frames and position candidates required for reliable tracking. Instead of using all possible anchor frames to generate all possible position candidates, the method selects a subset that provides sufficient robustness. This partial approach maintains tracking reliability while controlling the quantity of position candidates to avoid excessive computational burden.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentEP4465239A1A method and system for tracking a pixel in a video
Publication Date: 2024.11.20 TOYOTA JIDOSHA KK
  • EP4465239A1 patent drawingFigure 1~3
  • EP4465239A1 patent drawing
  • EP4465239A1 patent drawing

AI summary

A method for tracking at least one pixel from a reference frame (#0) to a current frame (#7) in a video comprising the reference frame (#0), the current frame (#7) and at least one intermediate frame (#1-#6) in-between, the pixel having a known position (30) in the reference frame (#0), the method comprising: - estimating a plurality of position candidates (36) for the pixel in the current frame (#7), wherein each of the position candidates (36) is obtained by chaining a position (32) of the at least one pixel in a respective anchor frame with an optical flow (34) from the respective anchor frame to the current frame (#7), wherein the anchor frame (#0, #3, #5, #6) is chosen among the reference frame (#0) and the at least one intermediate frame (#1-#6), and; - determining the position of the at least one pixel in the current frame (#7) by selecting one of the position candidates (36) or combining two or more of the position candidates (36).