Point Tracking Neural Network for Video Occlusion Handling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies struggle to effectively track arbitrary query points across video frames, particularly in handling occlusions and deformation of object surfaces.

Innovation Solution

A point tracking neural network is employed to process video sequences and query points, utilizing a visual backbone neural network to generate feature grids, and a decoder neural network to predict the spatial positions and occlusion scores of query points across frames.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional tracking methods are used, then the system is simpler, but tracking accuracy during occlusions and deformations deteriorates

Engineering Contradiction:

Improvetracking accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent replaces traditional mechanical/optical tracking methods with a neural network-based system. The point tracking neural network processes video frames and query points to predict spatial positions, substituting conventional tracking algorithms with deep learning-based prediction mechanisms that handle occlusions and deformations more effectively.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces feature grids as an intermediary representation between video frames and tracking predictions. The visual backbone neural network extracts feature grids from video frames, and the decoder neural network uses these feature grids along with query points to predict spatial positions, creating a mediating representation that improves tracking robustness.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If fine-grained occlusion tracking is implemented, then video understanding capability improves, but computational complexity increases

Engineering Contradiction:

Improvevideo understanding capabilityVSAvoidcomputational complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent segments the tracking problem into distinct components: query points representing specific locations of interest, feature grids capturing local visual features, and separate prediction streams for spatial position and occlusion status. This segmentation allows the system to focus computational resources on specific tracking challenges.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements local quality by processing individual query points independently through the neural network, allowing different regions of the video to be tracked with appropriate detail. The system extracts features from local neighborhoods around query points and generates predictions specific to each point's context, rather than processing the entire video uniformly.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS20250191194A1Tracking query points in videos using point tracking neural networks

Publication Date: 2025.06.12 GDM HOLDING LLC

AI summary

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for tracking query points in videos using a point tracking neural network.