Optical Flow Estimation via Realistic Distraction and Pseudo-Labeling

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional optical flow estimation systems face challenges in accurately estimating optical flow due to their inability to capture complex real-world variations and rely heavily on simple visual features, while also struggling with the scarcity of ground truth annotations, which limits their robustness and accuracy.

Innovation Solution

The approach involves generating realistic distractions by combining frames with distractor images from the same domain to create augmented frames, allowing the optical flow estimation model to learn semantically meaningful variations and leveraging unlabeled data through pseudo-labeling and cross-consistency regularization, thereby improving model robustness and accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If simple visual features are used for optical flow estimation, then the system complexity is reduced, but the accuracy and ability to capture complex real-world variations deteriorates

Engineering Contradiction:
Improvesystem complexityVSAvoidoptical flow estimation accuracy
Core Design Contradiction:
Device complexityVSMeasurement precision

Solution Approach 1:

The patent transforms the input data by applying distraction images and pseudo-labeling techniques, changing the parameters of the training data to include more varied and realistic scenarios. This allows the model to learn from enhanced data without increasing the fundamental complexity of the optical flow estimation system.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent creates pseudo-labels by copying and transforming existing ground truth annotations to generate additional training samples. This approach enriches the training data without requiring additional complex system components or increasing system complexity.

Inventive Principle:
Principle #26Copying

2Measurement precision

If ground truth annotations are extensively used for training, then the accuracy of optical flow estimation is improved, but the loss of time and resources for annotation increases

Engineering Contradiction:
Improveoptical flow estimation accuracyVSAvoidannotation time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent implements self-service through pseudo-labeling, where the model generates its own training labels by processing existing data and ground truth annotations. This automated label generation reduces the need for manual annotation while maintaining training quality, allowing the system to serve itself in creating training data.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent performs preliminary action by pre-processing ground truth annotations to create pseudo-labels before actual training. This advance preparation of training data reduces the need for extensive manual annotation during the training process, saving time and resources.

Inventive Principle:
Principle #10Preliminary action

3Ease of manufacture

If the model is trained only on labeled data, then the training process is simplified, but the model's robustness to real-world conditions deteriorates

Engineering Contradiction:
Improvetraining process simplicityVSAvoidmodel robustness
Core Design Contradiction:
Ease of manufactureVSReliability

Solution Approach 1:

The patent changes the parameters of the training data by introducing distraction images and pseudo-labels that simulate real-world variations. This enhances the model's exposure to diverse conditions while maintaining a relatively simple training framework, thus improving robustness without significantly complicating the training process.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent introduces distraction images as an intermediary element during training. These distraction images act as mediators that bridge the gap between simple labeled data and complex real-world scenarios, allowing the model to learn robust features without requiring complex training procedures.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS20240161312A1Realistic distraction and pseudo-labeling regularization for optical flow estimation
Publication Date: 2024.05.16 QUALCOMM INC
  • US20240161312A1 patent drawing
  • US20240161312A1 patent drawing
  • US20240161312A1 patent drawing

AI summary

A computer-implemented method includes generating a first augmented frame by combining a first image and a first frame of a first frame pair. The computer-implemented method also includes generating, via an optical flow estimation model, a first flow estimation based on a second frame of the first frame pair and the first augmented frame. The computer-implemented method further includes updating one or both of parameters or weights of the optical flow estimation model based on a first loss between the first flow estimation and a training target.