Deep Learning Video Frame Selection Heuristic Suppression

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Selecting representative images from a video efficiently is challenging due to the lack of labeled data for training machine learning algorithms and the need for real-time processing on mobile devices, where existing methods are either time-consuming or resource-intensive.

Innovation Solution

A combination of deep learning techniques and heuristic frame suppression is used to identify high-quality image frames, employing a similarity distance computation and suppression curve to select the top frames, which can operate in real-time on mobile devices with a small model size, reducing the need for extensive labeled data and minimizing processing time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If machine learning algorithms are used to select representative images from video, then selection accuracy can be improved, but processing time and computational resources increase significantly

Engineering Contradiction:

Improveselection accuracyVSAvoidprocessing time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent segments the video processing task into two distinct phases: an offline training phase where a machine learning model is trained using labeled data to learn representative image characteristics, and an online inference phase where the trained model rapidly scores and selects frames from new videos. This segmentation allows the computationally intensive training to be performed once, while subsequent video processing benefits from fast, pre-learned decision rules.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary action by pre-training the machine learning model with labeled video data before actual video processing. The training phase extracts features and learns patterns of representative frames in advance, creating a trained model that can be quickly applied to new videos without requiring extensive processing time during the actual selection task.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If extensive labeled data is used to train machine learning models, then model accuracy improves, but data preparation time and resources increase

Engineering Contradiction:

Improvemodel accuracyVSAvoiddata preparation time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses heuristic frame suppression that operates independently of the trained model to eliminate obviously non-representative frames. This self-service mechanism filters out poor quality frames based on simple criteria (such as temporal proximity to already-selected frames), reducing the burden on the machine learning model and allowing it to focus on more subtle discrimination tasks.

Inventive Principle:

Principle #25Self-service

3Measurement precision

If complex machine learning models are deployed on mobile devices, then selection quality improves, but device computational load and energy consumption increase

Engineering Contradiction:

Improveselection qualityVSAvoiddevice energy consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent employs heuristic frame suppression as a lightweight, computationally inexpensive method to quickly eliminate obviously non-representative frames before applying the more sophisticated trained model. This cheap filtering approach reduces the number of frames that require intensive processing, thereby lowering overall computational load and energy consumption on mobile devices.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

4Measurement precision

If all video frames are evaluated individually, then selection completeness improves, but processing efficiency decreases

Engineering Contradiction:

Improveselection completenessVSAvoidprocessing efficiency

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent extracts and removes obviously non-representative frames from consideration using heuristic suppression rules before applying the trained machine learning model. By taking out frames that are temporally close to already-selected frames or meet simple exclusion criteria, the system reduces the candidate pool without sacrificing the selection of truly representative frames, thereby improving processing efficiency.

Inventive Principle:

Principle #2Taking out (Extraction)