Video Dataset Augmentation via Subframe Pixel Change Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current data augmentation methods for video datasets in machine learning lack effective techniques to increase variation and efficiency, particularly in action detection, where manual cropping is time-consuming and reduces model accuracy.

Innovation Solution

A method that divides video frames into subframes, calculates cumulative pixel differences, and selects the subframe with the most changes to generate cropped video copies, which are then used to augment the training dataset, increasing data variation and efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual cropping is used to increase data variation, then model accuracy improves, but time consumption increases significantly

Engineering Contradiction:

Improvemodel accuracyVSAvoidtime consumption

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs automatic cropping by computing pixel differences and selecting subframes with the most changes, eliminating the need for manual cropping operations while maintaining data variation for model training

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

Manual cropping operations are replaced with an automated computational system that uses pixel difference calculations to identify and crop relevant subframes, substituting human labor with algorithmic processing

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If more video samples are collected to improve model accuracy, then detection accuracy improves, but data preparation effort increases

Engineering Contradiction:

Improvedetection accuracyVSAvoiddata preparation effort

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system creates multiple cropped copies from existing video frames by dividing frames into subframes and selecting those with the most pixel changes, generating additional training samples without collecting new video data

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system transforms existing video data by changing its spatial parameters through cropping operations, creating varied training samples from the same source video to increase data diversity without additional data collection

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11829443B2Augmentation of video datasets for machine learning training

Publication Date: 2023.11.28 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Disclosed are techniques for augmenting video datasets for training machine learning algorithms with additional video datasets that are cropped copies of the video datasets. Frames of a received video dataset are divided into a plurality of subframes. For each subframe, a count is tallied corresponding to the cumulative number of pixels changed across the frames of the received video. Counts are compared to determine which subframe includes the most changed pixels across the frames of the video dataset, which is selected as a cropping candidate. The cropping candidate is used to generate copies of the video dataset that are cropped to include at least the cropping candidate and exclude at least some of the remaining portions of each frame of the video dataset that are outside of the cropping candidate. In some embodiments, boundaries of cropping candidates are transformed to generate a plurality of cropped variations of the video dataset.