Video Stabilization Using Background Subtraction for Turbulence Correction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Long distance imaging applications face challenges with video stabilization and artefacts removal due to atmospheric turbulence, which causes geometric distortion and blur, especially in surveillance scenarios where moving objects are of interest, as existing methods struggle with random and temporal variations in turbulence effects, leading to false positives and negatives in object detection.

Innovation Solution

A real-time video stabilization method that determines background and moving object regions using a predetermined model, generates turbulence-corrected frames by fusing corrected background and object regions, and updates the background model based on processed frames to improve stability and reduce artefacts.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If multiple frames are used to remove turbulence effect, then geometric distortion is reduced, but moving objects become blurred

Engineering Contradiction:

Improvegeometric distortion correctionVSAvoidmoving object clarity

Core Design Contradiction:

Manufacturing precisionVSReliability

Solution Approach 1:

The patent divides the video frame into background regions and moving object regions, applying different processing methods to each. Background regions undergo turbulence correction using multiple frames, while moving object regions are preserved from the original frames to avoid blurring.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies different quality standards and processing techniques to different parts of the image. Background areas receive aggressive turbulence correction, while moving object areas maintain their original quality to preserve detail and avoid motion blur artifacts.

Inventive Principle:

Principle #3Local quality

2Difficulty of detecting and measuring

If background extraction is performed on turbulence-affected frames, then moving objects can be detected, but false positive errors increase due to apparent background movement

Engineering Contradiction:

Improvemoving object detectionVSAvoiddetection accuracy

Core Design Contradiction:

Difficulty of detecting and measuringVSReliability

Solution Approach 1:

The patent performs turbulence correction on background frames before conducting background extraction and moving object detection. This preliminary stabilization eliminates the apparent motion in background regions caused by turbulence, thereby reducing false positive detection errors.

Inventive Principle:

Principle #10Preliminary action

3Device complexity

If rigid frame registration is used to avoid problematic regions, then processing is simplified, but turbulence effect is not corrected due to random and local geometric distortion

Engineering Contradiction:

Improveprocessing complexityVSAvoidturbulence correction effectiveness

Core Design Contradiction:

Device complexityVSManufacturing precision

Solution Approach 1:

The patent segments the frame into reliable regions (with dense SURF features) and problematic regions, applying rigid registration only to reliable regions while using more sophisticated local distortion correction in problematic regions to address turbulence effects.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies different registration strategies to different regions of the frame. High-confidence regions undergo rigid registration for simplicity, while turbulence-affected regions receive localized non-rigid correction to maintain geometric accuracy.

Inventive Principle:

Principle #3Local quality

4Difficulty of detecting and measuring

If CNN semantic segmentation is used to classify objects, then moving objects can be identified, but the system requires clearly defined object classes and cannot monitor unexpected objects

Engineering Contradiction:

Improveobject classification accuracyVSAvoidobject detection flexibility

Core Design Contradiction:

Difficulty of detecting and measuringVSAdaptability or versatility

Solution Approach 1:

The patent inverts the traditional approach by first stabilizing the background and then detecting moving objects as deviations from the stabilized background. This eliminates the need for pre-defined object classes and enables detection of any moving object, including unexpected ones.

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentUS10600158B2Method of video stabilization using background subtraction

Publication Date: 2020.03.24 CANON KK

AI summary

The present disclosure provides a method of correcting for a turbulence effect in a video comprising a plurality of frames. The method comprises determining a first background region and a region corresponding to a moving object in a first frame of the plurality of frames using a predetermined background model. A second background region in a second frame of the plurality of frames is then determined using the predetermined background model. A turbulence-corrected background region from the first background region and the second background region is generated and the region corresponding to the moving object and the turbulence-corrected background region is fused to form a turbulence-corrected frame. The method then updates the predetermined background model based on the turbulence-corrected frame and corrects for the turbulence effect in the second frame using the updated predetermined background model.