Video Stabilization Using Motion Sensor Displacement

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing image stabilization methods for video signals in mobile devices require significant processing resources, making real-time stabilization challenging, especially when devices have limited processing capabilities and are subject to camera movement during video capture.

Innovation Solution

The method involves using data from motion sensors to determine camera displacement between exposure time midpoints of video frames, integrating angular velocity to calculate angular displacement, and compensating for motion by shifting frames without copying data, thus stabilizing the video signal efficiently and effectively.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If motion vectors are used for image stabilization, then video stabilization accuracy is improved, but processing resource consumption increases

Engineering Contradiction:

Improvevideo stabilization accuracyVSAvoidprocessing resource consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent extracts only the essential motion information needed for stabilization by using motion sensors to directly measure camera movement, rather than performing full motion vector analysis on video frames. This extraction approach provides sufficient stabilization accuracy while dramatically reducing processing requirements.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent replaces the computational mechanical system of motion vector calculation with a direct physical measurement system using motion sensors. The sensors physically measure camera acceleration and orientation, substituting complex image processing with straightforward sensor data acquisition and integration.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If real-time video stabilization is performed, then video quality is improved, but device complexity increases

Engineering Contradiction:

Improvevideo qualityVSAvoiddevice complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent performs preliminary action by continuously monitoring camera motion through sensors during video capture. By measuring and integrating motion data in advance, the system prepares compensation information before video frames are fully processed, enabling real-time stabilization without complex post-processing operations.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces motion sensors as an intermediary between the camera and the stabilization processing. These sensors act as mediators that directly measure camera motion and provide compensation data, simplifying the overall system architecture by decoupling motion measurement from video frame analysis.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Productivity

If motion sensor data is used for stabilization, then processing speed is improved, but measurement precision challenges arise

Engineering Contradiction:

Improveprocessing speedVSAvoiddisplacement estimation accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent transitions from two-dimensional pixel-based motion analysis to three-dimensional physical space measurement using motion sensors. By measuring acceleration and orientation in 3D space and integrating this data, the system achieves accurate displacement estimation while maintaining high processing speed, as sensor data requires simpler mathematical operations than frame-by-frame image analysis.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach provides accurate image stabilization with reduced processing demands, allowing for real-time stabilization in mobile devices by synchronizing sensor data with camera frames and using motion sensor data to determine displacement, resulting in improved pixel displacement estimation and reduced computational complexity.

Implementation Method 1

using a motion sensor associated with the camera to generate a plurality of samples representing motion of the camera

Methodology Applied

Scientific EffectMotion sensing: Accelerometer

Implementation Method 2

integrating angular velocity to calculate angular displacement

Methodology Applied

Scientific EffectAngular velocity integration:

Data Source

PatentUS10412305B2Video stabilization

Publication Date: 2019.09.10 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Method, device and computer program product for stabilizing a video signal. A plurality of frames of the video signal are captured using a camera. A motion sensor associated with the camera is used to generate a plurality of samples representing motion of the camera. The samples are used to determine a displacement of the camera between a first time and a second time, wherein the first time corresponds to an exposure time midpoint of a first frame of the video signal and the second time corresponds to an exposure time midpoint of a second frame of the video signal. The determined displacement is used to compensate for motion in the video signal between the first and second frames caused by the motion of the camera, to thereby stabilize the video signal.