Electronic Image Stabilization for Drone Video Analytics

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Video analytics algorithms, such as face tracking, struggle with accuracy due to vibration and rotation in handheld and flying cameras, making it difficult to maintain face orientation, especially in drones where gimbals are costly and impractical.

Innovation Solution

An apparatus that combines electronic image stabilization and horizon keeping with video analytics, using data from gyroscopic and gravity sensors to compensate for motion and maintain image stability, allowing accurate video analytics without the need for a gimbal.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a gimbal is used to stabilize the camera in drones, then image stability and video analytics accuracy are improved, but device complexity and cost increase significantly

Engineering Contradiction:

Improveimage stabilityVSAvoiddevice complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent replaces the mechanical gimbal stabilization system with an electronic image stabilization system that uses sensor data (accelerometer, gyroscope, magnetometer) to detect device motion and applies digital transformations to video frames to compensate for the motion, thereby eliminating the need for complex mechanical components while maintaining image stability

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces an intermediary processing system that acts as a mediator between the camera sensor and the final video output. This intermediary system captures raw video frames, applies stabilization transformations based on sensor data, and outputs corrected frames, effectively decoupling the stabilization function from mechanical hardware

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If a gimbal is used to stabilize the camera in drones, then image stability is improved, but cost increases significantly

Engineering Contradiction:

Improveimage stabilityVSAvoidcost

Core Design Contradiction:

ReliabilityVSEase of manufacture

Solution Approach 1:

The patent replaces the expensive mechanical gimbal system with software-based electronic image stabilization that processes video frames using sensor data, significantly reducing manufacturing costs while maintaining image stability functionality

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent uses inexpensive sensor components (accelerometer, gyroscope, magnetometer) and software processing to achieve stabilization, replacing the need for costly mechanical gimbal assemblies, making the solution economically viable for consumer-grade drones

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

3Adaptability or versatility

If video analytics algorithms are applied to unstable video signals, then object detection capability is maintained, but detection accuracy deteriorates due to vibration and rotation

Engineering Contradiction:

Improveobject detection capabilityVSAvoiddetection accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent applies preliminary stabilization processing to video frames before they are fed into video analytics algorithms. By pre-correcting for device motion using sensor data and transforming frames to compensate for rotation and vibration, the system ensures that subsequent object detection and tracking operations work with stabilized input, thereby improving detection accuracy

Inventive Principle:

Principle #10Preliminary action

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This solution enhances video analytics accuracy in drones by stabilizing images in real-time, improving face tracking and object detection, and reducing costs associated with gimbal systems.

Implementation Method 1

collect data samples from a gravity sensor (G-sensor) and a gyroscopic sensor

Methodology Applied

Scientific EffectGyroscope: Gyroscope

Implementation Method 2

generate a computed horizon level based on the data samples from the G-sensor and the data samples from the gyroscopic sensor

Methodology Applied

Scientific EffectGravitation: Gravitation

Data Source

PatentUS11258949B1Electronic image stabilization to improve video analytics accuracy

Publication Date: 2022.02.22 AMBARELLA INT LP

AI summary

An apparatus includes an input interface and a processor. The input interface may be configured to (i) receive a sequence of video frames of a targeted view of an environment and (ii) collect data samples from a gravity sensor (G-sensor) and a gyroscopic sensor. The processor may be configured to (i) detect axes transitions using the data samples from the gyroscopic sensor, (ii) perform, in real-time, electronic image stabilization, (iii) generate a computed horizon level based on the data samples from the G-sensor and the data samples from the gyroscopic sensor, (iv) perform, in real-time, a rotational offset compensation on stabilized captured image data of the sequence of video frames to maintain alignment of a respective horizon in each frame of the sequence of video frames with the computed horizon level, and (v) generate video analytics for the sequence of video frames using the compensated and stabilized captured image data.