Neural Network Video Stabilization for Jitter Correction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for correcting jittering in images captured by a shaking camera, such as those used in autonomous vehicles, either increase the camera's weight and cost or cause image distortion, and existing software techniques do not adequately address the optical flow of objects.

Innovation Solution

A method using multiple neural networks to estimate and correct jittering by generating adjusted images through jittering vectors, object motion vectors, and optical flow vectors, optimizing these vectors to minimize image distortion.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Stability of the object's composition

If physical correction methods are used to prevent jittering, then image stability is improved, but camera module weight increases and cost becomes expensive

Engineering Contradiction:

Improveimage stabilityVSAvoidcamera module weight

Core Design Contradiction:

Stability of the object's compositionVSWeight of moving object

Solution Approach 1:

The patent replaces physical/mechanical correction methods with a software-based neural network approach. The system uses deep learning models to detect and correct jittering in images captured by the camera, eliminating the need for additional physical stabilization components that would increase weight and cost.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces an intermediary processing system that includes a neural network model and processing circuitry. This intermediary detects jittering in captured images and generates corrected images by compensating for camera movement, serving as a mediator between the raw camera output and the final stabilized image.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Stability of the object's composition

If physical correction methods are used to prevent jittering, then image stability is improved, but device complexity and malfunction risk increase

Engineering Contradiction:

Improveimage stabilityVSAvoiddevice complexity

Core Design Contradiction:

Stability of the object's compositionVSDevice complexity

Solution Approach 1:

The patent replaces complex physical correction mechanisms with a software-based neural network approach. The system uses deep learning models to detect and correct jittering in images captured by the camera, eliminating the need for additional physical stabilization components that would increase weight and cost.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent enables the camera system to self-correct jittering through integrated neural network processing. The processing circuitry within the camera device itself performs jitter detection and correction, allowing the system to service its own stabilization needs without external intervention or additional complex components.

Inventive Principle:

Principle #25Self-service

3Weight of moving object

If software correction techniques are used to address jittering, then device weight is reduced, but image distortion increases

Engineering Contradiction:

Improvecamera module weightVSAvoidimage distortion

Core Design Contradiction:

Weight of moving objectVSManufacturing precision

Solution Approach 1:

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent employs parameter changes in the neural network processing to optimize correction accuracy. The system adjusts processing parameters such as optical flow calculations, feature point matching thresholds, and transformation matrix parameters to minimize image distortion while achieving effective jitter correction.

Inventive Principle:

Principle #35Parameter changes

4Device complexity

If conventional software techniques are used to correct jittering, then device complexity is reduced, but object tracking accuracy decreases due to large image distortion

Engineering Contradiction:

Improvedevice complexityVSAvoidobject tracking accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentEP3690811B1Learning method and learning device for removing jittering on video acquired through shaking camera by using a plurality of neural networks for fault tolerance and fluctuation robustness in extreme situations, and testing method and testing device using the same

Publication Date: 2025.10.29 STRADVISION

Figure 1
Figure 2
Figure 3

AI summary

A method for detecting jittering in videos generated by a shaken camera to remove the jittering on the videos using neural networks is provided for fault tolerance and fluctuation robustness in extreme situations. The method includes steps of: a computing device, generating each of t-th masks corresponding to each of objects in a t-th image; generating each of t-th object motion vectors of each of object pixels, included in the t-th image by applying at least one 2-nd neural network operation to each of the t-th masks, each of t-th cropped images, each of (t-1)-th masks, and each of (t-1)-th cropped images; and generating each of t-th jittering vectors corresponding to each of reference pixels among pixels in the t-th image by referring to each of the t-th object motion vectors. Thus, the method is used for video stabilization, object tracking with high precision, behavior estimation, motion decomposition, etc.