Virtual Gyroscope Image Stabilization for Fisheye Lenses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional electronic image stabilization (EIS) methods introduce undesirable artifacts in video or burst images, especially when non-rectilinear lens distortion is present, and are not suitable for consumer-grade action cameras due to bulkiness and cost.
Innovation Solution
A method and apparatus that transform images from a fisheye or other non-rectilinear lens into a spherical image space, align features using a virtual gyroscope model, and apply transformations to compensate for camera motion while preserving foreground motion, using a processor and non-transitory computer-readable storage medium to execute instructions for image stabilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional electronic image stabilization (EIS) is used to compensate for camera motion, then camera motion can be reduced, but undesirable artifacts are introduced into the video or burst images
Solution Approach 1:
The patent introduces an intermediary coordinate system (spherical or equirectangular projection) as a mediator between the captured non-rectilinear images and the final stabilized output. By transforming images into this intermediate coordinate space, aligning features there, and then transforming back, the system avoids direct manipulation in the distorted space, thereby reducing artifacts while maintaining stabilization effectiveness
Solution Approach 2:
The patent replaces the conventional EIS approach (which operates directly on image pixels through geometric transformations) with a coordinate system transformation approach. Instead of applying stabilization transformations directly to the distorted non-rectilinear images, the system substitutes a two-step transformation process: first to spherical/equirectangular space, then back to the target space, which preserves image quality while achieving stabilization
2Reliability
If gyroscope-based cameras or camera mounts are used to reduce camera motion, then unwanted camera motion is reduced or eliminated, but the device becomes bulky and expensive
Solution Approach 1:
The patent creates a virtual model (copy) of gyroscope-based stabilization that runs entirely in software. Instead of requiring physical gyroscopes and mechanical stabilization systems, the system copies the functional effect of hardware stabilization through computational methods - detecting camera motion from image sequences and applying corresponding corrections digitally, thereby achieving gyroscope-like stabilization without the bulk and cost
Solution Approach 2:
The patent substitutes mechanical stabilization systems (gyroscopes and camera mounts) with a purely computational approach. By replacing physical sensing and mechanical correction with image-based motion detection and digital transformation, the system eliminates the need for bulky hardware while achieving comparable stabilization results
3Reliability
If features are aligned across transformed images to stabilize content, then camera motion artifacts are reduced, but foreground motion may be distorted
Solution Approach 1:
The patent applies local quality by treating background and foreground differently during the alignment process. Background features are used to determine camera motion and guide the transformation, while foreground objects are preserved with their original motion characteristics. This selective approach ensures that stabilization corrects camera-induced motion without distorting intentional foreground movement
Solution Approach 2:
The patent segments the image content into background and foreground components. By identifying and separating these elements, the system can apply stabilization transformations selectively - aligning background features to compensate for camera motion while preserving the independent motion of foreground objects, thereby maintaining natural appearance of moving subjects
Data Source
AI summary
A target image captured from a fisheye lens or other lens with known distortion parameters may be transformed to align it to a reference image. Corresponding features may be detected in the target image and the reference image. The features may be transformed to a spherical coordinate space. In the spherical space, images may be re-pointed or rotated in three dimensions to align all or a subset of the features of the target image to the corresponding features of the reference image. For example, in a sequence of images, background features of the target image in the spherical image space may be aligned to background features of the reference image in the spherical image space to compensate for camera motion while preserving foreground motion. An inverse transformation may then be applied to bring the images back into the original image space.