Sensor aided image stabilization

Abstract

Techniques described herein provide a method for improved image and video stabilization using inertial sensors. Gyroscopes, accelerometers and magnetometers are examples of such inertial sensors. The movement of the camera causes shifts in the image captured. Image processing techniques may be used to track the shift in the image on a frame by frame basis. The movement of the camera may be tracked using inertial sensors. By calculating the degree of similarity between the image shift as predicted by image processing techniques with shift of the device estimated using one or more inertial sensor, the device may estimate the portions of the image that are stationary and those that are moving. Stationary portions of the image may be used to transform and align the images. For video stabilization, the realigned images may be combined to generate the video. For image stabilization, the realigned images may be either added or averaged to generate the de-blurred image.

Description

CROSS-REFERENCES TO RELATED APPLICATIONS[0001]This application claims priority to U.S. Provisional Application No. 61 / 552,382 entitled “SENSOR AIDED VIDEO AND IMAGE STABILIZATION,” filed Oct. 27, 2011, and is hereby incorporated by reference.BACKGROUND OF THE INVENTION[0002]Image stabilization aims to de-blur the image and video stabilization aims to stabilize videos to eliminate unintentional tremor and shake. Both video shake and image blur can result from motion introduced from the camera capturing the image. Shift and blur due to the motion introduced from camera shake can be minimized using known image analysis techniques to minimize offsets between consecutive frames of a video. However, these techniques from the related art suffer from an inherent limitation that they cannot distinguish between camera motion and subject moving in the field of view.[0003]Motion blur due to camera shake is a common problem in photography, especially in conditions involving zoom and low light. P...

AI Technical Summary

Benefits of technology

The patent describes a technique for stabilizing videos and images to reduce shake and blur caused by motion. The technique uses known image analysis techniques to minimize the offset between consecutive frames of a video, but it also uses data from inertial sensors to estimate which parts of the image are stationary and which parts are moving. These estimates are then used to align and combine multiple images to create a new image with significantly reduced blur. This technique helps to overcome the limitations of existing image stabilization methods, which can be affected by subject motion or sensor in-idealities.

Problems solved by technology

Both video shake and image blur can result from motion introduced from the camera capturing the image.

However, these techniques from the related art suffer from an inherent limitation that they cannot distinguish between camera motion and subject moving in the field of view.

Motion blur due to camera shake is a common problem in photography, especially in conditions involving zoom and low light.

Pressing a shutter release button on the camera can itself cause the camera to shake.

This problem is especially prevalent in compact digital cameras and cameras on cellular phones, where optical stabilization is not common.

If during this time—the exposure time—the image moves, either due to camera or object motion, the resulting image will exhibit motion blur.

The problem of motion blur due to camera shake is increased when a long focal length (zoom) is employed, since a small angular change of the camera creates a large displacement of the image.

The problem is exacerbated in situations when long exposure is needed, either due to lighting conditions, or due to the use of a small aperture.

This operation is computationally expensive and difficult to perform on a small mobile device.

Both video shake and image blur can result from motion introduced from the camera capturing the image.

However, these techniques from the related art suffer from an inherent limitation that they cannot distinguish between camera motion and subject moving in the field of view.

Instead of obtaining one image with a long exposure time, one can obtain multiple consecutive images with short exposure times. These images will typically be underexposed.

Images