Dynamic reconstruction of high resolution video from low-resolution color-filtered video (video-to-video super-resolution)

Abstract

In one aspect, the present invention provides a dynamic super-resolution technique that is computationally efficient. A recursive computation takes as input a previously computed super-resolved image derived from a sequence of low-resolution input frames. Combining this super-resolved image with a later low-resolution input frame in the sequence, the technique produces a new super-resolved image. By recursive application, a sequence of super-resolved images is produced. In a preferred embodiment, the technique uses a computationally simple and effective method based on adaptive filtering for computing a high resolution image and updating this high resolution image over time to produce an enhanced sequence of images. The method may be implemented as a general super-resolution software tool capable of handing a wide variety of input image data.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS [0001] This application claims priority from U.S. provisional patent application Ser. No. 60 / 637058 filed 12 / 16 / 2004, which is incorporated herein by reference. STATEMENT OF GOVERNMENT SPONSORED SUPPORT [0002] This invention was supported in part by the National Science Foundation under grant CCR-9984246 and by the U.S. Air Force under contract F49620-03-01-0387. The U.S. Government may have certain rights in the invention.FIELD OF THE INVENTION [0003] This invention relates generally to a class of digital image processing techniques known as digital image reconstruction. More particularly, it relates to methods for computing a temporal sequence of resolution-enhanced images from an original sequence of lower-resolution images. BACKGROUND OF THE INVENTION [0004] Super-resolution image reconstruction is a kind of digial image processing that increases the resolvable detail in images. The earliest techniques for super-resolution generated a stil...

AI Technical Summary

Problems solved by technology

The details of how to reconstruct the best high-resolution image from multiple low-resolution images is a complicated problem that has been an active topic of research for many years, and many different techniques have been proposed.

One reason the super-resolution reconstruction problem is so challenging is because the reconstruction process is, in mathematical terms, an under-constrained inverse problem.

Moreover, in cases where a unique solution is determined, it is not stable, i.e., small noise perturbations in the images can result in large differences in the super-resolved image.

Part of the challenge rests in selecting constraints that sufficiently restrict the solution space without an unacceptable increase in the computational complexity.

Another challenge is to select constraints that properly restrict the solution space to good high-resolution images f