Apparatus, system, and method for multi-patch based super-resolution from an image

Abstract

Embodiments of the present invention include apparatuses, systems and methods for multi-patch based super-resolution from a single video frame. Such embodiments include a scale-invariant self-similarity (SiSS) based super-resolution method. Instead of searching HR examples in a database or in LR image, the present embodiments may select the patches according to the SiSS characteristics of the patch itself, so that the computational complexity of the method may be reduced because there is not any search involved. To solve the problem of lack of relevant examples in natural images, the present embodiments may employ multi-shaped and multi-sized patches in HR image reconstruction. Additionally, embodiments may include steps for a hybrid weighing method for suppressing artifacts. Advantageously, certain embodiments of the method may be 10˜1,000 times faster than the example based SR approaches using patch searching and can achieve comparable HR image quality.

Description

TECHNICAL FIELD[0001]The present invention relates generally to image and video processing and, more particularly, to apparatuses, systems, and methods for super-resolution from an image.BACKGROUND OF THE INVENTION[0002]Super-resolution (SR) methods aim to recover new high-resolution (HR) information beyond the Nyquist frequency of the low-resolution (LR) image. SR methods are applicable in relation to HDTV, video communication, video surveillance, medical imaging, and other applications. Recently, example-based SR (also commonly referred to as “hallucination”) that reconstructs HR image from one single LR input image has emerged as a promising technology because it can overcome some limitations of the classical multi-image super-resolution methods and can be implemented with lower computation and memory costs.[0003]Example-based SR methods assume that the missing HR details can be learned and inferred from a representative training set or the LR image itself. For example, an image ...

AI Technical Summary

Benefits of technology

[0006]Embodiments of the present invention include apparatuses, systems and methods for multi-patch based super-resolution from a single image. As used herein, the term “image” means a visual representation of something obtained by capturing light or electromagnetic radiation corresponding to an object. The term image may include a still image or a frame of video. Such embodiments may include a scale-invariant self-similarity (SiSS) based super-resolution method. Instead of searching HR examples in a database or in LR image, the present embodiments may select the patches according to the SiSS characteristics of the patch itself, so that the computational complexity of the method may be reduced because there is not any search involved. To solve the problem of lack of relevant examples in natural images, the present embodiments may employ multi-shaped and multi-sized patches in HR image reconstruction. Additionally, embodiments may include steps for a hybrid weighing method for suppressing artifacts. Advantageously, certain embodiments of the method may be 10˜1,000 times faster than the example based SR approaches using patch searching and can achieve comparable HR image quality.

Problems solved by technology

These types of SR approaches are capable of producing plausible fine details across the image; however, a lack relevant examples in the database often causes noisy images and irregularities along curved edges.

Moreover, the use of larger databases is more time and memory consuming and effective hardware implementation may be a challenge.

Such example-based SR algorithms are often computationally intensive, because for each pixel or patch the methods require searching the high-resolution counterpart in a database / dictionary, an image pyramid or a small image region.

Although efforts have been made to reduce the computational complexity, it is still a major challenge to the commercial application of the SR technology.

Images