Super-Resolution Image Upscaling via Patch Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image super-resolution techniques face limitations in accurately increasing resolution, especially with single low-resolution images, due to challenges in estimating the Point Spread Function and registration, leading to suboptimal results and high computational complexity, and often produce unrealistic edges or over-smoothing.
Innovation Solution
A method involving three stages: interpolation-based upscaling, low-pass filtering, and a search for low-frequency matches between patches in the high-resolution and low-resolution images to accumulate high-frequency contributions, allowing for flexible upscaling factors and reduced noise introduction, without requiring a database or extensive training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If reconstruction-based super-resolution algorithms are used to increase resolution, then resolution can be improved, but the maximal increase is limited to around 1.6× under natural conditions
Solution Approach 1:
The patent introduces an intermediary database of high-resolution image patches that serves as a mediator between the low-resolution input and the super-resolved output. Instead of directly reconstructing high-resolution details from low-resolution data (which is limited to 1.6×), the method uses pre-collected high-resolution patches from the database as reference material to guide the enhancement process, enabling much higher resolution increases while maintaining image quality
Solution Approach 2:
The patent performs preliminary action by pre-collecting and storing high-resolution image patches in a database before the actual super-resolution process. These pre-prepared high-frequency details are then retrieved and integrated during enhancement, allowing the system to overcome the fundamental reconstruction limit and achieve higher resolution increases
2Manufacturing precision
If example-based super-resolution with a large database is used to improve resolution, then image quality can be enhanced, but computational cost increases excessively
Solution Approach 1:
The patent applies local quality by focusing the computational effort on local patch matching rather than processing entire images. The method divides the image into small patches (e.g., 8×8 or 16×16 pixels) and performs database searching and matching operations only on these local regions. This localized approach maintains high image quality through detailed patch-level processing while dramatically reducing computational cost compared to global processing methods
Solution Approach 2:
The patent segments the image processing task into smaller, manageable patches. By dividing the full-resolution image into multiple small patches and processing them independently with database matching, the method reduces the computational burden of each individual operation while maintaining overall image quality through the aggregation of locally optimized results
3Reliability
If classical optical flow estimation techniques are used for registration, then the method works well in quasi-synthetic examples, but it fails to robustly register consecutive frames in video sequences with general motion
Solution Approach 1:
The patent uses patch copying from the database instead of relying on optical flow estimation. High-resolution patches are copied directly from the pre-collected database based on similarity matching with corresponding low-resolution patches from the input image. This copying approach bypasses the registration problem entirely, as it does not require estimating motion between frames, and works robustly for any type of motion including general video sequences
4Device complexity
If single-image super-resolution methods are used to avoid database requirements, then computational complexity is reduced, but the methods still require extensive training data or produce unrealistic edges
Solution Approach 1:
The patent uses a database of high-resolution patches that can be considered as disposable reference material. Instead of requiring complex training processes or extensive training data sets that need to be repeatedly processed, the method uses pre-collected patches from the database as single-use references for each super-resolution operation. These patches are retrieved and used once for matching, then discarded, enabling the method to produce realistic edges without requiring complex training procedures or extensive training data
Data Source
AI summary
The invention relates to the improvement of the resolution of regularly sampled multi-dimensional signals, where a single low-resolution signal is available. These methods are generically referred to as example-based super-resolution or single-image super-resolution. The method for super-resolving a single image comprises three stages. First, an interpolation-based up-scaling of the input image is performed, followed by an equivalent low-pass filtering operation on the low-resolution image. The second stage comprises a search for low-frequency matches between an inspected patch in the high-resolution image and patches in a local neighborhood in the low-resolution low-frequency image, including partly overlapping patches, and accumulating the high-frequency contribution obtained from the low-resolution image. The third stage comprises adding the contributions of the low-frequency band of the high-resolution image and the extrapolated high-frequency band.