Structured Subspace Image Upsampling via Learned LR Dictionaries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for multiframe upsampling, such as super-resolution, face challenges in accurately estimating high-resolution images from low-resolution sequences due to the need for partial measurements and hardware modifications, particularly in dynamic scenes and non-optical imaging systems.
Innovation Solution
A structured subspace framework that learns pairs of low-resolution dictionaries to estimate high-resolution images directly, eliminating the requirement for partial measurements and hardware modifications, by representing high-resolution images in terms of specially structured basis matrices that span the polyphase components of low-resolution dictionaries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If motion estimation is used for multiframe upsampling, then high-resolution images can be estimated from low-resolution sequences, but accurate modeling of complex motion patterns requires high pixel density which creates a paradox
Solution Approach 1:
The patent replaces the motion estimation approach (which relies on sufficient pixel density for accurate motion modeling) with a signal representation approach using learned dictionaries. Instead of estimating motion patterns from pixel data, the method learns low-resolution dictionaries from the input sequence and uses them to directly represent and reconstruct high-resolution images, bypassing the need for high pixel density in motion modeling
Solution Approach 2:
The patent changes the fundamental parameters of the approach by transitioning from motion-based parameters (pixel density, motion vectors) to representation-based parameters (dictionary atoms, sparse coefficients). This parameter change allows the system to achieve accurate upsampling without being constrained by pixel density requirements for motion estimation
2Measurement precision
If specialized training sets are used to learn efficient dictionaries for specific image types, then dictionary efficiency is improved, but the learned dictionaries become useless for estimating other image types
Solution Approach 1:
The patent creates dictionaries that serve multiple functions and image types. By learning dictionaries from the actual input sequence rather than from specialized training sets, the resulting dictionaries are adapted to the specific content while still being able to represent the high-resolution images effectively. The dictionaries become universal for the given sequence rather than specialized for generic or specific image types
Solution Approach 2:
The patent performs preliminary learning of low-resolution dictionaries directly from the input low-resolution sequence before the upsampling process. This preliminary action creates dictionaries that are specifically adapted to the input data characteristics, ensuring both efficiency for the given sequence and versatility for the specific application at hand
3Measurement precision
If hardware modifications are made to obtain partial measurements for upsampling, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent replaces hardware-based solutions (additional sensors, optical modifications) with a computational approach using learned dictionaries. Instead of modifying the imaging hardware to capture partial measurements, the method uses signal processing and dictionary learning to reconstruct high-resolution images from the available low-resolution data, eliminating the need for hardware complexity increases
Solution Approach 2:
The patent introduces learned low-resolution dictionaries as an intermediary between the input low-resolution sequence and the output high-resolution images. These dictionaries serve as a computational mediator that enables accurate upsampling without requiring physical hardware modifications or partial measurements from additional sensors
Data Source
AI summary
A computational method is disclosed for producing a sequence of high-resolution (HR) images from an input sequence of low-resolution (LR) images. The method uses a structured subspace framework to learn pairs of LR dictionaries from the input LR sequence ‘and’ employ learned pairs of LR dictionaries into estimating HR images. The structured subspace framework itself is based on a pair of specially structured HR basis matrices, wherein a HR basis spans any HR image whose so-called polyphase components (PPCs) are spanned by the corresponding LR dictionary. The computational method may be used to denoise images, whether LR or HR images, by using the structured subspace framework to learn dictionaries of the images and estimate a denoised version of the images from the learned image dictionaries. The denoising process may be iterated until the noise is reduced below a desired threshold.