Super-Resolution Texture Swapping via Feature Map Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing super-resolution methods blur high-frequency texture features and introduce erroneous texture features by directly transferring pixel data from images with similar or same content, leading to a lack of perceptually sharp and realistic texture details in high-resolution images.
Innovation Solution
The method involves extracting content and texture features from low-resolution and reference images, generating texture feature maps, and swapping patches based on similarity scores to create a swapped texture map, which is then combined with the content feature map using a neural network trained for texture consistency, allowing for the generation of high-resolution images without the need for images with similar content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If super-resolution methods directly transfer pixel data from images with similar or same content, then the process is simple and fast, but erroneous texture features are introduced and texture consistency is lost
Solution Approach 1:
The patent segments the image processing into distinct feature maps: content feature maps (low-frequency) and texture feature maps (high-frequency). This segmentation allows independent processing of structural content and texture details, enabling accurate texture transfer without content constraints while maintaining processing efficiency.
Solution Approach 2:
The patent introduces texture feature maps as an intermediary representation between the source and target images. By operating in the texture feature map domain rather than directly transferring pixel data, the method achieves both efficiency and accuracy in texture feature transfer.
2Stability of the object's composition
If super-resolution methods use images constrained to have similar or same content, then content consistency is maintained, but high-frequency texture details are lost due to lack of detailed texture variation
Solution Approach 1:
The patent separates content features (low-frequency) from texture features (high-frequency) into different feature maps. This allows the content feature map to maintain structural consistency while the texture feature map captures detailed texture variations from reference images, resolving the contradiction between content stability and texture precision.
Solution Approach 2:
The patent applies different processing qualities to different parts of the image: content features are processed to maintain overall structural consistency, while texture features are processed to capture fine-grained local details. This local differentiation of quality requirements enables both content consistency and texture sharpness.
3Adaptability or versatility
If super-resolution methods transfer texture features from reference images without content constraints, then diverse texture options are available, but texture features may not match the target image content
Solution Approach 1:
The patent uses texture feature maps extracted from reference images as an intermediary representation. These feature maps capture texture characteristics in a content-agnostic manner, allowing diverse texture options to be transferred and adapted to the target image content through feature map operations rather than direct pixel transfer.
Solution Approach 2:
The patent transforms images into the feature map domain, changing the representation parameters from pixel values to texture features. This parameter transformation enables versatile texture feature selection from reference images while maintaining accurate matching with target content through feature space operations.
Data Source
AI summary
In implementations of super-resolution with reference images, a super-resolution image is generated based on reference images. Reference images are not constrained to have same or similar content as a low-resolution image being super-resolved. Texture features indicating high-frequency content are extracted into texture feature maps, and patches of texture feature maps of reference images are determined based on texture feature similarity. A content feature map indicating low-frequency content of an image is adaptively fused with a swapped texture feature map including patches of reference images with a neural network based on similarity of texture features. A user interfaces allows a user to select regions of multiple reference images to use for super-resolution. Hence, a super-resolution image can be generated with rich texture details incorporated from multiple reference images, even in the absence of reference images having similar content to an image being upscaled.