Guided Image Upsampling via Vector Contour Scaling

Solution Overview

Problem

Existing image upsampling methods often result in artifacts such as blurring, aliasing, and ringing, especially along sharp edges, due to the use of interpolation filters like bilinear and bicubic filters, which fail to effectively increase image resolution while preserving edge details.

Innovation Solution

The method involves generating vector contours from an initial image, scaling them to a higher resolution, and using a guiding image for joint upsampling to interpolate the image, which avoids power-of-two restrictions on scaling factors and maintains quality without loss, employing techniques like bitmap tracing and low discrepancy sequence sampling.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If interpolation filters (bilinear, bicubic) are used for image upsampling, then image resolution is increased, but artifacts such as blurring, aliasing, and ringing appear especially along sharp edges

Engineering Contradiction:

Improveimage resolutionVSAvoidartifacts (blurring, aliasing, ringing)

Core Design Contradiction:

Manufacturing precisionVSObject-affected harmful factors

Solution Approach 1:

The patent replaces traditional interpolation filter methods (mechanical filtering approach) with a learning-based neural network system. The neural network learns optimal upsampling operations from training data, substituting the deterministic mathematical filtering mechanism with a data-driven intelligent system that can adapt to different image content and preserve edges while reducing artifacts

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces a guiding image as an intermediary element in the upsampling process. This guiding image, generated at high resolution from the low-resolution input, serves as a reference to guide the upsampling of the original image, enabling the system to transfer edge information and structural details from the guiding image to the upsampled result, thereby reducing artifacts while maintaining resolution improvement

Inventive Principle:

Principle #24Intermediary (Mediator)

2Manufacturing precision

If traditional upsampling methods are used, then image resolution increases, but edge details are not preserved

Engineering Contradiction:

Improveimage resolutionVSAvoidedge detail preservation

Core Design Contradiction:

Manufacturing precisionVSMeasurement precision

Solution Approach 1:

The patent replaces traditional interpolation filter methods (mechanical filtering approach) with a learning-based neural network system. The neural network learns optimal upsampling operations from training data, substituting the deterministic mathematical filtering mechanism with a data-driven intelligent system that can adapt to different image content and preserve edges while reducing artifacts

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces a guiding image as an intermediary element in the upsampling process. This guiding image, generated at high resolution from the low-resolution input, serves as a reference to guide the upsampling of the original image, enabling the system to transfer edge information and structural details from the guiding image to the upsampled result, thereby reducing artifacts while maintaining resolution improvement

Inventive Principle:

Principle #24Intermediary (Mediator)

3Manufacturing precision

If learning-based upsampling is used, then image quality improves, but computational complexity and training requirements increase

Engineering Contradiction:

Improveimage qualityVSAvoidcomputational complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent segments the upsampling task into two distinct phases: (1) training phase where the neural network learns from paired low-resolution and high-resolution images, and (2) inference phase where the trained model performs upsampling. This segmentation allows the complex learning process to be separated from the actual upsampling operation, reducing the computational burden during image processing while maintaining high image quality

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary training of the neural network model using large datasets of image pairs before deployment. This preliminary action pre-computes the optimal upsampling transformations, so that during actual use, the system only needs to apply the learned model rather than performing complex computations in real-time, thereby reducing operational computational complexity while maintaining high image quality

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10402941B2Guided image upsampling using bitmap tracing

Publication Date: 2019.09.03 AT&T INTELLECTUAL PROPERTY I L P

AI summary

A method of increasing resolution of an image includes generating vector contours associated with a first image and scaling the vector contours to a second resolution. The first image has a first resolution, and the second resolution is greater than the first resolution. The vector contours are rendered to generate a guiding image at the second resolution, and a second image is generated from the first image based on the guiding image, by using joint upsampling. The second image is generated at the second resolution. The vector contours can be generated by bitmap tracing binary images at different quantization levels. An apparatus and computer readable device implementing the method of increasing the resolution of an image are also provided.