Neural Network Image Enhancement via Channel Correlation

Solution Overview

Problem

Current video compression techniques face challenges in achieving high compression ratios without sacrificing picture quality, particularly in lossy compression methods which introduce visible spatial artifacts at low bit rates, and there is a need for improved methods to enhance image or video quality during encoding and decoding processes.

Innovation Solution

The method involves using a neural network system to modify images by selecting a primary channel and processing it separately from secondary channels, with the secondary channels being processed based on the modified primary channel, allowing for adaptive image enhancement and improved quality through collaborative processing of color and feature channels.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of substance

If lossy image compression is used to achieve high compression rates, then the quantity of data is reduced, but visible spatial compression artifacts are introduced

Engineering Contradiction:

Improvedata quantityVSAvoidimage quality

Core Design Contradiction:

Loss of substanceVSManufacturing precision

Solution Approach 1:

The patent converts the harmful compression artifacts into beneficial enhancement opportunities by using them as input for neural network-based image enhancement. The degraded compressed image is processed through neural networks that learn to reconstruct and enhance the original image quality, turning the lossy compression disadvantage into a chance for advanced post-processing improvement.

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Solution Approach 2:

The patent introduces neural network-based image enhancement as an intermediary process between lossy compression and final image output. This intermediary enhancement stage processes the compressed image to remove artifacts and improve quality before delivery, effectively mediating between the conflicting requirements of high compression and high quality.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Manufacturing precision

If lossless image compression is used to perfectly reconstruct the original image, then image quality is preserved, but compression rates remain low

Engineering Contradiction:

Improveimage qualityVSAvoiddata quantity

Core Design Contradiction:

Manufacturing precisionVSLoss of substance

Solution Approach 1:

The patent segments the image processing into two distinct stages: first applying lossy compression to reduce data quantity, then applying neural network enhancement to restore and improve quality. This segmentation allows each stage to optimize for its specific function rather than trying to achieve both compression and quality preservation in a single process.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes the processing parameters by transitioning from traditional compression algorithms to neural network-based enhancement. The neural networks are trained with specific parameters and architectures (such as convolutional layers, activation functions, and loss functions) that enable them to effectively reconstruct and enhance images from compressed data, achieving quality levels that exceed the original compression trade-off.

Inventive Principle:

Principle #35Parameter changes

3Quantity of substance

If traditional video compression techniques are used to reduce data size for transmission, then bandwidth requirements are reduced, but picture quality deteriorates at low bit rates

Engineering Contradiction:

Improvevideo data sizeVSAvoidpicture quality

Core Design Contradiction:

Quantity of substanceVSManufacturing precision

Solution Approach 1:

The patent introduces neural network-based image enhancement as an intermediary process between lossy compression and final image output. This intermediary enhancement stage processes the compressed image to remove artifacts and improve quality before delivery, effectively mediating between the conflicting requirements of high compression and high quality.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent converts the harmful compression artifacts into beneficial enhancement opportunities by using them as input for neural network-based image enhancement. The degraded compressed image is processed through neural networks that learn to reconstruct and enhance the original image quality, turning the lossy compression disadvantage into a chance for advanced post-processing improvement.

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Data Source

PatentUS12047613B2Adaptive image enhancement using inter-channel correlation information

Publication Date: 2024.07.23 HUAWEI TECH CO LTD

AI summary

The present disclosure relates to image modification such as an image enhancement. The image enhancement may be applied for any image modification and it may be applied during or after image encoding and/or decoding, e.g. as a loop filter or a post filter. In particular, the image modification includes a multi-channel processing in which a primary channel is processed separately and secondary channels are processed based on the processed primary channel. The processing is based on a neural network. In order to enhance the image modification performance, prior to applying the modification, the image channels are analyzed and a primary channel and the secondary channels are determined, which may vary for multiples of images, images or image areas.