Block-level super-resolution video coding

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current video coding standards face inefficiencies in compressing high-resolution video frames, particularly in situations with low spatial correlation between pixels, such as 360 video or natural scenes with large texture, leading to suboptimal compression efficiency for I frames which can represent a significant portion of the bitstream.

Innovation Solution

The implementation of a super-resolution technique using a polyphase downsampling scheme during in-loop coding, where video blocks are downsampled to lower resolutions, allowing for more accurate predictions and efficient encoding by reconstructing high-resolution signals from multiple low-resolution components.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional intra-prediction is used for high-resolution video frames with low spatial correlation, then the encoding process is simple, but compression efficiency deteriorates

Engineering Contradiction:
Improvecompression efficiencyVSAvoidencoding complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent divides the high-resolution video frame into multiple lower-resolution blocks through downsampling, processes each block independently using conventional intra-prediction, and then reconstructs the high-resolution frame. This segmentation allows simple prediction methods to achieve high compression efficiency on complex content by breaking down the difficult task into manageable smaller units.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent transforms the problem from high-resolution direct encoding to low-resolution encoding by changing the resolution dimension. By downsampling the high-resolution frame to lower-resolution blocks, the system operates in a different dimensional space where conventional prediction methods are more effective, then maps back to the original high-resolution space through upsampling and residual coding.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Measurement precision

If video blocks are processed at high resolution, then prediction accuracy is high, but bitstream size increases

Engineering Contradiction:
Improveprediction accuracyVSAvoidbitstream size
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent segments the high-resolution video block into multiple lower-resolution blocks, processes each with reduced detail, and combines the results. This segmentation reduces the amount of data that needs to be transmitted while maintaining prediction accuracy through the super-resolution reconstruction process that synthesizes high-resolution predictions from multiple low-resolution components.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent discards some high-frequency detail information during downsampling to low-resolution blocks, processes these simplified representations, then recovers the high-resolution prediction through super-resolution techniques. The residual coding captures and transmits only the essential high-frequency information that was discarded, achieving compact bitstream representation.

Inventive Principle:
Principle #34Discarding and recovering

Data Source

PatentUS11438610B2Block-level super-resolution based video coding
Publication Date: 2022.09.06 KONINK KPN NV
  • US11438610B2 patent drawing
  • US11438610B2 patent drawing
  • US11438610B2 patent drawing

AI summary

A method of encoding video data into a bitstream comprising: determining from the plurality of video blocks of a video frame of a first resolution a current block and one or more reference blocks including reference samples for predicting original samples of the current block; determining predicted samples of the current block based on a downsampling scheme, the determining comprising computing samples of a first low-resolution block of a second resolution that is lower than the first resolution, and predicting samples of one or more second low-resolution blocks of the second resolution based on the samples of the first low resolution block, the samples of the first low-resolution block and the one or more second low-resolution blocks defining predicted samples of the current Block; and, determining residual samples of a residual block based on original samples of the current block and the predicted samples of the current block; transforming the samples of the residual block into a bitstream; and, embedding metadata associated with the current block in the bitstream for signalling a decoder apparatus that the predicted samples of the current block are based on the downsampling scheme.