Video Coding Dictionaries for Directional Residual Representation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video coding techniques, such as HEVC, face challenges in representing natural images and videos with directional features, leading to poor reconstruction and artifacts like ringing, due to their limited ability to capture directional orientations in residual signals.
Innovation Solution
The method involves training a set of overcomplete dictionaries using inter or intra predicted residual signals for improved representation, with a classifier determining the optimal number of dictionary elements through rate-distortion optimization, allowing for context-adaptive and dynamic dictionary updates during encoding and decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional video coding techniques (HEVC) are used, then the coding framework is simple and block-based prediction is efficient, but the representation of natural images with directional features is poor leading to artifacts like ringing
Solution Approach 1:
The patent transforms the residual signal from spatial domain to transform domain using learned transform matrices, changing the representation parameters to better capture directional features. This allows the system to maintain simple block-based processing while achieving superior reconstruction accuracy by adapting the transform parameters to the local characteristics of the residual signal
Solution Approach 2:
The patent combines multiple processing stages: block-based prediction, transform domain conversion using learned matrices, and sparse coding with overcomplete dictionaries. This composite approach integrates the simplicity of conventional coding frameworks with advanced directional feature representation, resolving the contradiction between framework simplicity and reconstruction accuracy
2Productivity
If overcomplete dictionaries with modulated Gabor functions are used, then coding gain is improved at low bit rates, but the representation of natural image features is insufficient introducing poor reconstruction
Solution Approach 1:
The patent employs dynamic dictionary selection and adaptation based on the characteristics of the transform domain residual signal. Instead of using fixed Gabor dictionaries, the system adapts the dictionary elements to match the directional orientations present in the actual residual signals, enabling both high coding gain and accurate reconstruction of natural image features
Solution Approach 2:
The patent performs preliminary transformation of the residual signal into the transform domain using learned matrices before applying sparse coding. This preliminary action prepares the signal in a form that better reveals directional features, allowing the subsequent dictionary-based coding to achieve both high compression efficiency and faithful reconstruction
3Adaptability or versatility
If curvelet transform is used, then multi-scale object representation is achieved, but the technique is non-adaptive leading to poor representation of directional orientations
Solution Approach 1:
The patent learns transform parameters (matrices) that adapt to the specific characteristics of the video content. This allows the system to maintain the multi-scale representation capability while adapting the transform parameters to accurately capture the directional orientations present in different regions and scales of the video signal
Solution Approach 2:
The system uses rate-distortion optimization as a feedback mechanism to select the optimal number of dictionary elements and transform parameters. This feedback loop ensures that the multi-scale representation is adapted to the actual directional features in the signal, achieving both versatility and precision
Data Source
AI summary
A video encoded as a bit stream is decoded by maintaining a set of dictionaries generated from decoded prediction residual signals, wherein elements of the set of dictionaries have associated indices. A current macroblock is entropy decoded and inverse quantized to produce decoded coefficients. For the current macroblock, a particular dictionary of the set of dictionaries is selected according to a prediction mode signaled in the bit stream, and particular elements of the particular dictionary are selected according to a copy mode signal in the bit stream and the associated index. The particular elements is scaled and combined, using the decoded coefficients, to reconstruct a current decoded macroblock prediction residual signal. Then, the current decoded macroblock prediction residual signal is combined with previously decoded macroblocks to generate an output macroblock of a reconstructed video, wherein the steps are performed in a decoder.