Video Encoding Transform Skip Mode Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video compression technologies, such as AVC/H.264, face inefficiencies in transforming and representing video data due to the statistical properties of residuals, which affect the energy compaction performance of transforms like DCT, especially when signal correlation is weak, leading to suboptimal compression and increased data requirements.
Innovation Solution
The introduction of Transform Skip Mode (TSM) allows for adaptive selection of transform modes, including 2D, 1D, and no transforms, with signaling of these modes to the decoder, and adaptive scanning strategies to optimize entropy coding, enabling efficient data representation by skipping transforms when beneficial and using scaling to maintain signal levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If DCT transform is used for video compression, then energy compaction performance is improved for highly correlated signals, but compression efficiency deteriorates when signal correlation is weak
Solution Approach 1:
The patent implements dynamic transform selection where the encoder adaptively chooses between DCT and DST transforms based on the statistical properties of residual signals. This is achieved through transform skip mode flags that enable conditional application of transforms, allowing the system to optimize for weakly correlated signals by selecting DST when DCT performance degrades, thereby resolving the contradiction between energy compaction and compression efficiency across varying signal conditions
Solution Approach 2:
The patent changes the transform parameter dynamically by selecting different transform types (DCT vs DST) and transform skip modes based on residual signal characteristics. The encoder analyzes signal correlation properties and adjusts the transform application accordingly, using mode flags to signal the selected transform type to the decoder, thus adapting the energy compaction strategy to match actual signal conditions rather than applying a fixed transform
2Loss of energy
If transform modes are fixed to maximize energy compaction, then energy compaction performance is improved, but adaptability to different signal statistics deteriorates
Solution Approach 1:
The system transitions from fixed transform modes to dynamic transform selection by introducing transform skip mode flags that enable real-time adaptation. The encoder evaluates residual signal statistics and dynamically selects between DCT, DST, or skip modes, with the corresponding mode flags signaling the choice to the decoder. This dynamic approach maintains optimal energy compaction for each signal type while adapting to varying statistical properties
Solution Approach 2:
The patent creates a universal transform framework that can handle multiple signal types and correlation characteristics through a single unified system. By incorporating both DCT and DST transforms with conditional skip modes, the system becomes multi-functional, capable of optimizing for highly correlated signals (DCT), weakly correlated signals (DST), or cases where transform application is unnecessary (skip mode), all within one adaptive framework
3Productivity
If transforms are applied to all blocks, then compression performance is optimized for correlated signals, but computational overhead increases and bit requirements increase for low correlation signals
Solution Approach 1:
The patent extracts the transform operation from being a mandatory universal step and makes it conditional. By introducing transform skip mode flags, the system selectively applies transforms only when beneficial, extracting unnecessary transform operations from the processing pipeline for low-correlation signals. This reduces computational overhead and bit requirements for blocks where transforms would not improve compression performance
Solution Approach 2:
Instead of applying transforms to all blocks (excessive action), the patent implements partial transform application based on signal characteristics. The encoder performs partial analysis of residual correlation and applies transforms only to blocks that benefit from them, skipping transforms for blocks where the computational cost outweighs the compression benefit, thus optimizing the balance between compression performance and computational overhead
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
Video encoding or decoding utilising a spatial transform operating on rows and columns of a block, with a set of transform skip modes including: transform on rows and columns; transform on rows only;transform on columns only;no transform. An indication of the selected mode is provided to the decoder. Coefficients are scaled by a factor dependent upon the norm of the transform vector of the skipped transform to bring the untransformed image values to the same level as transformed coefficients.