VVC Residual Sign Prediction Across Expanded Transform Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards, such as VVC, do not effectively predict and encode residual coefficient signs, leading to a significant portion of the bitstream being consumed by transmitting these signs, which hinders further bit rate gains and compression efficiency.
Innovation Solution
Implement a residual sign prediction method using gradient-based border residual sign prediction, multi-modal residual border prediction, and two-step residual sign prediction, allowing for prediction over an expanded region and increased number of signs, and utilizing a sorting order and extended border for improved accuracy and reduced computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If residual coefficient signs are transmitted as sign bits without prediction, then transmission is simple and straightforward, but the bitstream consumes a substantial portion of bits for these signs, reducing compression efficiency
Solution Approach 1:
The patent applies preliminary action by predicting residual coefficient signs before transmission. The encoder predicts signs of residual coefficients using neighboring coefficient signs and gradient information, and the decoder performs the same prediction to reconstruct signs without transmitting them explicitly. This preliminary prediction step eliminates the need to transmit sign bits for many coefficients, particularly those near block boundaries, thereby improving compression efficiency while maintaining decoding simplicity.
2Productivity
If residual coefficient signs are subject to prediction and coding, then bit rate gains are achieved, but prediction accuracy may be lost, offsetting the bit rate gains
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting prediction parameters based on local image characteristics. Specifically, the method calculates gradient values from neighboring pixels and uses these gradients to determine prediction modes and weighting factors. The prediction process adapts to different regions by changing parameters such as the number of neighboring samples used, the prediction mode selected, and the weighting applied to different neighbors. This adaptive parameter adjustment maintains high prediction accuracy across diverse image content while achieving significant bit rate gains.
Solution Approach 2:
The patent implements feedback mechanisms where the encoder evaluates prediction accuracy and adjusts prediction parameters accordingly. The decoder performs the same prediction using transmitted parameter information, and any prediction errors are compensated through residual coding. This feedback loop ensures that prediction accuracy is maintained while maximizing bit rate savings, as the system learns from actual reconstruction errors and refines its prediction strategy.
3Productivity
If prediction is performed over an expanded region with increased number of signs, then more bits are saved in transmission, but computational complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the prediction process into distinct stages and regions. The method segments the transform block into different zones (e.g., boundary regions vs. interior regions) and applies different prediction strategies to each segment. Boundary regions with high correlation to neighbors use aggressive prediction to save bits, while interior regions use simpler prediction or direct coding. This segmentation allows the system to achieve high overall bit savings while controlling computational complexity by applying complex prediction only where most beneficial.
Solution Approach 2:
The patent applies partial action by selectively predicting only those signs that will yield the greatest bit savings. Rather than attempting to predict all residual coefficient signs uniformly, the method identifies and prioritizes prediction for coefficients near block boundaries and in regions with high spatial correlation. This selective approach achieves significant bit rate gains while avoiding the computational overhead of predicting every single sign, thus balancing complexity and performance.
Data Source
AI summary
A VVC-standard encoder and a VVC-standard decoder are provided, implementing a residual sign prediction method utilizing a sorting order of residual coefficients and an expanded region of a TB. As sign prediction accuracy is higher for larger transform coefficient levels, a VVC-standard encoder and a VVC-standard decoder sort transform coefficient signs of a TB in a one-dimensional array, based on corresponding QIdx values instead of the residual coefficient level value. The first n signs according to corresponding QIdx values, ordered from largest to smallest, are predicted using a residual sign prediction method, and the rest of the signs are signaled by EP bins. A sign prediction area is also extended, without limitation to an upper-left 4×4 region within the transform block, but to a region up to 32×32 in size; a VVC-standard encoder signals the maximum dimensions of the region to a VVC-standard decoder in syntax structures of the block.