Video Coefficient Quantization Using Scalar and Trellis Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional scalar quantization methods independently process transform coefficients, reducing the effectiveness of subsequent entropy coding and overall encoding efficiency due to a lack of consideration for correlations between coefficients, leading to suboptimal compression during entropy coding.
Innovation Solution
Implementing trellis coded quantization (TCQ) that generates and evaluates different permutations of quantization indices based on a cost function balancing distortion and entropy coding efficiency, allowing for improved compression during entropy coding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If scalar quantization is applied independently to each transform coefficient, then the quantization process is simple and computationally efficient, but the effectiveness of subsequent entropy coding is substantially reduced and overall encoding efficiency decreases
Solution Approach 1:
The patent merges scalar quantization with trellis coded quantization by performing both operations on transform coefficients. The TCQ component considers correlations between adjacent coefficients and uses a trellis structure to generate quantization indices that exploit these correlations, while the scalar quantization component provides a baseline quantization. This combination allows the system to capture inter-coefficient relationships for improved entropy coding efficiency while maintaining the computational simplicity of scalar quantization through the separate processing paths.
2Loss of substance
If scalar quantization is used to reduce precision of transform coefficients, then data compression is achieved, but correlations between transform coefficients are not accounted for and opportunities for increased compression during entropy coding are lost
Solution Approach 1:
The patent introduces trellis coded quantization as an intermediary layer between the transform coefficients and entropy coding. The TCQ process analyzes correlations between adjacent transform coefficients and uses a trellis structure to generate quantization indices that preserve these correlation relationships. This intermediary processing step ensures that the quantization indices maintained correlation information, which then enables more effective entropy coding by exploiting the remaining correlations in the quantized data.
3Measurement precision
If different transform coefficients are mapped to different quantization indices during scalar quantization, then individual coefficient precision is maintained, but the values are not effectively compressed during entropy coding and overall encoding efficiency is reduced
Solution Approach 1:
The patent applies dynamic quantization strategies by using trellis coded quantization to adaptively determine quantization indices based on the relationships between adjacent coefficients. The TCQ trellis structure dynamically selects quantization paths that optimize both individual coefficient representation and inter-coefficient correlations. This dynamic approach allows the system to maintain individual coefficient precision where needed while simultaneously creating patterns in the quantization indices that enhance entropy coding compression efficiency.
Data Source
AI summary
In various embodiments, an encoder receives transform coefficients of prediction residues associated with video data. The encoder performs scalar quantization operations on the transform coefficients to generate first indices. The encoder performs trellis coded quantization operations on a first vector that includes the transform coefficients to generate a second vector that includes second indices. The encoder determines a first cost function value based on a third vector that includes the first indices and a second cost function value based on the second vector. The encoder sets a fourth vector that includes a plurality of quantization indices equal to either the second vector or the third vector based on whether the second cost function value is less than the first cost function value. The encoder performs entropy coding operations on the fourth vector to generate encoded video data.