Transform Coefficient Sub-sampling for Video Encoding Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High computational complexity in video encoding due to large numbers of transform coefficients, which hinders video encoding throughput in demanding applications like real-time video and multi-person gaming.
Innovation Solution
Sub-sampling and up-sampling of transform coefficients to reduce the number of coefficients that need to be quantized, implemented in the High Efficiency Video Coding (HEVC) standard, using sub-sample control parameters to determine which coefficients to remove, thereby decreasing computational burden.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all transform coefficients are quantized, then encoding accuracy is maintained, but computational complexity increases
Solution Approach 1:
The patent extracts and removes a subset of transform coefficients from the quantization process. Specifically, it identifies and removes coefficients below a certain magnitude threshold (e.g., |coeff| < threshold) or coefficients in less significant frequency bands, quantizing only the remaining coefficients. This extraction principle directly reduces the number of coefficients requiring quantization operations, thereby lowering computational complexity while preserving encoding accuracy for the most significant coefficients.
Solution Approach 2:
The patent applies different quantization treatments to different regions of the transform coefficient matrix. Instead of uniformly quantizing all coefficients, it applies selective quantization based on local properties such as coefficient magnitude, frequency position, or spatial location. For example, coefficients in high-frequency regions or with low magnitudes may be discarded or coarsely quantized, while coefficients in low-frequency regions or with high magnitudes are finely quantized. This local differentiation optimizes the balance between accuracy and complexity.
2Manufacturing precision
If all transform coefficients are quantized, then encoding quality is maintained, but encoding throughput decreases
Solution Approach 1:
The patent extracts and removes a subset of transform coefficients from the quantization process. Specifically, it identifies and removes coefficients below a certain magnitude threshold (e.g., |coeff| < threshold) or coefficients in less significant frequency bands, quantizing only the remaining coefficients. This extraction principle directly reduces the number of coefficients requiring quantization operations, thereby lowering computational complexity while preserving encoding quality for the most significant coefficients.
Solution Approach 2:
The patent applies partial quantization by processing only a subset of coefficients rather than all coefficients. It performs quantization on the most significant coefficients (e.g., low-frequency, high-magnitude coefficients) while omitting or coarsely processing less significant ones. This partial action approach achieves acceptable encoding quality by focusing computational resources on the coefficients that contribute most to visual quality, thereby improving encoding throughput.
3Device complexity
If the number of transform coefficients is reduced, then computational complexity decreases, but information loss increases
Solution Approach 1:
The patent applies different quantization treatments to different regions of the transform coefficient matrix. Instead of uniformly quantizing all coefficients, it applies selective quantization based on local properties such as coefficient magnitude, frequency position, or spatial location. For example, coefficients in high-frequency regions or with low magnitudes may be discarded or coarsely quantized, while coefficients in low-frequency regions or with high magnitudes are finely quantized. This local differentiation optimizes the balance between accuracy and complexity.
Solution Approach 2:
The patent changes the quantization parameters dynamically based on the characteristics of the transform coefficients. It adjusts quantization thresholds, step sizes, or selection criteria according to factors such as coefficient magnitude distribution, frequency content, or scene complexity. By adapting quantization parameters to the local characteristics of the coefficients, the patent minimizes information loss while maintaining reduced computational complexity.
Data Source
AI summary
Video compression encoding includes intra and inter prediction to reduce spatial and temporal redundancies in video. Prediction results or residuals represent differences between original video pixel values and predicted pixel values. The prediction residuals may be transformed into coefficients, referred to as transform coefficients, in the frequency domain. The transform coefficients may be quantized and entropy encoded. The transform coefficients can be sub-sampled prior to quantization to reduce their number. For example, sub-sampling may reduce more high frequency components than low frequency components represented in the transform coefficients. Therefore, sub-sampling reduces the number of transform coefficients that need to be quantized, reduces quantization complexity, and correspondingly increases throughput in the encoding.