Prediction Dependent Residual Scaling for Video Coding Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards, such as VVC, face challenges with increased computational complexity and on-chip memory requirements due to domain mappings in Luma Mapping with Chroma Scaling (LMCS), as well as latency issues from sequential applications of complex coding tools like DMVR, BDOF, and CIIP, which complicate chroma residual scaling.
Innovation Solution
The proposed Prediction Dependent Residual Scaling (PDRS) method scales prediction residuals directly without sample mapping, using luma prediction samples to derive scaling factors and maintain all decoding processes in the original domain, reducing the need for forward and inverse mapping operations and simplifying chroma residual scaling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If domain mappings are applied in LMCS for chroma residual scaling, then coding efficiency is improved, but computational complexity and on-chip memory requirements increase
Solution Approach 1:
The patent changes the parameter of scaling application from post-mapping domain to original domain. Instead of applying chroma residual scaling after luma mapping operations, the scaling is applied directly to chroma residuals in the original domain using scaling factors derived from luma prediction samples, thereby achieving coding efficiency without the computational overhead of multiple domain mappings
Solution Approach 2:
The patent extracts the chroma residual scaling operation from the post-mapping processing chain and performs it independently in the original domain. This separation eliminates the need for forward and inverse mapping operations that would otherwise be required, reducing computational complexity while maintaining the beneficial scaling effect
2Manufacturing precision
If forward and inverse mapping operations are performed in LMCS, then chroma residual scaling is achieved, but implementation complexity increases
Solution Approach 1:
The patent extracts the chroma residual scaling operation from the mapping-based processing chain and performs it directly in the original domain. By taking out the scaling operation and applying it before mapping, the patent eliminates the need for forward and inverse mapping operations, significantly reducing implementation complexity while preserving scaling accuracy
Solution Approach 2:
Instead of following the conventional approach of scaling after mapping (forward mapping → scaling → inverse mapping), the patent inverts the processing order by performing scaling in the original domain first, then applying mapping operations. This inversion eliminates redundant mapping operations and reduces implementation complexity
3Measurement precision
If complex coding tools like DMVR, BDOF, and CIIP are applied sequentially, then prediction accuracy is improved, but decoding latency increases
Solution Approach 1:
The patent performs chroma residual scaling as a preliminary action before applying complex coding tools like DMVR, BDOF, and CIIP. By scaling the chroma residuals early in the decoding process using readily available luma prediction samples, the patent enables parallel or pipelined processing of subsequent prediction refinement tools, thereby reducing overall decoding latency while maintaining prediction accuracy
Data Source
AI summary
Methods and devices are provided for reducing the decoding latency introduced by LMCS. In one method, one or more luma prediction sample values are selected from an output of a bilinear filter of Decoder-side Motion Vector Derivation (DMVR), the one or more selected luma prediction sample values are adjusted into luma prediction sample values with the same bit depth as an original coding bit depth of an input video, the luma prediction sample values with the same bit depth as the original coding bit depth of the input video are used to derive a scaling factor for decoding one or more chroma residual samples, the scaling factor is used to scale one or more chroma residual samples, and one or more chroma residual samples are reconstructed by adding the one or more scaled chroma residual samples and their corresponding chroma prediction samples.