CIIP Mode Constrained Application for Video Encoding Complexity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards, such as the now-current VVC, face challenges in computational complexity and encoding/decoding throughput due to the increased operations involved in the Combined Inter-and Intra-Prediction (CIIP) mode, particularly when using new inter-mode coding tools like Bi-Directional Optical Flow (BDOF) and Decoder-side Motion Vector Refinement (DMVR). Additionally, the inconsistent treatment of intra mode blocks and CIIP mode blocks in constructing the Most Probable Mode (MPM) candidate list affects coding efficiency.
Innovation Solution
The proposed solution involves constrained and adjusted applications of the CIIP mode. This includes determining whether a CU is bi-predicted or uni-predicted to constrain CIIP mode application, disabling certain inter-prediction techniques like BDOF and DMVR when CIIP is applied to bi-predicted CUs, and employing pre-defined criteria to select uni-prediction samples for combination with intra-prediction samples. Furthermore, the signaling of the CIIP flag is dependent on the prediction direction, and a unified criterion is used when forming the MPM candidate list for neighboring CUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Combined Inter-and Intra-Prediction (CIIP) mode is applied with BDOF and DMVR operations, then prediction accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent applies BDOF and DMVR operations selectively rather than universally. A determination mechanism checks whether to apply these operations based on the CIIP flag and other conditions, performing them only when beneficial. This partial application maintains prediction accuracy where needed while reducing overall computational complexity by skipping unnecessary operations in other blocks.
2Measurement precision
If CIIP mode is applied with BDOF and DMVR operations, then prediction accuracy is improved, but encoding/decoding throughput decreases
Solution Approach 1:
The patent implements conditional application of BDOF and DMVR operations through determination mechanisms that evaluate the CIIP flag and other parameters. By performing these computationally intensive operations only when determined necessary, the patent maintains high prediction accuracy for qualifying blocks while preserving encoding/decoding throughput by avoiding unnecessary processing in other blocks.
Solution Approach 2:
The patent introduces dynamic control mechanisms that adjust the application of BDOF and DMVR operations based on real-time conditions including the CIIP flag state and block characteristics. This dynamic approach allows the system to adaptively optimize between accuracy and throughput, switching between full operation, partial operation, or no operation based on current coding conditions.
3Adaptability or versatility
If inconsistent treatment of intra mode blocks and CIIP mode blocks is used in constructing MPM candidate list, then flexibility is maintained, but coding efficiency decreases
Solution Approach 1:
The patent establishes unified treatment rules for both intra mode blocks and CIIP mode blocks when constructing the MPM candidate list. Instead of maintaining separate inconsistent handling procedures, the patent applies homogeneous criteria to both block types, improving coding efficiency through consistent processing while maintaining sufficient flexibility to handle different block characteristics through the unified framework.
Data Source
AI summary
A method of video encoding is provided. The method includes obtaining, by an encoder, a first reference picture and a second reference picture associated with a current block, wherein the first reference picture is before a current picture and the second reference picture is after the current picture in display order. The method further includes determining, by the encoder, whether to apply a decoder-side motion vector refinement (DMVR) operation based on a combined inter and intra prediction (CIIP) flag for the current block. The method further includes determining, by the encoder, whether to apply a bi-directional optical flow (BDOF) operation based on the CIIP flag. The method further includes calculating, by the encoder, a prediction of the current block based on the determinations as to whether to apply the DMVR and BDOF operations.