VVC Intra CU Partitioning Using Multi-Task Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high computational complexity of H.266/VVC-based intra coding for screen content videos poses a challenge, as it significantly increases processing demands while maintaining coding performance.
Innovation Solution
A fast H.266/VVC-based intra coding unit (CU) partitioning method using multi-task learning, which involves a multi-task learning network model to predict CU partitioning types and coding modes, reducing unnecessary calculations and complexity without impacting coding efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If H.266/VVC standard encoder executes all 5,781 probabilities and evaluates all 6 CU partitioning choices, then coding performance is maintained, but computational complexity increases hugely
Solution Approach 1:
The patent applies preliminary action by using a trained neural network model to predict the optimal CU partitioning type and coding mode before the standard encoder executes full probability calculations. The network performs inference in advance to guide the encoding process, eliminating the need to evaluate all 5,781 probabilities and all 6 partitioning choices, thus reducing computational complexity while maintaining coding performance
Solution Approach 2:
The patent employs a lightweight neural network model that can be deployed on encoding devices to perform rapid predictions. This disposable prediction mechanism replaces the computationally expensive standard encoder evaluation process, providing accurate partitioning decisions without requiring the full computational resources of the standard H.266/VVC encoder
2Reliability
If H.266/VVC introduces IBC and PLT coding modes with flexible CU partitioning, then coding performance is significantly improved, but computational complexity increases hugely
Solution Approach 1:
The patent implements a multi-task learning network model that simultaneously predicts both the CU partitioning type and the coding mode (Intra, IBC, or PLT). This universal model handles multiple coding tasks in a single inference process, eliminating the need for separate evaluations of partitioning methods and coding mode selections, thus reducing computational complexity while achieving the coding performance improvements from flexible partitioning and multiple coding modes
Solution Approach 2:
The trained neural network performs preliminary prediction of both partitioning type and coding mode before the standard encoder processes the data. This advance prediction guides the encoding process to only execute necessary calculations for the predicted configuration, avoiding the computational burden of evaluating all partitioning and coding mode combinations
Data Source
AI summary
An H.266/VVC-based intra coding unit partitioning method for screen content based on multi-task learning and a device, the method includes: partitioning a 128×128 coding tree unit into 64×64 coding units, a multi-task learning network model comprises a trunk network configured to extract CU features, a first sub-network, and a second sub-network, inputting the CU features into the first sub-network and the second sub-network to predict a CU partitioning type and a coding mode, determining the predicted result in combination with the coding mode, a corresponding predicted probability of the coding mode, and a partitioning type of an adjacent CU, inputting the 64×64 CUs into the model to obtain a first predicted result, partitioning each of the 64×64 CUs into four 32×32 CUs in response to determining that the first predicted result is partition, inputting the four 32×32 CUs into the model to obtain a second predicted result.