Chroma Block Prediction Using Attention Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards like H.265/HEVC and CCLM in H.266/VVC fail to effectively remove spatial redundancy between current and adjacent blocks, limiting prediction accuracy and coding/decoding efficiency due to the use of a single linear model for luma and chroma components.
Innovation Solution
Employ an attention-mechanism-based artificial neural network model (ANNM) for predicting color components, considering the relationship between the first and reference color components to enhance prediction accuracy and reduce bitrate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If Cross-component Linear Model (CCLM) is used to predict luma and chroma components, then redundancy between luma and chroma components is removed, but spatial redundancy between current block and adjacent block is not removed, limiting prediction accuracy
Solution Approach 1:
The patent introduces an attention mechanism as an intermediary between the luma and chroma components. This attention mechanism processes the relationship between the current chroma block and adjacent blocks, enabling the model to selectively focus on relevant spatial information while maintaining the cross-component prediction relationship. The attention mechanism acts as a mediator that enhances the interaction between different color components and spatial regions, thereby removing both inter-component and spatial redundancy while improving prediction accuracy.
2Ease of operation
If a single linear model is adopted to predict luma and chroma components, then the prediction process is simple, but prediction accuracy between different color components is limited
Solution Approach 1:
The patent transforms the static single linear model into a dynamic neural network model with multiple layers and adjustable parameters. The model dynamically adapts to different block characteristics through learned weights and attention mechanisms, allowing it to capture complex non-linear relationships between luma and chroma components. This dynamic approach maintains operational efficiency through automated learning while significantly improving prediction accuracy by modeling intricate inter-component relationships.
Solution Approach 2:
The patent combines multiple functional components into a composite prediction model: convolutional layers for feature extraction, attention mechanisms for selective weighting, and fully connected layers for final prediction. This composite structure integrates the strengths of different model types, creating a unified framework that is both computationally efficient and highly accurate in predicting chroma components from luma information.
3Adaptability or versatility
If existing video coding standards are used, then compatibility is maintained, but coding and decoding efficiency is reduced due to limitations in color component prediction
Solution Approach 1:
The patent designs a universal neural network-based prediction model that can handle multiple color component prediction scenarios (luma-to-chroma, chroma-to-luma, and inter-component prediction). The model serves multiple functions within a single framework, improving coding efficiency across different video content types while maintaining adaptability to various coding standards and requirements.
Data Source
AI summary
The application discloses a picture prediction method, an encoder, a decoder and a computer storage medium. The method includes determining a prediction mode parameter of a current block; determining a reconstructed value of a first colour component of the current block and a value of a reference colour component of the current block, in response to the prediction mode parameter indicating determining a prediction value of the current block by using an attention-based artificial neural network model (ANNM) mode; and predicting, according to the reconstructed value of the first colour component of the current block and the value of the reference colour component of the current block, a second colour component of the current block, to determine a prediction value of the second colour component of the current block.