Multiple-Model Illumination Compensation in Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding techniques introduce errors in prediction blocks due to varying illumination within a single block, leading to increased residual data transmission, as they typically use a single linear model for each prediction direction.
Innovation Solution
Deriving and applying multiple linear models for each prediction block to account for illumination changes between samples, allowing for more accurate prediction and reduced residual data transmission by classifying neighboring samples into groups and using specific linear models for each group.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single linear model is used for each prediction direction, then the device complexity is reduced, but the prediction accuracy deteriorates due to varying illumination within a block
Solution Approach 1:
The patent divides the current block into multiple sub-blocks, allowing different linear models to be applied to different regions. This segmentation enables the system to capture local illumination variations within each sub-block while maintaining manageable model complexity at each location.
Solution Approach 2:
The patent applies different linear model parameters (scaling factors and offsets) to different sub-blocks based on local illumination characteristics. Each sub-block receives customized correction parameters derived from its specific neighborhood, improving prediction accuracy for regions with varying illumination without requiring a single complex global model.
2Measurement precision
If multiple linear models are derived and applied for each prediction block, then the prediction accuracy is improved, but the device complexity increases
Solution Approach 1:
By segmenting the block into sub-blocks, the patent reduces the number of models needed compared to applying a single model to the entire block. Each sub-block gets its own model, which is computationally more feasible than deriving multiple overlapping models for the whole block while still capturing illumination variations.
Solution Approach 2:
The patent applies linear models only to specific sub-blocks where illumination variation is detected, rather than uniformly applying multiple models to every pixel. This partial action approach improves accuracy where needed while avoiding unnecessary computational complexity in uniform regions.
3Loss of information
If multiple linear models are used for prediction, then the residual data amount is reduced, but the processing time increases
Solution Approach 1:
The segmentation into sub-blocks allows parallel processing of model derivation and application across different regions. This structure enables efficient implementation where multiple sub-blocks can be processed simultaneously, reducing the overall processing time penalty associated with using multiple models.
Solution Approach 2:
The patent derives linear model parameters in advance during the encoding process, storing them for use during decoding. This preliminary action during encoding reduces the computational burden during real-time decoding, as the complex model derivation is performed beforehand rather than during critical playback operations.
Data Source
Figure 1
Figure 2A~2B
Figure 3A~3B
AI summary
A method of processing video data includes classifying, by processing circuitry, luma samples of a neighboring luma block of a reference block and luma samples of a neighboring luma block of a current block into a plurality of groups and deriving, by the processing circuitry, one or more local illumination compensation parameters for each group of the plurality of groups to generate a plurality of local illumination compensation parameters for the current block. The method further includes deriving, by the processing circuitry, a plurality of linear models between the neighboring luma block of the reference block and the neighboring luma block of the current block using the plurality of local illumination compensation parameters for the current block and generating, by the processing circuitry, a prediction block using the plurality of linear models.