Local Illumination Compensation Using Multi-Mode Template Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The flexibility of the linear model used in legacy enhanced compression models (ECM) for illumination compensation (LIC) is limited, and the current and reference block templates may be noisy, failing to accurately represent the correlation between blocks.
Innovation Solution
A method is introduced to enhance illumination compensation by using multiple local illumination compensation (LIC) modes, including slope adjustment, multi-model modeling, extended template regions, and flexible template line selection, allowing for improved prediction of illumination based on multiple templates and models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If only one linear model is used for LIC in legacy ECM implementation, then device complexity is reduced, but illumination compensation accuracy deteriorates
Solution Approach 1:
The patent divides the illumination compensation process into multiple LIC modes (first LIC mode, second LIC mode, third LIC mode) corresponding to different template selection strategies. Each mode segments the problem by using different reference block templates (e.g., template from same reference block, template from different reference block, extended template regions) to handle various illumination variation scenarios, thereby improving accuracy without requiring a single complex model
Solution Approach 2:
The patent introduces dynamic mode selection where the encoder/decoder can choose among multiple LIC modes based on content characteristics. The system dynamically adapts between different template fetching strategies (e.g., when to use extended template regions, when to use standard templates) to optimize compensation accuracy for different scene conditions while maintaining manageable complexity through conditional logic
2Measurement precision
If current block template and reference block template are used directly, then processing speed is maintained, but measurement precision deteriorates due to noise
Solution Approach 1:
The patent applies preliminary filtering and validation actions to templates before they are used for illumination compensation. The system pre-processes template selection by evaluating template quality, applying noise reduction filters, and validating template correlation with the current block. This preliminary action ensures that only high-quality templates are used, improving measurement precision while the filtered results are then used efficiently in the compensation calculation
Solution Approach 2:
The patent introduces intermediary processing steps between template fetching and illumination compensation. These intermediaries include template validation mechanisms, quality assessment filters, and correlation checks that mediate between the raw templates and the final compensation application. This intermediary layer filters out noisy templates while preserving valid ones, improving accuracy without significantly impacting processing speed
3Measurement precision
If multiple LIC modes and extended template regions are used, then illumination compensation accuracy is improved, but device complexity increases
Solution Approach 1:
The patent applies local quality principles by using different LIC modes for different spatial regions and content types. Extended template regions are selectively applied based on local illumination characteristics, and different reference block templates are chosen for different areas of the current block. This localized approach improves prediction accuracy where needed while avoiding unnecessary complexity in regions where simpler methods suffice
Solution Approach 2:
The patent changes parameters such as template region boundaries, reference block selection, and mode indices dynamically based on content analysis. By adjusting these parameters conditionally (e.g., enabling extended templates only when illumination variation exceeds a threshold, selecting different reference blocks based on motion characteristics), the system achieves high accuracy when necessary while maintaining lower complexity for standard cases
Data Source
AI summary
A non-transitory machine-readable medium of an electronic device storing computer-executable instructions for decoding a bitstream is provided. The computer-executable instructions, when executed by a processor of the electronic device, cause the electronic device to receive an image frame of the bitstream; determine a first mode from a plurality of local illumination compensation (LIC) modes; determine a first template of a block unit of the image frame and at least one second template of at least one reference block based on the first mode; determine a first model based on the first template and the at least one second template; obtain a slope adjustment parameter; adjust the first model based on the slope adjustment parameter; and modify a prediction illumination of the block unit of the image frame based on the adjusted first model.