Color Remapping for Non-4:4:4 Video via Matrix Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video encoding and decoding technologies face significant challenges in processing non-4:4:4 video formats like 4:2:0 and 4:2:2 due to the resource-intensive requirements of upsampling and downsampling, which are necessary in traditional color-space conversion techniques.
Innovation Solution
The proposed techniques enable video encoding and decoding devices to perform color remapping by applying a 3×3 matrix directly to 4:2:0 content, omitting the need for upsampling to 4:4:4 and downsampling back, thereby reducing the computational and memory burdens.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional color-space conversion techniques are used on non-4:4:4 video, then color-space-based coding tools can be applied, but significant memory and processing resources are required due to upsampling and downsampling
Solution Approach 1:
The patent extracts and removes the unnecessary upsampling and downsampling steps from the traditional color-space conversion process. By directly applying color remapping to the subsampled chroma components in 4:2:0 format, the solution eliminates the intermediate 4:4:4 format conversion, thereby reducing memory and processing resource requirements while maintaining color-space conversion functionality
Solution Approach 2:
The patent changes the approach from spatial domain upsampling to frequency/domain-based color remapping. By applying 3×3 matrix operations directly to the subsampled chroma samples and using scaled-down luma samples for color difference calculations, the method transforms the conversion process to work efficiently with the existing subsampled format rather than requiring full-resolution conversion
2Adaptability or versatility
If upsampling from 4:2:0 to 4:4:4 and downsampling from 4:4:4 to 4:2:0 is performed, then color-space conversion can be achieved, but processing time and computational load increase significantly
Solution Approach 1:
The patent removes the time-consuming upsampling and downsampling operations from the color-space conversion pipeline. By applying color remapping directly to the 4:2:0 subsampled format through matrix operations on chroma components and scaled luma differences, the solution achieves color-space conversion without the intermediate format conversions that burden processing efficiency
Solution Approach 2:
The patent uses scaled-down copies of luma samples (every other sample) to compute color difference values instead of requiring full-resolution luma data. This copying approach with downsampling factor of 2 maintains sufficient accuracy for color remapping while reducing the computational load proportional to the reduction in sample quantity
Data Source
AI summary
A video decoder is configured to reconstruct a first version of a current picture; determine a transformed sample of the first component, a transformed sample of the second component, and a transformed sample of the third component by applying a first matrix to a triplet comprising the samples of the first, second, and third components; include the transformed samples of the first, second, and third components in a second version of the current picture; determine, based on the transformed samples of the first, second, and third components and a first row of a second matrix, a set of transformed samples for the third component, the set of transformed samples for the third component including samples of the third component that are immediately neighboring the transformed sample of the third component; and include the set of transformed samples for the third component in the second version of the current picture.