Matrix-Based Intra Prediction Downsampling for Lower Codec Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing matrix-based intra prediction methods in video coding, such as those used in Versatile Video Coding (VVC), suffer from increased computational complexity and latency due to the process of averaging boundary samples for downsampling, which also results in misalignment of reduced boundary samples with the matrix multiplication unit (MMU) output, leading to inefficiencies.
Innovation Solution
The proposed solution involves downsampling input boundary samples to generate reduced boundary samples that align with the MMU output, either through filtering or selective sampling without averaging, and performing matrix multiplication and linear interpolation in parallel to reduce complexity and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If boundary samples are averaged for downsampling in matrix based intra prediction, then prediction accuracy is improved, but computational complexity and latency increase
Solution Approach 1:
The patent extracts only the necessary boundary samples for downsampling without performing full averaging operations. Specifically, it selects every fourth boundary sample directly from the original boundary samples to form the downsampled boundary samples, eliminating the need for complex averaging calculations while maintaining sufficient prediction accuracy.
Solution Approach 2:
Instead of averaging multiple boundary samples to produce downsampled samples (conventional approach), the patent inverts the approach by directly selecting specific boundary samples at regular intervals. This inversion simplifies the downsampling process from a complex averaging operation to a simple sampling operation, reducing computational complexity while maintaining prediction quality.
2Measurement precision
If boundary samples are averaged for downsampling, then prediction quality is improved, but processing time increases
Solution Approach 1:
The patent extracts only the necessary boundary samples for downsampling without performing full averaging operations. Specifically, it selects every fourth boundary sample directly from the original boundary samples to form the downsampled boundary samples, eliminating the need for complex averaging calculations while maintaining sufficient prediction accuracy.
Solution Approach 2:
The patent skips the time-consuming averaging operations by directly selecting boundary samples at regular intervals (every fourth sample). This rushing through the downsampling process using simple selection instead of complex averaging significantly reduces processing time while maintaining adequate prediction quality.
3Measurement precision
If reduced boundary samples are derived through two-step averaging process, then prediction accuracy is maintained, but latency increases
Solution Approach 1:
The patent segments the boundary sampling process into direct selection of every fourth sample, eliminating the need for two-step averaging. This segmentation approach breaks down the complex two-step averaging process into a simpler single-step direct sampling operation, reducing latency while maintaining prediction accuracy.
Solution Approach 2:
The patent performs preliminary selection of boundary samples at regular intervals before the matrix multiplication operation. By pre-selecting every fourth boundary sample in advance, the patent eliminates the need for subsequent averaging operations, reducing overall processing latency while maintaining the necessary prediction accuracy.
4Measurement precision
If averaging is used to derive reduced boundary samples, then prediction quality is improved, but the number of operations increases
Solution Approach 1:
The patent extracts only the necessary boundary samples for downsampling without performing full averaging operations. Specifically, it selects every fourth boundary sample directly from the original boundary samples to form the downsampled boundary samples, eliminating the need for complex averaging calculations while maintaining sufficient prediction accuracy.
Solution Approach 2:
The patent replaces the expensive averaging operation with a cheap direct sampling operation. By selecting boundary samples directly without averaging, the patent uses a computationally inexpensive method that requires minimal operations (simple index selection) compared to the costly averaging process, thereby improving productivity with adequate prediction quality.
Data Source
AI summary
A prediction unit for an encoder or decoder implements matrix based intra prediction. Input boundary samples for a current block are downsampled to obtain reduced boundary samples for matrix multiplication and/or linear interpolation, or both. In one embodiment, downsampling is performed in a manner that aligns the reduced boundary samples with an output of a matrix multiplication unit of the prediction unit. In other embodiments, downsampling is performed without averaging. The embodiments reduce the complexity of the prediction unit and the latency of the encoder or decoder.