Matrix-Based Intra Prediction With Fixed-Point Mode Settings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in implementing matrix-based intra prediction modes is the need to approximate matrix-vector products using integer operations, which can lead to significant deviations from the trained floating-point operations, disrupting the data-driven approach and increasing bit stream costs.
Innovation Solution
Represent prediction matrix entries in a fixed point representation with a common bit-depth and apply a fixed right shift for all matrix-based intra-prediction modes, enabling efficient computation and memory management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point arithmetic is used for matrix-vector multiplication in MIP modes, then prediction accuracy is improved, but computational complexity and implementation difficulty increase
Solution Approach 1:
The patent changes the numerical representation parameter from floating-point to fixed-point arithmetic. This allows the system to maintain prediction accuracy while reducing computational complexity, as fixed-point operations are simpler to implement in hardware and software compared to floating-point operations.
Solution Approach 2:
The patent uses pre-computed integer matrices that approximate the optimal floating-point transformation matrices. These integer matrices are 'cheap' in terms of computational resources required to store and process them, while still providing sufficient prediction accuracy for practical applications.
2Adaptability or versatility
If different bit-depths are used for different MIP modes, then each mode can be optimized individually, but memory management and computation efficiency decrease
Solution Approach 1:
The patent applies a uniform bit-depth (e.g., 8-bit or 16-bit integers) across all MIP modes rather than using different bit-depths for different modes. This universal approach simplifies memory management and allows the use of shared multipliers and computational resources, significantly improving computation efficiency while still providing adequate precision for all modes.
3Productivity
If integer operations are used to approximate matrix-vector products, then computational efficiency is improved, but deviation from trained floating-point operations increases
Solution Approach 1:
The patent performs preliminary optimization of integer matrices to minimize the deviation from the optimal floating-point transformation matrices. By pre-computing and storing integer matrices that are optimized to approximate the floating-point operations, the system achieves both computational efficiency and acceptable prediction accuracy without requiring complex runtime corrections.
Data Source
AI summary
An apparatus includes at least one processor for decoding a block of a picture using intra-prediction that is configured to: decode, from a data stream, a mode index, the mode index indicating a matrix-based intra-prediction mode out of a plurality of matrix-based intra-prediction modes based on a size of the block, derive an input vector based on downsampling reference samples neighboring the block, determine a matrix based on the size of the block and the mode index, compute a respective output for each component of a matrix-vector product between the input vector and the determined matrix, the respective output computed by performing, a right shift by a number of bits which is independent of the matrix-based intra-prediction mode indicated by the mode index, and for each component of the matrix-vector product, use the respective output to predict a respective sample of the block.