Video Block Transform Selection With Inferred LFNST Signaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing video codec standardization process in Versatile Video Coding (VVC) involves extensive testing and signaling of multiple transform selection (MTS) and low frequency non-separable transform (LFNST) combinations, leading to increased encoder complexity and bit usage, without necessarily improving compression efficiency.
Innovation Solution
By limiting the number of MTS and LFNST combinations evaluated and reducing the need for signaling, the encoder and decoder complexity is reduced, with specific transform combinations being inferred or omitted based on predetermined indices, thereby optimizing the transform selection process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the encoder tests all allowed combinations of MTS and LFNST transforms, then the compression efficiency is improved, but the encoder complexity and run time increase significantly
Solution Approach 1:
The patent extracts and removes unlikely transform combinations from the evaluation set. Specifically, it identifies and eliminates combinations where LFNST is applied with certain MTS transforms based on predefined criteria, thereby reducing the number of combinations the encoder must test while preserving the most effective transformations for compression.
Solution Approach 2:
Instead of evaluating all possible transform combinations, the patent applies partial evaluation by limiting the search to a subset of combinations. The encoder performs transform selection only for the most promising combinations identified through predefined rules, rather than exhaustively testing every possible combination of MTS and LFNST transforms.
2Reliability
If the encoder signals all possible MTS and LFNST combinations, then the decoder can accurately reconstruct the transform applied, but the bit usage increases
Solution Approach 1:
The patent removes the need to signal certain transform combinations by inferring them at the decoder side. When specific MTS transforms are detected, the corresponding LFNST application is inferred without explicit signaling, thereby reducing the number of bits required while maintaining accurate transform reconstruction.
Solution Approach 2:
The decoder performs self-service by automatically inferring which LFNST transforms were applied based on the received MTS transform information. This eliminates the need for separate signaling bits, as the decoder can independently determine the transform combination used during encoding through predefined inference rules.
3Manufacturing precision
If the encoder evaluates all MTS and LFNST combinations, then the best compression is achieved, but the memory usage and processing time increase
Solution Approach 1:
The patent extracts and removes computationally expensive transform combinations from the evaluation process. By identifying and eliminating combinations that are unlikely to provide optimal compression based on predefined criteria, the encoder reduces the number of transforms that need to be computed and evaluated, thereby reducing run time while maintaining compression quality.
Solution Approach 2:
The patent performs preliminary filtering of transform combinations before the actual encoding process. By pre-defining which combinations are worth evaluating and which can be skipped, the encoder prepares a reduced set of candidates in advance, avoiding unnecessary computational work during the time-critical encoding phase.
4Adaptability or versatility
If multiple LFNST transform sets are maintained for different intra prediction directions, then the adaptability is improved, but the device complexity increases
Solution Approach 1:
The patent makes the decoder universal by enabling it to handle multiple transform combinations through a single unified process. Instead of requiring separate decoding paths for different LFNST sets, the decoder uses a general-purpose inference mechanism that works for all transform types, thereby maintaining adaptability while reducing complexity.
Data Source
Figure 1
Figure 2~4
Figure 5
AI summary
A method, decoder apparatus, and computer programs are provided. A bit stream having a video block encoded with a multiple transform selection (MTS) index is received. The MTS index is parsed to determine a value of it. Responsive to the MTS index value being a predetermined value (or one of a predetermined set of indices), a LFNST index is parsed from the bit stream to determine a value of it, the value providing an indication whether a LFNST transform set is to be used in decoding the video block. Responsive to the MTS index value not being the predetermined value (or one of the set of indices), the value of the LFNST index is set to a default value indicating no LFNST transform is to be used in the decoding. The video block is decoded based on the values of the MTS index and the LFNST index.