Video Decoding Neural Post-Filter for Chroma Tensor Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding and decoding technologies, such as H.264/AVC and H.265/HEVC, lack explicit definitions for the input and output of post-filter processing, particularly regarding color space and chroma sub-sampling, preventing effective specification and processing of neural network filters.
Innovation Solution
A video decoding apparatus and coding apparatus that derive input and output tensors for neural network models based on parameters indicating the number of channels, and utilize additional input information to generate images, with loop processing and chroma sub-sampling adjustments, enabling inference of chroma formats without analyzing the neural network model directly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural network post-filter processing is added to video decoding, then image quality is improved, but the complexity of specifying and processing the filter increases
Solution Approach 1:
The patent applies parameter changes by defining specific parameters for the neural network post-filter processing, including chroma format parameters (chroma_format_idc), chroma sub-sampling parameters (sub_width_c, sub_height_c), and tensor channel parameters (numTensors). These parameter definitions enable precise control and specification of the post-filter processing without requiring complex analysis of the neural network model itself, thereby improving image quality while managing specification complexity.
2Measurement precision
If the neural network model is analyzed to determine processing capabilities, then accurate capability determination is achieved, but processing time and complexity increase
Solution Approach 1:
The patent implements preliminary action by pre-defining the parameters and data structures needed for neural network post-filter processing. The chroma format parameters, chroma sub-sampling parameters, and tensor channel parameters are established in advance through standardized definitions, allowing the decoding apparatus to quickly determine processing capabilities without performing time-consuming analysis of the neural network model. This preliminary parameter setup enables fast capability determination while maintaining accuracy.
3Adaptability or versatility
If chroma sub-sampling is not explicitly defined, then processing flexibility is maintained, but processing precision and reliability decrease
Solution Approach 1:
The patent applies local quality by explicitly defining chroma sub-sampling parameters (sub_width_c, sub_height_c) and chroma format parameters (chroma_format_idc) for specific regions and processing stages. These local parameter definitions ensure that chroma processing is performed with precise and reliable settings appropriate for each specific case, while still allowing flexibility to adjust parameters based on the input image characteristics and processing requirements.
Data Source
AI summary
A problem with known SEI is that a relationship between a target image and an input tensor of a neural network model and a relationship between an output tensor and an output image are undefined or insufficient, and thus processing cannot be performed by simply using a model. Another problem is that in a case that the color component of the output image and the color component of the target image are different from each other, processing cannot be performed. An aspect of the present invention provides a video decoding apparatus including a prediction image derivation unit configured to decode a prediction image, and a residual decoder configured to decode a residual, wherein an input tensor is derived from a parameter indicating the number of channels of the input tensor and the output tensor of the neural network model or an image is derived from the output tensor.