Neural Video Post-Processing for Resolution Downsampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video processing technologies lack the capability to perform resolution downsampling, limiting the functionality of neural-network post-processing filters (NNPFs) in video coding systems.
Innovation Solution
Implementing a neural-network post-processing filter (NNPF) that supports both resolution upsampling and downsampling by applying it on video pictures, allowing for flexible conversion between video and bitstreams, with indications specifying the purpose of the NNPF, including options for decreasing the width or height of the pictures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If NNPF is applied for resolution upsampling only, then the video processing capability is limited to upsampling operations, but the functionality and versatility of the NNPF is insufficient
Solution Approach 1:
The patent extends the NNPF to perform multiple functions including both upsampling and downsampling operations. By configuring the NNPF with different purpose indicators (first indication for downsampling, second indication for upsampling), a single filter structure can adapt to various resolution conversion needs, eliminating the requirement for separate dedicated filters for each operation and thereby improving versatility without proportionally increasing system complexity
2Adaptability or versatility
If NNPF supports both upsampling and downsampling, then the functionality of NNPF is enhanced, but the complexity of the video processing system increases
Solution Approach 1:
The patent introduces dynamic configuration mechanisms where the NNPF's operation mode (upsampling or downsampling) can be changed through purpose indications in the bitstream. This allows the system to adapt its processing capability dynamically based on the specific video content and requirements, enabling multi-functionality while maintaining a relatively simple base structure that doesn't require separate hardware for each operation type
3Loss of information
If only resolution upsampling is supported, then the video processing capability is simpler, but the loss of information during downsampling operations cannot be addressed
Solution Approach 1:
The patent replaces traditional mechanical or algorithmic downsampling methods with a neural-network-based approach. The NNPF uses learned patterns and intelligent processing to perform downsampling while minimizing information loss, achieving better quality results compared to conventional methods. The neural network can be trained to preserve important visual information during resolution conversion, addressing the information loss problem while maintaining processing efficiency through optimized network inference
Data Source
AI summary
Embodiments of the present disclosure provide a solution for video processing. A method for video processing is proposed. The method comprises: performing a conversion between a video and a bitstream of the video, wherein a neural-network post-processing filter (NNPF) is applied on at least one picture associated with the video, the bitstream comprises a first indication indicating a purpose of the NNPF, and a first candidate of a plurality of candidates for the purpose at least indicates that at least one of the following is applicable: decreasing a width of the at least one picture, or decreasing a height of the at least one picture.


