Scene-Adaptive Video Post-Processing With Online Filter Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies suffer from compression artifacts that degrade visual quality, and deep learning-based post-processing methods are sub-optimal due to their inability to adapt to varying content, resolution, and other characteristics.
Innovation Solution
Implement scene-adaptive and online learning techniques to update the parameters of a post-processing filter network by training it based on actual content being encoded, allowing the network to adapt dynamically during video processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If deep learning-based post-processing methods are used, then visual quality is improved, but adaptability to varying content and resolution deteriorates due to fixed parameters
Solution Approach 1:
The patent implements dynamic parameter adaptation by enabling the post-processing network to update its parameters online based on the actual content being processed. The system determines which parameters to update by analyzing gradient changes during training, allowing the filter to adapt dynamically to different video content, resolutions, and encoding conditions rather than relying on fixed pre-trained parameters.
Solution Approach 2:
The patent changes the parameters of the post-processing filter network based on the actual video content being decoded. By selecting parameters with the largest accumulated gradient changes for updating and transmitting these updated parameters to the playback device, the system enables parameter adaptation to different content characteristics, resolving the contradiction between maintaining high visual quality and achieving adaptability to varying content.
2Manufacturing precision
If post-processing filter network with updated parameters is used, then distortion is reduced at lower bitrates, but computational complexity and processing time increase
Solution Approach 1:
The patent applies partial action by updating only a subset of parameters rather than all parameters of the post-processing network. By selecting parameters with the largest accumulated gradient changes for updating and keeping other parameters fixed, the system reduces the computational burden while still achieving significant distortion reduction at lower bitrates, thus resolving the contradiction between quality improvement and complexity increase.
Solution Approach 2:
The capture device performs the computationally intensive parameter training and updating in advance during the encoding phase. By determining and updating the parameters before transmission, and then simply applying the pre-updated parameters at the playback device, the system shifts the computational complexity to the encoding end, reducing the real-time processing burden at the decoding end while still achieving lower distortion.
3Adaptability or versatility
If online learning techniques are used to update parameters during video processing, then scene adaptability is improved, but processing time and computational resources increase
Solution Approach 1:
The patent performs the online learning and parameter updating during the encoding phase at the capture device before video transmission. By completing the adaptive training process in advance, the system achieves scene adaptability without adding processing time to the real-time decoding process at the playback device, thus resolving the contradiction between adaptability and processing time.
Solution Approach 2:
The system performs partial online learning by updating only the necessary subset of parameters rather than all parameters during the encoding process. This selective parameter updating reduces the computational resources and time required for online learning while still achieving effective scene adaptability, resolving the contradiction between adaptability improvement and resource consumption.
Data Source
AI summary
A video capture device may be encode a set of original pictures to create encoded video data, decode the encoded video data to create a set of reconstructed pictures, determine a subset of parameters to update from among a plurality of parameters of a post-processing filter network, including using the set of original pictures as ground truth and the set of reconstructed pictures as input to the post-processing filter network, update the subset of parameters to generate updated parameters, and send the encoded video data and the updated parameters to a playback device.