ML Refinement Networks for Video Compression Artifact Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video codecs introduce noticeable compression artifacts during lossy compression, such as blocking, ringing, blurring, and mosquito noise, which degrade the quality of high spatial resolution and high frame rate videos when encoded at low bitrates.
Innovation Solution
Implementing machine learning (ML) refinement networks for post-processing to mitigate compression artifacts by refining decoded video, and using ML encoder and decoder networks for hybrid compression to enhance video quality and improve rate-distortion performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If lossy compression is used to reduce bitrate, then transmission efficiency is improved, but compression artifacts are introduced that degrade video quality
Solution Approach 1:
The video processing system is segmented into multiple independent components: traditional video encoder, ML encoder network, traditional video decoder, and ML refinement network. Each component handles specific aspects of compression and quality enhancement, allowing the system to optimize bitrate reduction while maintaining video quality through specialized processing stages.
Solution Approach 2:
Machine learning networks serve as intermediary components between the traditional encoder and decoder. The ML encoder network transforms video before traditional encoding to improve compression efficiency, while the ML refinement network processes decoded video to remove artifacts. These intermediaries enable quality preservation during lossy compression by adding intelligent processing layers.
2Device complexity
If traditional video encoding is used to maintain compatibility, then device complexity is reduced, but compression artifacts such as blocking and ringing are noticeable
Solution Approach 1:
The patent replaces traditional mechanical signal processing methods with machine learning-based approaches. Instead of using conventional filtering and post-processing techniques to reduce artifacts, the system employs trained neural networks that learn optimal artifact reduction strategies, providing superior performance while maintaining reasonable complexity.
Solution Approach 2:
The system changes the operational parameters of video encoding by introducing ML-based transformation and refinement stages. The ML encoder network learns optimal transformation parameters, and the refinement network adjusts quality parameters dynamically, enabling artifact reduction without significantly increasing overall system complexity.
3Manufacturing precision
If high bitrate encoding is used to preserve video quality, then video quality is maintained, but transmission efficiency decreases
Solution Approach 1:
The ML encoder network performs preliminary transformation of the video signal before traditional encoding. This preliminary action optimizes the video representation for compression, enabling more efficient bitrate utilization and reducing the overall bitrate required to achieve target quality levels.
Solution Approach 2:
The system implements feedback loops where the ML refinement network analyzes decoded video quality and adjusts processing parameters accordingly. This feedback mechanism enables dynamic optimization of compression efficiency, maintaining video quality while adapting bitrate usage to actual quality requirements.
Data Source
AI summary
Innovations in machine learning (“ML”) networks used in video processing scenarios are described. For example, an ML refinement network can be used to refine video after a video decoder has reconstructed the video. Using the ML refinement network for post-processing can mitigate compression artifacts introduced during encoding and otherwise improve the quality of the reconstructed video. Or, as another example, an ML encoder network and ML decoder network can be used, in combination with a core video encoder and core video decoder, for hybrid compression and corresponding decompression. In the hybrid compression, the ML encoder network can transform video before encoding in order to boost rate-distortion performance of the core video encoder. In corresponding decompression, the ML decoder network can enhance reconstructed video after decoding, thereby compensating for transformations applied by the ML encoder network, mitigating compression artifacts, and otherwise improving the quality of the reconstructed video.