Hybrid Video Compression Networks for Low-Bitrate Artifact Mitigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video codecs introduce noticeable compression artifacts such as blocking, ringing, blurring, and mosquito noise during lossy compression, particularly in high spatial resolution and high frame rate videos encoded at low bitrates, leading to degraded video quality.
Innovation Solution
Implementing machine learning (ML) refinement networks for post-processing to mitigate compression artifacts by training the networks to identify and reduce these artifacts based on differences between input and reconstructed video, using feedback loops for improved quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If lossy compression is used to reduce bitrate, then bandwidth and storage requirements are reduced, but compression artifacts (blocking, ringing, blurring, mosquito noise) are introduced degrading video quality
Solution Approach 1:
A machine learning refinement network is introduced as an intermediary component between the decoder and the final video output. This ML network processes the decoded video frames to identify and reduce compression artifacts while preserving visual quality, effectively mediating between the compressed representation and the perceived video quality
Solution Approach 2:
Traditional artifact reduction methods based on fixed mathematical filters are replaced with a machine learning-based system that can adaptively learn and remove artifacts. The ML refinement network substitutes conventional signal processing mechanisms with intelligent algorithms that generalize across different compression scenarios
2Manufacturing precision
If traditional video encoding algorithms are used, then computational complexity is manageable, but compression efficiency and video quality at low bitrates are insufficient
Solution Approach 1:
The video processing system is segmented into distinct functional components: traditional video encoder, decoder, and ML refinement network. This segmentation allows each component to specialize in specific tasks, with the ML network handling the computationally intensive artifact reduction while traditional codecs handle compression and decompression
Solution Approach 2:
The ML refinement network is trained in advance on large datasets of compressed and original video pairs to learn artifact patterns and removal strategies. This preliminary training phase enables the network to perform efficient inference during actual video processing, reducing the computational burden during runtime
Data Source
AI summary
Innovations in machine learning (“ML”) networks used in video processing scenarios are described. For example, an ML refinement network can be used to refine video after a video decoder has reconstructed the video. Using the ML refinement network for post-processing can mitigate compression artifacts introduced during encoding and otherwise improve the quality of the reconstructed video. Or, as another example, an ML encoder network and ML decoder network can be used, in combination with a core video encoder and core video decoder, for hybrid compression and corresponding decompression. In the hybrid compression, the ML encoder network can transform video before encoding in order to boost rate-distortion performance of the core video encoder. In corresponding decompression, the ML decoder network can enhance reconstructed video after decoding, thereby compensating for transformations applied by the ML encoder network, mitigating compression artifacts, and otherwise improving the quality of the reconstructed video.