Block-wise Neural Post-filtering for Video Compression Artifacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies face limitations in efficiently compressing video data, particularly in reducing redundancy and achieving optimal rate-distortion performance, due to the complexity of optimizing individual modules within hybrid video codecs and the need for adaptive compression strategies that account for varying content.
Innovation Solution
The implementation of a block-wise content-adaptive online training method using neural networks for neural image compression (NIC), which includes post-filtering and deblocking processes, allows for the optimization of video encoding and decoding by updating neural network parameters based on specific content blocks, thereby improving compression efficiency and quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional video coding technologies are used, then device complexity is reduced, but rate-distortion performance and compression efficiency deteriorate
Solution Approach 1:
The patent implements dynamic neural network parameter updates during video encoding and decoding operations. The neural network parameters are adapted online based on the actual video content being processed, allowing the system to optimize rate-distortion performance dynamically rather than using fixed pre-trained parameters. This dynamic adaptation resolves the contradiction by enabling high performance while managing complexity through content-aware parameter adjustment.
Solution Approach 2:
The patent changes neural network parameters adaptively during the coding process based on video content characteristics. By modifying parameters such as quantization matrices and transform scales dynamically according to the actual video content, the system achieves improved rate-distortion performance without requiring a complete redesign of the codec architecture, thus managing device complexity.
2Productivity
If content-adaptive neural network training is implemented, then compression efficiency is improved, but processing time increases
Solution Approach 1:
The patent applies partial content-adaptive training by updating only specific neural network parameters that are most beneficial for the current video content, rather than retraining the entire network. This selective parameter update approach improves compression efficiency for the specific content while avoiding the excessive processing time that would result from complete retraining, thus resolving the contradiction.
Solution Approach 2:
The patent performs preliminary online training of neural network parameters using previously decoded video frames before applying them to current frame compression. This preliminary adaptation allows the system to have compression parameters ready in advance, reducing the actual processing time during encoding while still achieving content-adaptive optimization.
3Manufacturing precision
If block-wise adaptive processing is applied, then compression quality is improved, but computational complexity increases
Solution Approach 1:
The patent divides the video content into blocks and applies adaptive neural network processing at the block level rather than processing the entire video frame uniformly. This segmentation allows quality improvement in regions that benefit most from adaptive processing while reducing computational complexity in regions where simpler processing suffices, thus resolving the contradiction between quality and complexity.
Solution Approach 2:
The patent applies different processing qualities to different regions of video blocks based on local content characteristics. By identifying regions that require higher quality processing (such as regions with important visual information) and applying adaptive neural network processing only there, while using simpler processing in other regions, the system improves overall compression quality without proportionally increasing computational complexity.
Data Source
AI summary
Aspects of the disclosure provide a method, an apparatus, and a non-transitory computer-readable storage medium for video decoding. The apparatus can include processing circuitry. The processing circuitry is configured to reconstruct blocks of an image from a coded video bitstream. The processing circuitry can perform post-processing on one of a plurality of regions of first two neighboring reconstructed blocks of the reconstructed blocks with at least one post-processing neural network (NN). The first two neighboring reconstructed blocks have a first shared boundary and include a boundary region having samples on both sides of the first shared boundary. The plurality of regions of the first two neighboring reconstructed blocks includes the boundary region and non-boundary regions that are outside the boundary region. The one of the plurality of regions is replaced with the post-processed one of the plurality of regions of the first two neighboring reconstructed blocks.