Neural Network Video Filter Clipping Subsampled Color Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies face challenges in efficiently processing and compressing video data, particularly in reducing redundancy and improving picture quality, especially when dealing with high-resolution and high-bitrate video streams, which require significant bandwidth and storage space.
Innovation Solution
The implementation of a neural network-based in-loop filter that clips color component values of video data into a valid range before processing, utilizing a dense residual convolutional neural network (DRNLF) to enhance picture quality and compression efficiency, and converting subsampled formats to non-subsampled formats for processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If video data is compressed using traditional methods, then bandwidth and storage requirements are reduced, but picture quality deteriorates due to increased distortion
Solution Approach 1:
The patent applies clipping operations to color component values before they are fed into the neural network filter. This preliminary action ensures that the input data is within valid ranges, preventing distortion artifacts and improving the effectiveness of subsequent filtering operations, thereby maintaining picture quality at lower bitrates
Solution Approach 2:
The patent transforms subsampled color formats (e.g., 4:2:0, 4:2:2) to non-subsampled formats (e.g., 4:4:4) before neural network processing. This parameter change restores full color resolution, allowing the neural network to operate on complete color information and produce higher quality output at compressed bitrates
2Manufacturing precision
If neural network based filtering is applied to improve picture quality, then processing complexity increases, but compression efficiency may be reduced
Solution Approach 1:
The patent performs preprocessing operations (clipping color values to valid ranges, converting subsampled formats to non-subsampled formats) before neural network filtering. This prepares the data in an optimal format, allowing the neural network to process more efficiently and achieve better results with fewer computational resources
Solution Approach 2:
The patent introduces intermediate processing steps (clipping operations, format conversion) as mediators between the compressed video data and the neural network filter. These intermediaries ensure data validity and optimal format, improving neural network efficiency while maintaining manageable overall system complexity
3Speed
If color component values are not clipped before neural network processing, then processing speed is maintained, but training and inference accuracy deteriorates
Solution Approach 1:
The patent applies clipping operations as a preliminary step before neural network processing. This ensures color component values are within valid ranges, significantly improving training and inference accuracy. The clipping operation is computationally simple, so it adds minimal processing time while dramatically improving neural network performance
Data Source
AI summary
Aspects of the disclosure provide methods and apparatuses for video processing. In some examples, an apparatus for video processing includes processing circuitry. The processing circuitry converts a picture in a subsampled format in a color space into a non subsampled format in the color space. Then, the processing circuitry clips values of a color component of the picture in the non subsampled format before providing the picture in the non subsampled format as an input to a neural network based filter.