Weakly-Connected-Dense-Attention Neural Network for Video Artifact Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video compression methods, such as JPEG, HEVC, and VVC, result in irreversible information loss and compression artifacts like blocking, blurring, and banding, especially at high compression ratios, and deep-learning based methods using pyramid or block stacking networks are limited in their ability to fully utilize residual features.
Innovation Solution
The proposed solution is a weakly-connected-dense-attention-neural-network (WCDANN) framework that uses multiple weakly-connected-dense-attention-blocks (WCDABs) with residual attention blocks and channel-spatial-attention-blocks to extract and enhance residual features, reducing model parameters through depth-wise separable convolutions and promoting residual information circulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If deep-learning based methods using pyramid structure are used to remove compression artifacts, then image quality is improved, but device complexity and computational cost increase significantly
Solution Approach 1:
The network is divided into multiple stages with progressive resolution enhancement. Each stage processes features at a specific scale and progressively refines the output, breaking down the complex task of artifact removal into manageable sequential steps that reduce overall computational burden while maintaining quality.
Solution Approach 2:
Different processing strategies are applied to different regions and scales of the image. The network focuses computational resources on areas with compression artifacts while preserving already-quality regions, and uses different feature extraction approaches for various spatial frequencies.
2Manufacturing precision
If multiple deep-learning based methods with many convolution operations are used to process information at different scales, then compression artifacts are reduced, but computational cost and processing time increase
Solution Approach 1:
The computational task is segmented across multiple processing stages, where each stage handles specific frequency bands or spatial regions. This allows for optimized resource allocation at each stage rather than uniformly applying heavy computation across the entire image.
Solution Approach 2:
The network applies processing only where needed - focusing computational effort on regions containing compression artifacts rather than processing the entire image uniformly. This partial action approach reduces overall computational cost while maintaining effectiveness.
3Reliability
If frameworks based on block stacking with dense or residual blocks are used, then feature learning is enhanced, but the number of network parameters increases significantly
Solution Approach 1:
The network dynamically adjusts the number and type of parameters used at different processing stages based on the input characteristics and compression level. This allows the model to maintain high feature learning capability while adapting the parameter quantity to match the actual processing needs.
Solution Approach 2:
Full dense or residual block structures are applied selectively only when and where they provide necessary feature learning benefits. In regions or stages where simpler processing suffices, the network uses reduced-parameter configurations, thereby reducing overall parameter count while maintaining reliability.
4Productivity
If existing residual learning methods are used that only select partially residual characteristics in time, then processing speed is maintained, but the ability to fully utilize residual features is limited
Solution Approach 1:
The network maintains continuous residual feature extraction and utilization across all processing stages rather than selectively processing only certain time points or frames. This continuous action ensures that residual information is fully exploited throughout the entire processing pipeline without sacrificing processing speed.
Solution Approach 2:
Residual features are extracted and prepared in advance at each processing stage, allowing the network to fully utilize them before final reconstruction. This preliminary preparation of residual information ensures complete utilization without requiring additional processing time during the critical reconstruction phase.
Data Source
AI summary
Methods and a system for video processing are provided. The method includes receiving an input image; extracting shallow features of the input image through a head network; determining, based on the shallow features, residual features of the input image and enhancing a portion of the residual features by two or more weakly-connected-dense-attention-blocks (WCDABs); reconstructing the residual features to form a residual map; and adding the residual map to the input image to generate a reconstructed image.