Compressed video quality enhancement method based on spatio-temporal information balance

Abstract

The invention discloses a compressed video quality enhancement method based on spatio-temporal information balance, which is applied to the field of video processing and aims to solve the problem that the video quality is reduced in the existing compression technology. According to the method, a plug-and-play spatio-temporal information balance module is adopted, spatial features and time features are extracted, and the extracted time features and spatial features are aligned again in a feature space; noise and redundant time information introduced by alignment can be effectively removed; meanwhile, the space-time information proportion can be adaptively balanced, and the unified modeling problem of the PQF and the non-PQF is solved. And the effect of the existing video quality enhancement method can be obviously improved.

Description

technical field [0001] The invention belongs to the field of video processing, and particularly relates to a video compression technology. Background technique [0002] The most commonly used H.265 / HEVC standard now adopts a block-based hybrid coding framework, and its core processes include: predictive coding, transform coding, quantization and entropy coding, and block-based prediction. Transform and quantization operations ignore block-to-block correlations, resulting in coded reconstructed images exhibiting blockiness, i.e. apparent discontinuities at block boundaries that can be perceived by the human eye (when the step size is large and the bit rate is low, these The effect is more obvious); at the same time, quantization is based on block expansion in the transform domain, and this quantization process is irreversible. In addition, high-precision interpolation in motion compensation is prone to ringing effects. Due to the accumulation of errors in the inter-frame co...

AI Technical Summary

Problems solved by technology

However any error in the optical flow calculation will introduce new artifacts around the image structure in the aligned adjacent frames

At the same time, accurate optical flow estimation is itself challenging and time-consuming, so the quality enhancement effect of this invention is still limited

And a separate model is used for PQF, although it can improve the enhancement effect of PQF, it will significantly increase the computational cost

[0008] To sum up, due to the limitations of storage devices and transmission bandwidth, video is usually compressed in a lossy manner

However, although lossy coding reduces the size of the video, it inevitably introduces noise and artifacts. These noises will not only reduce the quality of user experience, but also affect the algorithm performance of some computer vision tasks; the existing video enhancement The method can be roughly divided into two types: single frame enhancement and multi-frame enhancement. Although the single frame enhancement method can be applied to the image at the same time, it ignores the inter-frame information in the video, and the enhancement effect is limited.

Multi-frame methods exploit the temporal information of the video but largely ignore the spatial information

Images