Non-Uniform Sparse Sampling Video Super-resolution Method

Abstract

The invention discloses a non-uniform sparse sampling video super-resolution method, which belongs to the field of video super-resolution technology and includes shot detection and key frame extraction, original image blur processing and down-sampling, and a visual model of non-uniform sampling. Key technologies such as establishment, construction of sparse dictionaries, and video reconstruction use non-uniform sparse sampling methods based on retinal foveal vision to achieve differential sparse sampling of video sequences, greatly reducing the number of atoms, and generating them in real time or based on mobile terminals. The resolution of the device screen is calculated a priori with a high- and low-resolution dual dictionary (high-resolution reference copy) as the onboard metadata. The nonlinear Mipmap interpolation method supported by the hardware is used to simulate and generate the Foveation image in real time, which can obtain a Foveation image similar to the Gaussian pyramid method. results with lower computational cost.

Description

technical field [0001] The invention relates to the technical field of video super-resolution, in particular to a non-uniform sparse sampling video super-resolution method. Background technique [0002] Today's mobile devices have become an integral part of our daily lives. Mature wireless communication technology and video codec enable video streaming to run on mobile devices, so people can easily access digital content anytime and anywhere through mobile devices, such as online TV programs, music videos, sports reports and news programs, which Promote the integration of computers, radio and television networks, and various communication systems, make cross-platform video communication gradually become the mainstream method, and initially provide powerful multimedia engines and full high-definition video technology (full high-definition video) support. [0003] However, due to limitations of wireless network bandwidth and terminal processing capabilities, there are still m...

AI Technical Summary

Problems solved by technology

[0010] (2) Current interpolation techniques or upsampling techniques have limitations in terms of reconstruction quality and computation time

Compared with image upsampling, current real-time video upsampling techniques produce more obvious visual artifacts and cannot fully utilize existing computing resources

At the same time, since the target boundary and important details are usually unknown, it will increase the calculation time of upsampling

[0011] (3) Due to the lack of high-frequency prior information and the limitation of computational complexity, the effect of super-resolution technology is unsatisfactory, far lower than the level that theoretical analysis should achieve

In practical applications, this is often limited by the computational complexity, and the individual restoration of each frame of image fails to effectively utilize the useful information of the restored image of the adjacent frame, resulting in a large waste of computation

The method based on iterative and adaptive filtering reduces the complexity of the algorithm by effectively utilizing the effective information of the restored high-resolution image, but this is at the cost of reducing the restoration performance

At present, the problems existing in the super-resolution reconstruction of compressed video mainly focus on three aspects: first, the effectiveness and universal adaptability of the compressed video model need further research; second, the effect of conventional reconstruction methods is not satisfactory, It is far lower than the level that theoretical analysis should achieve; third, the computational complexity of almost all reconstruction algorithms is too high to be implemented in real time

[0012] (4) The traditional wavelet transform cannot effectively capture the smoothness of image edges and contours, so it cannot achieve simplified or sparse image representation

The main difficulties include: how to implement sparse coding in video has certain technical barriers; sparse coding of heterogeneous data in heterogeneous networks has certain difficulties; video data is usually massive data, how to effectively expand the sparse coding algorithm and improve the efficiency of the algorithm , is also a very difficult problem

Images