Video super-resolution reconstruction method based on multi-frame fusion optical flow

Abstract

The invention discloses a video super-resolution reconstruction method based on multi-frame fusion optical flow. The video super-resolution reconstruction method comprises the steps of collecting a data set, constructing a motion compensation network and reconstructing the network in a super-resolution manner. In a multi-frame fusion optical flow network, for multiple input frames, intra-frame spatial correlation can be fully utilized, loss details can be compensated, fused optical flow is used for motion compensation, and compensation frames are similar to a learning target. In a super-resolution reconstruction network, a three-dimensional scale feature extraction layer and a space-time residual module are used to extract image features of a compensation frame, and sub-pixel convolution is used to obtain a high-resolution video frame. And end-to-end training is carried out on the multi-frame fusion optical flow network and the video super-resolution reconstruction network at the sametime. Space-time information between acquired video frames can express features of video frame information fusion, and high-resolution video frames with good effects are reconstructed. The method canbe applied to the technical fields of satellite images, video monitoring, medical imaging, military science and technology and the like.

Description

technical field [0001] The invention relates to the technical field of video super-resolution, in particular to a video super-resolution reconstruction method based on multi-frame fusion optical flow and spatiotemporal residual dense blocks. Background technique [0002] Video super-resolution methods, which generate high-resolution videos from low-resolution videos, have been extensively studied for decades as a typical computer vision problem. In recent years, with the emergence of a large number of high-definition display devices and the emergence of ultra-high-definition resolution, the development of video super-resolution has been further promoted. At the same time, it also has wide application prospects in satellite images, video surveillance, medical imaging, and military technology, and has become one of the hot research issues in the field of computer vision. [0003] Traditional super-resolution methods are based on interpolation methods, such as nearest neighbor...

AI Technical Summary

Problems solved by technology

Liu et al. designed a time-adaptive neural network to adaptively learn the optimal scale of time-dependence, but currently only designed a simple three-layer convolutional neural network structure, which limits the performance

[0006] There are still some problems in the current video super-resolution method: the images in the single video frame super-resolution method are independent, and each image in the video frame is processed separately , and finally synthesize the entire video, these methods ignore the temporal correlation between video frames and lose a lot of details

Although the multi-video frame super-resolution method considers the time correlation between video frames, these method models increase a lot of computational costs, which limits the development of video super-resolution to a certain extent.

Images