Video super-resolution method based on convolutional neural network and mixed resolution

Abstract

The invention belongs to the technical field of computer vision, and particularly provides a video super-resolution method based on a convolutional neural network and mixed resolution. The method comprises the following steps of firstly, prediorically revsering a part of frame in a video sequence to serve as high-resolution frame data, subjecting other frames to degradation processing to serve aslow-resolution image frames, and combinding the two frames are to form a hybrid resolution video; secondly, training a degradation network based on the convolutional neural network to extract featureinformation of a degradation factor, and obtaining a degradation feature map of the low-resolution image frame by using the trained model; and then, taking the low-resolution image frame and the high-resolution frame and degradation feature map related to the low-resolution image frame as input data, performing training based on a convolutional neural network to obtain a super-resolution network model, and obtaining an output high-resolution video. The convolutional neural network and the hybrid resolution model are combined, and image texture details and degradation factors can be analyzed ina targeted manner, so that the super-resolution accuracy is improved.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and in particular relates to a video super-resolution method based on a convolutional neural network and mixed resolutions. Background technique [0002] With the continuous development of multimedia technology, video applications such as online live broadcast and high-resolution TV have gradually become the mainstream media for people's life and entertainment; however, video systems are often limited by various objective conditions, including video acquisition equipment with insufficient precision and limited network bandwidth. And terminals with insufficient processing capabilities, etc., which make it difficult for the video system to provide sufficient high-resolution video sources. [0003] In order to solve the above problems, super-resolution technology can be used in the video system, so that video applications with limited objective conditions can also provide high-quality video ...

AI Technical Summary

Problems solved by technology

[0004] At present, many video super-resolution methods have been proposed, but due to the characteristics of video image frames and the diversity of video scenes, the super-resolution results are not completely satisfactory.

Existing video super-resolution methods generally use redundant information between adjacent frames to complement each other to restore the high-frequency information missing from the image frame; however, whether it is the current image frame or adjacent frames, they are all low-resolution images. There is also a lack of high-frequency information containing texture details, which may make it difficult to recover this part of the information during the entire super-resolution process; at the same time, the existing video super-resolution methods generally use 5 consecutive low-resolution frames as network input, each Frames are repeatedly input multiple times, which increases the overall computational redundancy for restoring the entire video sequence; in addition, in practical applications, the degradation methods of image frames are complex and diverse, and most current video super-resolution methods only consider bicubic downsampling. Degradation methods, once the degradation methods involved in the video also include other degradation factors, the results of these super-resolution methods will degrade

Images