Frame Loop Video Super Resolution

Abstract

The present disclosure provides systems and methods for increasing the resolution of imagery. In an example embodiment, a computer-implemented method includes obtaining a current low-resolution image frame. The method includes obtaining a previously estimated high resolution image frame that is a high resolution estimate of a previous low resolution image frame. The method includes warping a previously estimated high resolution image frame based on a current low resolution image frame. The method includes inputting a warped previously estimated high-resolution image frame and a current low-resolution image frame into a machine learning frame estimation model. The method includes receiving as an output of the machine learning frame estimation model a current estimated high resolution image frame, the current estimated high resolution image frame being a high resolution estimate of the current low resolution image frame.

Description

technical field [0001] This disclosure relates generally to machine learning. More specifically, the present disclosure relates to performing super-resolution on imagery, such as video, using machine learning models. Background technique [0002] Super-resolution is a classic problem in image processing that addresses the problem of how to construct a high-resolution (high -resolution, HR) version. With the rise of deep learning, super-resolution has received a lot of attention from the research community in the past few years. While in the case of single-image super-resolution usually only high-frequency details are reconstructed from spatial statistics, temporal relationships in the input can be exploited for video super-resolution to improve reconstruction. Therefore, some super-resolution techniques try to combine information from as many LR frames as possible to achieve the best video super-resolution results. [0003] Certain video super-resolution methods address ...

AI Technical Summary

Problems solved by technology

However, this scheme has several weaknesses

First, this scheme is computationally expensive since each input frame needs to be processed several times

Second, generating each output frame separately reduces the system's ability to produce temporally consistent frames, which can lead to unpleasant flickering artifacts

That is, each output frame is estimated independently conditional on the input frame, limiting the system's ability to produce temporally consistent results

Images