Method and device for realizing multi-scale optical flow pixel transformation by utilizing dilution convolution

Abstract

The invention relates to a method and a device for realizing multi-scale optical flow pixel transformation by utilizing dilution convolution. According to an embodiment, the invention provides a method for realizing multi-scale optical flow pixel transformation by using dilution convolution for a pyramid optical flow estimator. The pyramid optical flow estimator comprises at least two layers of optical flow estimators, and the method comprises: enabling each layer of optical flow estimator to use a known image frame and an upper layer of prediction image frame generated by an upper layer of optical flow estimator as input to estimate a current layer of optical flow field with a current layer of working scale; determining a current layer convolution kernel corresponding to the current layeroptical flow field; diluting the convolution kernel of the current layer to obtain a diluted convolution kernel of the current layer; and performing optical flow pixel transformation processing on the prediction image frame of the previous layer by using the dilution convolution kernel of the current layer so as to obtain the prediction image frame of the current layer, wherein the prediction image frame of the previous layer used as the input of the optical flow estimator of the first layer is zero, and the working scales of the optical flow fields of each layer are different from each other.

Description

technical field [0001] The present invention generally relates to the field of image processing, and more particularly, to a method and device for realizing multi-scale optical flow pixel transformation using dilute convolution for an optical flow field pyramid, and an electronic device. Background technique [0002] Video prediction can be widely used in various fields, for example, it can be used in assisted driving to predict the future driving environment based on the current driving environment, so as to take corresponding driving strategies in advance. A commonly used video prediction method involves the use of optical flow fields, which describe the displacement vectors of corresponding pixels between adjacent image frames in an image sequence, Figure 1A An exemplary posterior optical flow field is shown, Figure 1B An exemplary prior optical flow field is shown. like Figure 1A As shown, when the previous image frame 1 and the next image frame 2 are known, the opti...

AI Technical Summary

Problems solved by technology

In the process of scaling these images, pixels will inevitably be blurred, which will affect the prediction accuracy and is not conducive to the subsequent use of prediction results.

Images