Real-time optical flow estimation method based on lightweight convolutional neural network

Abstract

The invention discloses a real-time optical flow estimation method based on a lightweight convolutional neural network, and the method comprises the steps: giving two adjacent frames of images, and constructing a multi-scale feature pyramid with shared parameters; on the basis of the constructed feature pyramid, constructing a U-shaped network structure of a first frame of image by adopting a deconvolution operation to perform multi-scale information fusion; initializing the lowest resolution optical flow field to be zero, and performing deformation operation based on bilinear sampling on a second frame matching feature after the optical flow estimated by the second low resolution is up-sampled; carrying out local similarity calculation based on an inner product on the features of the first frame and the deformed features of the second frame, constructing a matching cost, and carrying out cost aggregation; taking the multi-scale features, the up-sampled optical flow field and the matching cost features after cost aggregation as the input of an optical flow regression network, and estimating the optical flow field under the resolution; and repeating until the optical flow field under the highest resolution is estimated. According to the invention, optical flow estimation is more accurate, and the model is lightweight, efficient, real-time and rapid.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a real-time optical flow estimation method based on a lightweight convolutional neural network. Background technique [0002] Optical flow estimation is a fundamental research task in computer vision, which is the bridge and link connecting images and videos. The core idea is to estimate the pixel-by-pixel correspondence given two frames of images before and after. This can also be approximately understood as a projected motion field of a 3D object on a 2D image plane. Optical flow plays an important role in behavior understanding, video processing, motion prediction, multi-view 3D reconstruction, autonomous driving, and real-time localization and mapping (SLAM). Therefore, how to estimate optical flow (especially dense optical flow) accurately and quickly in the field of computer vision is particularly important. [0003] The traditional optical flow estimation method...

AI Technical Summary

Problems solved by technology

However, it still relies on traditional methods and cannot perform fast inference in real time

Images