End-to-end optical flow estimation method based on multi-stage loss

Abstract

The invention discloses an end-to-end optical flow estimation method based on multistage loss, and the method comprises the steps: sending two adjacent images into the same feature extraction convolutional neural network, carrying out the feature extraction, and obtaining a multi-scale feature map of the two frames of images; carrying out correlation analysis operation on the two image feature maps under each scale so as to obtain multi-scale loss information; combining the loss amount information obtained under the same scale, the feature map of the first frame of image under the scale and the optical flow information obtained by the previous stage of prediction together, sending the combined information into an optical flow prediction convolutional neural network, obtaining a residual flow under the scale, and adding the residual flow with an upper sampling result of the optical flow information of the previous stage to obtain the optical flow information of the scale; and carrying out feature fusion operation on the optical flow information of the second-level scale and the input two frames of images, and sending the fused information to a motion edge optimization network to obtain a final optical flow prediction result. By using the method, the optical flow estimation algorithm precision and efficiency can be improved.

Description

technical field [0001] The invention relates to the field of optical flow estimation in computer vision, in particular to end-to-end optical flow estimation, in particular to an end-to-end optical flow estimation method based on multi-level loss. Background technique [0002] Optical flow characterizes the apparent motion of image brightness patterns, which is usually caused by the relative motion of the observer and the scene. Optical flow estimation is one of the classic research topics in the field of computer vision. As a low-level visual task, optical flow is widely used in high-level visual tasks, such as video action recognition, video target detection and tracking, etc.; , a high-performance optical flow estimation algorithm is of great significance for applications based on optical flow, such as video editing, robot navigation, etc. [0003] Optical flow field is the projection of motion field on two-dimensional space, which is a low-level representation of motion....

AI Technical Summary

Problems solved by technology

Among these methods, most of them first use CNN to extract high-level semantic information of the image, and then use the traditional method based on region matching for optical flow estimation. However, these methods often require complex post-processing and occupy a lot of running time. Does not meet the speed requirements of tasks such as video detection and tracking

Images