Multi-scale feature optical flow learning calculation method based on self-attention mechanism

Abstract

The invention discloses a multi-scale feature optical flow learning calculation method based on a self-attention mechanism, and the method comprises the steps: firstly selecting any two continuous frames of images in an input image sequence, carrying out the pyramid feature extraction of the selected two frames of images, and solving a sequence initial optical flow field; secondly, performing feature fusion on the initial optical flow field and the corresponding features thereof, and respectively capturing attention dependence relationships by superposing the fusion features and the features of each layer of the pyramid corresponding to the fusion features and utilizing a self-attention mechanism; after superposition of channel layers is carried out, carrying out feature extraction calculation to solve a residual optical flow field; therefore, the optical flow calculation precision of the model at the image boundary or the moving edge in the large-displacement motion state is further improved. The boundary blur phenomenon caused by large displacement motion in image sequence optical flow calculation is improved, and the method has higher calculation precision and better applicability for complex scenes and large displacement image sequences.

Description

technical field [0001] The invention relates to an image sequence optical flow calculation technology, in particular to a multi-scale feature optical flow learning calculation method based on a self-attention mechanism. Background technique [0002] Optical flow is the two-dimensional instantaneous velocity of moving objects or pixel points on the surface of the scene on the projection plane. It not only contains the motion parameters of the moving target and the scene in the image, but also carries the structural information of the target and the scene. The purpose of studying optical flow calculation is to restore the motion and structure information of target objects and scenes from image sequences, and then apply it to more advanced visual tasks. In recent years, with the rapid development of deep learning theory and technology, the convolutional neural network model has been widely used in the research of optical flow computing technology. Because this type of method ha...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a multi-scale feature optical flow learning calculation method based on the self-attention mechanism to solve the technical problem of blurring the motion boundary of the convolutional neural network model image based on the feature pyramid under large displacement motion

Images