Multi-scale feature optical flow learning calculation method based on self-attention mechanism

A technology of multi-scale features and calculation methods, which is applied in computing, image data processing, instruments, etc., can solve problems such as motion boundary blur, and achieve the effect of improving the boundary blur phenomenon

Active Publication Date: 2020-06-26
NANCHANG HANGKONG UNIVERSITY
View PDF10 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a multi-scale feature optical flow learning calculation method based on the self-attention mechanism to...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-scale feature optical flow learning calculation method based on self-attention mechanism
  • Multi-scale feature optical flow learning calculation method based on self-attention mechanism
  • Multi-scale feature optical flow learning calculation method based on self-attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] The present invention will be further described below in conjunction with drawings and embodiments. see Figure 1a to Figure 5 , a multi-scale feature optical flow learning calculation method based on the self-attention mechanism, using the Temple3 image sequence optical flow calculation experiment to illustrate:

[0030] 1) input Figure 1a and Figure 1b It is two consecutive frames of images in the Temple3 image sequence; where: Figure 1a is the first frame image, Figure 1b is the second frame image, corresponding to a resolution of 448×512;

[0031] 2) Carry out K=6 layer pyramid feature extraction to the selected two frames of images, and solve the initial optical flow field of the sequence, the initial optical flow field is as follows figure 2 shown;

[0032] 3) Use 3×3 convolution operation to perform feature fusion on the initial optical flow field and its corresponding features, and down-sample the fused features to match the size of the optical flow fea...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a multi-scale feature optical flow learning calculation method based on a self-attention mechanism, and the method comprises the steps: firstly selecting any two continuous frames of images in an input image sequence, carrying out the pyramid feature extraction of the selected two frames of images, and solving a sequence initial optical flow field; secondly, performing feature fusion on the initial optical flow field and the corresponding features thereof, and respectively capturing attention dependence relationships by superposing the fusion features and the features of each layer of the pyramid corresponding to the fusion features and utilizing a self-attention mechanism; after superposition of channel layers is carried out, carrying out feature extraction calculation to solve a residual optical flow field; therefore, the optical flow calculation precision of the model at the image boundary or the moving edge in the large-displacement motion state is further improved. The boundary blur phenomenon caused by large displacement motion in image sequence optical flow calculation is improved, and the method has higher calculation precision and better applicability for complex scenes and large displacement image sequences.

Description

technical field [0001] The invention relates to an image sequence optical flow calculation technology, in particular to a multi-scale feature optical flow learning calculation method based on a self-attention mechanism. Background technique [0002] Optical flow is the two-dimensional instantaneous velocity of moving objects or pixel points on the surface of the scene on the projection plane. It not only contains the motion parameters of the moving target and the scene in the image, but also carries the structural information of the target and the scene. The purpose of studying optical flow calculation is to restore the motion and structure information of target objects and scenes from image sequences, and then apply it to more advanced visual tasks. In recent years, with the rapid development of deep learning theory and technology, the convolutional neural network model has been widely used in the research of optical flow computing technology. Because this type of method ha...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T7/246G06T7/269
CPCG06T7/246G06T7/269G06T2207/10016G06T2207/20084G06T2207/20081G06T2207/20016
Inventor 张聪炫周仲凯陈震黎明江少锋
Owner NANCHANG HANGKONG UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap