Image matching method based on self-attention deep neural network

Abstract

The invention relates to an image matching method based on a self-attention deep neural network. The method comprises the following steps: firstly, constructing a data set and carrying out feature enhancement on data in the data set; for input features, firstly using a Point CN module for extracting initial global features, downsampling through a differentiable pooling layer, then transmitting theinitial global features to a sequential awareness network to better learn global information, and then transmitting the initial global features to a differentiable unpooling layer to carry out upsampling to obtain better global information; and transmitting the initial global features extracted by the PointCN module to a self-attention layer for operation, so that enhanced feature information canbe obtained; carrying out splicing processing on the obtained better global information, and obtaining a preliminary prediction result through the PointCN module; and calculating the preliminary prediction result through a weighted 8-point algorithm to obtain an essential matrix. The network can replace RANSAC (Random Sample Consensus) to carry out post-processing on matching points extracted bySIFT (Scale Invariant Feature Transform) algorithms so as to improve the matching precision.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to an image matching method based on a self-attention deep neural network. Background technique [0002] Image matching is an important research field in computer vision. It is widely used in preprocessing in many fields, such as target recognition, target tracking, super-resolution image reconstruction, visual navigation, image stitching, 3D reconstruction, visual positioning, scene depth calculation, etc. It mainly consists of two parts, constructing initial matching pairs and removing false matches. [0003] Among the existing image matching methods, we can divide them into parametric methods, non-parametric methods and learning-based methods. Parameter-based methods are popular strategies for solving matching problems, such as RANSAC and its variants: PROSAC and USAC. Specifically, it first randomizes the minimum subset sampling of the dataset, generates a homography ...

AI Technical Summary

Problems solved by technology

However, these methods have two fundamental drawbacks: 1) they (parametric-based methods) cannot work effectively when the ratio of outliers to the total matching is low; 2) they cannot express complex models with non-parametric methods to mine Local information for corresponding selection of feature pairs

The network model of MooYi et al. uses context normalization to capture global context information and embed context information into nodes, but because context normalization is easily affected by other matching pairs, the robustness is not high

Although the method based on deep learning has been able to achieve good results on various data sets, the spatial information corresponding to the local feature points in the network layer is often ignored. Making good use of local and global relationships will effectively improve the model matching accuracy. Therefore, How to make good use of the relationship between global context and local context is an important and challenging task

Images