Image semantic feature matching method based on geometric consistency

Abstract

The invention discloses an image semantic feature matching method based on geometric consistency. The method comprises the steps of semantic feature extraction, feature matching initialization, feature matching positioning optimization, image semantic alignment and the like, wherein the semantic feature extraction is to extract high-level semantic features by using a convolutional neural network to construct a five-layer semantic feature pyramid; the feature matching initialization is to design a semantic feature matching constraint rule on the top layer of a feature pyramid based on geometricconsistency, and construct an energy function; the feature matching positioning optimization is used for improving the positioning precision of feature matching, and the accuracy of feature matchingpairs is improved layer by layer through a pyramid back propagation algorithm; and finally, geometric transformation model parameters between the to-be-matched images is estimated by adopting a localgeometric transformation model, and image deformation is performed to realize image semantic alignment. The method can improve the precision of semantic feature matching, and achieves the alignment ofthe geometric attitude and orientation of the foreground target.

Description

technical field [0001] The present invention relates to image feature matching technology, more specifically, relates to a method for image semantic feature matching, which establishes a point-to-point feature matching relationship between different target images with the same attribute category label, and belongs to the field of digital image processing and computer vision . Background technique [0002] Image feature matching refers to retrieving salient feature information in images in the same or similar scene of image content information, and using feature descriptors to quantify the feature information, and then determining the features between images according to the degree of similarity between feature descriptors. Point-to-point matching relationship with features. Image feature matching plays a very important role in computer vision fields such as image stitching, 3D reconstruction, and SLAM. The present invention relates to image semantic feature matching, which...

AI Technical Summary

Problems solved by technology

However, the descriptor used in this method still focuses on the color, gradient and other information of the low-level texture, which has low robustness, and needs to use a preprocessing algorithm to extract data blocks containing foreground target information. highly dependent

In addition, it still uses optical flow for the alignment model, and the distortion and distortion are serious when the image is deformed, and the alignment effect is poor.

Jing Liao et al. published the paper "Visual Attribute Transfer through Deep Image Analogy" (ACM Transactions on Graphics, 36(4):1–15, 2017), which uses convolutional neural networks to extract semantic features and feature descriptors, but is directly based on feature descriptions The feature descriptors are used for nearest neighbor matching, and the dependence on the feature descriptors is too high, and the obtained feature descriptor information is relatively redundant, and there is no quantification mechanism for significant features, so that more mismatches are generated.

In addition, it uses the semantic feature matching results to directly calculate the global homography matrix model with 8 degrees of freedom parameters between the images to be matched, the alignment ability is poor, and the local correlation of the images is not fully utilized

[0004] Analysis of the existing related semantic matching technology found that the image semantic matching technology is not mature enough, and there is still a lot of room for improvement. There are still the following challenges: 1) When feature extraction, the existing methods still mainly use traditional SIFT and other methods. The technical route of artificial features pays too much attention to the texture information such as the color and gradient of the local neighborhood, and the discrimination of the feature descriptor is not high enough; 2) The feature descriptor information is relatively redundant, and the existing methods directly rely on the feature descriptor for similarity estimation , the mismatching is high, and the geometric information between the images to be matched is not fully utilized; 3) When the images are aligned, the existing methods directly use the optical flow or the global homography matrix model, the distortion and distortion caused by the image deformation are more obvious, and the alignment effect Difference

Images