An image matching method based on deep learning
A technology of deep learning and matching method, which is applied in the field of image matching based on deep learning, and can solve the problem of limited computing cost, sensitivity to texture features, and incompatibility between computing cost and matching accuracy, observable object motion freedom and matching pixel density, etc. problem, to achieve the effect of increasing the search range and fast dense matching
Active Publication Date: 2022-07-26
郑健青
View PDF4 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Deep learning can usually achieve fast feature point matching, but limited by the computational cost brought by high-resolution and large-scale search space, existing deep learning methods cannot achieve two large-scale or non-parametric coordinate transformations. Fast Dense Matching Between Images
[0006] Some matching methods use parametric coordinate transformation, observed objects, scenes, and prior conditions of the shooting process to limit the search range, such as patent CN102654902, which uses the translation and rotation information transmitted from top to bottom of image pyramids of different scales to achieve fast matching; such as ECCV in 2018 The cost body (Cost Volume) adopted by MVSNet included in the conference to achieve stereo matching not only requires camera calibration based on homography but also limits the range of parallax; for example, patent CN201180057099.3 is also limited based on the homography relationship in stereoscopic view projection search range
But the acquisition of these prior information also increases the cost or limits the usage scenarios
[0007] Therefore, the current dense image matching method is usually based on the consistency constraints of the pixel correspondence between the two images in the neighborhood, using a large range of low-resolution scale matching to limit the search range for high-resolution scale matching, and through high-resolution Resolution scale matching refines the matching results, such as matching the pixel correspondence between two frames of images based on the pyramid-layered Horn–Schunck optical flow method. However, this method 1) requires iterative optimization during the application process, so it is time-consuming. 2) Assume that the luminosity is constant and iteratively optimize through this assumption, without considering the semantic characteristics of pixels, resulting in sensitivity to the light source, object material, and texture features of the object. 3) The large-scale movement of dense small objects in the image will destroy the neighbor Intra-domain consistency assumption, so it is difficult to fully express the movement of multiple small objects in different directions densely in the low-resolution scale optical flow map, which leads to the loss of pixel correspondence in the matching process. 4) For different resolution scales The matching effect is not ideal; the FlowNet method published by Philipp Fischer et al. at the 2015 ICCV conference, based on the convolutional neural network of the encoder-decoder, additionally includes the correlation coefficient of the pixel-by-pixel calculation of the feature vector between the two images, Eddy Ilg proposed FlowNet2 at the CVPR conference in 2016, which builds an end-to-end learning optical flow estimation model by stacking multiple convolutional neural networks based on the encoder-decoder structure, in which each encoder-decoder network The predicted optical flow transforms the image and then enters the next encoder-decoder network for more refined matching, while the first encoder-decoder network follows the above FlowNet structure to achieve pixel correspondence in the global range of the image Relational search, these two methods based on deep learning can better solve the problems in the previous traditional image matching methods, but among them: 1) Calculating the correlation coefficient pixel by pixel between two images requires the computational complexity of the square of the number of pixels in the original image, 2) The input of the stacked encoder-decoder network is used to predict the optical flow close to the resolution of the original image, so that the convolutional neural network needs a larger effective receptive field (Receptive Field) for searching for pixel correspondences with larger spatial distances. ) requires a large amount of calculation, so it is difficult to apply to high-definition and high-dimensional image matching due to the limitation of hardware equipment
[0008] To sum up, in the existing image matching methods, the calculation cost and matching accuracy, the degree of freedom of motion of the observable object, and the density of matching pixels are generally incompatible. From the perspective of practicability and economy, they are not ideal.
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0050] The present invention will be further described below with reference to the accompanying drawings and typical embodiments.
[0051] exist Figure 1 to Figure 6 In the present invention, an image matching method based on deep learning is constructed by building a deep learning model including a feature extraction module 1, a feature fusion module 2 and a feature matching module 3, and by extracting and fusing different resolution features. Resolution fusion feature map, from large search range to small search range, from low computational density to high computational density, fine matching and resampling iteration, and then through the loss function set as needed to optimize the learning model parameters, It is realized by outputting the optimized model parameters and their matching results. The computational density is defined as the number of pixel connections within the search range of the input feature map corresponding to each pixel in the deep learning neural netw...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to an image matching method based on deep learning, which is to obtain a high-resolution fusion feature map by building a deep learning model including a feature extraction module, a feature fusion module and a feature matching module, and fusing different resolution features therein. Combined with the refined matching and resampling iteration of the neural network layer model with the spatially spaced connection structure, the matching search range can be increased without increasing the computational complexity, and the learning model parameters are adjusted based on the loss function set on demand. Optimize, and finally output the optimized model parameters and their matching results. Since the degree of freedom of high-resolution pixels in the matching process is preserved, it is easier to obtain the pixel correspondence of scale-transformed objects, so as to ensure the reliability of each estimated pixel correspondence, which can not only assist in the calculation of pixel correspondences in different layers Fusion can also be used to interpolate or adjust pixels for which the corresponding relationship cannot be found correctly based on the matching results of the neighborhood.
Description
technical field [0001] The invention relates to the technical field of image data processing, in particular to an image matching method based on deep learning. Background technique [0002] In recent years, with the increasing level of science and technology, the pattern of automation and intelligence in various industries is increasingly formed, and the artificial intelligence technology that follows is booming. The main purpose is to make machines and computers perceive, understand and act like humans. Among them, visual perception, as one of the most important perception technologies, occupies a pivotal position under the artificial intelligence boom and promotes the rapid development of computer vision technology. At the same time, how to understand the differences and connections between multiple visual objects, and how to process the perceived information according to specific needs has become one of the research hotspots in the field of computer vision, and image matc...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & AuthorityPatents(China)
IPC IPC(8): G06V10/75G06V10/80G06V10/82G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06N3/045G06F18/22G06F18/25
Inventor郑健青黄保茹
Owner郑健青