Image local feature matching method and system

Abstract

The application discloses a kind of image local feature matching method and system, the method includes: two images to be matched input scale isomorphism main stem network, extracts coarse-grained feature map and fine-grained feature map;The coarse-grained feature map of two images to be matched is sent into information fusion module processing respectively;Feature matching is carried out using differentiable matching layer, to obtain confidence matrix, to obtain coarse matching;For each coarse matching, fine matching is carried out in the region of corresponding fine-grained feature map, to obtain final matching prediction.The application introduces scale isomorphism convolution into the current image matching framework without detector, solves the problem of image matching difficulty when there is large scale change in image, has better matching ability and matching precision, and the effect is remarkable when matching image has large scale change.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an image local feature matching method and system. BACKGROUND

[0002] Many basic computer vision tasks, such as Structure From Motion (SfM), Simultaneous localization and mapping (SLAM), relative pose estimation and visual positioning, are centered on matching two or more views of the same scene. The image matching task aims to identify and align the contents or structures with the same / similar attributes in two images at the pixel level.

[0003] Existing image matching methods can be divided into three types: Detect then Describe framework, Describe to Detect framework and Detector free framework. The Detect then Describe framework first detects the key points of the image, then generates patches around the key points, and sends the patches into the feature extraction network to extract features; the Describe to Detect framework first uses CNN to extract dense descriptors, and then determines the possibility of the descriptor as a key point based on the distinguishability of the descriptor; the Detector free framework does not need to detect the key points of the image, uses CNN or Transformer to extract feature maps of different levels, and then performs coarse-level matching according to the coarse-grained feature maps, and finally refines the matching based on the fine-grained feature maps. This method can produce repeatable dense matching because it directly calculates the similarity of all feature vectors between the feature maps of the images to be matched, which is more conducive to generating corresponding relationships in low-texture areas and repeated patterns. At present, this detector-free method is superior to the detector-based method in most cases.

[0004] However, in the image matching task, the two images to be matched in many scenes have some scale changes. Such scale changes are the natural result of the change in the distance between the object and the camera. In addition, it is well known that the feature descriptors extracted by CNN are only translation equivariant, not scale equivariant, which means that CNN lacks a specialized method for handling scale changes, which also limits the performance of CNN to some extent in handling images. Therefore, when handling image data with obvious scale changes, CNN often cannot play its due role. Therefore, how to improve the image matching ability and accuracy is a technical problem that needs to be solved.

[0005] The above merely serves to assist in understanding the technical solutions of the present application and does not represent an acknowledgement that the above is prior art. SUMMARY

[0006] The main purpose of the present application is to provide an image local feature matching method and system, aiming to better solve the problem of image matching, improve the performance of the image matching model with significant scale change, have better matching ability and matching precision, and have significant effect when matching images with large scale change.

[0007] To achieve the above purpose, the present application provides an image local feature matching method, which comprises the following steps:

[0008] Input two images to be matched into a scale-invariant backbone network, and extract coarse-grained feature maps and fine-grained feature maps;

[0009] The coarse-grained feature maps of the two images to be matched are sent into an information fusion module for processing respectively;

[0010] Feature matching is performed using a differentiable matching layer to obtain a confidence matrix, thereby obtaining coarse matching;

[0011] For each coarse matching, fine matching is performed in the corresponding region of the fine-grained feature map to obtain a final matching prediction.

[0012] Optionally, the method further comprises: establishing a scale-invariant backbone network based on a wide residual network with a feature pyramid structure and a convolution with scale invariance.

[0013] Optionally, the extraction of coarse-grained feature maps and fine-grained feature maps specifically comprises: extracting multi-level features from the two images to be matched using a scale-invariant backbone network; and determining coarse-grained feature maps and fine-grained feature maps based on the extracted features.

[0014] Optionally, the method further comprises: establishing an information fusion module comprising a self-attention layer and a cross-attention layer, wherein the information fusion module performs information fusion and transmission on the coarse-grained feature maps of the two images to be matched through the self-attention layer and the cross-attention layer.

[0015] Optionally, the information fusion module adopts a linear transformer model; the linear transformer model uses the formula sim(Q, K) = φ(Q)·φ(K) TThis is used to calculate the similarity between vectors participating in the attention layer calculation; where Q and K represent the vectors participating in the attention layer calculation, sim represents the similarity between vectors, φ = elu + 1 represents the kernel function that processes vectors using the elu function, ● represents the dot product operation of vectors, and T represents the transpose operation; this operation replaces the operation of multiplying Q and K vectors and then using the softmax function to calculate similarity in the traditional transformer model.

[0016] Optionally, a differentiable matching layer can be used for feature matching to obtain a confidence matrix, thereby obtaining a coarse match. Specifically, this includes: calculating the similarity matrix of two images to be matched, applying softmax in two dimensions to obtain the probability matrix of soft nearest neighbor matching, selecting matches with confidence scores higher than a threshold, filtering outlier coarse matches using the nearest neighbor criterion, and obtaining the final predicted coarse match.

[0017] Optionally, the loss function for the coarse matching is:

[0018]

[0019] in, This represents the correspondence generated using the mutual nearest neighbor rule for supervision through coarse truth matching. This indicates the result of a coarse match. In the confidence matrix The numerical value of the position; this formula is the minimization of Negative log-likelihood loss in the grid.

[0020] Optionally, for each coarse match, fine matching is performed in the corresponding fine-grained feature map region to obtain the final matching prediction. Specifically, this includes: for each coarse match, locating it at the corresponding position on the fine-grained feature map, cropping two sets of local windows, using the information fusion module to transform the cropped features in each window, correlating the vector of the center position of the transformed A map feature with all vectors in the B feature vector to obtain a heatmap, calculating the expectation of the probability distribution, and obtaining the final position.

[0021] Optionally, the loss function for fine matching is:

[0022]

[0023] Among them, M f Indicates the final match prediction; This indicates the correspondence between the final matching predictions; For the fine-grained matching of the truth value, || ||2 represents the L2 norm. The uncertainty of each pair of matching relationships is measured by the variance of the corresponding heatmap for each query point.

[0024] Further, in order to achieve the above object, the present application also provides an image local feature matching system, which comprises:

[0025] an extraction module, configured to input two images to be matched into a scale-invariant backbone network, and extract coarse-grained feature maps and fine-grained feature maps;

[0026] a fusion module, configured to send the coarse-grained feature maps of the two images to be matched into an information fusion module respectively for processing;

[0027] a coarse matching module, configured to perform feature matching by using a differentiable matching layer, obtain a confidence matrix, and thus obtain coarse matching;

[0028] a fine matching module, configured to perform fine matching on a region of the corresponding fine-grained feature map for each coarse matching, and obtain a final matching prediction.

[0029] Further, in order to achieve the above object, the present application also provides an image local feature matching device, which comprises a memory, a processor, and an image local feature matching program stored in the memory and executable on the processor, and the image local feature matching program implements the steps of the above image local feature matching method when executed by the processor.

[0030] Further, in order to achieve the above object, the present application also provides a storage medium, which stores an image local feature matching program, and the image local feature matching program implements the steps of the above image local feature matching method when executed by a processor.

[0031] The image local feature matching method and system provided by the embodiment of the present application comprise the following steps: inputting two images to be matched into a scale-invariant backbone network, and extracting coarse-grained feature maps and fine-grained feature maps; sending the coarse-grained feature maps of the two images to be matched into an information fusion module respectively for processing; performing feature matching by using a differentiable matching layer, obtaining a confidence matrix, and thus obtaining coarse matching; and performing fine matching on a region of the corresponding fine-grained feature map for each coarse matching, and obtaining a final matching prediction. The present application introduces scale-invariant convolution into the current image matching framework without detector, solves the problem of difficult image matching when there is large scale change in the image, has better matching ability and matching precision, and has remarkable effect when the matching image has large scale change. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The figure is a schematic diagram of the device structure of the hardware running environment related to the embodiment of the present application;

[0033] Figure 2 The figure is a flowchart of the image local feature matching method of the embodiment of the present application;

[0034] Figure 3 This is a schematic diagram illustrating the principle of image local feature matching in an embodiment of the present invention;

[0035] Figure 4 This is a model diagram of the image local feature matching backbone network in an embodiment of the present invention.

[0036] Figure 5 This is a schematic diagram comparing image matching results according to an embodiment of the present invention;

[0037] Figure 6 This is a comparison chart of pose estimation AUC metrics from embodiments of the present invention.

[0038] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0039] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0041] like Figure 1 As shown, Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention.

[0042] like Figure 1 As shown, the device may include: a processor 1001, such as a CPU; a communication bus 1002; a user interface 1003; a network interface 1004; and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0043] Those skilled in the art will understand that Figure 1The structure of the device shown in the figure does not constitute a limitation on the device, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0044] As shown in Figure 1 The memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and an image local feature matching program.

[0045] In the terminal shown in Figure 1 The network interface 1004 is mainly used for connecting the background server and communicating data with the background server; the user interface 1003 is mainly used for connecting the client (user end) and communicating data with the client; and the processor 1001 can be used to call the image local feature matching program stored in the memory 1005 and perform the following operations:

[0046] The two images to be matched are input into the scale-invariant backbone network to extract coarse-grained feature maps and fine-grained feature maps;

[0047] The coarse-grained feature maps of the two images to be matched are sent into the information fusion module for processing respectively;

[0048] Feature matching is performed using the differentiable matching layer to obtain a confidence matrix, thereby obtaining coarse matching;

[0049] For each coarse matching, fine matching is performed in the corresponding region of the fine-grained feature map to obtain a final matching prediction.

[0050] The specific embodiments of the application applied to the device are basically the same as the following embodiments of the image local feature matching method, and are not repeated here.

[0051] The embodiment of the application provides an image local feature matching method, referring to Figure 2 and Figure 3 , Figure 2 The flowchart of the embodiment of the image local feature matching method of the application, Figure 3 The principle diagram of the image local feature matching of the application.

[0052] In this embodiment, the image local feature matching method includes the following steps:

[0053] S1: input two images to be matched into a scale-invariant backbone network to extract coarse-grained and fine-grained feature maps;

[0054] Specifically, S1 includes:

[0055] S11: refer to Figure 4The application designs a scale-invariant version of WideResnet network to extract scale-invariant features. The main design ideas mainly include the following points:

[0056] The most advanced image dense matching method does not perform very well in the case of large scale change of the image to be matched. The method proposed in SESN is selected to construct a controllable neural network that is scale-invariant: a controllable convolution kernel is used to construct a scale-invariant convolution layer, and other common block operations are generalized to be scale-invariant. This method is scale-invariant for any discrete scale transformation factor. A modified version of WideResNet is used to replace the CNN backbone. Compared with the basic block of WideResNet and our model, we use the Conv H→H proposed in SESN to replace the ordinary convolution layer.

[0057] S12: Use the scale-invariant version of the WideResNet structure with the FPN structure to extract multi-level features from two images and use features of two granularities. The feature map of 1 / 8 original image size is defined as a coarse-grained feature map, and the feature map of 1 / 2 original image size is defined as a fine-grained feature map. Specifically, the network receives data with a size of (640, 480, 3) during actual inference, and the ConvZ2H layer of the designed scale-invariant CNN feature extraction network is improved to four dimensions, that is, the feature map at this time already contains the information of the scale dimension. After 3 blocks, the size of the feature map gradually decreases, and the number of channels gradually increases, and the backbone network can calculate the feature map of the image under different scales. Finally, a coarse-grained feature with a length of 80, a width of 60, and a channel number of 256 and a fine-grained feature map with a length of 320, a width of 240, and a channel number of 128 are obtained, and the Dropout operation is used to prevent network overfitting. The coarse-grained and fine-grained feature maps at this time have invariance under a certain range of scale changes.

[0058] S2: The coarse-grained features of the two images are respectively sent into the information fusion module for processing. The attention mechanism expression of the traditional Transformer is as follows: Attention(Q, K, V) = softmax(QK TV. The information fusion module is composed of self-attention layers and cross-attention layers. Specifically, the information fusion module includes a stack of attention modules of the NC layer, each of which first performs self-attention and then performs cross-attention to complete the information transmission between the features of the two images. Among them, the self-attention performs information transmission interaction within the image, and the cross-attention performs information transmission interaction between images. In order to reduce the computational complexity, the traditional transformer is replaced by a linear transformer. The linear transformer uses sim(Q,K) = φ(Q) · φ(K) T , where φ = elu + 1 instead of the original Q, K matrix multiplication followed by the softmax operation. In this way, the computational complexity is reduced from O(N2) to O(N). After processing by the information fusion module, more easily matched image features can be obtained.

[0059] S3: Perform feature matching using the differentiable matching layer to obtain a confidence matrix, thereby obtaining a coarse match;

[0060] First, the cosine similarity matrix of the two images to be matched is calculated, and then the softmax operation is applied in both dimensions (referred to as dual-softmax) to obtain a probability matrix of soft mutual nearest neighbor matching. Each row or column in the matrix represents the probability of matching a pixel in image A or B with all pixels in the corresponding image B or A. Based on this matrix, matches with a confidence higher than a certain threshold are selected, and further filtered using the mutual nearest neighbor (MNN) criterion to filter possible outliers of coarse matches. Thus, the final predicted coarse match is obtained.

[0061] where the loss function Loss designed for coarse matching is:

[0062]

[0063] where, denotes the corresponding relationship of the true value coarse match generated using the mutual nearest neighbor rule for supervision, denotes the matching result of the coarse match, denotes the value of the position in the confidence matrix; this formula is the negative log-likelihood loss in the grid to be minimized.

[0064] S4: For each coarse match, perform refinement matching in the corresponding fine feature map area to obtain the final matching prediction, which includes the following steps:

[0065] ​​S41: For each coarse match, a correlation-based method is used to refine the match. Specifically, it is first positioned to the corresponding position on the fine feature map, and then two sets of local windows are cropped. After that, the cropped features in each window are transformed by the information fusion module. After that, the vector of the center position of the transformed A feature is correlated with all the vectors in the B feature vector (matrix multiplication and softmax), and a heatmap is obtained. The heatmap represents the matching probability of each pixel in the neighborhood of ^j with ^i. Then the expectation of the probability distribution is calculated to obtain the final position, as shown in Figure 5

[0066] where the loss function Loss of the fine match is designed as:

[0067]

[0068] where M f represents the final matching prediction; represents the corresponding relationship of the final matching prediction; is the true value of the fine matching corresponding position, and || ||2 represents the l2 norm, is the variance of the corresponding heat map for each query point, which measures the uncertainty of each matching relationship.

[0069] S42: An evaluation index is designed for objective evaluation.

[0070] The pose estimation task is used as a test task to evaluate the performance of the model. The AUC of the error of the calculated camera pose at the threshold (5°, 10°, 20°) is calculated:

[0071]

[0072] where, represents the serial number of the i-th predicted sample (the probability score is arranged from small to large); M and N are the number of positive and negative samples, respectively; represents only the serial number of the positive sample is added up. The pose error is defined as the maximum value of the angle error in rotation and translation. The fundamental matrix is solved from the predicted matching by RANSAC to recover the camera pose.

[0073] In another embodiment, under the condition that other objective environments are the same, a comparative experiment is conducted with the image matching method LoFTR without detector, so as to verify the image matching effect of the method designed in the application.

[0074] ​The Megadepth dataset is an outdoor scene dataset using network pictures, and its test set (10 scenes) is divided into four subsets according to the size of the overlap rate (referring to the co-visible region of two images) of the image pairs [1, 2], [2, 3], [3, 4], [4, +∞], wherein the last three subsets are merged into [2, +∞] and regarded as a set of image pairs with significant scale changes. The three subsets [2, 3], [3, 4], [4, +∞] in the Megadepth dataset are used as test sets in this application to detect the robustness of the model to image scale changes in three scale changes. The AUC value at the threshold (5°, 10°, 20°) is used to evaluate the quality of the network in the image matching task.

[0075] As shown in Figure 6 , using the objective evaluation index AUC described in S4, the image matching method of the present application is compared with the classical LoFTR, and the AUC value of the present application is higher in each scale change tested, indicating that the image matching accuracy of the method designed in the present application is very high, and the effect is very ideal.

[0076] In this embodiment, a key point structure body is provided to determine the winding point by using the position information and sequence information of the key points in the key point structure body, and then to draw the target winding pattern. By obtaining the target pattern in the original winding layout data file and / or standard pattern file, the information of the key points in the target pattern is extracted to obtain the callable key point structure body, so as to call the key point structure body to draw the target winding layout. The portability of the winding layout data file and the calling characteristics of the key point structure body are utilized in this embodiment to improve the drawing efficiency of the winding layout.

[0077] In another embodiment, an image local feature matching system embodiment is also provided, which comprises:

[0078] The extraction module is configured to input two images to be matched into the scale-invariant backbone network to extract coarse-grained feature maps and fine-grained feature maps.

[0079] The fusion module is configured to send the coarse-grained feature maps of the two images to be matched into the information fusion module for processing, respectively.

[0080] The coarse matching module is configured to perform feature matching by using the differentiable matching layer to obtain a confidence matrix, so as to obtain coarse matching.

[0081] The fine matching module is configured to perform fine matching on the corresponding region of the fine-grained feature map for each coarse matching to obtain a final matching prediction.

[0082] Other embodiments or specific implementations of the image local feature matching system of the present application can refer to the above-mentioned method embodiments, and will not be described here again.

[0083] In addition, the present application also provides an image local feature matching device, characterized in that the image local feature matching device comprises a memory, a processor, and an image local feature matching program stored in the memory and executable on the processor, wherein the image local feature matching program is executed by the processor to implement the image local feature matching method of each embodiment of the present application.

[0084] The specific implementation of the image local feature matching device of the present application is basically the same as the above-mentioned image local feature matching method embodiments, and will not be described here again.

[0085] In addition, the present application also provides a readable storage medium, which comprises a computer readable storage medium, and an image local feature matching program is stored on the computer readable storage medium. The readable storage medium can be a memory 1005 in a terminal, and can also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The readable storage medium comprises a plurality of instructions for causing an image local feature matching device with a processor to execute the image local feature matching method of each embodiment of the present application. Figure 1

[0086] The specific implementation of the image local feature matching program in the readable storage medium of the present application is basically the same as the above-mentioned image local feature matching method embodiments, and will not be described here again.

[0087] It can be understood that, in the description of the present application, the description of the terms "an embodiment", "another embodiment", "other embodiments", or "the first embodiment to the Nth embodiment" means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are contained in at least one embodiment or example of the present application. In the present application, the illustrative description of the above-mentioned terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0088] ​It should be noted that, in this document, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0089] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0090] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and the necessary general hardware platform, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) as described above, and includes a number of instructions for making a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in the various embodiments of the present application.

[0091] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, which is made by using the content of the specification and drawings of the present application, is also included in the patent protection scope of the present application.

Claims

1. An image local feature matching method, characterized in that, The method includes the following steps: The two images to be matched are input into a scale-variable backbone network to extract coarse-grained feature maps and fine-grained feature maps. The coarse-grained feature maps of the two images to be matched are respectively sent to the information fusion module for processing; Feature matching is performed using a differentiable matching layer to obtain a confidence matrix, thereby obtaining a coarse match; For each coarse match, a fine match is performed in the corresponding fine-grained feature map region to obtain the final match prediction; The method further includes: establishing a scale-equivariant backbone network based on a wide residual network with a feature pyramid structure and scale-equivariant convolutions; constructing scale-equivariant convolutional layers using controllable convolutional kernels; and replacing the CNN backbone with a modified version of WideResNet; wherein the modified version of WideResNet is configured to use the ConvH2H convolution operator in the SESN scale-equivariant guided network to replace ordinary convolutional layers to obtain the WideResNet network structure. The information fusion module consists of a self-attention layer and a cross-attention layer. The information fusion module includes a stack of attention modules in the NC layer. Each attention module first performs self-attention and then cross-attention to complete the information transfer between the features of the two images. Self-attention performs information transfer and interaction within the image, while cross-attention performs information transfer and interaction between the images.

2. The image local feature matching method as described in claim 1, characterized in that, Extracting coarse-grained and fine-grained feature maps specifically includes: using a scale-variable backbone network to extract multi-level features from two matched images; and determining coarse-grained and fine-grained feature maps based on the extracted features.

3. The image local feature matching method as described in claim 1, characterized in that, The information fusion module employs a linear transformer model; the linear transformer model uses the formula... This is used to calculate the similarity between vectors participating in the attention layer computation; where Q,K represent the vectors participating in the attention layer computation, and sim represents the similarity between the vectors. This indicates a kernel function that uses ELU functions to process vectors. The dot product operation represents vectors; T represents the transpose operation; this operation replaces the operation of multiplying Q and K vectors in the traditional transformer model and then using the softmax function to calculate similarity.

4. The image local feature matching method as described in claim 1, characterized in that, Feature matching is performed using a differentiable matching layer to obtain the confidence matrix. The process involves: calculating the similarity matrix of the two images to be matched; applying softmax in two dimensions to obtain the probability matrix of the nearest neighbor matching; selecting matches with confidence scores higher than a threshold; filtering outliers using the nearest neighbor criterion; and finally obtaining the predicted coarse match.

5. The image local feature matching method as described in claim 4, characterized in that, The loss function for the coarse matching is: 、 in, This represents the correspondence generated using the mutual nearest neighbor rule for supervision, based on coarse truth matching. This indicates the result of a coarse match. In the confidence matrix The numerical value of the position; this formula is the minimization of Negative log-likelihood loss in the grid.

6. The image local feature matching method as described in claim 1, characterized in that, For each coarse match, a fine match is performed in the corresponding fine-grained feature map region to obtain the final match prediction. Specifically, this includes: for each coarse match, locating it to the corresponding position on the fine-grained feature map, cropping two sets of local windows, using the information fusion module to transform the cropped features within each window, correlating the vector of the center position of the transformed A map features with all vectors in the B map feature vector to obtain a heatmap, calculating the expectation of the probability distribution, and obtaining the final position.

7. The image local feature matching method as described in claim 6, characterized in that, The loss function for fine matching is: 、 in, Indicates the final match prediction; This indicates the correspondence between the final matching predictions; For the fine-grained matching of the true value, the corresponding position is... Describing the l2 norm, The uncertainty of each pair of matching relationships is measured by the variance of the corresponding heatmap for each query point.

8. An image local feature matching system, characterized in that, The system is used to perform the image local feature matching method as described in any one of claims 1-7, the system comprising: The extraction module is used to input two images to be matched into a scale-variable backbone network to extract coarse-grained feature maps and fine-grained feature maps; The fusion module is used to send the coarse-grained feature maps of the two images to be matched into the information fusion module for processing. The coarse matching module is used to perform feature matching using a differentiable matching layer to obtain a confidence matrix, thereby obtaining a coarse match. The fine-matching module is used to perform fine-matching in the corresponding fine-grained feature map region for each coarse match, so as to obtain the final matching prediction.

Images