Deep learning image feature matching method and system based on preference guided filtering

Abstract

The present invention relates to a deep learning image feature matching method based on preference guided filtering, comprising the following steps: Step 1: for a given image pair I and I' , using SIFT algorithm to establish the initial feature matching set S ; Step 2: Use the preference-guided filtering network to remove mismatches and obtain the probability set of matching as internal points W ; Step 3: The probability set obtained in step 2 W Considered as a weight set, the essential matrix is ​​estimated by the weighted eight-point algorithm; Step 4: Recover the accurate relative posture according to the essential matrix obtained in step 3. The method and system are conducive to improving the accuracy of image feature matching.

Description

Technical Field

[0001] The present invention belongs to the technical field of image matching, and in particular relates to a deep learning image feature matching method and system based on preference guided filtering. Background Art

[0002] Image feature matching is a basic and critical step in computer vision. Applications such as face detection, structure from motion, image stitching, simultaneous localization and mapping, and 3D reconstruction all require stable and reliable image feature matching technology as a prerequisite and guarantee. It is also an important topic in the fields of pattern recognition, image analysis, security, and remote sensing. Usually, image matching includes three steps, namely, feature point extraction and description, establishing an initial matching set, and removing false matches. Due to large perspective changes, illumination changes, and severe occlusion in image pairs, the initial feature matching set contains a large number of outliers. As a key post-processing step, false match removal can retain the correct matches (inliers) from the initial feature matching set and remove outliers to improve the matching accuracy. Therefore, studying an image matching technology with high accuracy, high efficiency, and high robustness has important and extensive theoretical significance and application value.

[0003] Image feature matching is the process of extracting feature points with the same structural content in two matching images and correctly matching them. Generally, image feature matching methods can be divided into three categories according to the problem-solving ideas, namely direct matching strategy, indirect matching strategy, and deep learning strategy. The central idea of ​​the direct matching strategy is to extract image feature matching into the problem of correctly matching two feature point sets, and usually use an assignment matrix to indicate the possibility of matching between points. The main methods of this strategy are the corresponding matrix estimation algorithm and the graph matching algorithm. The biggest difference between the indirect matching strategy and the direct matching strategy is the use of local feature descriptors, such as scale-invariant feature transform (SIFT) and speeded up robust features (SURF). After the initial feature matching can be established using the similarity criterion of local feature descriptors, the geometric relationship between feature matching is used to eliminate false matching, such as random sampling consensus algorithm (RANSAC) and local preserving matching (LPM). In recent years, due to the powerful feature acquisition and representation capabilities of deep learning, deep learning strategies have become popular in image feature matching and have achieved good results on large data sets. For example, LFGC uses the rough matches estimated by feature learning methods as network input, uses multi-layer perceptrons (MLPs) and regularization to process disordered matching data, and constructs a corresponding loss function to optimize the neural network, and finally outputs each rough match as the possible probability of a correct match, thereby achieving the purpose of removing false matches. OANet captures the local context information of the match (the potential relationship between matches) through differentiable pooling and inverse pooling operations. NM-Net designs a specific compatible neighbor algorithm based on affine information to obtain more reliable local context. However, the initial matches estimated by existing feature methods usually contain a large number of false matches, especially for some challenging data sets. These false matches hinder the network from capturing accurate and sufficient context information for network learning. At the same time, these methods use MLP s The regularization and regularization operations treat each match indiscriminately, so they cannot effectively avoid the negative impact of wrong matches, which will lead to suboptimal performance of the network. Therefore, how to avoid the negative impact of outliers during the network learning stage and discriminate each feature match is a challenging problem, and further analysis and processing of it will help improve image matching accuracy. Summary of the invention

[0004] The object of the present invention is to provide a deep learning image feature matching method and system based on preference guided filtering, which method and system are conducive to improving the accuracy of image feature matching.

[0005] To achieve the above object, the technical solution adopted by the present invention is: a deep learning image feature matching method based on preference guided filtering, comprising the following steps:

[0006] Step 1: For a given pair of images I and I′, the SIFT algorithm is used to establish an initial feature matching set S;

[0007] Step 2: Use the preference-guided filtering network to remove false matches and obtain the probability set W of matching as an internal point;

[0008] Step 3: Treat the probability set W obtained in step 2 as a weight set and estimate the essential matrix using the weighted eight-point algorithm

[0009] Step 4: Restore the accurate relative posture based on the essential matrix obtained in step 3.

[0010] Further, step 1 includes the following steps:

[0011] Step 1.1: For two given image pairs I and I′ with the same or similar scenes, extract the key point coordinates and corresponding feature descriptors in the image pairs I and I′ using the SIFT algorithm:

[0012] K, D, K′, D′=SIFT(I, I′)

[0013] Among them, K and K′ represent the key point coordinates of the two images respectively, D and D′ are their corresponding feature descriptors, and SIFT() represents the SIFT algorithm;

[0014] Step 1.2: According to the nearest neighbor similarity principle of feature descriptors, establish the initial feature matching set S:

[0015]

[0016] Where NCC() represents the nearest neighbor search algorithm; i represents the i-th match, N represents the number of matches, represents the dimension of the feature matching set, where 4 represents the coordinate dimension, k = (u i , v i ) and k′=(u′ i , v′ i ) represent s i Normalized keypoint coordinates in image pair I and I′.

[0017] Further, step 2 includes the following steps:

[0018] Step 2.1: Take the initial feature matching set S as the input of the preference guided filtering network and process it using a grouped residual attention module GRA to obtain the preference score of the initial feature matching.

[0019] Step 2.2: For the initial feature matching preference score ps1 obtained, extract the previous matches, and obtain a candidate matching set in The candidate matching set As the input of the next GRA module, it is processed by the next GRA module to obtain the preference score of the candidate match.

[0020] Step 2.3: Add the preference scores ps1 and ps2 to obtain the preference score ps used by the preference-guided filtering network for matching classification, and calculate the corresponding inlier probability set.

[0021] Furthermore, the GRA module includes two multilayer perceptrons, six GRA blocks and a sequentially available block OA; the input is dimensionally upgraded through a multilayer perceptron to obtain features The obtained features are then processed by three GRA blocks, one OA block, and three GRA blocks in sequence. The output of the last GRA block is reduced in dimension by another multi-layer perceptron, and finally the preference score is output;

[0022] The processing of GRA blocks includes the following steps:

[0023] Step 2.1.1: For the input feature map Where C and N are the number of channels and matches respectively, and the feature map is evenly divided into g groups along its channel dimension, denoted by x i , i∈{1, 2, ..., g}; each feature map subset Have the same channel and space size;

[0024] Step 2.1.2: Apply spatial attention to the first feature map subset x1 to generate spatial relationships; use average pooling and maximum pooling operations along its channel dimension to aggregate the channel information of feature map subset x1, and then concatenate them to generate an efficient spatial feature The formula is as follows:

[0025] F s =concat(Avg Pool0(x1), Max Pool0(x1))

[0026] Among them, concat() represents the concatenation operation along the channel dimension; AvgPool0() and MaxPool0() represent the average pooling operation and the maximum pooling operation along the feature 0th dimension respectively;

[0027] Step 2.1.3: F sThe spatial attention weights are obtained by passing through an MLP layer, a batch normalization layer and a sigmoid activation function respectively. The formula is as follows:

[0028] w s =SA(x1)=σ(BN(MLP0(F s )))

[0029] Among them, MLP0() represents a size of The MLP layer is used to reduce the feature channel dimension to 1, BN() represents the batch normalization layer, σ() is the sigmoid activation function, and SA() represents the entire spatial attention operation;

[0030] Step 2.1.4: Each feature map subset is processed by a corresponding PointCN block, denoted as PCN i (); At the same time, use the obtained spatial attention weight w s To enhance the inlier representation capability of each input feature group; in addition, in order to increase the diversity of subset output feature representation, a hierarchical residual-like connection method is used to connect all feature map subsets; use y i Indicates PCN i The output of () is as follows:

[0031]

[0032] in, Represents element-by-element multiplication; uses grouping operations to explore contextual information from different aspects;

[0033] Step 2.1.5: Connect all feature map subsets along the channel dimension to obtain the output feature y, which is formulated as follows:

[0034] y = concat(y1, y2, ..., y g )

[0035] Step 2.1.6: Explore the relationship on the channel by performing channel attention operation on the output feature y; mainly use average and maximum pooling operations to aggregate the spatial information of the output feature y, and then add them together to generate an efficient channel feature The formula is as follows:

[0036] F c =AvgPool 12 (y)+MaxPool 12 (y)

[0037] Among them, AvgPool 12 () and MaxPool 12() refer to the average pooling operation and the maximum pooling operation along the feature space dimensions, i.e., the first and second dimensions, respectively;

[0038] Step 2.1.7: Calculate channel attention weights through a bottleneck structure The formula is as follows:

[0039] w c =CA(y)=σ(BN(MLP2(ReLU(BN(MLP1(F s )))))))

[0040] Among them, MLP1() and MLP2() represent the size of and The two MLP layers are used to learn channel weights and reduce parameter overhead; CA() represents the entire channel attention operation, and the channel attention weights are used to enhance the representation ability of important channels of feature y;

[0041] Step 2.1.8: Perform element-by-element AND operation with the input feature map x to obtain the output feature Y, which can be expressed as a function as follows:

[0042]

[0043] Among them, Y is the output feature with strong representation ability obtained by the grouped residual attention operation;

[0044] Step 2.1.9: In order to increase the communication between feature map subsets and fuse their information, the channel shuffle operation proposed by ShuffleNet is introduced to obtain a new feature map Y out , which can be expressed as a function as follows:

[0045] Y out =CS(Y)

[0046] The OA block is mainly used to capture complex local contexts, which includes a DiffPool layer to aggregate feature matches, several Order-Aware filter layers to explore the global context, and a DiffUnpool layer to restore the size and order of the original matches.

[0047] Furthermore, in step 2.3, the preference scores ps1 and ps2 are added to obtain the preference score ps of the preference-guided filtering network for matching classification, and the corresponding internal point probability set is calculated, and the calculation formula is expressed as follows:

[0048] W=tanh(ReLU(ps)), ps=αps1+ps2,

[0049]

[0050] in, and denote two GRA modules with parameters θ1 and θ2 respectively; ps1 and ps2 denote the preference scores learned from the two GRA modules respectively; α is a learnable weight initialized to 0, which is used to control the influence of the first GRA module; ReLU and tanh are two activation functions used to calculate the inlier probability set W = {w1, w2, ..., w N},w i ∈[0,1), where w i Indicates the initial match s i as the possibility of an inlier; com() is a custom function used to complete and align the preference scores of the second GRA module to make it consistent with Have the same size and order.

[0051] Furthermore, in step 3, the calculation formula for predicting the essential matrix by the weighted eight-point algorithm is:

[0052]

[0053] in, represents the essential matrix predicted by the weighted eight-point algorithm, and g() represents the weighted eight-point algorithm.

[0054] The present invention also provides a deep learning image feature matching system based on preference guided filtering, comprising a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0055] Compared with the prior art, the present invention has the following beneficial effects: a deep learning image feature matching method and system based on preference-guided filtering is proposed. The method designs an iterative filtering structure, which can learn the matching preference score to guide the matching filtering strategy, thereby greatly reducing the negative impact of false matches and fully capturing accurate context information. At the same time, the method constructs a new grouped residual attention module, in which the grouped residual attention block is used as a new backbone structure of the network, and uses a series of carefully designed operations to capture sufficient context information and enhance the representation ability of correct matches and important channels. The network proposed by the method can accurately remove false matches when the initial matching set contains a large number of outliers, and estimate the corresponding relative posture, which is crucial for computer vision tasks such as motion recovery structure, image stitching, simultaneous positioning and mapping, and three-dimensional reconstruction. At the same time, compared with existing networks, significant performance improvements have been achieved on challenging outdoor and indoor datasets, reaching the current state-of-the-art level. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The figure is a schematic diagram of a method flow of an embodiment of the present invention.

[0057] Figure 2 This is a network framework diagram of an embodiment of the present invention.

[0058] Figure 3 2. It is a structural diagram of the grouped residual attention module in an embodiment of the present invention.

[0059] Figure 4 It is the structural block of the grouped residual attention in the embodiment of the present invention.

[0060] Figure 5 Schematic diagram of channel shuffling operation in an embodiment of the present invention.

[0061] Figure 6 1 is a rendering of feature matching performed on the outdoor public dataset YFCC100M in an embodiment of the present invention; wherein (a) and (b) are renderings under different matching conditions, respectively. DETAILED DESCRIPTION

[0062] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0063] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0064] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0065] like Figure 1 As shown, this embodiment provides a deep learning image feature matching method based on preference guided filtering, comprising the following steps:

[0066] Step 1: For a given pair of images I and I′, the SIFT algorithm is used to establish an initial feature matching set S.

[0067] Step 2: Use the preference-guided filtering network to eliminate false matches and obtain the probability set W of matching as internal points.

[0068] Step 3: Treat the probability set W obtained in step 2 as a weight set and estimate the essential matrix using the weighted eight-point algorithm

[0069] Step 4: Restore the accurate relative posture (rotation and translation) based on the essential matrix obtained in step 3.

[0070] In this embodiment, step 1 includes the following steps:

[0071] Step 1.1: For two given image pairs I and I′ with the same or similar scenes, extract the key point coordinates and corresponding feature descriptors in the image pairs I and I′ using the SIFT algorithm:

[0072] K, D, K′, D′=SIFT(I, I′)

[0073] Among them, K and K′ represent the key point coordinates of the two images respectively, D and D′ are their corresponding feature descriptors, and SIFT() represents the SIFT algorithm.

[0074] Step 1.2: According to the nearest neighbor similarity principle of feature descriptors, establish the initial feature matching set S:

[0075]

[0076] Where NCC() represents the nearest neighbor search algorithm; i represents the i-th match, N represents the number of matches, represents the dimension of the feature matching set, where 4 represents the coordinate dimension, k = (u i , v i ) and k′=(u′ i , v′ i ) represent s i Normalized keypoint coordinates in image pair I and I′.

[0077] like Figure 2 As shown, step 2 specifically includes the following steps:

[0078] Step 2.1: Take the obtained N×4 initial feature matching set S as the input of the preference guided filtering network, and use a grouped residual attention module GRA to process it to obtain the preference score of the initial feature matching

[0079] Step 2.2: For the initial feature matching preference score ps1 obtained, extract the previous matches, and obtain a candidate matching set in The candidate matching set As the input of the next GRA module, it is processed by the next GRA module to obtain the preference score of the candidate match.

[0080] like Figure 3 As shown in Figure 1, the GRA module consists of two multilayer perceptrons, six GRA blocks, and an sequentially accessible (OA) block. The input is dimensionally upgraded through a multilayer perceptron to obtain features. The obtained features are then processed through three GRA blocks, one OA block, and three GRA blocks in sequence. The output of the last GRA block is reduced in dimension through another multi-layer perceptron, and finally the preference score is output.

[0081] like Figure 4 As shown, the processing of the GRA block includes the following steps:

[0082] Step 2.1.1: For the input feature map Where C and N are the number of channels and matches respectively, and the feature map is evenly divided into g groups along its channel dimension, denoted by x i , i∈{1, 2, ..., g}. Each feature map subset Have the same channel and space sizes.

[0083] Step 2.1.2: Apply spatial attention to the first feature map subset x1 to generate spatial relationships. Use average pooling and maximum pooling operations along its channel dimension to aggregate the channel information of feature map subset x1, and then concatenate them to generate an efficient spatial feature The formula is as follows:

[0084] F s =concat(Avg Pool0(x1), Max Pool0(x1))

[0085] Among them, concat() represents the concatenation operation along the channel dimension; AvgPool0() and MaxPool0() represent the average pooling operation and the maximum pooling operation along the 0th dimension of the feature, respectively.

[0086] Step 2.1.3: F s The spatial attention weights are obtained by passing through an MLP layer, a batch normalization layer and a sigmoid activation function respectively. The formula is as follows:

[0087] w s =SA(x1)=σ(BN(MLP0(F s )))

[0088] Among them, MLP0() represents a size of The MLP layer is used to reduce the feature channel dimension to 1, BN() represents the batch normalization layer, σ() is the sigmoid activation function, and SA() represents the entire spatial attention operation.

[0089] Step 2.1.4: Each feature map subset is processed by a corresponding PointCN block, denoted as PCN i (). At the same time, use the obtained spatial attention weight w s To enhance the inlier representation capability of each input feature group. In addition, in order to increase the diversity of subset output feature representation, a hierarchical residual-like connection is used to connect all feature map subsets. i Indicates PCN i The output of () is as follows:

[0090]

[0091] in, Represents element-by-element multiplication; using grouping operations to explore contextual information from different aspects is beneficial to the learning of neural networks; in addition, this hierarchical residual-like connection method can make each PCN i () can receive the feature information of the previous PointCN block, and due to the combinatorial explosion effect, generate output features with strong representation ability.

[0092] Step 2.1.5: Connect all feature map subsets along the channel dimension to obtain the output feature y, which is formulated as follows:

[0093] y = concat(y1, y2, ..., y g )

[0094] Step 2.1.6: Explore the relationship on the channel by performing channel attention operation on the output feature y. The average and maximum pooling operations are mainly used to aggregate the spatial information of the output feature y, and then add them together to generate an efficient channel feature The formula is as follows:

[0095] F c =AvgPool 12 (y)+MaxPool 12 (y)

[0096] Among them, AvgPool 12 () and MaxPool 12 () refer to the average pooling operation and the maximum pooling operation along the feature space dimensions, i.e., the 1st and 2nd dimensions, respectively. Here, we use an element-wise sum operation to fuse the information of the two channel features to achieve parameter efficiency.

[0097] Step 2.1.7: Calculate channel attention weights through a bottleneck structure The formula is as follows:

[0098] w c =CA(y)=σ(BN(MLP2(ReLU(BN(MLP1(F s )))))))

[0099] Among them, MLP1() and MLP2() represent the size of and The two MLP layers are used to learn channel weights and reduce parameter overhead; CA() represents the entire channel attention operation, and the channel attention weights are used to enhance the representation ability of important channels of feature y.

[0100] Step 2.1.8: Perform element-by-element AND operation with the input feature map x to obtain the output feature Y, which can be expressed as a function as follows:

[0101]

[0102] Among them, Y is the output feature with strong representation ability obtained after the grouped residual attention operation.

[0103] Step 2.1.9: In order to increase the communication between feature map subsets and fuse their information, the channel shuffle operation proposed by ShuffleNet is introduced, such as Figure 5 As shown, a new feature map Y is obtained out , which can be expressed as a function as follows:

[0104] Y out =CS(Y)

[0105] The OA block is mainly used to capture complex local contexts, which includes a DiffPool layer to aggregate feature matches, several Order-Aware filter layers to explore the global context, and a DiffUnpool layer to restore the size and order of the original matches.

[0106] Step 2.3: Add the preference scores ps1 and ps2 to obtain the preference score ps used by the preference-guided filtering network for matching classification, and calculate the corresponding internal point probability set; the calculation formula is as follows:

[0107] W=tanh(ReLU(ps)), ps=αps1+ps2,

[0108]

[0109] in, and denote two GRA modules with parameters θ1 and θ2 respectively; ps1 and ps2 denote the preference scores learned from the two GRA modules respectively; α is a learnable weight initialized to 0, which is used to control the influence of the first GRA module; ReLU and tanh are two activation functions used to calculate the inlier probability set W = {w1, w2, ..., w N},w i ∈[0,1), where w i Indicates the initial match s i as the possibility of an inlier; com() is a custom function that completes and aligns the preference scores of the second GRA module (removed matches are set to 0) to match Have the same size and order.

[0110] In this embodiment, in step 3, the calculation formula for predicting the essential matrix by the weighted eight-point algorithm is:

[0111]

[0112] in, represents the essential matrix predicted by the weighted eight-point algorithm, and g() represents the weighted eight-point algorithm.

[0113] This embodiment also provides a deep learning image feature matching system based on preference guided filtering, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above-mentioned method steps can be implemented.

[0114] In this embodiment, RANSAC, OANet++ and this method are selected to carry out relative posture estimation experiments in the YFCC100M dataset, with mean average precision (mAP) as the evaluation index, and the error thresholds are selected as 5° and 20°. The technical effects are shown in the comparison table, and this method has the best performance.

[0115] Technical Effect Comparison Table

[0116]

[0117]

[0118] The method of the present invention is tested on an outdoor public data set, and the experimental results show that the present invention achieves the best effect among all the technologies involved in the comparison. It can be seen from practice that the present invention can be widely used in the fields of remote sensing image stitching and three-dimensional reconstruction.

[0119] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0120] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0123] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A deep learning image feature matching method based on preference guided filtering, characterized in that: The following steps are involved: Step 1: For a given pair of images I and I', use the SIFT algorithm to establish an initial feature matching set S; Step 2: Use the preference-guided filtering network to remove false matches and obtain the probability set W of matching as an internal point; Step 3: Treat the probability set W obtained in step 2 as a weight set and estimate the essential matrix using the weighted eight-point algorithm Step 4: Restore the accurate relative posture according to the essential matrix obtained in step 3; Step 1 includes the following steps: Step 1.1: For two given image pairs I and I' with the same or similar scenes, extract the key point coordinates and corresponding feature descriptors in the image pairs I and I' using the SIFT algorithm: K,D,K',D'=SIFT(I,I') Where K and K' represent the key point coordinates of the two images, D and D' are their corresponding feature descriptors, and SIFT() represents the SIFT algorithm; Step 1.2: According to the nearest neighbor similarity principle of feature descriptors, establish the initial feature matching set S: Among them, NNC() represents the nearest neighbor search algorithm; s i represents the i-th match, N represents the number of matches, Represents the dimension of the feature matching set, where 4 represents the coordinate dimension, (u i ,v i ) and (u' i ,v' i ) represent s i Normalized keypoint coordinates in image pairs I and I'; Step 2 includes the following steps: Step 2.1: Take the initial feature matching set S as the input of the preference guided filtering network and process it using a grouped residual attention module GRA to obtain the preference score of the initial feature matching. Step 2.2: For the initial feature matching preference score ps1 obtained, extract the previous matches, and obtain a candidate matching set in The candidate matching set As the input of the next GRA module, it is processed by the next GRA module to obtain the preference score of the candidate match. Step 2.3: Add the preference scores ps1 and ps2 to obtain the preference score ps used by the preference-guided filtering network for matching classification, and calculate the corresponding inlier probability set.

2. The deep learning image feature matching method based on preference guided filtering according to claim 1 is characterized in that: The GRA module consists of two multilayer perceptrons, six GRA blocks, and a sequentially available block OA; the input is dimensionally upgraded through a multilayer perceptron to obtain features The obtained features are then processed by three GRA blocks, one OA block, and three GRA blocks in sequence. The output of the last GRA block is reduced in dimension by another multi-layer perceptron, and finally the preference score is output; The processing of a GRA block consists of the following steps: Step 2.1.1: For the input feature map Where C and N are the number of channels and matches respectively, and the feature map is evenly divided into g groups along its channel dimension, denoted by x i ,i∈{1,2,…,g}; each feature map subset Have the same channel and space size; Step 2.1.2: Apply spatial attention to the first feature map subset x1 to generate spatial relationships; use average pooling and maximum pooling operations along its channel dimension to aggregate the channel information of feature map subset x1, and then concatenate them to generate an efficient spatial feature The formula is as follows: F s =concat(Avg Pool0(x1),Max Pool0(x1)) Among them, concat() represents the concatenation operation along the channel dimension; AvgPool0() and MaxPool0() represent the average pooling operation and the maximum pooling operation along the feature 0th dimension respectively; Step 2.1.3: F s The spatial attention weights are obtained by passing through an MLP layer, a batch normalization layer and a sigmoid activation function respectively. The formula is as follows: w s =SA(x1)=σ(BN(MLP0(F s ))) Among them, MLP0() represents a size of The MLP layer is used to reduce the feature channel dimension to 1, BN() represents the batch normalization layer, σ() is the sigmoid activation function, and SA() represents the entire spatial attention operation; Step 2.1.4: Each feature map subset is processed by a corresponding PointCN block, denoted as PCN i (); At the same time, use the obtained spatial attention weight w s To enhance the inlier representation capability of each input feature group; in addition, in order to increase the diversity of subset output feature representation, a hierarchical residual-like connection method is used to connect all feature map subsets; use y i Indicates PCN i The output of () is as follows: in, Represents element-by-element multiplication; uses grouping operations to explore contextual information from different aspects; Step 2.1.5: Connect all feature map subsets along the channel dimension to obtain the output feature y, which is formulated as follows: y=concat(y1,y2,…,y g ) Step 2.1.6: Explore the relationship on the channel by performing channel attention operation on the output feature y; mainly use average and maximum pooling operations to aggregate the spatial information of the output feature y, and then add them together to generate an efficient channel feature The formula is as follows: F c =AvgPool 12 (y)+MaxPool 12 (y) Among them, AvgPool 12 () and MaxPool 12 () refer to the average pooling operation and the maximum pooling operation along the feature space dimensions, i.e., the first and second dimensions, respectively; Step 2.1.7: Calculate channel attention weights through a bottleneck structure The formula is as follows: w c =CA(y)=σ(BN(MLP2(ReLU(BN(MLP1(F s )))))))) Among them, MLP1() and MLP2() represent the size of and The two MLP layers are used to learn channel weights and reduce parameter overhead; CA() represents the entire channel attention operation, and the channel attention weights are used to enhance the representation ability of important channels of feature y; Step 2.1.8: Perform element-by-element AND operation with the input feature map x to obtain the output feature Y, which can be expressed as a function as follows: Among them, Y is the output feature with strong representation ability obtained by the grouped residual attention operation; Step 2.1.9: In order to increase the communication between feature map subsets and fuse their information, the channel shuffle operation proposed by ShuffleNet is introduced to obtain a new feature map Y out , which can be expressed as a function as follows: AND out =CS(Y) The OA block is mainly used to capture complex local contexts, which includes a DiffPool layer to aggregate feature matches, several Order-Aware filter layers to explore the global context, and a DiffUnpool layer to restore the size and order of the original matches.

3. The deep learning image feature matching method based on preference guided filtering according to claim 1 is characterized in that: In step 2.3, the preference scores ps1 and ps2 are added to obtain the preference score ps of the preference-guided filtering network for matching classification, and the corresponding internal point probability set is calculated, and the calculation formula is expressed as follows: W=tanh(ReLU(ps)),ps=αps1+ps2, in, and denote two GRA modules with parameters θ1 and θ2 respectively; ps1 and ps2 denote the preference scores learned from the two GRA modules respectively; α is a learnable weight initialized to 0, which is used to control the influence of the first GRA module; ReLU and tanh are two activation functions used to calculate the inlier probability set W = {w1,w2,…,w N {,w i ∈[0,1), where w i Indicates the initial match s i as the possibility of an inlier; com() is a custom function used to complete and align the preference scores of the second GRA module to make it consistent with Have the same size and order.

4. The deep learning image feature matching method based on preference guided filtering according to claim 1 is characterized in that: In step 3, the calculation formula for predicting the essential matrix using the weighted eight-point algorithm is: in, represents the essential matrix predicted by the weighted eight-point algorithm, and g() represents the weighted eight-point algorithm.

5. A deep learning image feature matching system based on preference guided filtering, characterized in that: The method comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps as described in any one of claims 1 to 4 can be implemented.

Images