An abnormal matching removal method and system based on a multi-layer graph neural network

Abstract

This invention discloses an anomaly matching removal method and system based on a multi-layer graph neural network. The method includes the following steps: extracting key points and feature descriptors from two input images to construct an initial set of matching points; constructing basic features; obtaining fused features based on a geometric context fusion module; obtaining output features based on a multi-branch feedforward network; after processing by a PointCN module, clustering operation, recovery operation, and PointCN module, obtaining a confidence prediction value through a confidence prediction module; obtaining the final fused features based on a cross-stage graph consensus mechanism; fusing the sparse graphs corresponding to each matching point in each stage to obtain a cross-stage fused graph, and performing feature alignment; performing parallel aggregation of similar neighborhood features of the current matching point; filtering matching pairs based on the inlier probability threshold of the matching pair; and optimizing the matching relationship using the inlier probability as a weight. This invention can achieve cross-stage information alignment and dynamic collaboration, improving matching accuracy.

Description

Technical Field

[0001] This invention relates to the field of image matching technology, and specifically to an anomaly matching removal method and system based on a multi-layer graph neural network. Background Technology

[0002] Traditional image matching, such as feature-point-based matching, relies on accurate feature extraction and matching algorithms. However, with increasing image noise, changing viewpoints, varying lighting conditions, and object occlusion, the accuracy and robustness of traditional image matching in complex scenes often fail to meet practical requirements. This is especially true in high-noise and extreme scenarios, where matching results are frequently affected by anomalous matches, leading to increased errors in the final estimation. Therefore, removing these anomalous matches and improving matching accuracy and stability has always been a key research focus in the field of image matching.

[0003] In the conventional process of image matching, geometric constraint optimization is usually combined after the matching point set is generated to improve the accuracy of matching. However, traditional image matching often fails to guarantee the stability and robustness of matching in complex scenarios such as changes in lighting, differences in viewpoints, repetitive textures, and dynamic occlusion, which may lead to a significant increase in the error rate and thus affect the overall matching quality. In addition, most existing geometric constraint optimization methods rely on fixed geometric constraint rules, which are difficult to adapt to the matching requirements of different scenarios. This may lead to the incorrect removal of some correct matching points or the retention of incorrect matching points, thereby reducing the matching accuracy. Summary of the Invention

[0004] To overcome the shortcomings and deficiencies of existing technologies, this invention provides an anomaly matching removal method and system based on multi-layer graph neural networks. Addressing the problem of geometric constraint optimization in dual-view matching tasks, this invention improves feature matching accuracy by combining global context information and local geometric relationships through a geometric context fusion module and a multi-branch feedforward network. A cross-stage multi-graph consensus module is constructed, achieving dynamic fusion of feature information and geometric consistency modeling through a cross-stage sparse graph structure. This aligns matching features from different stages to enhance the consistency of geometric information and the stability of matching, thereby improving the robustness and matching accuracy of the model in scenarios with a high proportion of outliers.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] This invention provides an anomaly matching removal method based on a multi-layer graph neural network, comprising the following steps:

[0007] Extract key points and feature descriptors from the two input images to construct an initial set of matching points;

[0008] The local features of the initial matching points are extracted through convolution operations and then normalized to obtain the basic features F.

[0009] A geometric context fusion module is constructed, and the fused feature F is obtained based on the global context attention weights and the geometric enhancement attention weights. out ;

[0010] Construct a multi-branch feedforward network, extract features at different scales based on the multi-branch structure, and fuse the results of each branch to obtain the output features.

[0011] Output features After processing by two consecutive PointCN modules, local features are obtained. These local features are then subjected to clustering and restoration operations to obtain restored features. The restored features are then processed by two consecutive PointCN modules. The processing results are input into the confidence prediction module to obtain the confidence prediction value for each pair of matching relationships at the current stage.

[0012] A cross-stage graph consensus mechanism is constructed to stitch together the features of each stage to obtain the final fused feature;

[0013] A sparse graph is constructed for each matching point. The sparse graphs of each stage are fused to obtain a cross-stage fusion graph. Feature alignment is performed on the cross-stage fusion graph. Similar neighborhood features of the current matching point are aggregated in parallel. The inlier probability of each matching pair is calculated. Matching pairs are selected based on the inlier probability threshold. The inlier probability is used as a weight to optimize the matching relationship.

[0014] As a preferred technical solution, key points and feature descriptors are extracted from the two input images to construct a preliminary matching point set. Specific steps include:

[0015] The nearest neighbor matching strategy is adopted. An initial set of correspondences is generated based on feature descriptors. Matching pairs are obtained by matching key points through feature descriptor similarity. The coordinates of the matching pairs are normalized to obtain a preliminary set of matching points.

[0016] As a preferred technical solution, normalization processing is performed to obtain the basic feature F, which is specifically expressed as:

[0017] F = BN(IN(Conv(S)))

[0018] Where BN represents batch normalization, IN represents instance normalization, Conv represents convolutional layer, and S represents the initial set of correspondences.

[0019] As a preferred technical solution, a geometric context fusion module is constructed to obtain the output feature F based on the global context attention weight and the geometric enhancement attention weight. out Specifically, it includes:

[0020] The basic feature F is mapped to three different subspaces, generating a query vector Q, a key vector K, and a value vector V, respectively, as follows:

[0021] Q = ReLU(BN(MLP(F)))

[0022] K = ReLU(BN(MLP(F)))

[0023] V = ReLU(BN(MLP(F)))

[0024] Where MLP stands for Multilayer Perceptron, BN stands for Batch Normalization, and ReLU stands for Activation Function;

[0025] Calculate the global context attention weights A F :

[0026]

[0027] Where Softmax represents the Softmax function. Indicates the scaling factor;

[0028] The keypoint coordinates of the two input images are combined into matrices P1 and P2, respectively, and then a geometric feature P is constructed using a multilayer perceptron.

[0029] P = MLP(P1) + MLP(P2)

[0030] MLP stands for Multilayer Perceptron;

[0031] Computational geometry-enhanced attention weights A G :

[0032] A G =Softmax(Q·P) T )

[0033] Fusion feature F out Represented as:

[0034] F out =(A F +A G )·V

[0035] Where · denotes matrix multiplication.

[0036] As a preferred technical solution, a multi-branch feedforward network is constructed. Based on the multi-branch structure, features at different scales are extracted, and the results of each branch are fused to obtain the output features. Specifically, it is expressed as follows:

[0037]

[0038] Wherein, CBGC represents the local convolutional branch, GAP represents the global average pooling branch, GMP represents the global max pooling branch, and LN represents layer normalization.

[0039] As a preferred technical solution, in each PointCN module, the features are processed by instance normalization, batch normalization and GELU activation function, and then feature transformation is performed through convolution. Each PointCN module adds the input and output through residual connection.

[0040] As a preferred technical solution, a cross-stage graph consensus mechanism is constructed to concatenate the features of each stage to obtain the final fused feature, specifically represented as follows:

[0041]

[0042] Where Z represents the final fused feature, SE represents the channel attention mechanism, and concat represents the feature concatenation operation. This represents the output characteristics from different stages.

[0043] As a preferred technical solution, a corresponding sparse graph is constructed for each matching point, and the sparse graphs of each stage are fused to obtain a cross-stage fused graph, specifically including:

[0044] Based on the k-nearest neighbor algorithm, output at each stage For each matching point in the feature space, select the k most similar neighbors to determine its local neighborhood;

[0045] For each matching point, construct a corresponding sparse graph, represented as follows:

[0046]

[0047] Among them, G i Represents a sparse graph. Let ε represent the set of the i-th matching point and its selected k neighboring points, i.e., the set of nodes in the graph. i Represents the set of edges between nodes;

[0048] The cross-stage fusion diagram is represented as follows:

[0049]

[0050] in, This represents the graph features extracted from the first M-2 stages. These represent the graph features extracted from the previous stage. This represents a cross-stage fusion diagram.

[0051] As a preferred technical solution, feature alignment is performed on the cross-stage fusion graph, and parallel aggregation is performed on the similar neighborhood features of the current matching point, specifically including:

[0052] Feature alignment of the cross-stage fusion map using a multilayer perceptron is represented as follows:

[0053]

[0054] Among them, c f The features are aligned; MLP stands for Multilayer Perceptron. Represents a cross-stage fusion diagram;

[0055] Parallel aggregation of neighborhood features similar to the current matching point is represented as:

[0056]

[0057] in, W represents the features of the i-th matching point after convolution in the n-th ring region. n The weight matrix is ​​a learnable matrix. The features of the j-th neighbor of the i-th node are... All from c f b n These are bias terms, where p and n represent indices for different stages and feature points.

[0058] This invention also provides an anomaly matching removal system based on a multi-layer graph neural network, comprising: an initial matching point set construction module, a basic feature construction module, a geometric context fusion module, a multi-branch feedforward network, a PointCN module, a confidence prediction module, a cross-stage graph consensus module, a sparse graph construction module, a cross-stage fusion module, a feature alignment module, an aggregation module, an interior point probability calculation module, and a matching relationship optimization module;

[0059] The initial matching point set construction module is used to extract key points and feature descriptors from the two input images and construct an initial matching point set.

[0060] The basic feature construction module is used to extract local features of the initial matching points through convolution operations, perform normalization processing, and obtain basic features F;

[0061] The geometric context fusion module is used to obtain the fusion feature F based on the global context attention weight and the geometric enhancement attention weight. out ;

[0062] The multi-branch feedforward network is used to extract features at different scales based on a multi-branch structure, and the results of each branch are fused to obtain the output features.

[0063] Output features After processing by two consecutive PointCN modules, local features are obtained. These local features are then subjected to clustering and restoration operations to obtain restored features. The restored features are then processed by two consecutive PointCN modules. The processing results are input into the confidence prediction module, which outputs the confidence prediction value for each pair of matching relationships at the current stage.

[0064] The cross-stage graph consensus module is used to construct a cross-stage graph consensus mechanism, which splices together the features of each stage to obtain the final fused feature.

[0065] The sparse graph construction module is used to construct a corresponding sparse graph for each matching point;

[0066] The cross-stage fusion module is used to fuse the sparse graphs of each stage to obtain a cross-stage fused graph;

[0067] The feature alignment module is used to align features in the cross-stage fusion graph;

[0068] The aggregation module is used to perform parallel aggregation of similar neighborhood features of the current matching point;

[0069] The in-point probability calculation module is used to calculate the in-point probability of each matching pair;

[0070] The matching relationship optimization module is used to filter matching pairs based on the inlier probability threshold and optimize the matching relationship by using the inlier probability as a weight.

[0071] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0072] (1) Based on the cross-stage multi-graph consensus module, this invention effectively integrates feature information from different stages, significantly improves the geometric consistency and global stability between matching points, breaks the limitation of traditional methods that only focus on feature optimization within a stage, realizes cross-stage information alignment and dynamic collaboration, and greatly improves the matching robustness and accuracy of the model in complex scenarios and with a high proportion of outliers.

[0073] (2) Based on the geometric context fusion module and the multi-branch feedforward network, this invention realizes the collaborative modeling of global and local features, which significantly enhances the model's ability to express complex spatial relationships. It not only improves the accuracy and robustness of anomaly matching removal, but also has excellent adaptability and robustness in diverse scenarios and high-noise environments. Attached Figure Description

[0074] Figure 1 This is a flowchart illustrating the anomaly matching removal method based on a multi-layer graph neural network according to the present invention.

[0075] Figure 2 This is a schematic diagram of the overall architecture of the anomaly matching and removal system based on a multi-layer graph neural network according to the present invention. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0077] Example 1

[0078] like Figure 1 As shown, this invention provides an anomaly matching removal method based on a multi-layer graph neural network, comprising the following steps:

[0079] S1: Extract features from the two input images, extract key points in the images and calculate their feature descriptors to form an initial set of matching points;

[0080] S11: Extract key points and their descriptors from the two input images based on feature detection algorithms (such as SIFT algorithm, SuperPoint algorithm, etc.);

[0081] S12: Generate an initial set of correspondences x based on feature descriptors using a nearest neighbor matching strategy. i y i The matching pairs are key points in the image matched by feature descriptor similarity, and these matching pairs are subjected to coordinate normalization.

[0082] S2: Intra-stage feature extraction and fusion;

[0083] S21: The initial matching points are input into the convolutional layer. The local features of the matching points are extracted through convolution operations, further enhancing the geometric information of the matching points. Subsequently, to eliminate differences between different samples, instance normalization is applied to the convolutional layer for standardization, and then batch normalization is used to uniformly normalize the features of the entire batch. Specifically:

[0084] F = BN(IN(Conv(S)))

[0085] Where F represents the robust basic feature obtained after instance normalization and batch normalization, BN represents batch normalization, IN represents instance normalization, Conv represents convolutional layer, and S represents the initial set of correspondences.

[0086] S22: For the features after preliminary normalization, this embodiment constructs a geometric context fusion module, which realizes the dynamic fusion of global semantic information and local geometric information through a dual attention mechanism, specifically including:

[0087] S221: To model and map global context information, a multilayer perceptron combined with batch normalization and ReLU activation function is first used to map the input feature F to three different subspaces, generating a query vector Q, a key vector K, and a value vector V respectively.

[0088] Q = ReLU(BN(MLP(F)))

[0089] K = ReLU(BN(MLP(F)))

[0090] V = ReLU(BN(MLP(F)))

[0091] Where MLP stands for Multilayer Perceptron, BN stands for Batch Normalization, and ReLU stands for Activation Function;

[0092] Next, in order to capture global semantic information, attention weights are calculated using similarity, specifically through scaled dot product attention:

[0093]

[0094] Among them, A F This represents the global context attention weights calculated based on the query vector Q and the key vector K. This represents the scaling factor, ensuring that the attention score remains stable during training. This step allows each feature to integrate global contextual information.

[0095] S222: To address the shortcomings of existing anomaly removal methods in modeling local geometric relationships, the geometric context fusion module employs a positional attention mechanism to explicitly encode the geometric information of key points in the matching pairs. Specifically, the initial matching pair set S = {s1, s2, ..., s...} is... N Each pair of matching points s in} i =(x i ,y i ,x′ i ,y′ i ), directly extract the coordinates (x, y) of key points in images 1 and 2. i ,y i ) and (x′ i ,y′ i Then, the coordinates (x, y) of all key points in image 1 are... i ,y i Combined into a matrix Coordinates (x′) of all key points in Image 2 i ,y′ i Combined into a matrix Subsequently, the data is mapped to a high-dimensional space using a multilayer perceptron, and the two are then added together to obtain the comprehensive geometric feature P:

[0096] P = MLP(P1) + MLP(P2)

[0097] Subsequently, the geometric enhancement attention weight A is calculated using the dot product of the query vector Q from step S221 and the geometric feature P from step S222. G :

[0098] A G =Softmax(Q·P) T )

[0099] S223: Finally, the global context attention weights A F With geometrically enhanced attention weights A G The fusion is performed and multiplied with the value vector V to obtain the fused feature F. out :

[0100] F out =(A F +A G )·V

[0101] S23: In order to further integrate and enhance the fused features, a multi-branch feedforward network is proposed for the geometric context fusion module. The multi-branch feedforward network aims to capture multi-scale information and improve the network's generalization ability.

[0102] The multi-branch feedforward network employs a multi-branch structure, extracting features at different scales through local convolutional branches (CBGC), global average pooling (GAP), and global max pooling (GMP). Then, convolution, batch normalization, GELU, and other convolutions are used to process the different branches. Finally, the results from each branch are fused element-wise to obtain the final output features.

[0103]

[0104] Among them, F out The terms represent fused features. CBGC indicates that the feature is processed sequentially through a convolutional layer, batch normalization (BN), GELU activation function, and finally another convolutional layer. GAP represents global average pooling, and GMP represents global max pooling. This represents the final fused features obtained through a multi-branch feedforward network;

[0105] S24: Subsequently, in order to further extract and aggregate local features, the local information of the fused features is deeply mined;

[0106] Specifically, the final fused features obtained from the multi-branch feedforward network As input, the feature is processed by two consecutive PointCN modules. In each PointCN module, the feature undergoes instance normalization, batch normalization, and a GELU activation function, followed by feature transformation via convolution. Each module directly adds the input and output through a residual connection (shortcut), ensuring that the original feature information is preserved during feature propagation. Specifically, the PointCN1 module first extracts preliminary local features, and then its output is fed into the PointCN2 module for deeper feature refinement. After these two consecutive processing steps, the resulting local features retain the detailed information of the original features while enhancing feature consistency and robustness.

[0107] S25: The local features extracted in step S24 are summarized and integrated through clustering operations, and similar local features are aggregated together; then, the clustered results are mapped back to the original feature space through restoration operations, thereby ensuring that the original local structural information is completely preserved. This step not only effectively reduces feature redundancy, but also enhances the consistency of local structure.

[0108] S26: For the features restored in step S25, two consecutive PointCN modules, similar to those in step S24, are used again for in-depth refinement. First, the clustered and restored features are fed into the PointCN3 module for preliminary feature enhancement; then, the output of the PointCN3 module is passed to the PointCN4 module to further mine and enhance local geometric details.

[0109] Finally, by using the geometric context fusion module and the multi-branch feedforward network, the refined local features extracted in the previous stages are effectively fused with global semantic information through the interaction mechanism of global context attention and local geometric information.

[0110] Specifically, the geometric context fusion module enhances the expressive power of features by capturing the interaction between global context and local geometric features; at the same time, the multi-branch feedforward network further improves the diversity and robustness of features by processing information at different scales in parallel, thereby providing more accurate and stable feature representations for subsequent matching optimization and abnormal matching removal.

[0111] S27: Input the features obtained in the current stage (the processing results of steps S21 to S26) into the confidence prediction module. Specifically, a convolutional layer (Conv) followed by a ReLU activation function, and finally a Tanh activation function, is used to output the confidence score for all matching relationships. This process effectively compresses the feature channels and maps each pair of matching relationships to a confidence value in the range of 0 to 1, ultimately obtaining the current confidence prediction, which serves as the input to the cross-stage multi-graph consensus module.

[0112] S3: Construct a cross-stage graph consensus mechanism to integrate feature information from different stages;

[0113] The preceding steps mainly focus on feature extraction and optimization within a stage. However, existing methods often neglect feature consistency between multiple stages, which can lead to information mismatch between stages and thus affect the accuracy of the final result. To make up for this deficiency, this embodiment obtains the local and global feature representations of each stage from the multi-level feature extraction module in the cross-stage multi-graph consensus module. By fusing feature information from different stages, the geometric consistency and information flow stability between stages are ensured.

[0114] S31: Extract the outputs of the two multi-branch feedforward networks from the previous stage (M-1) and the final output of this stage. Specifically, extract the corresponding local and global features from steps S22, S26, and S27 of the previous stage (M-1) as features. The features are then aligned and integrated using 1×1 convolutions to ensure consistency of geometric relationships and stability of information flow between stages. Through fusion and optimization, the robustness and accuracy of subsequent stages are improved.

[0115] S32: For earlier stages (M-2, M-3, ..., 1 stages), only the final output features of each stage are extracted, i.e., the results of step S27 in each stage. The features of each stage are integrated into a unified representation through a concatenation operation, and the concatenated features are adaptively weighted and aligned using the Squeeze-and-Excitation (SE) mechanism to ensure that the features of different stages achieve optimal fusion in the same representation space. The specific expression is as follows:

[0116]

[0117] in, The output features represent the output features from different stages 1, 2, ..., M-2. These features typically correspond to the results of step S27 in each stage. The feature with the subscript 3 represents the final fusion output of each stage. concat indicates that the features are concatenated (concatenated according to the feature dimension). Z is the final fusion feature after processing by the channel attention mechanism Squeeze-and-Excitation (SE). In this embodiment, the current stage, stage M-1, stage M-2, ..., stage 1 refer to the feature output of the model under different iteration steps. The features output from each stage participate in concatenation and weighted fusion, which enhances the expressive power of the feature space.

[0118] S33: Subsequently, in order to match the local geometric relationships between points, the k-nearest neighbor algorithm is used to output the results for each stage. For each matching point in the graph, select its k most similar neighbors in the feature space to determine its local neighborhood. Based on these neighborhood relationships, construct a corresponding sparse graph for each matching point i. in, Let represent the set of the i-th matching point and its selected k neighboring points, i.e., the set of nodes in the graph. The set of edges between these nodes is represented, specifically, the connection between i and each of its neighboring points, reflecting their similarity or geometric relationship. In this way, the local graph can clearly reflect the spatial structure of the matching points.

[0119] S34: Subsequently, in order to fuse the local sparse graphs from different stages, a unified cross-stage fused graph is constructed. Specifically, the features constructed at each stage are integrated together through a concatenation operation, the expression of which is:

[0120]

[0121] in, This represents the graph features extracted from the first M-2 stages. These represent the graph features extracted from the previous stage, which are then combined into a complete cross-stage feature map through a concat operation.

[0122] S35: Fusion graph based on step S34 A dynamic weighting strategy is employed to process the features of each node. Specifically, a multilayer perceptron (MLP) is first used to align the features of the fused graph, effectively eliminating differences in the dimensionality and distribution of features across different nodes, thus achieving consistency and uniformity in the feature space. This is specifically represented as follows:

[0123]

[0124] S36: Subsequently, in order to preserve the local relationships between nodes, a circular convolution operation is introduced based on the nearest neighbor local geometric relationships aligned in step S35. This operation aggregates similar neighborhood features of the current matching point in parallel. This aggregation method not only fuses the node's own information but also effectively integrates local geometric features within the neighborhood. Its expression is:

[0125]

[0126] in, W represents the features of the i-th matching point after convolution in the n-th ring region. n The weight matrix is ​​a learnable matrix. For the feature of the j-th neighbor of the i-th node, here All from c f b n These are bias terms, where p and n represent indices for different stages and feature points;

[0127] S4: After feature fusion, cross-stage graph consensus construction and feature optimization in the aforementioned modules, this step will process the final output features to generate the final probability of each matching pair; then, using the geometric constraints between interior points, the final basic matrix will be calculated and optimized based on the filtered matching relationship.

[0128] The entire process consists of the following steps:

[0129] S41: First, sort the inlier probabilities calculated for each matching pair using the pruning module, then filter out the half of the matching pairs with the lowest probabilities and retain the matching relationships with higher probabilities.

[0130] S42: The pruned matching relationships are then input into a multilayer perceptron (MLP) for further feature compression. Specifically, the MLP compresses the number of feature channels for each matching pair to 1, resulting in a scalar output. This scalar represents the probability that the matching pair is an inlier. In this way, the MLP can extract the probability information of whether a pair is an inlier from complex feature representations.

[0131] S43: The matching relationships are then further filtered by setting a threshold. Specifically, an appropriate inlier probability threshold is set, and matching pairs with probabilities below this threshold are removed. In this way, only those matching pairs with higher probabilities and geometric consistency are included in the final matching set. Finally, the output is the correct matching relationship after filtering and geometric constraint optimization.

[0132] S44: After the matching pair set filtered in step S42 is completed, the results are input into a weighted eight-point algorithm, which optimizes the matching relationship by using the inlier probability as weight. The fundamental matrix is ​​then calculated, describing the geometric relationship between the two viewpoints. This fundamental matrix not only verifies the geometric consistency of the matching points but also provides the geometric basis for subsequent tasks.

[0133] Example 2

[0134] like Figure 2 As shown, this embodiment provides an anomaly matching removal system based on a multi-layer graph neural network, used to implement the anomaly matching removal method based on a multi-layer graph neural network in Embodiment 1 above. The system includes: an initial matching point set construction module, a basic feature construction module, a geometric context fusion module, a multi-branch feedforward network, a PointCN module, a confidence prediction module, a cross-stage graph consensus module, a sparse graph construction module, a cross-stage fusion module, a feature alignment module, an aggregation module, an interior point probability calculation module, and a matching relationship optimization module.

[0135] In this embodiment, the initial matching point set construction module is used to extract key points and feature descriptors from the two input images to construct an initial matching point set;

[0136] In this embodiment, the basic feature construction module is used to extract the local features of the initial matching points through convolution operations, perform normalization processing, and obtain the basic features F;

[0137] In this embodiment, the geometric context fusion module is used to obtain the fused feature F based on the global context attention weights and the geometric enhancement attention weights. out ;

[0138] In this embodiment, a multi-branch feedforward network is used to extract features at different scales based on a multi-branch structure, and the results of each branch are fused to obtain the output features.

[0139] In this embodiment, the output features After processing by two consecutive PointCN modules, local features are obtained. These local features are then subjected to clustering and restoration operations to obtain restored features. The restored features are then processed by two consecutive PointCN modules. The processing results are input into the confidence prediction module, which outputs the confidence prediction value for each pair of matching relationships at the current stage.

[0140] In this embodiment, the cross-stage graph consensus module is used to construct a cross-stage graph consensus mechanism, which splices the features of each stage to obtain the final fused feature;

[0141] In this embodiment, the sparse graph construction module is used to construct a corresponding sparse graph for each matching point;

[0142] In this embodiment, the cross-stage fusion module is used to fuse the sparse graphs of each stage to obtain a cross-stage fused graph;

[0143] In this embodiment, the feature alignment module is used to perform feature alignment on the cross-stage fusion graph;

[0144] In this embodiment, the aggregation module is used to perform parallel aggregation of neighborhood features similar to the current matching point;

[0145] In this embodiment, the inlier probability calculation module is used to calculate the inlier probability of each matching pair;

[0146] In this embodiment, the matching relationship optimization module is used to filter matching pairs based on the inlier probability threshold and optimize the matching relationship by using the inlier probability as a weight.

[0147] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. An anomaly matching removal method based on a multi-layer graph neural network, characterized in that, Includes the following steps: Extract key points and feature descriptors from the two input images to construct an initial set of matching points; Local features of the initial matching points are extracted through convolution operations and then normalized to obtain the basic features. ; A geometric context fusion module is constructed to obtain fused features based on global context attention weights and geometric enhancement attention weights. Specifically, it includes: Basic features Mapped to three different subspaces, each generating a query vector. Key vector Sum value vector , is represented as: ； ； ； in, This represents a multilayer perceptron. Indicates batch normalization, Indicates the activation function; Calculate global context attention weights : ； in, express function, Indicates the scaling factor; Combine the keypoint coordinates of the two input images into matrices. sum matrix Geometric features are constructed using a multilayer perceptron. : ； in, This represents a multilayer perceptron; Computational geometry-enhanced attention weights : ； Fusion features Represented as: ； in, Indicates matrix multiplication; Construct a multi-branch feedforward network, extract features at different scales based on the multi-branch structure, and fuse the results of each branch to obtain the output features. ; Output features After processing by two consecutive PointCN modules, local features are obtained. These local features are then subjected to clustering and restoration operations to obtain restored features. The restored features are then processed by two consecutive PointCN modules. The processing results are input into the confidence prediction module to obtain the in-point confidence prediction value for each pair of matching relationships at the current stage. A cross-stage graph consensus mechanism is constructed to concatenate the features of each stage to obtain the final fused feature, specifically represented as follows: ； in, Indicates the final fusion characteristics, This indicates the channel attention mechanism. This indicates a feature concatenation operation. This represents the output characteristics from different stages, where M represents the current stage; A sparse graph is constructed for each matching point. The sparse graphs of each stage are fused to obtain a cross-stage fusion graph. Feature alignment is performed on the cross-stage fusion graph. Similar neighborhood features of the current matching point are aggregated in parallel. The inlier confidence prediction value of each matching pair across stages is calculated. Matching pairs are selected based on the inlier probability threshold. The inlier confidence prediction value is used as a weight to optimize the matching relationship.

2. The anomaly matching removal method based on a multi-layer graph neural network according to claim 1, characterized in that, Extract key points and feature descriptors from the two input images to construct a preliminary set of matching points. Specific steps include: The nearest neighbor matching strategy is adopted. An initial set of correspondences is generated based on feature descriptors. Matching pairs are obtained by matching key points through feature descriptor similarity. The coordinates of the matching pairs are normalized to obtain a preliminary set of matching points.

3. The anomaly matching removal method based on a multi-layer graph neural network according to claim 1, characterized in that, Normalization is performed to obtain the basic features. Specifically, it is expressed as: ； in, This indicates batch normalization processing. This indicates instance normalization processing. Indicates a convolutional layer. This represents the initial set of correspondences.

4. The anomaly matching removal method based on a multi-layer graph neural network according to claim 1, characterized in that, Construct a multi-branch feedforward network, extract features at different scales based on the multi-branch structure, and fuse the results of each branch to obtain the output features. Specifically, it is expressed as: ； in, Indicates a local convolution branch. This indicates the global average pooling branch. This represents the global max-pooling branch. Representation layer normalization.

5. The anomaly matching removal method based on a multi-layer graph neural network according to claim 1, characterized in that, In each PointCN module, features are processed by instance normalization, batch normalization, and GELU activation function, and then transformed by convolution. Each PointCN module adds the input and output through residual connections.

6. The anomaly matching removal method based on a multi-layer graph neural network according to claim 1, characterized in that, A sparse graph is constructed for each matching point, and the sparse graphs from each stage are fused to obtain a cross-stage fused graph, specifically including: Based on the k-nearest neighbor algorithm, output at each stage For each matching point in the feature space, the one most similar to it is selected. Each neighbor is used to determine its local neighborhood, where M-1 represents the previous stage, and the feature with index 3 represents the final fusion output of each stage; For each matching point, construct a corresponding sparse graph, represented as follows: ； in, Represents a sparse graph. Indicates the first Matching points and their selections The set of neighboring points, i.e., the set of nodes in the graph. Represents the set of edges between nodes; The cross-stage fusion diagram is represented as follows: ； in, This represents the graph features extracted from the first M-2 stages. , , These represent the graph features extracted from the previous stage. This represents a cross-stage fusion diagram.

7. The anomaly matching removal method based on a multi-layer graph neural network according to claim 1, characterized in that, Feature alignment is performed on the cross-stage fusion graph, and parallel aggregation of similar neighborhood features of the current matching point is performed, specifically including: Feature alignment of the cross-stage fusion map using a multilayer perceptron is represented as follows: ； in, Indicates the aligned features. This represents a multilayer perceptron. Represents a cross-stage fusion diagram; Parallel aggregation of neighborhood features similar to the current matching point is represented as: ； in, Indicates the first The matching point at the th ... Features aggregated from the ring regions through convolution The weight matrix is ​​a learnable matrix. For the first The node of the first Characteristics of a neighbor, All from , It is a bias term. and Indexes representing different stages and feature points.

8. An anomaly matching removal system based on a multi-layer graph neural network, characterized in that, The method for implementing the anomaly matching removal method based on a multi-layer graph neural network according to any one of claims 1-7 includes: an initial matching point set construction module, a basic feature construction module, a geometric context fusion module, a multi-branch feedforward network, a PointCN module, a confidence prediction module, a cross-stage graph consensus module, a sparse graph construction module, a cross-stage fusion module, a feature alignment module, an aggregation module, an interior point probability calculation module, and a matching relationship optimization module. The initial matching point set construction module is used to extract key points and feature descriptors from the two input images and construct an initial matching point set. The basic feature construction module is used to extract local features of the initial matching points through convolution operations, perform normalization processing, and obtain basic features. ; The geometric context fusion module is used to obtain fused features based on global context attention weights and geometric enhancement attention weights. ; The multi-branch feedforward network is used to extract features at different scales based on a multi-branch structure, and the results of each branch are fused to obtain the output features. ; Output features After processing by two consecutive PointCN modules, local features are obtained. These local features are then subjected to clustering and restoration operations to obtain restored features. The restored features are then processed by two consecutive PointCN modules. The processing results are input into the confidence prediction module, which is used to output the in-point confidence prediction value for each pair of matching relationships in the current stage. The cross-stage graph consensus module is used to construct a cross-stage graph consensus mechanism, which splices together the features of each stage to obtain the final fused feature. The sparse graph construction module is used to construct a corresponding sparse graph for each matching point; The cross-stage fusion module is used to fuse the sparse graphs of each stage to obtain a cross-stage fused graph; The feature alignment module is used to align features in the cross-stage fusion graph; The aggregation module is used to perform parallel aggregation of similar neighborhood features of the current matching point; The in-point probability calculation module is used to calculate the in-point confidence prediction value for each matching pair across stages; The matching relationship optimization module is used to filter matching pairs based on the inlier probability threshold and optimize the matching relationship by using the inlier confidence prediction value as a weight.

Images