Remote sensing image feature matching and splicing method based on improved LoFTR algorithm

Abstract

The invention discloses a remote sensing image feature matching and splicing method based on an improved LoFTR algorithm, and the method comprises the steps: S1, obtaining a to-be-spliced remote sensing image pair, and respectively extracting a coarse-grained feature map and a fine-grained feature map from the remote sensing image pair; s2, performing dimension feature flattening and position coding processing to obtain a sequence form feature vector, acquiring matching feature correlation and a rough matching point set by using an improved LoFTR algorithm module, and constructing a local similarity matrix; s3, obtaining a sub-pixel-level matching point set under the scale of the fine-grained feature map; and S4, carrying out weighted fusion of overlapped regions on the remote sensing image pair. According to the method, two stages of rough matching and fine matching are adopted, local fine-grained optimization is carried out on candidate areas in the high-resolution feature map on the basis of a rough matching result in the fine matching stage, a high-precision matching result is finally obtained in combination with a sub-pixel-level optimization strategy, and the consistency of splicing boundaries and the global fusion quality are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a remote sensing image feature matching and splicing method based on an improved LoFTR algorithm. Background Art

[0002] Image feature matching and stitching are key technologies in computer vision, particularly in remote sensing image processing. They are widely used in tasks such as topographic mapping, urban planning, and disaster monitoring. The high resolution and large size of remote sensing images require Transformer operations to be performed on the entire feature map, significantly reducing computational efficiency. Traditional methods rely on feature detectors (such as SIFT and ORB) to extract keypoints and match descriptors, then stitch the images together through geometric transformations. However, feature detection is not stable in scenes with low texture, repetitive textures, or drastic illumination changes, resulting in misalignment or ghosting in the stitched results. Furthermore, noise interference (such as cloud occlusion and sensor noise) and geometric distortion (such as terrain undulation and viewpoint changes) in remote sensing images can disrupt feature correlation, leading to spatial variance in the fine-grained matching module, ultimately causing local misalignment or blurring in the stitched image. Furthermore, the multi-scale nature of remote sensing images (such as the mixture of dense urban areas and open agricultural areas) and complex scenes (such as the mixed distribution of buildings, vegetation, and water bodies) make traditional matching strategies difficult to adapt, further exacerbating geometric inconsistencies in the stitched results. Traditional methods often have defects such as poor application effect and low computational efficiency in remote sensing image feature point matching and stitching. It is urgent to study new methods to improve computational efficiency, noise robustness and multi-scale adaptability, and promote the further development of remote sensing image stitching technology. Summary of the Invention

[0003] The purpose of the present invention is to provide a remote sensing image feature matching and splicing method based on an improved LoFTR algorithm, screen out image pairs with a high overlap rate for processing, improve the computational efficiency and processing accuracy of the subsequent matching process, introduce a customized direction-aware feature encoding module in the feature extraction stage, and enhance the adaptability of the extracted features to common geometric transformations; the method of the present invention includes two stages: coarse matching and fine matching. In the coarse matching stage, low-resolution feature maps are modeled, global context information is obtained, and potential matching point pairs are preliminarily screened out; in the fine matching stage, local fine-grained optimization is performed on candidate areas in high-resolution feature maps based on the coarse matching results, the correlation of local features is further enhanced, a matching heat map is generated, and a sub-pixel optimization strategy is combined to finally obtain high-precision matching results, effectively improving the consistency of splicing boundaries and the global fusion quality.

[0004] The purpose of the present invention is achieved through the following technical solutions:

[0005] A remote sensing image feature matching and stitching method based on an improved LoFTR algorithm, the method comprising:

[0006] S1. Obtain a pair of remote sensing images to be stitched, and extract a coarse-grained feature map and a fine-grained feature map for image A and image B of the remote sensing image pair, respectively. The coarse-grained feature map includes a first-level scale feature map and a second-level scale feature map with sizes from small to large.

[0007] S2. Flatten the first-level scale feature maps of images A and B and perform position encoding processing to obtain sequence feature vectors. Use the self-attention module of the improved LoFTR algorithm module to obtain the attention weights and enhanced features of images A and B respectively. Use the cross-attention module of the LoFTR algorithm module to obtain the matching feature correlation between images A and B. Based on the feature correlation, construct the matching point confidence matrix and coarse matching point set of images A and B; extract local feature blocks from the second-level scale feature maps of images A and B with the determined matching point pairs in the coarse matching point set as the center and construct a local similarity matrix;

[0008] S3. Using a dynamic window to intercept the window area of ​​the fine-grained feature map, sampling and matching point pair information of the first-level scale feature map and the second-level scale feature map of the window area, and obtaining a sub-pixel matching point set at the scale of the fine-grained feature map. The sub-pixel matching point set includes sub-pixel precision matching positions and corresponding matching confidences;

[0009] S4. Perform weighted fusion of the overlapping areas of image A and image B using matching confidence.

[0010] In order to better implement the present invention, in method S1, the remote sensing image pairs to be stitched are screened and processed using the following method:

[0011] S11, gridding the original remote sensing image to obtain a plurality of grids, calculating the local gradient of each grid, calculating the average gradient A of all grids, and screening the original remote sensing images whose average gradient A is higher than a set threshold A;

[0012] S12. Counting each grid adjacent to the edge contour of the original remote sensing image and calculating the average gradient B, screening the original remote sensing images whose average gradient B is greater than the set threshold B; and correspondingly forming a remote sensing image pair to be stitched, the remote sensing image pair including image A and image B.

[0013] Preferably, in method S1, image A and image B of the remote sensing image pair are feature extracted using a feature extraction module including a direction-aware feature extraction module, and the features include a coarse-grained feature map and a fine-grained feature map, and the first-level scale feature map, the second-level scale feature map, and the fine-grained feature map are 1 / 16, 1 / 8, and 1 / 2 scale features, respectively; the direction-aware feature extraction module performs a direction-sensitive convolution operation on the input image to obtain a direction feature vector at a spatial position u and in a direction d, and constructs a feature mapping function of the direction alignment relationship of the features extracted by the feature extraction module when a rotation transformation θ occurs.

[0014] Preferably, in method S2, the self-attention module captures the long-range dependency and local context information of the feature vector sequence containing position information, generates a vector matrix of query Q, key K, and value V, calculates the dot product similarity between query Q and key K to obtain the attention weight, and outputs the enhanced feature vector as the weighted fusion value V. , ,in is the transpose of key K, is the scaling factor.

[0015] Preferably, in method S2, the cross-attention module obtains the cross-image feature correlation of the remote sensing image pair according to the following method: ;in is the query matrix from image A, is the key matrix from image B, is the matrix of values ​​from image B, is the transpose of the key matrix from image B, is the scaling factor; the similarity matrix of the remote sensing image pair is constructed based on the cross-image feature correlation of the remote sensing image pair;

[0016] Perform two softmax operations on the similarity matrix as follows:

[0017] Fix the position of image A and perform softmax operation on each row of the similarity matrix to calculate the conditional probability of all positions of image B , is the conditional probability of position i in image A and position j in image B;

[0018] Fix the position of image B and perform softmax operation on each column of the similarity matrix to calculate the conditional probability of all positions of image A , is the conditional probability of position i in image A and position j in image B;

[0019] The confidence expression of image A position i and image B position j as matching points in the matching point confidence matrix is ​​as follows: ;

[0020] The matching point pairs whose results of two softmax operations are consistent and whose confidence level is higher than the threshold are selected as the coarse matching point set.

[0021] Preferably, in method S2, when using confidence to screen matching point pairs, the following local search window method is used to screen matching points:

[0022] Create a local search window for each location , ,in The maximum window size is set. is the adjustment coefficient of window scaling, For matching point pairs The matching confidence of

[0023] Preferably, in method S2, a local window is used to extract local feature blocks from the second-level scale feature maps of image A and image B respectively, with the determined matching point pair in the coarse matching point set as the center. The expression of feature similarity with the corresponding position of image B is as follows: ,in For local window Internal position index; are the corresponding offsets of the matching points, 、 are the features corresponding to image A and image B respectively; feature similarity processing is performed on all local feature blocks to construct a local similarity matrix, and then two softmax operations are performed on the local similarity matrix to calculate the matching confidence matrix.

[0024] Preferably, the method for obtaining the sub-pixel matching point set in the window area is as follows:

[0025] Take the window center vector of image A as the current point to be matched, calculate the cosine similarity between it and all pixels in the window area of ​​image B, generate a local heat map, perform normalization and matching processing through the softmax function, and obtain the probability of matching point pairs between image A and image B. Calculate the matching confidence of the matching point pairs as follows : ,in is the coarse matching confidence in the matching point confidence matrix, is the variance of the local heat map, is the scaling factor; the sub-pixel matching point set is grouped according to the matching point pairs, the sub-pixel precision matching positions of the matching point pairs, and the matching confidence.

[0026] Preferably, in method S4, matching point pairs with high matching confidence are selected from the sub-pixel matching point set, and the spatial transformation of image B relative to image A is modeled using a homography matrix to align image A with image B at the pixel level.

[0027] Preferably, Gaussian diffusion method and / or bilateral filtering method are used to generate pixel-level sparse confidence maps for spatially aligned images A and B, respectively; weighted fusion is performed on the overlapping area of ​​images A and B, and the weighted fusion expression is as follows:

[0028] ,in 、 After spatial alignment, image A and image B are at positions The pixel value at ; 、 are the confidence scores of image A and image B in the sub-pixel matching point set, is a constant.

[0029] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0030] (1) The present invention achieves progressive feature matching from global to local through two stages: coarse-grained feature matching and fine-grained feature matching. The coarse-grained feature matching stage can quickly capture global context information, preliminarily screen out potential matching point pairs, and provide a general direction for subsequent fine matching. Based on the coarse matching, the fine feature matching stage performs fine processing on high-resolution local areas, further optimizes local feature correlation, generates a heat map, and obtains accurate matching positions through sub-pixel level optimization. The present invention adopts a hierarchical matching strategy that fully considers the characteristics of remote sensing images, effectively avoids local feature matching errors, and improves matching accuracy. The present invention uses an adaptive weighted fusion method to splice feature-matched images, ensuring the geometric consistency of the splicing results, and providing high-quality spliced ​​images for the practical application of remote sensing images in resource monitoring, disaster assessment and other fields.

[0031] (2) The present invention addresses the feature extraction challenges brought about by common noise interference, occlusion phenomena, and complex backgrounds (such as vegetation, buildings, and water overlap) in remote sensing images. The present invention adopts a feature extraction module that includes a direction-aware feature extraction module. The direction-aware feature extraction module effectively enhances the stability and discrimination ability of feature expression under multiple geometric transformation conditions by integrating deformation robustness mechanisms such as rotation modeling, scale adaptation, and affine, so that high-quality feature extraction effects can still be maintained in scenes with interference factors such as cloud occlusion and sensor noise. The present invention expands the output of a layer of scale feature map based on the original feature extraction, so that the multi-scale structure of the image can be captured more comprehensively. Combined with the context modeling mechanism, it effectively integrates local details and global information, further improving the adaptability of the model in complex ground backgrounds. The above design significantly reduces the impact of noise and background confusion on feature matching accuracy, providing a solid technical foundation for subsequent high-precision feature matching and image stitching. The present invention has strong robustness and can adapt to various interference conditions in complex remote sensing scenes.

[0032] (3) The present invention performs data preprocessing in the initial stage and screens image areas with a high overlap rate to carry out processing work, avoiding unnecessary calculations in non-critical areas and greatly reducing the amount of calculation; in the feature matching link, the present invention designs a dynamic matching window mechanism, which can adaptively adjust the matching range according to the local feature density; in feature-sparse areas, the matching window can be expanded to increase the matching probability, and in feature-dense areas, the window can be narrowed to reduce redundant calculations, thereby improving the overall matching efficiency and enabling more efficient real-time processing on resource-constrained satellite or UAV platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Flowchart of the remote sensing image feature matching and stitching method in the embodiment;

[0034] Figure 2 Schematic diagram of the remote sensing image feature matching and stitching method in the embodiment;

[0035] Figure 3 Schematic diagram of the principle of obtaining enhanced features by processing the first-level scale feature map in the embodiment;

[0036] Figure 4 Schematic diagram of the principle of the method for obtaining sub-pixel matching point sets in the embodiment;

[0037] Figure 5 Schematic diagram of the result of feature matching using the method of the present invention in an embodiment. DETAILED DESCRIPTION

[0038] Below in conjunction with embodiment, the present invention is described in further detail:

[0039] Example

[0040] like Figure 1 、 Figure 2 As shown, a remote sensing image feature matching and stitching method based on an improved LoFTR algorithm includes:

[0041] S1. Obtain a pair of remote sensing images to be stitched. In some embodiments, the pair of remote sensing images to be stitched is screened and processed using the following method:

[0042] S11. Grid the original remote sensing image to obtain several grids, calculate the local gradient of each grid (each grid is processed separately to ensure the spatial balance of the distribution of feature points to avoid the concentration of features in local areas), calculate the average gradient A of all grids (measure the image quality of the original remote sensing image in terms of texture richness), and screen the original remote sensing images with an average gradient A higher than the set threshold A (if the average gradient A is lower than the set threshold A, it is considered that its texture is weak, the information is insufficient, and there is background noise interference, which needs to be excluded).

[0043] S12. Count the grids adjacent to the edge contour of the original remote sensing image and calculate the average gradient B, and select the original remote sensing images whose average gradient B is greater than the set threshold B; use the retained original remote sensing images to form a remote sensing image pair to be stitched, where the remote sensing image pair includes image A and image B.

[0044] A coarse-grained feature map and a fine-grained feature map are respectively extracted for image A and image B of the remote sensing image pair. The coarse-grained feature map includes a first-level scale feature map and a second-level scale feature map with sizes ranging from small to large. In this embodiment, for example, the first-level scale feature map, the second-level scale feature map, and the fine-grained feature map are scale features of 1 / 16, 1 / 8, and 1 / 2 of the original remote sensing image size, respectively. That is, the first-level scale feature map is a scale feature of 1 / 16, the second-level scale feature map is a scale feature of 1 / 8, and the fine-grained feature map (also called the third-level scale feature map) is a scale feature of 1 / 2.

[0045] In some embodiments, images A and B of the remote sensing image pair are subjected to feature extraction using a feature extraction module including a direction-aware feature extraction module (DAFEM module for short). The feature extraction module uses a customized direction-aware feature extraction module as a backbone network to enhance the stability of feature representation under rotation, scale change, and local geometric perturbations. The features extracted by the feature extraction module include coarse-grained feature maps and fine-grained feature maps. The direction-aware feature extraction module performs a direction-sensitive convolution operation on the input image to obtain a direction feature vector at a spatial position u and in a direction d, and constructs a feature mapping function for the direction alignment relationship of the features extracted by the feature extraction module when a rotation transformation θ occurs. The expression of the feature mapping function is as follows: ,in is the feature map when the position x in the image is rotated by θ, Represents the feature at position x in the image Or the operation of rotating the image around the center by an angle θ. This embodiment constructs a feature mapping function that can effectively ensure the spatial consistency of image features under different rotation angles, improve the matching error caused by viewing angle or target rotation in remote sensing images, and enhance the adaptability and robustness of the model to complex scenes.

[0046] S2. Flatten the first-level scale feature maps of image A and image B and perform position encoding processing (position encoding preferably uses two-dimensional sinusoidal position encoding, which can effectively express the absolute spatial position information of each pixel in the image and enhance the ability to recognize spatial position information) to obtain a sequence feature vector, such as Figure 3 As shown, the self-attention module (full name Self-Attention in English, mainly used for internal processing of a single image of image A or image B) of the improved LoFTR algorithm module is used to obtain the attention weights and enhanced features of image A and image B respectively. In some embodiments, the self-attention module captures the long-range dependency and local context information of the feature vector sequence containing position information (with position encoding added), generates a vector matrix of query Q, key K, and value V, calculates the dot product similarity between query Q and key K to obtain the attention weight, and outputs the weighted fusion value V as the enhanced feature. , ,in is the transpose of key K, is the scaling factor.

[0047] like Figure 3As shown, the cross-attention module (full name: Cross-Attention, used between two images, image A and image B, to make the feature points of one image focus on the feature points of the other image, establishing cross-image association) of the LoFTR algorithm module is used to obtain the matching feature correlation between images A and B. Based on the feature correlation, the matching point confidence matrix and coarse matching point set of images A and B are constructed. In some embodiments, the cross-attention module obtains the cross-image feature correlation of the remote sensing image pair according to the following method: ,in Indicates relevance, is the query matrix from image A, is the key matrix from image B, is the matrix of values ​​from image B, is the transpose of the key matrix from image B, is the scaling factor; the similarity matrix of the remote sensing image pair is constructed based on the cross-image feature correlation of the remote sensing image pair. The example expression of the similarity matrix is , Indicates the similarity between the feature vector at position i in image A and the feature vector at position j in image B.

[0048] Perform two softmax operations on the similarity matrix as follows:

[0049] Fix the position of image A and perform softmax operation on each row of the similarity matrix to calculate the conditional probability of all positions of image B , is the conditional probability of position i in image A and position j in image B.

[0050] Fix the position of image B and perform softmax operation on each column of the similarity matrix to calculate the conditional probability of all positions of image A , is the conditional probability of position i in image A and position j in image B.

[0051] The confidence expression of image A position i and image B position j as matching points in the matching point confidence matrix is ​​as follows: .

[0052] Filter the matching point pairs whose two softmax operations are consistent in both directions and whose confidence is higher than the threshold as the coarse matching point set Preferably, in some embodiments, when using confidence to screen matching point pairs, the following local search window method is used to screen matching points:

[0053] Create a local search window for each location , ,in The maximum window size is set. is the adjustment coefficient of window scaling (used to control the influence of confidence on window scaling), For matching point pairs For matching point pairs with lower confidence, the corresponding Approaching , that is, expand the search range to increase the matching recall; and for matching point pairs with higher confidence, The local search window in this embodiment is a dynamic search window. This dynamic search window mechanism effectively alleviates the shortcomings of fixed windows in complex remote sensing image matching, especially in the presence of scale changes, occlusions, or weak texture areas. It can achieve a more refined local matching strategy and enhance the overall robustness of the system.

[0054] The local feature blocks of the second-level scale feature maps of images A and B are extracted with the matching point pairs determined in the coarse matching point set as the center, and the local similarity matrix is ​​constructed. In some embodiments, the local feature blocks of the second-level scale feature maps of images A and B are extracted with the matching point pairs determined in the coarse matching point set as the center using a local window. The expression of feature similarity with the corresponding position of image B is as follows: ,in For local window Internal position index, local window Using the aforementioned dynamic search window, features within the dynamic window are fed into a local Transformer refinement network. The architecture includes a four-layer multi-head attention mechanism (each layer contains four heads, with 256 hidden layer parameters) and a two-layer local cross-attention module (for A / B feature interaction within the current window). Position encoding: A two-dimensional relative position encoding based on the window center is added to the extracted local feature blocks. are the corresponding offsets of the matching points, 、 are the features corresponding to image A and image B respectively. All local feature blocks are processed for feature similarity and a local similarity matrix is ​​constructed based on this. Then, two softmax operations are performed on the local similarity matrix to calculate the matching confidence matrix.

[0055] S3. Use a dynamic window to intercept the window area of ​​the fine-grained feature map, sample the first-level scale feature map and the second-level scale feature map of the window area, and match the point pair information (sampling the first-level scale feature map and the second-level scale feature map and relying on stacking Nf=1 layers of self-attention and cross-attention to achieve matching point pair information at the scale of the fine-grained feature map). Obtain a sub-pixel matching point set at the scale of the fine-grained feature map. The sub-pixel matching point set includes sub-pixel precision matching positions and corresponding matching confidences. In some embodiments, the method for obtaining the sub-pixel matching point set of the window area is as follows:

[0056] Take the window center vector of image A as the current point to be matched, calculate the cosine similarity between it and all pixels in the window area of ​​image B, generate a local heat map, perform normalization and matching processing through the softmax function, and obtain the probability of matching point pairs between image A and image B. Calculate the matching confidence of the matching point pairs as follows : ,in is the coarse matching confidence in the matching point confidence matrix (e.g. Figure 4 As shown, this embodiment uses coarse matching confidence, which is a matching point confidence matrix. The coarse matching confidence is part of the coarse matching result, which includes the matching point confidence matrix and the coarse matching point set). is the variance of the local heat map, which is used to quantify the concentration of matching positions. The smaller the value, the more concentrated the probability of matching points is, and the higher the final confidence level is. is the scaling factor (the value range of this embodiment is 1 to 5 for example). The sub-pixel matching point set is grouped according to the matching point pairs, the sub-pixel precision matching positions of the matching point pairs, and the matching confidence, such as Figure 5 As shown, the sub-pixel matching point set is represented by a line according to the sub-pixel precision matching position ( Figure 5 The green lines in the middle are the corresponding lines connecting the matching point pairs at the sub-pixel precision matching positions, which facilitates the splicing and fusion of images A and B in the overlapping areas). The sub-pixel matching point set obtains the set of sub-pixel precision matching positions and matching confidences. An example of a sub-pixel matching point set is as follows: , i is the matching point position, , 、 For images With image The sub-pixel position of the match in is the corresponding matching confidence

[0057] S4. The overlapping area of ​​image A and image B is weightedly fused using matching confidence. In this embodiment, sub-pixel matching points are selected to concentrate matching points with high confidence, and the spatial transformation of image B relative to image A is modeled using a homography matrix. Image A and image B are spatially aligned at the pixel level, and image B is transformed to the same coordinate system as image A to achieve spatial alignment of the two remote sensing images at the pixel level, laying the foundation for subsequent image fusion. Gaussian diffusion method and / or bilateral filtering method are used to generate pixel-level sparse confidence maps for spatially aligned images A and B (this embodiment uses pixel-level sparse confidence maps to create conditions for the local adaptive fusion strategy of subsequent weighted fusion and seamless splicing, avoiding problems such as brightness mutation and structural fracture caused by improper boundary processing in traditional fusion methods); weighted fusion is performed in the overlapping area of ​​image A and image B (weighted fusion of pixel values ​​of the two images in the overlapping area), and the weighted fusion expression is as follows: ,in 、 After spatial alignment, image A and image B are at positions The pixel value at . 、 are the confidence scores of image A and image B in the sub-pixel matching point set, is a small non-zero constant. For non-overlapping areas, the original image pixel values ​​are retained without participating in weighted fusion. To further improve the transition smoothness of the edges of the fused area, this embodiment introduces a gradient-weighted edge-guided filtering strategy to adjust the fusion weight distribution in overlapping edge areas, reducing boundary gaps and brightness mutations.

[0058] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A remote sensing image feature matching and stitching method based on an improved LoFTR algorithm, characterized by: The methods include: S1. Obtain a pair of remote sensing images to be stitched, and extract a coarse-grained feature map and a fine-grained feature map for image A and image B of the remote sensing image pair, respectively. The coarse-grained feature map includes a first-level scale feature map and a second-level scale feature map with sizes from small to large. S2. Flatten the first-level scale feature maps of images A and B and perform position encoding processing to obtain sequence feature vectors. Use the self-attention module of the improved LoFTR algorithm module to obtain the attention weights and enhanced features of images A and B respectively. Use the cross-attention module of the LoFTR algorithm module to obtain the matching feature correlation between images A and B. Based on the feature correlation, construct the matching point confidence matrix and coarse matching point set of images A and B; extract local feature blocks from the second-level scale feature maps of images A and B with the determined matching point pairs in the coarse matching point set as the center and construct a local similarity matrix; S3. Using a dynamic window to intercept the window area of ​​the fine-grained feature map, sampling and matching point pair information of the first-level scale feature map and the second-level scale feature map of the window area, and obtaining a sub-pixel matching point set at the scale of the fine-grained feature map. The sub-pixel matching point set includes sub-pixel precision matching positions and corresponding matching confidences; S4. Perform weighted fusion of the overlapping areas of image A and image B using matching confidence.

2. The remote sensing image feature matching and stitching method based on the improved LoFTR algorithm according to claim 1, characterized in that: In method S1, the remote sensing image pairs to be stitched are screened and processed using the following method: S11, gridding the original remote sensing image to obtain a plurality of grids, calculating the local gradient of each grid, calculating the average gradient A of all grids, and screening the original remote sensing images whose average gradient A is higher than a set threshold A; S12. Counting each grid adjacent to the edge contour of the original remote sensing image and calculating the average gradient B, screening the original remote sensing images whose average gradient B is greater than the set threshold B; and correspondingly forming a remote sensing image pair to be stitched, the remote sensing image pair including image A and image B.

3. The remote sensing image feature matching and stitching method based on the improved LoFTR algorithm according to claim 1, characterized in that: In method S1, image A and image B of the remote sensing image pair are subjected to feature extraction using a feature extraction module including a direction-aware feature extraction module. The features include a coarse-grained feature map and a fine-grained feature map. The first-level scale feature map, the second-level scale feature map, and the fine-grained feature map are 1 / 16, 1 / 8, and 1 / 2 scale features, respectively. The direction-aware feature extraction module performs a direction-sensitive convolution operation on the input image to obtain a direction feature vector at a spatial position u and in a direction d, and constructs a feature mapping function of the direction alignment relationship of the features extracted by the feature extraction module when a rotation transformation θ occurs.

4. The remote sensing image feature matching and stitching method based on the improved LoFTR algorithm according to claim 1, characterized in that: In method S2, the self-attention module captures the long-range dependency and local context information of the feature vector sequence containing position information, generates a vector matrix of query Q, key K, and value V, calculates the dot product similarity between query Q and key K to obtain the attention weight, and outputs the enhanced feature vector V as the weighted fusion value. , ,in is the transpose of key K, is the scaling factor.

5. The remote sensing image feature matching and stitching method based on the improved LoFTR algorithm according to claim 1 or 4, characterized in that: In method S2, the cross-attention module obtains the cross-image feature correlation of the remote sensing image pair according to the following method: ;in is the query matrix from image A, is the key matrix from image B, is the matrix of values ​​from image B, is the transpose of the key matrix from image B, is the scaling factor; the similarity matrix of the remote sensing image pair is constructed based on the cross-image feature correlation of the remote sensing image pair; Perform two softmax operations on the similarity matrix as follows: Fix the position of image A and perform softmax operation on each row of the similarity matrix to calculate the conditional probability of all positions of image B , is the conditional probability of position i in image A and position j in image B; Fix the position of image B and perform softmax operation on each column of the similarity matrix to calculate the conditional probability of all positions of image A , is the conditional probability of position i in image A and position j in image B; The confidence expression of image A position i and image B position j as matching points in the matching point confidence matrix is ​​as follows: ； The matching point pairs whose results of two softmax operations are consistent and whose confidence level is higher than the threshold are selected as the coarse matching point set.

6. The remote sensing image feature matching and stitching method based on the improved LoFTR algorithm according to claim 5, characterized in that: In method S2, when using confidence to screen matching point pairs, the following local search window method is used to screen matching points: Create a local search window for each location , ,in The maximum window size is set. is the adjustment coefficient of window scaling, For matching point pairs The matching confidence.

7. The remote sensing image feature matching and stitching method based on the improved LoFTR algorithm according to claim 1, characterized in that: In method S2, a local window is used to extract local feature blocks from the second-level scale feature maps of image A and image B, with the determined matching point pair in the coarse matching point set as the center. The expression of feature similarity with the corresponding position of image B is as follows: ,in For local window Internal position index; are the corresponding offsets of the matching points, 、 are the features corresponding to image A and image B respectively; feature similarity processing is performed on all local feature blocks to construct a local similarity matrix, and then two softmax operations are performed on the local similarity matrix to calculate the matching confidence matrix.

8. The remote sensing image feature matching and stitching method based on the improved LoFTR algorithm according to claim 1, characterized in that: The method for obtaining the sub-pixel matching point set in the window area is as follows: Take the window center vector of image A as the current point to be matched, calculate the cosine similarity between it and all pixels in the window area of ​​image B, generate a local heat map, perform normalization and matching processing through the softmax function, and obtain the probability of matching point pairs between image A and image B. Calculate the matching confidence of the matching point pairs as follows : ,in is the coarse matching confidence in the matching point confidence matrix, is the variance of the local heat map, is the scaling factor; the sub-pixel matching point set is grouped according to the matching point pairs, the sub-pixel precision matching positions of the matching point pairs, and the matching confidence.

9. The remote sensing image feature matching and stitching method based on the improved LoFTR algorithm according to claim 1, characterized in that: In method S4, matching point pairs with high matching confidence in the sub-pixel matching point set are selected, and the spatial transformation of image B relative to image A is modeled using a homography matrix to align image A with image B at the pixel level.

10. The remote sensing image feature matching and stitching method based on the improved LoFTR algorithm according to claim 9, characterized in that: Gaussian diffusion method and / or bilateral filtering method are used to generate pixel-level sparse confidence maps for spatially aligned images A and B respectively; weighted fusion is performed on the overlapping area of ​​images A and B. The weighted fusion expression is as follows: ,in 、 After spatial alignment, image A and image B are at positions The pixel value at ; 、 are the confidence scores of image A and image B in the sub-pixel matching point set, is a constant.

Images