A cross-view image matching method based on DALGlue

Abstract

This invention proposes a cross-view image matching method based on DALGlue. For images of the same target scene obtained from both UAV and satellite carriers, image matching is performed, and based on the matching results, two tasks are completed: UAV target localization and UAV navigation. This invention addresses the shortcomings of existing matching methods in complex UAV flight scenarios (satellite denial, weak texture, dynamic occlusion, multiple altitude differences), such as insufficient robustness, poor cross-modal adaptability, and the need to improve positioning and navigation accuracy. By employing a modality adaptive conversion layer, a cross-view feature alignment module, a hybrid loss function, and a post-processing refinement module, the entire process of image preprocessing, feature extraction, and matching allocation is optimized, achieving high-precision and high-real-time matching between UAV views and satellite / reference view images. This invention uses the SUES-200 dataset for training and testing, and divides the dataset into multiple dimensions such as height, viewpoint, and environment to construct a hard negative sample set, proving the effectiveness of the method. Compared with other deep learning-based geolocation methods, this invention has the advantages of high matching accuracy, excellent real-time performance, and strong robustness. It can be widely applied to various scenarios such as high-speed navigation of UAVs, low-altitude inspection, and satellite denial positioning, significantly improving the actual landing performance of UAV positioning and navigation.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and unmanned aerial vehicle (UAV) technology, and in particular to a cross-view image matching method based on DALGlue, which is applicable to scenarios such as UAV-satellite cross-view image matching, UAV high-precision target positioning, UAV autonomous navigation, and low-altitude inspection, and is especially suitable for complex application scenarios such as high-speed flight, satellite denial, and multimodal applications. Background Technology

[0002] Cross-view matching is a crucial problem in computer vision, with wide applications in image processing, navigation, remote sensing, and positioning. Satellites, aircraft, and drones are the primary carriers for acquiring remote sensing images. Compared to the other two carriers, drones offer advantages such as fast response time, high image resolution, strong application scalability, and low operating costs, making them a primary source of input images for cross-view matching tasks. A standard cross-view matching task works as follows: given a query image taken at one view, the system aims to find the most relevant image in another view from a large pool of candidate images (a library). Since the candidate images in the library are all geo-labeled, we can predict the location of the target location based on the retrieved geo-labels.

[0003] Traditional cross-view matching methods rely on handcrafted feature descriptors such as SIFT, SURF, and ORB. However, these feature extraction methods are not robust and are susceptible to uncertainties such as lighting and occlusion, especially for drones flying at high altitudes. Due to the significant differences between the acquired image and the satellite view image, mismatches or missing matches often occur frequently. Since the release of the University-1652 dataset, deep learning methods have made significant progress in recent years. Deep learning-based cross-view image matching methods have the advantage of strong robustness. Among them, the DALGlue high-efficiency image matching method, based on a CNN+linear attention architecture, has the advantages of fast matching speed and good real-time performance, and can be adapted to high-speed flight scenarios of UAVs. However, this method still has the following shortcomings, making it difficult to meet the high-precision and robust requirements of UAV positioning and navigation: First, it lacks modal adaptive processing capabilities and cannot effectively adapt to multimodal UAV images such as infrared and visible light. Cross-modal feature differences lead to a decrease in matching accuracy. Second, it does not set an explicit cross-view feature alignment mechanism. The difference in perspective between UAV view (tilted top view) and satellite view (orthogonal top view) can easily lead to feature misalignment and matching errors. Third, it uses a single loss function for training, which has weak discrimination ability for scenes with similar geographical locations and visual features, and is prone to mismatches. Fourth, it lacks a post-processing stage for matching results. Matching errors directly affect the accuracy of UAV positioning and navigation, making it difficult to implement in practical applications. While traditional Swing Transformer cross-view matching methods can extract global features, they suffer from high computational complexity and slow inference speed, making them unsuitable for the real-time requirements of high-speed UAV flight. Other traditional feature optimization methods, although low-cost, lack sufficient matching accuracy and struggle to handle complex scenarios. Therefore, there is an urgent need to design an efficient cross-view image matching method based on DALGlue to address the aforementioned technical pain points and improve its adaptability and performance in UAV positioning and navigation scenarios. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention provides a cross-view image matching method based on DALGlue, which solves the problems of poor cross-modal adaptability, cross-view feature misalignment, weak similar scene discrimination, and insufficient positioning and navigation accuracy of existing DALGlue methods, and realizes high-precision, high-real-time, and strong robustness of UAV target positioning and navigation.

[0005] The technical solution adopted in this invention is: a cross-view image matching method based on DALGlue, which completes two core tasks of UAV target localization and UAV navigation. The method includes the following steps:

[0006] Step 1, Dataset Preprocessing: The SUES-200 dataset is divided into training and testing sets according to height, viewpoint and environmental features, and a hard negative sample set is constructed. The infrared modal dataset is expanded and preprocessed with feature alignment to provide a sample basis for training the cross-view image matching method based on DALGlue that is adapted to UAV positioning and navigation scenarios.

[0007] Step 2, Image preprocessing optimization: Dual-tree complex wavelet transform is used to preprocess the real-time images (visible light / infrared) acquired by the UAV and satellite / reference images to enhance the image structure information and fine details. A modality adaptive conversion layer is set to achieve the unification of multimodal feature space and solve the problems of edge blurring and cross-modal feature differences during the dynamic flight of the UAV.

[0008] Step 3, Multi-scale feature extraction and aggregation: Construct an adaptive spatial feature fusion module, extract multi-scale features of the image through CNN and simultaneously calculate feature points and descriptors, introduce a linear attention mechanism to aggregate image features, and set up a cross-view feature alignment module to achieve bidirectional alignment of UAV view and satellite / baseline view features, thereby improving feature correlation and discriminability;

[0009] Step 4, Model Training and Optimal Matching: The Sinkhorn algorithm is used to calculate the matching assignment matrix and output the optimal feature correspondence. A hybrid loss function is constructed by combining cross-entropy loss, contrastive loss and hard negative sample mining loss. The entire network is trained end-to-end to filter out noisy matching pairs.

[0010] Step 5, Post-processing of matching results and task implementation: The matching accuracy is verified by an adaptive distance metric algorithm, the model performance is evaluated by multi-dimensional indicators, the matching results are post-processed and refined, the real-time coordinates of the UAV are calculated by combining the geographic labels of the baseline map to complete the target positioning, and navigation instructions are generated based on the positioning results to complete the path navigation.

[0011] Furthermore, in step 1, the SUES-200 dataset is divided into a training set and a test set. 120 designated locations are used for training, and 80 locations are used as test data. Each location contains drone images taken at four altitudes: 150m, 200m, 250m, and 300m. Each altitude contains 50 drone images taken from different shooting angles and one satellite image. The data is further divided according to the shooting angle (top / bottom / side angle) and environmental features (dense buildings / weak texture / vegetation cover / light changes) to allow the model to fully learn the image features of different altitudes, different angles, and different environments, thereby improving the robustness of the model.

[0012] To accomplish the dual tasks of drone navigation and drone localization, the test data includes query drone datasets, query satellite datasets, image library drone datasets, and image library satellite datasets. The image library dataset contains the test data, with training data and hard negative samples added as obfuscation to increase the difficulty of matching. The hard negative sample set selects scene images with geographically proximate locations and highly similar visual features (such as similar buildings in the same area or similar scenes at different heights from the same viewpoint), adding them to the image library as obfuscation to improve the model's ability to distinguish similar scenes.

[0013] To accomplish the dual tasks of infrared UAV navigation and UAV localization, the dataset was expanded by adding infrared UAV images of the target building. The infrared modal dataset underwent dual preprocessing: first, illumination / occlusion / noise enhancement was performed to improve the model's robustness to complex scenes in infrared images; second, modal feature alignment was performed, mapping the infrared single-channel grayscale features to the visible light three-channel feature space to reduce cross-modal feature distribution differences and adapt to the needs of multimodal UAV localization and navigation.

[0014] Furthermore, in step 2, the real-time images (visible light / infrared) acquired by the UAV and the satellite / reference image are input into the preprocessing module. Preprocessing is completed through dual-tree complex wavelet transform and mode adaptive conversion layer, specifically including the following steps:

[0015] First, the RGB three-channel visible light images, infrared single-channel images, and satellite reference images acquired in real time by the UAV are preprocessed using dual-tree complex wavelet transform. By decomposing the low-frequency and high-frequency components of the image, the structural details and edge information of the image are preserved, while suppressing image noise. This effectively solves the edge blurring problem caused by motion blur during the dynamic flight of the UAV, providing high-quality input for subsequent feature extraction and matching.

[0016] Then, a modality adaptive transformation layer is set up to perform differentiated processing for images of different modalities: for infrared single-channel images, the single-channel to three-channel conversion is first completed through pixel mapping, and then the modality feature mapping is completed through a lightweight convolutional layer to map the infrared features to a feature space consistent with visible light; for visible light images and satellite images, the feature dimension is unified through linear transformation (set to 96 / 192, dynamically adjusted according to image resolution) to ensure the feature matching of images of different modalities and different views, and solve the problem of poor cross-modal adaptability of the existing DALGlue method.

[0017] Furthermore, in step 3, the preprocessed image is input into the feature extraction and aggregation module, where feature processing is completed through the adaptive spatial feature fusion module, linear attention mechanism, and cross-view feature alignment module. Specifically, this includes the following steps:

[0018] First, the adaptive spatial feature fusion module adopts a multi-scale CNN structure to extract shallow detail features (such as building edges and road textures), mid-level texture features (such as regional texture distribution), and deep global features (such as geographical layout) of the image. It simultaneously calculates feature points and descriptors of features at each scale. Through an adaptive weight allocation mechanism, higher weights are assigned to detail features and second-highest weights are assigned to global features, so as to achieve multi-scale feature complementarity and solve the problem of losing key information in single-scale features.

[0019] Then, the linear attention mechanism simplifies the computation process of traditional attention, avoids complex matrix multiplication operations, reduces computational costs and improves inference speed, while aggregating multi-scale features, improving feature correlation, adapting to the resource constraints of UAV onboard equipment, and ensuring the real-time requirements of UAVs during high-speed flight.

[0020] Finally, a cross-view feature alignment module is set up to solve the problem of no explicit alignment of cross-view features in the existing DALGlue method. Specifically, cross-attention calculation is performed on the aggregated features of the UAV branch as the query vector Q and the aggregated features of the satellite / baseline map branch as the key-value pair K / V to obtain UAV features guided by satellite / baseline map features. At the same time, reverse cross-attention calculation is performed, using the aggregated features of the satellite / baseline map branch as the query vector Q and the aggregated features of the UAV branch as the key-value pair K / V to obtain satellite / baseline map features guided by UAV features. The bidirectionally associated features are residually joined with the original aggregated features to achieve accurate cross-view feature alignment while retaining the discriminativeness of the original features and mitigating matching bias caused by viewpoint differences.

[0021] Furthermore, in step 4, the aligned features are input into the training and matching module, and the model training and optimal matching are completed through the Sinkhom algorithm and a hybrid loss function. This specifically includes the following steps:

[0022] First, the Sinkhorn algorithm iteratively optimizes the matching assignment matrix, sets an iteration threshold (default 100 times) and a similarity threshold (default 0.8), filters out noisy matching pairs with similarity below the threshold, and outputs the optimal feature correspondence between the UAV image and the satellite / baseline image, thereby improving matching accuracy.

[0023] Then construct the hybrid loss function L mix =L ce +L con +L hn This addresses the weakness of the existing DALGlue method's single loss function in terms of discriminative ability. Specifically, L... ce The cross-entropy loss of the two branches is used to calculate the difference between the model's predicted probability and the true label, using the following formula: L conTo compare losses, the distance between UAV-satellite / baseline map features (positive sample pairs) at the same location is shortened, while the distance between features at different locations (negative sample pairs) is widened, thus enhancing the discriminative power of the features; L hn For the hard negative sample mining loss, the feature distance between the hard negative sample and the query sample is calculated for the hard negative sample set constructed in step 1, which further improves the model's ability to distinguish similar scenes.

[0024] Finally, the sum of the hybrid losses of the two branches (UAV branch and satellite / baseline map branch) L is calculated. total =L s +L d (L s For satellite / baseline map branching loss, L d (Uniform branch hybrid loss) is used to optimize the entire network through backpropagation, achieving end-to-end training of the model.

[0025] Furthermore, in step 5, the matching results are finalized through adaptive distance metric, multi-dimensional evaluation, and post-processing refinement, specifically including the following steps:

[0026] First, the fusion distance D = α × D is calculated using an adaptive distance metric algorithm. cos +(1-α)×D ham D cos The cosine distance between feature vectors measures similarity at the feature level; D ham Hamming distance is used to measure the similarity of spatial structures such as buildings and roads in an image by extracting spatial structure feature maps from shallow detail features. α is an adaptive weight (0 < a < 1), which is learned through the test set. α is larger in regular scenes (emphasizing feature similarity) and smaller in weak texture scenes (emphasizing spatial structure similarity). The matching accuracy is verified by sorting the fusion distance D.

[0027] Then, the model is evaluated from multiple dimensions using recall, mean precision, precision, and F1 score to comprehensively measure the model's matching accuracy and recall capability: Recall measures the recall capability of true matching images. If a true matching image of the query image appears before the (K+1)th image in the ranking of the matching results, the value of Recall@K is set to 1; otherwise, it is 0. K is set to 1, 5, 10, and 1% of the number of images in the library; Mean Precision (AP) is the area under the precision-recall curve, where m is the number of true matching images in the query, and T is the area under the curve. h F is the number of true matching images before the h-th true matching image. h is the number of false matching images before the h-th true matching image; precision@K measures the correctness of the first K matching results; F1 value is the harmonic mean of precision and recall.

[0028] Then, a multi-feature voting mechanism is used to vote on the top K matching results, and the result with the highest number of votes is taken as the preliminary matching result to filter out isolated erroneous matches. Combined with the geographic tags of the image library, the geographic location rationality of the preliminary matching results is checked, and abnormal results with excessive deviation from the geographic location of the query image are removed. Subpixel-level feature point alignment is performed on the final matched UAV-satellite / baseline image to further improve the coordinate accuracy of UAV positioning.

[0029] Finally, by combining the geographic labels of the baseline map, the real-time coordinates of the UAV are calculated using the PnP algorithm to complete the target localization of the UAV. Based on the real-time localization results, the image features of the preset navigation path are compared, and the current image and the path node image are matched in real time using the cross-view image matching method based on DALGlue. The degree of deviation of the UAV is judged, and navigation commands such as turning and speed adjustment are generated to ensure that the UAV flies along the preset path and completes the UAV path navigation.

[0030] Compared with existing technologies, the beneficial effects of this invention are: 1. Setting up a modality adaptive conversion layer solves the problem of poor cross-modal adaptability of the existing DALGlue method, realizes the unification of feature spaces of visible light and infrared multimodal images, and adapts to the positioning and navigation needs of multimodal UAVs; 2. Setting up a cross-view feature alignment module explicitly models the bidirectional correspondence between UAV and satellite / baseline image features, solves the feature misalignment problem caused by viewpoint differences, and improves matching accuracy; 3. Constructing a hybrid loss function of cross-entropy + comparison + hard negative sample mining improves the model's ability to distinguish similar scenes and complex environments, and reduces false matching; 4. Designing post-processing for matching results. In the refinement phase, multi-feature voting, geographic location constraints, and sub-pixel alignment are used to further improve the accuracy of UAV positioning and navigation, facilitating practical application; 5. The CNN+linear attention architecture based on the DALGlue method balances matching accuracy and real-time performance, adapts to high-speed UAV flight scenarios, and does not rely on Transformer, reducing computational costs and airborne resource consumption; 6. Adapted to the SUES-200 dataset, through multi-dimensional dataset partitioning and hard negative sample training, the model has strong robustness to different altitudes, different perspectives, and complex environments, making it more suitable for practical application scenarios of UAV positioning and navigation compared to existing methods.

[0031] The present invention will now be further described with reference to the accompanying drawings. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the overall process of the cross-view image matching method based on DALGlue of the present invention;

[0033] Figure 2 This is a diagram of the adaptive spatial feature fusion and linear attention aggregation module architecture in this invention;

[0034] Figure 3 This is a schematic diagram illustrating the working principle of the cross-view feature alignment module in this invention;

[0035] Figure 4 This is a visualization of the UAV positioning task of the present invention;

[0036] Figure 5 This is a visualization of the UAV navigation task of the present invention;

[0037] Figure 6 This is a visualization of the infrared UAV positioning task of the present invention.

[0038] Figure 7 This is a visualization of the infrared UAV navigation mission of the present invention. Detailed Implementation

[0039] The invention will now be further described with reference to the accompanying drawings.

[0040] This invention provides a cross-view image matching method based on DALGlue, which is used to complete two tasks: UAV target localization and UAV navigation. Addressing the technical pain points of existing DALGlue methods, this invention improves matching accuracy, robustness and cross-modal adaptability through multi-dimensional design, making it suitable for complex UAV flight scenarios.

[0041] Step 1, Dataset Preprocessing: The SUES-200 dataset is divided into training and testing sets. 120 designated locations are used for training, and 80 locations are used as test data. Each location contains drone images taken at four altitudes: 150m, 200m, 250m, and 300m. Each altitude contains 50 drone images taken from different shooting angles and one satellite image. The data is further divided according to shooting angle (top / bottom / side angle) and environmental features (dense buildings / weak texture / vegetation cover / light variation). A hard negative sample set is constructed, and scene images with geographical proximity and highly similar visual features are added to the image library to obfuscate the data. The infrared modal dataset is expanded by adding infrared drone images of the target buildings. Preprocessing is performed on illumination / occlusion / noise enhancement and modal feature alignment to map the infrared features to the visible light feature space.

[0042] Figure 1 This is a schematic diagram of the overall process for implementing the method of this invention. A dual-branch network structure with hybrid loss is used to train the model. The feature map is obtained after processing by the preprocessing module, feature extraction and aggregation module, and optimal matching is achieved through the Sinkhorn algorithm, combined with the hybrid loss function for training. In the testing phase, an adaptive distance metric algorithm is used to calculate the fusion distance between the query image and candidate images in the image library. After multi-dimensional evaluation, the matching results are post-processed and refined, finally outputting high-precision matching results for UAV positioning and navigation.

[0043] Step 2, Image Preprocessing Optimization: Visible light images, infrared images, and satellite reference images acquired in real time by the UAV are all preprocessed using dual-tree complex wavelet transform to decompose the low-frequency and high-frequency components of the image, preserve structural details, and suppress noise; a modality adaptive conversion layer is set up, the infrared single-channel image is first converted to three channels and modality mapping is completed through a lightweight convolutional layer, and the visible light and satellite images are unified in feature dimension (set to 192 to adapt to the high-definition images of the UAV) through linear transformation to achieve multimodal feature space unification.

[0044] Step 3, Multi-scale Feature Extraction and Aggregation: The adaptive spatial feature fusion module adopts a 3-layer CNN structure to extract shallow detail features, mid-layer texture features, and deep global features respectively. Feature points and descriptors are calculated simultaneously, and adaptive weights are assigned through the Sigmoid activation function (0.5 for detail features, 0.3 for mid-layer texture, and 0.2 for global features) to achieve multi-scale feature fusion. The linear attention mechanism simplifies attention calculation and aggregates the fused features. The cross-view feature alignment module performs bidirectional cross-attention calculation, bidirectionally associating UAV features and satellite features and performing residual connections to achieve feature alignment.

[0045] Figure 2 This is the architecture diagram for the adaptive spatial feature fusion and linear attention aggregation module. Figure 3 This is a schematic diagram illustrating the working principle of the cross-view feature alignment module.

[0046] Step 4, Model Training and Optimal Matching: The Sinkhorn algorithm is used to calculate the matching assignment matrix, with 100 iterations and a similarity threshold of 0.8, filtering out noisy matching pairs; a hybrid loss function L is constructed. mix =L ce +L con +L hn L ce Weight 0.4, L con Weight 0.3, L hn With a weight of 0.3, calculate the sum of the bi-branch hybrid losses, L. total =L s +L d End-to-end training was performed using the Adam optimizer (learning rate 1e-4) for 50 iterations until the loss converged.

[0047] Step 5, Post-processing of matching results and task implementation: In the adaptive distance metric algorithm, α is set to 0.6 (normal scene), and adjusted to 0.4 for weak texture scene. The fusion distance D is calculated and sorted. Multi-dimensional evaluation is performed using recall @K (K=1, 5, 10), average precision AP, precision @K, and F1 score. Abnormal results are filtered through multi-feature voting (voting for the top 10 matching results) and geographic location constraints (deviation threshold of 5m). Sub-pixel-level feature alignment improves positioning accuracy. Combined with the geographic labels of the baseline map, the real-time coordinates of the UAV are calculated using the PnP algorithm to complete the positioning. Navigation commands (such as "adjust 5° to the left" and "go straight for 100m") are generated based on the positioning results to complete navigation.

[0048] Figure 4 This is a visualization of the UAV positioning task in this invention. Figure 5 This is a visualization of the UAV navigation task in this invention. Figure 6 This is a visualization of the infrared UAV positioning task in this invention. Figure 7 This is a visualization of the infrared UAV navigation task in this invention. For each target scene to be matched, ten of the most suitable candidate images are selected. Correct matching results are highlighted in yellow boxes, and incorrect matching results are highlighted in blue boxes, with the matching confidence level (fusion distance value) marked next to the boxes.

[0049] Experimental Verification: This invention was trained and tested using the SUES-200 dataset. The results were compared with the original DALGlue method and the traditional Swin Transformer method. The results are as follows: The matching accuracy (MMA) of the cross-view image matching method based on DALGlue in this invention is improved by 8.6% compared to the original DALGlue and by 10.2% compared to Swin Transformer. In the Megadepth-1500 benchmark test, the AUC@5° / 10° / 20° reached 62.34, 78.52, and 88.76 respectively, all superior to the two comparison methods. The inference speed maintains the advantage of the original DALGlue and can adapt to the maximum flight speed of 7 m / s for UAVs. In weak texture, dynamic occlusion, satellite rejection, and multimodal scenarios, the matching success rate is improved by more than 15% compared to the original DALGlue, and the robustness is significantly enhanced.

[0050] While the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Those skilled in the art can make various modifications and refinements without departing from the spirit and scope of the invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A cross-view image matching method based on DALGlue, the method comprising the following steps: Step 1: Divide the SUES-200 dataset into training and testing sets according to height, viewpoint and environmental features, and construct a hard negative sample set. Expand and preprocess the infrared modal dataset with feature alignment to provide a sample basis for training the cross-view image matching method based on DALGlue that is adapted to UAV positioning and navigation scenarios. Step 2: The Dual-Tree Complex Wavelet Transform is used to preprocess the real-time images (visible light / infrared) and satellite / reference images acquired by the UAV to enhance the image structure information and fine details. A modality adaptive conversion layer is set up to achieve the unification of multimodal feature space and solve the problems of edge blurring and cross-modal feature differences during the dynamic flight of the UAV. Step 3: Construct an adaptive spatial feature fusion module. Extract multi-scale features of the image through CNN and simultaneously calculate feature points and descriptors. Introduce a linear attention mechanism to aggregate image features. Set up a cross-view feature alignment module to achieve bidirectional alignment of UAV view and satellite / baseline view features, thereby improving feature correlation and discriminability. Step 4: The Sinkhorn algorithm is used to calculate the matching assignment matrix and output the optimal feature correspondence. A hybrid loss function is constructed by combining cross-entropy loss, contrastive loss and hard negative sample mining loss. The entire network is trained end-to-end to filter out noisy matching pairs. Step 5, Post-processing of matching results and task implementation: The matching accuracy is verified by an adaptive distance metric algorithm. Recall, mean precision, accuracy and F1 score are used for multi-dimensional evaluation. The matching results are post-processed and refined. The real-time coordinates of the UAV are calculated by combining the geographic labels of the reference map to complete the target positioning. Navigation instructions are generated based on the positioning results to complete the path navigation.

2. The cross-view image matching method based on DALGlue according to claim 1, characterized in that: The specific process of step 1 is as follows: the SUES-200 dataset is divided into a training set and a test set. 120 designated locations are used for training, and 80 locations are used as test data. Each location contains drone images taken at four altitudes of 150m, 200m, 250m, and 300m. Each altitude contains 50 drone images taken from different shooting angles and one satellite image. The data is then further divided according to the shooting angle (top / bottom / side angle) and environmental features (dense buildings / weak texture / vegetation cover / light changes). To accomplish the tasks of drone navigation and drone localization, the test data includes query drone datasets, query satellite datasets, image library drone datasets, and image library satellite datasets. The image library dataset contains test data, with added training data and hard negative samples as confusion. Hard negative samples are scene images that are geographically close and have highly similar visual features to increase the difficulty of matching. To accomplish the tasks of infrared drone navigation and drone localization, the dataset was expanded by adding infrared drone images of target buildings. The infrared images underwent illumination / occlusion / noise enhancement and modal feature alignment preprocessing to map infrared features to the visible light feature space.

3. The cross-view image matching method based on DALGlue according to claim 1, characterized in that: The specific process of step 2 is as follows: Step 1: The RGB three-channel visible light images, infrared single-channel images, and satellite reference images acquired in real time by the UAV are all preprocessed using dual-tree complex wavelet transform to decompose the low-frequency and high-frequency components of the images, preserve the image structure details, and suppress noise, thus solving the problem of edge blurring during the dynamic flight of the UAV. Step 2: Set up a modality adaptive transformation layer. First, perform single-channel to three-channel processing on the infrared single-channel image, and then complete the modality feature mapping through a lightweight convolutional layer to map the infrared features to a feature space consistent with visible light. For visible light images and satellite images, unify the feature dimensions through linear transformation to ensure the feature matching of different modalities and different view images.

4. The cross-view image matching method based on DALGlue according to claim 1, characterized in that: The specific process of step 3 is as follows: Step 1: The adaptive spatial feature fusion module adopts a multi-scale CNN structure to extract shallow detail features, mid-level texture features and deep global features of the image respectively, and simultaneously calculates feature points and descriptors of each scale feature. Multi-scale feature complementarity is achieved through adaptive weight allocation. Step 2: The linear attention mechanism simplifies the attention calculation process, reduces computational costs, aggregates multi-scale features, improves feature correlation, and adapts to the resource constraints of UAV onboard equipment. Step 3: The cross-view feature alignment module uses the aggregated features of the UAV branch as the query vector Q and the aggregated features of the satellite / baseline map branch as the key-value pair K / V to perform cross-attention calculation. At the same time, it performs reverse cross-attention calculation and performs residual connection on the bidirectionally associated features to achieve accurate alignment of UAV-satellite / baseline map features and solve the matching deviation caused by the difference in viewpoint.

5. The cross-view image matching method based on DALGlue according to claim 1, characterized in that: The specific process of step 4 is as follows: Step 1: The Sinkhorn algorithm iteratively optimizes the matching assignment matrix, sets the iteration number threshold and similarity threshold, filters noisy matching pairs, and outputs the optimal feature correspondence between the UAV image and the satellite / baseline image. Step two: mixed loss function L mix = L ce + L con + L hn where L ce is the binary branch cross-entropy loss, which calculates the difference between the model's predicted probability and the true label, with the formula: Cross-entropy loss function: L con For contrast loss, the feature distance of the same position positive sample pair is shortened, and the feature distance of the different position negative sample pair is lengthened; L hn For hard negative sample mining loss, the model is optimized for the hard negative sample set to improve the discrimination ability of similar scenes. Step 3: Calculate the sum of the two-branch mixed losses, L. total =L s +L d L s L represents the hybrid loss of the satellite / baseline map branch. d This represents the mixed loss of the drone branch, which is optimized across the entire network through backpropagation.

6. The cross-view image matching method based on DALGlue according to claim 1, characterized in that: The specific process of step 5 is as follows: Step 1: Calculate the fusion distance D = α × D using the adaptive distance metric algorithm. cos +(1-α)×D ham D cos For characteristic cosine distance, D ham The spatial structure Hamming distance is used, and α is an adaptive weight (0 < α < 1), which is dynamically adjusted according to the scene type. It finds the most similar feature vector set between the query image and the image in the library. The smaller the fusion distance, the higher the similarity between the features and the structure. Step 2: Evaluate the network model using a multi-dimensional approach, including recall, average precision, precision, and F1 score. Recall rate: Average accuracy: where p0 = 1, Precision is the proportion of correct matches among the top K matches, and F1 score is the harmonic mean of precision and recall. Step 3: Filter out abnormal matching results through a multi-feature voting mechanism and geographic location constraints, and perform sub-pixel-level feature alignment on the final matched image to improve positioning accuracy; By combining the geographic labels of the baseline map, the real-time coordinates of the UAV are calculated using the PnP algorithm to complete the target positioning. Based on the real-time positioning results, the image features of the preset navigation path are compared to generate navigation commands such as turning and speed adjustment to complete the UAV path navigation.

7. The cross-view image matching method based on DALGlue according to claim 1, characterized in that: The proposed cross-view image matching method based on DALGIue does not rely on the Transformer architecture. Its core is built on CNN + linear attention, which balances matching accuracy and real-time performance. It can adapt to the flight speed of drones up to 7m / s and maintains high robustness in weak texture, dynamic occlusion, satellite denial, and multimodal scenarios.

Images