Unmanned aerial vehicle assisted ground and satellite cross-view image geolocation method and system
By using UAV view assistance, a shared feature extraction network, and progressive alignment relationships, the problems of large differences between ground views and satellite views and high complexity of existing methods are solved, achieving efficient cross-view positioning.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUNAN UNIV
- Filing Date
- 2026-04-24
- Publication Date
- 2026-05-29
AI Technical Summary
In existing cross-view geolocation methods, there are significant differences between ground views and satellite views, making direct matching difficult. Furthermore, UAV-assisted positioning methods have complex structures and high inference overhead.
By using UAV views as an intermediate bridge, a shared feature extraction network is used to extract features from ground and satellite images, a progressive cross-view alignment relationship is established, and a UAV candidate feature aggregation and fusion mechanism is introduced to improve the retrieval accuracy from ground to satellite.
Without increasing model complexity, the accuracy of ground-to-satellite cross-view retrieval is improved, and inference overhead is reduced, achieving efficient cross-view positioning.
Smart Images
Figure CN122115579A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and remote sensing geolocation technology, specifically relating to a UAV-assisted ground and satellite cross-view image geolocation method and system. Background Technology
[0002] Cross-view geolocation aims to determine the true geographical location of a target based on images acquired from different perspectives, and has significant application value in scenarios such as intelligent navigation, unmanned system positioning, urban perception, remote sensing interpretation, and emergency search and rescue. Existing typical tasks usually use ground-view images as query images to retrieve the corresponding target location from a satellite image database. However, there are significant perspective differences between ground and satellite images. Ground images typically only show building facades, partial roads, or near-field targets, and are easily affected by factors such as occlusion, lighting variations, and limited field of view; while satellite images present an overhead view of the overall spatial layout, focusing more on roof shapes, road distribution, and building structures. Therefore, there are significant differences between ground and satellite views in terms of geometric relationships, spatial structure, and semantic distribution, making direct matching between the two difficult. Most existing cross-view geolocation methods adopt a direct matching paradigm between ground and aerial images, improving retrieval performance by learning perspective-invariant features. While existing methods have improved performance through spatially perceptual feature aggregation, local pattern modeling, Transformer global modeling, and stronger pre-trained backbone networks, most remain confined to a direct two-view matching framework, failing to adequately bridge the significant view gap between ground and satellite. UAV imagery, situated between ground and satellite views, provides more complete spatial layout information than ground images while preserving richer close-range structural details than satellite images, thus serving as a crucial bridge between them. Some studies have attempted to utilize UAV views to assist ground-to-satellite retrieval, but these methods typically rely on additional functional modules or complex auxiliary processing, easily increasing model complexity and online inference overhead. Therefore, effectively utilizing UAV views as auxiliary information to improve the accuracy of ground-to-satellite cross-view positioning without excessively increasing model complexity and inference overhead has become a critical technical challenge. Summary of the Invention
[0003] The technical problem to be solved by this invention is to provide a ground-to-satellite cross-view image geolocation method and system assisted by UAVs, which addresses the above-mentioned problems in the prior art. This invention aims to solve the problems of large differences between ground views and satellite views, difficulty in direct matching, and the complex structure and high inference overhead of existing UAV-assisted positioning methods, thereby improving the accuracy of ground-to-satellite cross-view retrieval and positioning without excessively increasing the complexity of the model.
[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A drone-assisted geolocation method using ground and satellite cross-view images includes the following steps: S101, use a shared feature extraction network to extract features from the input ground query image; S102, based on the cosine similarity between the features of the ground query image and the features of the drone images in the drone image library, find the top images with the highest similarity. Drone image features of candidate drone images; S103, will be forward The drone image aggregate features are obtained by weighted aggregation of drone image features from 1 candidate drone images; S104, which fuses UAV image aggregation features and ground query image features into a fused feature; S105. Based on the cosine similarity between the fused features and the satellite image features in the satellite image library, find the satellite image with the highest similarity and output its corresponding geographical location as the geolocation result of the ground query image.
[0005] Optionally, in step S103, the previous The function expression for obtaining the aggregated drone image features from the weighted aggregation of drone image features from candidate drone images is as follows: ; ; in, Aggregate features for UAV images, The number of candidate drone images, Let be the weight of the i-th candidate drone image. The drone image features of the i-th candidate drone image are obtained by inputting the i-th candidate drone image into the shared feature extraction network. and This is an index for candidate drone images.
[0006] Optionally, the function expression for fusing the UAV image aggregation features and the ground query image features into a fused feature in step S104 is as follows: ; in, As a feature of fusion, For ground query image features, Aggregate features for UAV images.
[0007] Optionally, the functional expression for the similarity between the fused features and the satellite image features of satellite images in the satellite image library in step S105 is as follows: ; ; in, To fuse features with the cosine similarity between satellite image features and satellite image features in the satellite image library, The normalized fusion characteristics As a feature of fusion, for L2 norm, for transpose, The satellite image features are satellite image features from satellite images in the satellite image library, which are obtained by inputting satellite images from the satellite image library into the shared feature extraction network.
[0008] Optionally, the training of the shared feature extraction network includes: S201: Obtain the corresponding ground images, UAV images and satellite images from the input, and extract features using a shared feature extraction network to obtain the extracted ground image features, UAV image features and satellite image features; S202, establish a contrastive loss for the ground-to-UAV branch for the first cross-view alignment between ground image features and UAV image features, and a contrastive loss for the UAV-to-satellite branch for the second cross-view alignment between UAV image features and satellite image features. Use the total loss composed of the two losses to optimize the shared feature extraction network, thereby obtaining a shared feature extraction network that can be used for cross-view representation learning of ground images, UAV images and satellite images.
[0009] Optionally, the functional expression for the contrast loss of the ground-to-UAV branch is: ; The functional expression for the contrast loss from the UAV to the satellite branch is: ; The functional expression for the total loss composed of the two types of losses is: ; in, For the comparison loss from the ground to the drone branch, To normalize the temperature-scaled cross-entropy loss, For ground image features, For drone image features, For satellite image features, For the comparison loss from drone to satellite branch, For the total loss, and These are the weight parameters.
[0010] Optionally, the shared feature extraction network consists of a backbone network and a pooling layer.
[0011] Furthermore, the present invention also provides a UAV-assisted ground and satellite cross-view image geolocation system, including a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the UAV-assisted ground and satellite cross-view image geolocation method.
[0012] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the UAV-assisted ground and satellite cross-view image geolocation method via a processor.
[0013] In addition, the present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the UAV-assisted ground and satellite cross-view image geolocation method via a processor.
[0014] Compared with existing technologies, the present invention can mainly achieve the following beneficial effects: The UAV-assisted ground and satellite cross-view image geolocation method of the present invention utilizes the intermediate transitional view characteristic of the UAV view located between the ground view and the satellite view to establish a progressive cross-view alignment relationship between ground-UAV-satellite during the training phase, and introduces a UAV candidate feature aggregation and fusion mechanism during the inference phase. This can solve the problems of large differences between ground view and satellite view, difficulty in direct matching, and the complex structure and high inference overhead of existing UAV-assisted positioning methods. Without excessively increasing the model complexity, it can improve the accuracy of ground-to-satellite cross-view retrieval and positioning. Attached Figure Description
[0015] Figure 1 This is a schematic diagram illustrating the basic principle of the method in the reasoning stage of an embodiment of the present invention.
[0016] Figure 2 This is a schematic diagram illustrating the basic principle of the method in the training phase of an embodiment of the present invention. Detailed Implementation
[0017] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.
[0018] like Figure 1As shown, the UAV-assisted ground and satellite cross-view image geolocation method in this embodiment includes the following steps: S101, extract ground query image features from the input ground query image using a shared feature extraction network. ; S102, Based on ground query image features The cosine similarity between the features of drone images and those in the drone image library is used to find the images with the highest similarity. The drone image features of each candidate drone image can be represented as: ; in, ~ They are respectively number 1 to Drone image features of candidate drone images; such as Figure 1 As shown in this embodiment, the number of candidate drone images The value is 5; S103, will be forward The drone image aggregate features are obtained by weighted aggregation of drone image features from 1 candidate drone images; S104, which fuses UAV image aggregation features and ground query image features into a fused feature; S105. Based on the cosine similarity between the fused features and the satellite image features in the satellite image library, find the satellite image with the highest similarity and output its corresponding geographical location as the geolocation result of the ground query image.
[0019] To comprehensively utilize multiple candidate drone images obtained from retrieval, this embodiment employs a ranking-weighted aggregation method to construct aggregated drone features. In step S103, the previous The function expression for obtaining the aggregated drone image features from the weighted aggregation of drone image features from candidate drone images is as follows: ;
[0020] in, Aggregate features for UAV images, The number of candidate drone images, Let be the weight of the i-th candidate drone image. Let be the drone image features of the i-th candidate drone image, which are obtained by inputting the i-th candidate drone image into the shared feature extraction network. To ensure that higher-ranked candidates have a greater weight in the aggregation result, the weights are assigned in a decreasing manner related to the ranking position, i.e.: ; in, and This is an index for candidate drone images.
[0021] In step S104 of this embodiment, the function expression for fusing the UAV image aggregation features and the ground query image features into a fused feature is as follows: ; in, As a feature of fusion, For ground query image features, Aggregate features for UAV images.
[0022] In step S105 of this embodiment, the functional expression for the similarity between the fused features and the satellite image features in the satellite image library is as follows: ; ; in, To fuse features with the cosine similarity between satellite image features and satellite image features in the satellite image library, The fusion features are normalized (the fusion features are normalized to ensure the consistency of the final retrieval representation). As a feature of fusion, for L2 norm, for transpose, The satellite image features in the satellite image library are obtained by inputting the satellite images in the satellite image library into the shared feature extraction network. Based on the similarity results, candidate images in the satellite image library are sorted, and the most matching satellite image is output as the target retrieval result, with its corresponding geographical location used as the final positioning result.
[0023] Figure 2 This is a schematic diagram illustrating the basic principle of the method in this embodiment during the training phase. The training of the shared feature extraction network includes: S201: Obtain the input ground images, UAV images, and satellite images that have a corresponding relationship (same geographical location), and extract features using a shared feature extraction network to obtain extracted ground image features, UAV image features, and satellite image features; for the same geographical location, obtain its corresponding ground image, UAV image, and satellite image, denoted as... , and A shared feature extraction network is employed. Feature encoding is performed on the above three types of images to obtain the corresponding ground image features, UAV image features, and satellite image features. , and Its expression is: ; S202, establish a contrastive loss for the ground-to-UAV branch for first cross-view alignment between ground image features and UAV image features, and a contrastive loss for the UAV-to-satellite branch for second cross-view alignment between UAV image features and satellite image features. Use the total loss composed of these two losses to optimize the shared feature extraction network, thereby obtaining a shared feature extraction network that can be used for cross-view representation learning of ground images, UAV images, and satellite images. The first cross-view alignment is used to constrain ground image features. With drone image features Consistency between them; second cross-view alignment is used to constrain UAV image features With satellite image features Consistency between them. Through the above settings, the optimization path of the model is progressively aligned in the order of "ground view - UAV view - satellite view", so that the UAV view plays the role of an intermediate bridge in the relationship between the three views.
[0024] In this embodiment, the functional expression for the contrast loss of the ground-to-UAV branch is: ; The functional expression for the contrast loss from the UAV to the satellite branch is: ; The functional expression for the total loss composed of the two types of losses is: ; in, For the comparison loss from the ground to the drone branch, To normalize the temperature-scaled cross-entropy loss, For ground image features, For drone image features, For satellite image features, For the comparison loss from drone to satellite branch, For the total loss, and ...
[0025] The shared feature extraction network can employ any required feature extraction network. For example, as an optional implementation, the shared feature extraction network consists of a backbone network and pooling layers. In one optional embodiment, the shared feature extraction network uses a visual Transformer-based backbone network followed by pooling layers to obtain a global embedding representation; in this embodiment, the backbone network uses DINOv2, and the pooling layers use GeM pooling (generalized average pooling) layers. Furthermore, considering the distribution differences between ground and aerial images, a view-dependent normalization strategy is used before feature normalization, where the ground branch has an independent normalization layer, and the UAV and satellite branches share the aerial normalization layer.
[0026] To verify the effectiveness of the UAV-assisted ground and satellite cross-view image geolocation method in this embodiment, the publicly available University-1652 dataset was used for experimental verification. This dataset contains three perspectives: ground images, UAV images, and satellite images. 701 buildings were used for training, and 951 buildings were used for testing. In the experiments, ground images were used as query images, satellite images as the retrieval database, and UAV images were used as intermediate auxiliary views. Evaluation metrics included recall (Recall@1, Recall@5, and Recall@10) and mean precision (mAP), considering only the top 1, 5, and 10 rankings. Higher values for these metrics indicate better cross-view retrieval and geolocation performance. Besides retrieval accuracy, online inference efficiency is also an important indicator of practicality. Relevant evaluation metrics included the number of network model parameters, FLOPs, average query latency, and additional online overhead. Table 1 shows the performance comparison results of the method in this embodiment with existing representative methods (including the U1652 method, LPN method, Sample4Geo method, PLCD method, CV-Cities method, From Street to Orbit method, SkyLink method, and VICI method) on the University-1652 dataset. Table 2 shows the comparison of the direct ground-to-satellite retrieval baseline method with the method in this embodiment in terms of computational complexity and online inference efficiency, including the number of parameters, FLOPs, average query latency, additional overhead, and Recall@1 metric.
[0027] Table 1: Comparison of objective performance of the method in this embodiment with existing methods on the University-1652 dataset.
[0028] As shown in Table 1, compared to the existing methods listed in the table, the method in this embodiment shows a significant improvement in recall@1 and mean precision, indicating that the method in this embodiment performs better in terms of the quality of the top ranking of search results and the overall ranking stability. This also shows that by introducing the UAV intermediate view and performing candidate feature aggregation and fusion, the method in this embodiment can provide additional auxiliary spatial structure information for ground-to-satellite retrieval. Therefore, the method in this embodiment performs better on multiple evaluation metrics, demonstrating that the proposed UAV-assisted cross-view positioning scheme can effectively improve the performance of ground-to-satellite retrieval.
[0029] Table 2: Comparison of computational complexity and online inference efficiency between the method in this embodiment and the baseline method.
[0030] As can be seen from Table 2, the method in this embodiment introduces only a small online inference overhead while keeping the number of model parameters and the amount of backbone computation basically unchanged, and at the same time improves the retrieval and localization performance, indicating that the method in this embodiment has a good balance between accuracy and efficiency.
[0031] In summary, the UAV-assisted ground-to-satellite cross-view image geolocation method of this embodiment acquires ground images, UAV images, and satellite images corresponding to the same geographical location; it extracts ground features, UAV features, and satellite features respectively using a shared feature extraction network; during the training phase, it establishes progressive cross-view alignment targets from ground to UAV and from UAV to satellite, and performs joint optimization by combining contrastive learning; during the inference phase, for the input ground query image, it first retrieves Top-K candidate UAV features from the UAV image library, then aggregates the candidate UAV features using a ranking and weighting method, and fuses them with ground features; finally, it uses the fused features to complete the retrieval and positioning in the satellite image library. This invention uses the UAV view as an intermediate bridge between the ground view and the satellite view, and introduces a lightweight auxiliary retrieval and fusion mechanism without changing the backbone feature extraction network structure, thereby improving the accuracy of ground-to-satellite cross-view retrieval. This method has a simple structure, low inference overhead, and practical value.
[0032] Furthermore, this embodiment also provides a UAV-assisted ground and satellite cross-view image geolocation system, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the UAV-assisted ground and satellite cross-view image geolocation method. This embodiment also provides a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the UAV-assisted ground and satellite cross-view image geolocation method via a processor. This embodiment also provides a computer program product, including a computer program or instructions programmed or configured to execute the UAV-assisted ground and satellite cross-view image geolocation method via a processor.
[0033] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0034] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.
Claims
1. A UAV-assisted geolocation method using ground and satellite cross-view images, characterized in that, Includes the following steps: S101, use a shared feature extraction network to extract features from the input ground query image; S102, based on the cosine similarity between the features of the ground query image and the features of the drone images in the drone image library, find the top images with the highest similarity. Drone image features of candidate drone images; S103, will be forward The drone image aggregate features are obtained by weighted aggregation of drone image features from 1 candidate drone images; S104, which fuses UAV image aggregation features and ground query image features into a fused feature; S105. Based on the cosine similarity between the fused features and the satellite image features in the satellite image library, find the satellite image with the highest similarity and output its corresponding geographical location as the geolocation result of the ground query image.
2. The UAV-assisted ground and satellite cross-view image geolocation method according to claim 1, characterized in that, In step S103, the front The function expression for obtaining the aggregated drone image features from the weighted aggregation of drone image features from candidate drone images is as follows: ； ； in, Aggregate features for drone images, The number of candidate drone images, Let be the weight of the i-th candidate drone image. The drone image features of the i-th candidate drone image are obtained by inputting the i-th candidate drone image into the shared feature extraction network. and This is an index for candidate drone images.
3. The UAV-assisted ground and satellite cross-view image geolocation method according to claim 1, characterized in that, The function expression for fusing the UAV image aggregation features and the ground query image features into a fused feature in step S104 is as follows: ； in, As a feature of fusion, For ground query image features, Aggregate features for drone images.
4. The UAV-assisted ground and satellite cross-view image geolocation method according to claim 1, characterized in that, In step S105, the functional expression for the similarity between the fused features and the satellite image features of satellite images in the satellite image library is as follows: ； ； in, To fuse features with the cosine similarity between satellite image features and satellite image features in the satellite image library, The normalized fusion characteristics As a feature of fusion, for L2 norm, for transpose, The satellite image features are satellite image features from satellite images in the satellite image library, which are obtained by inputting satellite images from the satellite image library into the shared feature extraction network.
5. The UAV-assisted ground and satellite cross-view image geolocation method according to any one of claims 1 to 4, characterized in that, The training of the shared feature extraction network includes: S201: Obtain the corresponding ground images, UAV images and satellite images from the input, and extract features using a shared feature extraction network to obtain the extracted ground image features, UAV image features and satellite image features; S202, establish a contrastive loss for the ground-to-UAV branch for the first cross-view alignment between ground image features and UAV image features, and a contrastive loss for the UAV-to-satellite branch for the second cross-view alignment between UAV image features and satellite image features. Use the total loss composed of the two losses to optimize the shared feature extraction network, thereby obtaining a shared feature extraction network that can be used for cross-view representation learning of ground images, UAV images and satellite images.
6. The UAV-assisted ground and satellite cross-view image geolocation method according to claim 5, characterized in that, The functional expression for the contrast loss of the ground-to-UAV branch is: ； The functional expression for the contrast loss from the UAV to the satellite branch is: ； The functional expression for the total loss composed of the two types of losses is: ； in, For the comparison loss from the ground to the drone branch, To normalize the temperature-scaled cross-entropy loss, For ground image features, For drone image features, For satellite image features, For the comparison loss from drone to satellite branch, For the total loss, and These are the weight parameters.
7. The UAV-assisted ground and satellite cross-view image geolocation method according to claim 1, characterized in that, The shared feature extraction network consists of a backbone network and a pooling layer.
8. A UAV-assisted ground and satellite cross-view image geolocation system, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to execute the UAV-assisted ground and satellite cross-view image geolocation method according to any one of claims 1 to 7.
9. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the UAV-assisted ground and satellite cross-view image geolocation method as described in any one of claims 1 to 7.
10. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the UAV-assisted ground and satellite cross-view image geolocation method as described in any one of claims 1 to 7.