An arbitrary inclination unmanned aerial vehicle-satellite geolocation method and system

Abstract

The present application belongs to the technical field of computer vision and remote sensing image processing, and specifically discloses a method and system for unmanned aerial vehicle-satellite geographic positioning at any inclination angle, which comprises the following steps: constructing an unmanned aerial vehicle-satellite geographic positioning model at any inclination angle based on structure perception weighting and view consistency; inputting unmanned aerial vehicle images or satellite images into the unmanned aerial vehicle-satellite geographic positioning model at any inclination angle based on structure perception weighting and view consistency for unmanned aerial vehicle-satellite image matching, and then realizing unmanned aerial vehicle-satellite geographic positioning at any inclination angle. The present application enhances the feature expression of the model for discriminative regions through a structure perception context weighting module. The present application solves the problems of performance degradation and insufficient generalization ability of the prior art when facing the geometric distortion, occlusion interference, feature redundancy and multi-view consistency problems in the task of unmanned aerial vehicle-satellite geographic positioning at any inclination angle, so as to better realize the task of unmanned aerial vehicle-satellite geographic positioning at any inclination angle.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and remote sensing image processing technology, specifically relating to an arbitrary tilt angle UAV-satellite geolocation method and system. Background Technology

[0002] Cross-view geolocalization is a key spatial perception technology that aims to determine the specific location of a target in the real world by comparing an input image (such as a drone image) with a geotagged reference image (such as a satellite image). This technology has broad application prospects in various fields such as drone autonomous navigation, emergency rescue, target localization, and smart cities. Traditional cross-view geolocalization methods mainly rely on handcrafted feature descriptors (such as SIFT, ORB, etc.) or template matching strategies. While effective in scenarios with minimal viewpoint changes and simple environmental structures, their feature representation capabilities and generalization performance are limited, making it difficult to meet the matching requirements of cross-view, cross-modal, and large-scale environments.

[0003] With the development of deep learning technology, feature extraction and matching methods based on convolutional neural networks and Transformers have gradually become mainstream. Representative works such as CVM-Net and DSM employ a dual-branch network structure to process ground and aerial images separately, and achieve cross-view matching through feature aggregation and similarity measurement. Furthermore, the introduction of datasets such as University-1652 and DenseUAV has promoted the development of UAV-satellite geolocation tasks, enabling models to be trained and evaluated over a wider range of viewpoints and altitudes. These methods typically rely on global pooling strategies and pairwise contrastive learning mechanisms (such as InfoNCE loss), achieving significant results in matching images viewed from below (UAV tilt angles are typically less than 30° vertically). However, they are still based on the "viewpoint consistency assumption," which assumes that UAV images and satellite images are highly aligned in visual structure and semantic content, ignoring the fundamental challenges posed by the large tilt angle shooting conditions commonly encountered in real-world applications.

[0004] In practical drone operations, influenced by flight path planning, obstacle avoidance requirements, and mission type, drone cameras often cannot maintain a vertical downward shooting angle. Instead, they capture images at large tilt angles (e.g., 70°–85° from the vertical), resulting in severe perspective projection distortion, scale inconsistencies, occlusion, and significant appearance differences. The spatial correspondence between these images and satellite images is greatly weakened, repetitive patterns (e.g., rooftops, roads) become more interfering, and discriminative structural features (e.g., building outlines, road intersections) become sparse or incomplete due to the tilted perspective. Although a few studies have attempted to introduce attention mechanisms or multi-level feature fusion to enhance the model's ability to perceive local structures, these methods generally lack explicit modeling of feature redundancy and fail to fully utilize the inherent semantic consistency of multi-view images under a unified geographical location.

[0005] Furthermore, most existing geolocation methods are built around near-downward perspectives, covering mainly urban and campus scenes with limited environmental types and lacking complex terrains such as mountains, ports, and coastlines, failing to fully reflect the geographical diversity and structural complexity of the real world. More importantly, their evaluation methods are typically based on a "one-to-one" perfect matching assumption, assuming that each UAV image corresponds to only one perfectly aligned satellite image patch. However, real-world UAV images often partially overlap with multiple satellite image patches, requiring fuzzy matching and spatial reasoning from a large candidate database, which places higher demands on the model's discriminativeness and robustness. Experiments show that models trained on existing datasets experience a sharp performance drop when directly applied to images with large tilt angles, fully exposing the insufficient generalization ability of existing methods under extreme perspectives.

[0006] Therefore, in the face of geometric distortion, occlusion interference, feature redundancy, and multiple views in UAV-satellite geolocation tasks at arbitrary tilt angles... Figure 1 For the consistency issue, existing technologies have not yet provided a systematic solution, and there is an urgent need for a new geolocation framework that can integrate structural awareness and multi-view semantic understanding. Summary of the Invention

[0007] The purpose of this invention is to solve the problems of geometric distortion, occlusion interference, feature redundancy, and multiple views in existing technologies for UAV-satellite geolocation tasks with arbitrary tilt angles. Figure 1 When dealing with inconsistency issues, there are problems of performance degradation and insufficient generalization ability. To address this, we propose an arbitrary tilt angle UAV-satellite geolocation method and system that can effectively handle geometric distortion, occlusion interference, and appearance differences caused by extreme viewing angles.

[0008] The technical solution of the present invention is as follows: Firstly, an arbitrary tilt angle UAV-satellite geolocation method, comprising the following steps:

[0009] Acquire drone and satellite images;

[0010] Constructing a structure-aware weighted and visual Figure 1 A model for arbitrary tilting UAVs-satellite geolocation;

[0011] Input UAV images and satellite images into a structure-aware weighted and visual input method. Figure 1 The model of arbitrary tilt UAV-satellite geolocation is used to match UAV-satellite images, thereby achieving arbitrary tilt UAV-satellite geolocation.

[0012] Based on structure-aware weighted and visual Figure 1 The arbitrary tilt UAV-satellite geolocation model is a deep visual feature extraction network that integrates a structure-aware context-weighted module;

[0013] The input of the structure-aware context-weighted module is connected to the output of the visual backbone network in the deep visual feature extraction network. It is used to adaptively suppress redundant regions in the image and highlight discriminative local structural features, thereby obtaining the final structured representation of the image.

[0014] As a preferred approach, the structure-aware context-weighted module adaptively suppresses redundant regions in the image and highlights discriminative local structural features, thereby obtaining the final structured representation of the image. The specific method is as follows:

[0015] Extract local and global feature sets from UAV or satellite images using a visual encoder;

[0016] By using two independent two-layer multilayer perceptron projection networks, local features and global features are projected onto a d-dimensional shared embedding space to obtain the projected local features and global features.

[0017] Calculate the self-similarity matrix between the local features after each projection;

[0018] Calculate the redundancy weight for each spatial location based on the self-similarity matrix;

[0019] K learnable cluster centers are introduced, the cluster assignment score for each feature is calculated, and the cluster assignment score is modulated using redundant weights to obtain soft assignment weights.

[0020] Each cluster is calculated based on soft-assigned weights. The weighted summation of all projected local features in the matrix yields the first... Descriptors for each cluster;

[0021] All cluster descriptors are concatenated with the projected global features to form the final structured representation of the UAV or satellite image. , Represents the global features after projection. Indicates splicing, Indicates the first Descriptors for each cluster, .

[0022] Preferably, the redundancy weight The calculation formula is:

[0023]

[0024] in, Represents the first in the self-similarity matrix Line number Column elements, This represents the number of spatial locations in the neighborhood of the projected local feature. and For learnable parameters, This refers to the Sigmod function;

[0025] The soft allocation weight The calculation formula is:

[0026]

[0027] in, Represents an exponential function. Indicates the first The local features after projection and the first The similarity of cluster centers This represents a constant, less than a preset threshold, introduced to prevent the denominator from being zero. Indicates the variable index in the summation of the denominator. Indicates the first The local features after projection and the first The similarity of the cluster centers;

[0028] No. Descriptors of clusters The calculation formula is:

[0029]

[0030] in, Indicates the first Local features after projection.

[0031] As a preferred option, construct a structure-aware weighted and visual... Figure 1 The arbitrary tilt angle UAV-satellite geolocation model includes the following steps:

[0032] Construct an arbitrary tilt UAV-satellite geolocation model based on structure perception weighting, and perform initial training on the arbitrary tilt UAV-satellite geolocation model based on structure perception weighting;

[0033] Using visual Figure 1 To achieve this strategy, the structure-aware weighted UAV-satellite geolocation model, which had undergone initial training, was retrained using semantic groups as units, to obtain a structure-aware weighted and visual-based model. Figure 1 A model for arbitrary tilting UAVs-satellite geolocation.

[0034] As a preferred option, visual Figure 1 The specific method for retraining the structure-aware weighted UAV-satellite geolocation model based on the initial training, using semantic groups as units, is as follows:

[0035] For each geographic location, collect one satellite image and M drone images taken from different tilt angles to form a semantic group;

[0036] Using the structure-aware context-weighted module, the final structured representation of all images within the group is extracted;

[0037] All final structured representation samples within a group are defined as positive samples, and final structured representation samples between groups are defined as negative samples. Multi-similarity loss is used for optimization to obtain an incrementally optimized UAV-satellite geolocation model with arbitrary tilt angle.

[0038] Preferably, the formula for the multi-similarity loss is:

[0039]

[0040] in, This represents the loss due to multiple similarities. Represents the positive sample set. Represents the negative sample set. Indicates a positive sample. Indicates a negative sample. Represents cosine similarity. Indicates anchor point sample, and This represents the weighting function for difficult samples.

[0041] Preferably, the method further includes:

[0042] Through many-to-many evaluation and partial matching mechanisms, the structure-aware weighted and visual... Figure 1 The performance of the arbitrary tilt angle UAV-satellite geolocation model was evaluated.

[0043] Preferably, the many-to-many evaluation and partial matching mechanism is as follows:

[0044] The complete satellite base map of the test area is segmented using a sliding window to generate multiple partially overlapping satellite image patches, thereby constructing a dense candidate library;

[0045] Calculate the cosine similarity between the embedding representation of each queried drone image and the embedding representation of all satellite image patches in the dense candidate library;

[0046] Retrieve the top L candidate satellite image patches with the highest cosine similarity, and calculate the intersection-union ratio between each candidate satellite image patch and the actual location of the query UAV image;

[0047] If the intersection-union ratio of a candidate satellite image patch exceeds a predetermined threshold, the search is successful; otherwise, the search is unsuccessful.

[0048] Based on the search results, the recall rate and average precision of all queried drone images were statistically analyzed to obtain a result based on structure-aware weighted and visual... Figure 1 The comprehensive evaluation results of the arbitrary tilt angle UAV-satellite geolocation model.

[0049] The beneficial effects of this invention are:

[0050] 1. This invention proposes a method based on structure-aware weighted and visual... Figure 1 This paper presents a satellite geolocation method for UAVs with arbitrary tilt angles. By fusing a structured perception feature aggregation mechanism with a group-level consistency learning strategy and employing a realistic many-to-many evaluation approach, it effectively addresses the performance degradation caused by geometric distortion, occlusion, and appearance differences at extreme tilt angles. This method overcomes the reliance of existing technologies on the "downward-looking perspective assumption" and the "perfect matching assumption," significantly improving positioning accuracy, robustness, and generalization ability in complex real-world scenarios.

[0051] 2. This invention adaptively suppresses redundant regions in images (such as numerous similar rooftops and road surfaces) and highlights discriminative local structural features (such as building edges, road intersections, and unique landforms) through a structure-aware context-weighted module. Combined with visual... Figure 1 The learning strategy imposes semantic consistency constraints on multi-view images of the same location, enabling the model to penetrate the interference of viewpoint changes and capture invariant features across modalities.

[0052] 3. The feature representations learned in this invention possess strong viewpoint invariance and semantic robustness, thanks to the view... Figure 1 The learning strategy fully explores positive samples from multiple perspectives and optimizes them specifically for difficult negative samples during the training phase. The model no longer relies on specific perspectives or appearance cues, but learns to extract high-level semantic features that are insensitive to geometric changes.

[0053] 4. The "many-to-many evaluation and partial matching mechanism" adopted in this invention completely abandons the unrealistic assumption of the traditional "one-to-one perfect matching," simulating the complex scenario of partial overlap between UAV images and multiple satellite image patches in real-world applications. This mechanism requires the model to possess strong spatial reasoning capabilities, enabling it to identify the most reasonable matching item in spatial relationships from a large number of highly similar candidate patches, thus allowing the evaluation results to more accurately reflect the model's performance in the actual system. Simultaneously, the solution provided by this invention does not rely on precise initial alignment and can handle fuzzy and partial matching queries, providing reliable technical support for high-precision autonomous positioning and navigation of UAVs in complex geographical environments such as urban canyons, mountains, and coastal areas, thus possessing higher practical application value and deployment feasibility.

[0054] Secondly, an arbitrary tilt angle UAV-satellite geolocation system includes:

[0055] The image acquisition module is used to acquire images from drones and satellites;

[0056] The model building module is used to build models based on structure-aware weighted and visual... Figure 1 A model for arbitrary tilting UAVs-satellite geolocation;

[0057] The matching and localization module is used to input UAV images and satellite images into a structure-aware weighted and visual input system. Figure 1 The model of arbitrary tilt UAV-satellite geolocation is used to match UAV-satellite images, thereby achieving arbitrary tilt UAV-satellite geolocation.

[0058] Thirdly, a non-transitory computer-readable storage medium is provided that stores computer instructions for causing a computer to perform the method as described in the first aspect. Attached Figure Description

[0059] Figure 1 The diagram shows a flowchart of an arbitrary tilt UAV-satellite geolocation method.

[0060] Figure 2 The diagram shows a structure-aware weighted average and view-based weighted average. Figure 1 Overall framework diagram of the UAV-satellite geolocation method for arbitrary tilt angle learning.

[0061] Figure 3 The results shown are comparative experimental results of the method of this invention and the traditional methods CAMP and Sample4GEO. Detailed Implementation

[0062] Exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the embodiments shown and described in the drawings are merely exemplary and are intended to illustrate the principles and spirit of the invention, and are not intended to limit the scope of the invention.

[0063] Example 1:

[0064] This invention designs a structure-aware weighted and visual method. Figure 1 This paper proposes a satellite geolocation method for arbitrary tilt UAVs, aiming to suppress redundant region interference and enhance discriminative structural representation through a structure-aware feature weighting mechanism, and to utilize multi-view... Figure 1 This method employs a learning strategy to uncover semantic consistency between images from different perspectives within the same geographic location, thereby achieving more robust, accurate, and highly generalizable UAV-satellite image matching at any tilt angle. It is applicable to real-world scenarios such as UAV autonomous navigation, target localization, and disaster emergency response. The method primarily comprises two core components: the first is a structure-aware context-weighted module, which enhances the model's feature representation of discriminative regions and suppresses interference from repetitive structures and occluded areas by performing redundant clustering and weight allocation on local features; the second is a view... Figure 1 The learning strategy involves constructing semantic groups from multi-view drone images and corresponding satellite images of the same geographical location, thereby achieving consistency constraints within groups and learning differences between groups, thus improving the model's adaptability to changes in perspective.

[0065] like Figure 1 As shown, an arbitrary tilt UAV-satellite geolocation method includes the following steps:

[0066] S1. Acquire drone and satellite images;

[0067] S2. Construct a structure-aware weighted and view-based system. Figure 1 A model for arbitrary tilting UAVs-satellite geolocation;

[0068] S3. Input UAV images and satellite images into a structure-aware weighted and visual input system. Figure 1 The model of arbitrary tilt UAV-satellite geolocation is used to match UAV-satellite images, thereby achieving arbitrary tilt UAV-satellite geolocation.

[0069] In this embodiment, the structure-aware weighted and visual... Figure 1 The arbitrary tilt UAV-satellite geolocation model is a deep visual feature extraction network that integrates a structure-aware context-weighted module;

[0070] The input of the structure-aware context-weighted module is connected to the output of the visual backbone network in the deep visual feature extraction network. It is used to adaptively suppress redundant regions in the image and highlight discriminative local structural features, thereby obtaining the final structured representation of the image.

[0071] In this embodiment, the structure-aware context-weighted module adaptively suppresses redundant regions in the image and highlights discriminative local structural features, thereby obtaining the final structured representation of the image. The specific method is as follows:

[0072] Extracting local feature sets from UAV or satellite images using a visual encoder. and global feature t, where, Indicates the first Local features, Represents the total number of local features. , , Let represent an l-dimensional real vector space.

[0073] Through two independent two-layer multilayer sensor projection networks and Local and global features are mapped to a d-dimensional shared embedding space, respectively:

[0074] in, Indicates the first Local features after projection Represents the global features after projection. Represents a d-dimensional shared embedding space;

[0075] Calculate the self-similarity matrix among all projected local features. Each element in the self-similarity matrix Used to quantify spatial redundancy;

[0076] Calculate the redundancy weight for each spatial location based on the self-similarity matrix. :

[0077]

[0078] in, and For learnable parameters, This refers to the Sigmod function;

[0079] K learnable cluster centers are introduced, and the cluster assignment score for each feature is calculated. Redundant weights are then used to modulate the cluster assignment scores to obtain soft assignment weights. :

[0080]

[0081] in, Represents an exponential function. Indicates the first The local features after projection and the first The similarity of cluster centers This represents a constant, less than a preset threshold, introduced to prevent the denominator from being zero. Indicates the variable index in the summation of the denominator. Indicates the first The local features after projection and the first The similarity of the cluster centers;

[0082] Each cluster is calculated based on soft-assigned weights. The descriptor for each cluster is obtained by weighted summation of all features. :

[0083]

[0084] All cluster descriptors are concatenated with the projected global features to form the final structured representation of the UAV or satellite image. , Indicates splicing.

[0085] In this embodiment, a structure-aware weighted average and visual... Figure 1 The arbitrary tilt angle UAV-satellite geolocation model includes the following steps:

[0086] By integrating the structure-aware context-weighted module into a traditional deep visual feature extraction network, an arbitrary tilt UAV-satellite geolocation model based on structure-aware weighting is constructed.

[0087] Initial training was conducted on an arbitrary tilt UAV-satellite geolocation model based on structure-aware weighting.

[0088] Using visual Figure 1 To achieve this strategy, the structure-aware weighted UAV-satellite geolocation model, which had undergone initial training, was retrained using semantic groups as units, to obtain a structure-aware weighted and visual-based model. Figure 1 A model for arbitrary tilting UAVs-satellite geolocation.

[0089] In this embodiment, a view is used. Figure 1 The specific method for retraining the structure-aware weighted UAV-satellite geolocation model based on the initial training, using semantic groups as units, is as follows:

[0090] For each geographic location, collect one satellite image and M drone images taken from different large tilt angles to form a semantic group;

[0091] The feature extraction and structure-aware context-weighted module with shared weights is used to extract the embedding representation of all images in the group, i.e. the final structured representation.

[0092] We define all embedded representation samples within a group as positive samples and embedded representation samples between groups as negative samples. We use multi-similarity loss for optimization to obtain an incrementally optimized UAV-satellite geolocation model with arbitrary tilt angle.

[0093] The formula for multi-similarity loss is:

[0094]

[0095] in, This represents the loss due to multiple similarities. Represents the positive sample set. Represents the negative sample set. Indicates a positive sample. Indicates a negative sample. Represents cosine similarity. Indicates anchor point sample, and This represents the weighting function for difficult samples. Difficult samples refer to samples whose features, after feature extraction by the model, lie on the classification boundary. Specifically, in this task, for a drone image to be localized, a difficult positive sample is one that shares the same geographical location as the anchor image but exhibits variations in time, season, weather, etc.; a difficult negative sample is one that is located at a different location than the anchor image but happens to be highly similar to the buildings, road layouts, etc., within the captured area.

[0096] In this embodiment, the method further includes:

[0097] Through many-to-many evaluation and partial matching mechanisms, the structure-aware weighted and visual... Figure 1 The performance of the arbitrary tilt UAV-satellite geolocation model was evaluated, specifically as follows:

[0098] The complete satellite base map of the test area is segmented using a sliding window to generate multiple partially overlapping satellite image patches, thus constructing a dense candidate library;

[0099] Calculate the cosine similarity between the embedding representation of each query drone image and the embeddings of all satellite image patches in the dense candidate library;

[0100] Retrieve the top L candidate blocks with the highest cosine similarity and calculate the intersection-union ratio between each candidate block and the true location of the query image;

[0101] If the intersection-union ratio of a candidate block exceeds a predetermined threshold, the search is successful; otherwise, the search is unsuccessful.

[0102] Statistical analysis of recall and mean precision for all queries yields results based on structure-aware weighted and view-based algorithms. Figure 1 The comprehensive evaluation results of the arbitrary tilt angle UAV-satellite geolocation model.

[0103] like Figure 2 The diagram shows a structure-aware weighted average and view-based weighted average. Figure 1 The overall framework diagram of the arbitrary tilt UAV-satellite geolocation method for learning is shown. The left half of the model is a structure-aware context-weighted module. The input UAV or satellite image is used by a visual encoder to extract a set of local and global features. Then, two independent multilayer perceptrons project the local and global features into a low-dimensional shared embedding space and calculate the self-similarity matrix between local features to quantify spatial redundancy, thereby generating redundant weights for each location. Finally, learnable semantic clustering centers are introduced, and the clustering score is modulated using redundant weights. Multiple clustering descriptors are formed through weighted aggregation and concatenated with the projected global features to generate a robust image representation that preserves global semantics while highlighting discriminative local structures. The right half is the visual... Figure 1 The learning strategy constructs a semantic group from a satellite image and M multi-view UAV images of the same geographic location. During training, constraints are no longer imposed on individual image pairs, but on semantic groups. This ensures that all samples within a group are close to each other in the feature space, while samples between groups are far apart. A multi-similarity loss function is used to mine difficult sample pairs to optimize the model, thereby forcing the model to learn viewpoint-invariant feature representations, enhancing its adaptability to extreme viewpoint changes and its generalization performance in cross-modal matching.

[0104] In this embodiment, to address the challenges posed by severe perspective distortion, occlusion, and numerous repetitive structures (such as similar roofs and roads) in UAV images at arbitrary tilt angles, this invention proposes a structure-aware context-weighted module. This module can be integrated into existing deep visual feature extraction networks to achieve adaptive and learnable feature selection and aggregation.

[0105] The structure-aware context-weighted module operates in a feedforward manner, first receiving raw local feature maps and global feature tokens extracted from a visual backbone network (such as ViT). Then, through two independent lightweight projection networks, the local features and global tokens are mapped into a shared, low-dimensional embedding space. This step not only reduces subsequent computational complexity but, more importantly, provides adapted feature representations for subsequent structured modeling.

[0106] To quantify feature redundancy, the structure-aware context-weighted module calculates pairwise similarities between all spatial location features, forming a self-similarity matrix. This self-similarity matrix characterizes the visual similarity between different regions within an image. Based on this matrix, a redundancy weight is further calculated for each local location. This weight reflects the overall similarity between the features at each local location and other regions in the image. A higher redundancy weight indicates a more general and less unique feature in the corresponding region, such as large areas of asphalt pavement or continuously arranged standardized rooftops. Conversely, a lower redundancy weight indicates a more unique and discriminative feature in the corresponding region, such as road intersections, uniquely shaped building outlines, or abrupt changes in natural terrain.

[0107] To further achieve high-level semantic aggregation, this invention introduces a set of learnable semantic cluster centers, each representing a potential semantic pattern or structural type in the image. Through a lightweight scoring network, each local feature receives an assigned score belonging to each cluster center. Crucially, the final soft assignment weights are modulated by the aforementioned redundant weights, significantly suppressing the contribution of highly redundant features to the formation of any cluster center. Finally, the descriptor for each cluster center is constructed from a weighted average of its member features. The final image representation is then formed by concatenating the modulated cluster descriptors with a projected global token, creating a robust feature representation that incorporates both global contextual information and emphasizes local discriminative structure.

[0108] In this embodiment, considering the characteristic that there are multiple significantly different observation angles at the same geographical location under arbitrary tilt angle conditions, the present invention proposes a viewing angle... Figure 1 This learning strategy fundamentally changes the traditional supervised learning model based on "image pairs".

[0109] The core idea of ​​this strategy is to construct "semantic groups" as the basic unit of training. Each semantic group contains a satellite image from the target area and a set of drone images taken from multiple different angles above the horizon at the same geographical location. Although these images differ greatly in viewpoint, scale, and appearance, they depict the exact same physical scene.

[0110] During training, the model no longer simply learns the correspondence between a single drone image and a satellite image, but is forced to understand the inherent consistency among all samples within the entire group. Specifically, all image feature embeddings from the same semantic group, regardless of whether they originate from a satellite modality or any drone viewpoint, are considered positive samples in the feature space and should be close to each other; while feature embeddings from different semantic groups are considered negative samples and should be far from each other.

[0111] To achieve this goal, this invention employs a group-based multi-similarity loss function for optimization. This loss function not only considers the absolute similarity between sample pairs but also focuses on identifying difficult sample pairs to improve the model's discriminative ability. It assigns higher weights to positive sample pairs with low similarity to anchor samples, prompting the model to better bridge those difficult-to-align perspectives; simultaneously, it also focuses on negative sample pairs with high similarity to anchor samples (i.e., easily confused negative samples), thereby enhancing the model's ability to distinguish different geographical locations.

[0112] Through this group-level supervision signal, the model is forced to abandon its dependence on specific viewpoints or appearance cues, and instead learn to mine deep, invariant scene structural semantics that span different viewpoints. This makes the final learned feature representation possess strong viewpoint invariance, effectively associating the performance of the same location under satellite views and arbitrarily large-angle drone views, greatly improving the model's generalization ability and matching accuracy under extreme viewpoints.

[0113] In this embodiment, to ensure that the capability assessment of the proposed method closely reflects the complexity and challenges of real-world applications, this invention designs and implements a novel, groundbreaking many-to-many assessment and partial matching mechanism. This mechanism fundamentally abandons the idealized assumption of "perfect one-to-one matching" commonly used in existing benchmarks, and instead simulates the real search and matching scenarios faced by UAVs in actual operations.

[0114] In the evaluation framework of this invention, the complete satellite base map of each test area is first segmented into multiple partially overlapping image patches using a sliding window method, constructing a dense candidate image library. For each UAV query image, the system retrieves its L most similar satellite image patches, and then determines whether the retrieval is correct based on the spatial overlap criterion.

[0115] Specifically, a retrieval is considered successful when the intersection-over-union (IoU) ratio between a satellite image patch and the actual geographic location of a drone image exceeds a predetermined threshold. This evaluation mechanism reflects a key understanding: in real-world scenarios, there may be multiple reasonable matching candidates, rather than a single correct answer. This makes the evaluation results more reflective of the model's performance in actual deployments.

[0116] This many-to-many evaluation method requires models to possess stronger discriminative and spatial reasoning capabilities. The model not only needs to understand visual similarities between images but also needs an implicit understanding of spatial relationships, enabling it to identify the most likely correct match from multiple candidate blocks that partially match. This is fundamentally different from traditional evaluation protocols that assume perfect alignment, providing a more reliable and practical performance evaluation standard for the widespread application of geolocation technology.

[0117] In this embodiment, data from various scenarios, including urban buildings, rural buildings, and port vessels, were selected to conduct comparative experiments on the method of this invention and the traditional methods CAMP and Sample4GEO. The results are as follows: Figure 3 As shown in the diagram, blue boxes indicate drone images to be located, green boxes represent correctly located results, and red boxes represent incorrectly located results. The comparison shows that the method proposed in this invention significantly outperforms the other two methods in the Top 3 search results: in four sets of test data, this method only resulted in one incorrect search, while the CAMP and Sample4GEO methods each resulted in eight incorrect search results.

[0118] This invention proposes a structure-aware weighted and visual method. Figure 1 This paper proposes a high-inclination UAV-satellite geolocation method. By fusing a structured perception feature aggregation mechanism with a group-level consistency learning strategy and employing a realistic many-to-many evaluation approach, it effectively addresses the performance degradation caused by geometric distortion, occlusion, and appearance differences under extreme high inclination angles. This method overcomes the reliance of existing technologies on the "downward view assumption" and "perfect matching assumption," significantly improving positioning accuracy, robustness, and generalization ability in complex real-world scenarios. Its effectiveness is mainly reflected in the following three aspects:

[0119] (1) Positioning accuracy is significantly improved

[0120] This invention adaptively suppresses redundant regions in images (such as numerous similar rooftops and road surfaces) and highlights discriminative local structural features (such as building edges, road intersections, and unique landforms) through a structure-aware context-weighted module. Combined with visual... Figure 1 The learning strategy imposes semantic consistency constraints on multi-view images of the same location, enabling the model to penetrate the interference of viewpoint changes and capture invariant features across modalities. On tasks specifically designed for steep tilt scenes, the method of this invention significantly outperforms existing mainstream methods in key metrics such as recall and mean precision, demonstrating its superior matching accuracy under extreme viewpoints.

[0121] (2) It has stronger robustness and generalization ability.

[0122] The feature representations learned in this invention possess strong viewpoint invariance and semantic robustness. This is thanks to the view... Figure 1The learning strategy fully leverages multi-view positive samples and optimizes for challenging negative samples during the training phase. The model no longer relies on specific viewpoints or appearance cues, but instead learns to extract high-level semantic features insensitive to geometric changes. This advantage is fully validated in zero-shot transfer experiments: models trained solely on large-angle data, without any fine-tuning, consistently outperform state-of-the-art methods trained specifically on near-downward-view datasets such as University-1652 and SUES-200, demonstrating superior cross-dataset and cross-scene generalization capabilities.

[0123] (3) It has higher practical application value and more reliable evaluation results.

[0124] The "many-to-many evaluation and partial matching mechanism" employed in this invention completely abandons the unrealistic assumption of the traditional "one-to-one perfect matching," simulating the complex scenario of partial overlap between UAV images and multiple satellite image patches in real-world applications. This mechanism requires the model to possess strong spatial reasoning capabilities, enabling it to identify the most reasonable spatial match from a large number of highly similar candidate patches, thus allowing the evaluation results to more accurately reflect the model's performance in actual systems. Furthermore, the solution provided by this invention does not rely on precise initial alignment and can handle fuzzy and partial matching queries, providing reliable technical support for high-precision autonomous positioning and navigation of UAVs in complex geographical environments such as urban canyons, mountains, and coastal areas, thus possessing higher practical application value and deployment feasibility.

[0125] Example 2:

[0126] Based on Embodiment 1, this embodiment of the invention provides an arbitrary tilt angle UAV-satellite geolocation system, which can be used to implement the arbitrary tilt angle UAV-satellite geolocation method as described in the foregoing embodiments. The system includes:

[0127] The image acquisition module is used to acquire images from drones or satellites;

[0128] The model building module is used to build models based on structure-aware weighted and visual... Figure 1 A model for arbitrary tilting UAVs-satellite geolocation;

[0129] The matching and positioning module is used to input UAV images or satellite images into a structure-aware weighted and visual input system. Figure 1 The model of arbitrary tilt UAV-satellite geolocation is used to match UAV-satellite images, thereby achieving arbitrary tilt UAV-satellite geolocation.

[0130] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0131] In an exemplary embodiment, the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the arbitrary tilt UAV-satellite geolocation method as described in Embodiment 1 above.

[0132] In an exemplary embodiment, the readable storage medium may be a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the arbitrary tilt UAV-satellite geolocation method according to Embodiment 1 above.

[0133] In an exemplary embodiment, the computer program product includes a computer program that, when executed by a processor, implements the arbitrary tilt angle UAV-satellite geolocation method according to Embodiment 1 above.

[0134] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0138] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0139] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for arbitrary inclination unmanned aerial vehicle-satellite geolocation, characterized in that, The method comprises the following steps: acquiring unmanned aerial vehicle images and satellite images; constructing a structure perception weighting and view consistent arbitrary angle unmanned aerial vehicle-satellite geographic positioning model, specifically, for each geographic location, collecting a satellite image and M unmanned aerial vehicle images taken from different angles to form a semantic group; using a view consistent strategy, retraining the structure perception weighting based arbitrary angle unmanned aerial vehicle-satellite geographic positioning model that has completed initial training in units of semantic groups to obtain the structure perception weighting and view consistent arbitrary angle unmanned aerial vehicle-satellite geographic positioning model; inputting the unmanned aerial vehicle images and the satellite images into the structure perception weighting and view consistent arbitrary angle unmanned aerial vehicle-satellite geographic positioning model to perform unmanned aerial vehicle-satellite image matching, and then realizing arbitrary angle unmanned aerial vehicle-satellite geographic positioning; the structure perception weighting and view consistent arbitrary angle unmanned aerial vehicle-satellite geographic positioning model is a deep visual feature extraction network integrated with a structure perception context weighting module; the input end of the structure perception context weighting module is connected with the output end of a visual backbone network in the deep visual feature extraction network, and is used for adaptively suppressing redundant regions in images and highlighting discriminative local structure features, and then obtaining a final structured representation of the images; the method that the structure perception context weighting module adaptively suppresses redundant regions in images and highlights discriminative local structure features, and then obtains a final structured representation of the images specifically comprises: extracting a local feature set and a global feature of the unmanned aerial vehicle image or the satellite image through a visual encoder; projecting the local feature and the global feature into a d-dimensional shared embedding space through two independent two-layer multilayer perception machine projection networks to obtain projected local features and global features; calculating a self-similarity matrix between the projected local features; calculating a redundancy weight of each spatial position according to the self-similarity matrix; introducing K learnable clustering centers, calculating a clustering assignment score of each feature, and modulating the clustering assignment score by using the redundancy weight to obtain a soft assignment weight; According to the soft assignment weight, each cluster is calculated The weighted sum of all projected local features in the descriptor of the first cluster is obtained. concatenating all clustered descriptors with the projected global features to form a final structured representation of the drone image or satellite image , representing the projected global features, representing concatenating, representing the descriptors of the th cluster, ; The redundancy weight The calculation formula is: wherein, denotes the element of the self-similarity matrix in the i-th row and j-th column, denotes the element of the self-similarity matrix in the i-th row and j-th column, denotes the element of the self-similarity matrix in the i-th row and j-th column, denotes the number of spatial positions in the neighborhood of the projected local feature, and are learnable parameters, denotes the Sigmod function; The soft allocation weights The calculation formula is: wherein, denotes an exponential function, denotes the similarity of the -th projected local feature to the -th cluster center, denotes a constant smaller than a pre-set threshold introduced to prevent the denominator from being zero, denotes the variable index in the denominator summation, denotes the similarity of the -th projected local feature to the -th cluster center; The formula for calculating the descriptor of the first cluster is: The formula for calculating the descriptor of the first cluster is:​ wherein, represents the local feature of the th projected image.

2. The arbitrary inclination unmanned aircraft-satellite geo-location method of claim 1, wherein, constructing the structure perception weighting and view consistent arbitrary angle unmanned aerial vehicle-satellite geographic positioning model specifically comprises the following steps: constructing a structure perception weighting based arbitrary angle unmanned aerial vehicle-satellite geographic positioning model, and performing initial training on the structure perception weighting based arbitrary angle unmanned aerial vehicle-satellite geographic positioning model; using a view consistent strategy, retraining the structure perception weighting based arbitrary angle unmanned aerial vehicle-satellite geographic positioning model that has completed initial training in units of semantic groups to obtain the structure perception weighting and view consistent arbitrary angle unmanned aerial vehicle-satellite geographic positioning model.

3. The arbitrary inclination unmanned aircraft-satellite geo-location method of claim 2, wherein, the method of retraining the structure perception weighting based arbitrary angle unmanned aerial vehicle-satellite geographic positioning model that has completed initial training in units of semantic groups using a view consistent strategy specifically comprises: for each geographic location, collecting a satellite image and M unmanned aerial vehicle images taken from different angles to form a semantic group; using the structure perception context weighting module to extract a final structured representation of all images in the group; All final structured representation samples in the group are positive samples, and the final structured representation samples between groups are negative samples, a multi-similarity loss is used for optimization, and an incremental optimized arbitrary inclination UAV-satellite geolocation model is obtained.

4. The arbitrary inclination unmanned aircraft-satellite geo-location method of claim 3, wherein, The formula of the multi-similarity loss is: wherein, represents a multi-similarity loss, represents a positive sample set, represents a negative sample set, represents a positive sample, represents a negative sample, represents a cosine similarity, represents an anchor sample, and represents a weighting function for difficult samples.

5. The arbitrary inclination unmanned aircraft-satellite geo-location method of claim 1, wherein, The method further comprises: The performance of the arbitrary inclination UAV-satellite geolocation model based on structure perception weighting and view consistency is evaluated by a many-to-many evaluation and partial matching mechanism.

6. The arbitrary inclination unmanned aircraft-satellite geo-location method of claim 5, wherein, The many-to-many evaluation and partial matching mechanism is specifically: A plurality of partially overlapped satellite image blocks are generated by segmenting the complete satellite base map of the test area by using a sliding window, and a dense candidate library is constructed; Cosine similarity between the embedding representation of each query UAV image and the embedding representation of all satellite image blocks in the dense candidate library is calculated; The first L candidate satellite image blocks with the highest cosine similarity are retrieved, and the intersection over union between each candidate satellite image block and the true position of the query UAV image is calculated; If the intersection over union of a certain candidate satellite image block exceeds a predetermined threshold, the retrieval is successful; if not, the retrieval is unsuccessful; According to the retrieval result, the recall rate and the average precision of all query UAV images are calculated to obtain the comprehensive evaluation result of the arbitrary inclination UAV-satellite geolocation model based on structure perception weighting and view consistency.

7. An arbitrary inclination unmanned plane-satellite geo-positioning system based on the arbitrary inclination unmanned plane-satellite geo-positioning method according to any one of claims 1-6, characterized by, It comprises: An image acquisition module for acquiring UAV images and satellite images; A model construction module for constructing an arbitrary inclination UAV-satellite geolocation model based on structure perception weighting and view consistency; A matching positioning module for inputting the UAV images and the satellite images into the arbitrary inclination UAV-satellite geolocation model based on structure perception weighting and view consistency for UAV-satellite image matching, and then realizing arbitrary inclination UAV-satellite geolocation.

8. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to make the computer execute the method according to any one of claims 1-6.

Images