A cross-view geographical positioning retrieval method and device for unmanned aerial vehicles and satellites

Abstract

The application is a cross-view geographical positioning retrieval method and device for unmanned aerial vehicles and satellites. The cross-view geographical positioning retrieval method for unmanned aerial vehicles and satellites comprises the following steps: S1, acquiring a set of unmanned aerial vehicle view images and satellite view images, and determining a query image and a gallery image; S2, preprocessing the query image and the gallery image; S3, inputting the preprocessed query image and the gallery image into a basic visual coding network to obtain global mark features and spatial patch token features of the query image and the gallery image; S4, normalizing the global mark features to obtain global features; S5, dividing the spatial patch token features into multiple concentric square ring local regions, and then performing attention residual pooling, projection and normalization, and learning weighted combination to obtain local features; and S6, calculating the similarity between the query image and the gallery image according to the global features and the local features, sorting, and outputting a retrieval result. The application improves the cross-view geographical positioning retrieval performance between the unmanned aerial vehicle view and the satellite view.

Description

Technical Field

[0001] This invention belongs to the field of cross-view geolocation technology, specifically relating to a cross-view geolocation retrieval method and device for UAVs and satellites. Background Technology

[0002] With the development of UAV remote sensing, satellite remote sensing, smart cities, emergency inspection, and geographic information systems, the scale of geographic image data collected by different platforms continues to grow. UAV-view images typically feature flexible shooting angles, rich target details, and clear local textures, while satellite-view images typically feature wide coverage, stable top-down structure, and complete spatial layout. Due to significant differences between the two types of images in imaging height, viewing direction, scale range, target morphology, background distribution, and spatial structure, establishing a stable and accurate correspondence between UAV and satellite images is a crucial technical challenge in the field of cross-view geolocation and retrieval.

[0003] Existing cross-view geolocation methods typically map UAV images and satellite images to the same feature space and perform target retrieval through feature similarity calculation. However, in complex urban, campus building, and road environments, relying solely on a single global feature is susceptible to changes in shooting angle, local occlusion, scale differences, and background interference, leading to mismatches between similar buildings, roads, or scenes. Especially when the target subject occupies only a local area of ​​the image, or when spatial cues such as surrounding roads, green spaces, and building outlines play a crucial role in localization, global features struggle to fully represent the fine-grained geographic structure within the cross-view image.

[0004] In recent years, deep vision models based on visual Transformers and large-scale self-supervised pre-training have demonstrated strong semantic extraction capabilities in image representation tasks. However, directly using their global output for UAV-satellite cross-view retrieval still suffers from insufficient utilization of local structures. The patch features within the model contain local spatial information such as building structures, road boundaries, and surrounding areas. Without effective region partitioning, local aggregation, and weight modeling mechanisms, local geographic cues can easily be overwhelmed by global semantic features. On the other hand, simple grid partitioning or forced local alignment may introduce unstable matching due to cross-view geometric differences.

[0005] Furthermore, in practical retrieval applications, cross-view geolocation not only requires models with high matching accuracy but also necessitates comprehensive software support, including image preprocessing, model weight loading, retrieval direction configuration, feature descriptor selection, similarity calculation, Top-K result output, experimental metric display, and training log management. Existing solutions often focus on algorithm training or offline evaluation, lacking an integrated visualization system for UAV-to-satellite and satellite-to-UAV bidirectional retrieval tasks, resulting in cumbersome model deployment, retrieval verification, and result analysis processes.

[0006] Therefore, there is a need for a UAV-satellite cross-view geolocation retrieval technology that can combine deep visual features, global semantic information, local spatial structure information, and bidirectional retrieval processes to enhance the representation of geographic targets under complex viewpoint differences, improve the stability of similar target matching, and provide unified software support for actual retrieval demonstrations, experimental evaluations, and model management. Summary of the Invention

[0007] The first objective of this invention is to provide a cross-view geolocation retrieval method for UAVs and satellites. Based on deep visual feature extraction, CLS anchored square ring local region representation, and local priority similarity ranking, it can be used for bidirectional geotarget retrieval, similar region matching, and Top-K candidate positioning result generation between UAV view images and satellite view images.

[0008] To achieve the above objectives, the technical solution adopted is as follows:

[0009] A cross-view geolocation retrieval method for UAVs and satellites includes the following steps:

[0010] S1: Obtain the set of images from the perspective of the UAV and the set of images from the perspective of the satellite, and determine the query images and images in the image library according to the preset search direction;

[0011] S2: Perform unified preprocessing on the query image and the image library image to obtain preprocessed query image and image library image;

[0012] S3: Input the preprocessed query image and library image into the basic visual coding network to obtain the global label features and spatial patch token features of the query image, and the global label features and spatial patch token features of the library image;

[0013] S4: Normalize the global marker features of the query image and the image library image respectively to obtain the global features of the query image and the global features of the image library image.

[0014] S5: After dividing the spatial patch token features of the query image and the image library image into multiple concentric square ring local regions, attention residual pooling is performed on each square ring local region to obtain region context features; then the region context features are projected and normalized to obtain local region descriptors; the local region descriptors are learned and weighted to obtain local features of the query image and local features of the image library image.

[0015] S6: Based on the global and local features of the query image and the image in the library, calculate the similarity between the query image and the image in the library, sort them in descending order of similarity, and output the Top-K search results.

[0016] Furthermore, in step S1, when the search direction is UAV → satellite, the UAV image is used as the query image and the satellite image is used as the image in the image library; when the search direction is satellite → UAV, the satellite image is used as the query image and the UAV image is used as the image in the image library.

[0017] In step S2, for any input image The preprocessing process is represented as follows: ;in, This represents the image preprocessing function. and Indicates the model input dimensions. and These represent the normalized mean and standard deviation, respectively.

[0018] In step S3, the basic visual coding network adopts the DINOv2 visual Transformer network, and its output can be expressed as: ;

[0019] in, Represents the basic visual coding network. Represents global labeling features. Indicates the characteristics of a register token. For spatial patch token features, This indicates the number of features in the patch token. Indicates the feature channel dimension.

[0020] Furthermore, in step S5, the process of dividing the spatial patch token features into multiple concentric square ring local regions is as follows: assuming the patch tokens are arranged as follows... The spatial grid has a number of square ring regions. , No. The mask corresponding to each square ring is: Then we have: ;in, Represents a point with the image center as the reference and a side length of . a square area ,and .

[0021] Furthermore, in step S5, the process of obtaining the region context features is as follows:

[0022] First calculate the first The mean characteristic of the patch tokens within each annular region is given by the following formula:

[0023] ;

[0024] in, Indicates the first Each ring region contains the number of patch tokens;

[0025] Then, attention residual pooling is performed on the patch tokens within the square ring region. The attention weight is represented by the following formula:

[0026] ;

[0027] in, and Indicates the first Attention parameters corresponding to each square ring region Presentation layer normalization operation;

[0028] No. The attention aggregation feature of each square ring region is represented as follows: ;

[0029] Finally, the first The regional context features of each square ring region are represented as follows: ;in, Indicates the first The residual strength coefficient of each square ring region.

[0030] Furthermore, in step S5, the local features are represented as follows: ;

[0031] in, Indicates feature concatenation operation; For local region descriptors, the context features of each ring region are used. The result is obtained by projection and normalization; For the first The weight of each square ring region;

[0032] The The calculation formula is: ;in, Indicates the first Learnable weight parameters corresponding to each square ring region This indicates the number of square ring regions.

[0033] Furthermore, in step S5, the total training loss includes a bidirectional contrastive learning loss of local features and global features, and its formula is:

[0034] ;

[0035] in, Bidirectional contrastive learning loss representing local features. The bidirectional contrastive learning loss represents the global features. and They represent the first Local loss weights and global loss weights corresponding to each training phase;

[0036] The formula for the bidirectional contrastive learning loss is:

[0037] ;

[0038] in, Indicates the number of samples in the batch. Indicates the learnable temperature coefficient. Descriptor The corresponding feature representation.

[0039] Furthermore, in step S6, the formula for calculating the similarity between the query image and the images in the image library is as follows:

[0040] ;

[0041] in, To query images, For the first A library of images, Represents the global similarity weight. Indicates global feature similarity. Indicates local feature similarity;

[0042] The formulas used to calculate the global feature similarity and local feature similarity are as follows: .

[0043] The second objective of this invention is to provide a cross-view geolocation retrieval device for both drones and satellites.

[0044] To achieve the above objectives, the technical solution adopted is as follows:

[0045] A cross-view geolocation retrieval device for UAVs and satellites, used to implement the aforementioned cross-view geolocation retrieval method, includes:

[0046] The image acquisition module is used to acquire query images and images from the image library;

[0047] The image preprocessing module is used to perform unified preprocessing on the query image and the image library image;

[0048] The basic visual feature extraction module is used to input the preprocessed query image and library image into the basic visual coding network to obtain the global label features and spatial patch token features of the query image and library image, respectively.

[0049] The feature extraction module includes a CLS global anchor feature extraction module, a square ring local region representation module, and a local priority feature generation module. The CLS global anchor feature extraction module is used to construct global features based on the global marker features. The square ring local region representation module and the local priority feature generation module are used to construct multiple square ring local features based on the spatial patch token features.

[0050] The search and ranking module is used to perform similarity calculations and ranking based on the search features selected by the user.

[0051] The results display module is used to display the results.

[0052] A third objective of this invention is to provide a computer device comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the aforementioned cross-view geolocation retrieval method.

[0053] The fourth objective of this invention is to provide a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the aforementioned cross-view geolocation retrieval method.

[0054] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0055] 1. The technical solution of this invention proposes a CLS-anchored square ring local region representation mechanism, which divides the image patch token into multiple square ring regions in a way that expands outward from the center. This can simultaneously model local geographic clues such as building main body, road boundary, green space and surrounding structure, and alleviate the problem of insufficient fine-grained spatial information caused by relying solely on global features.

[0056] 2. In the technical solution of the present invention, a local priority cross-view retrieval feature is adopted. Multiple square ring local descriptors are weighted, spliced ​​and normalized for similarity calculation, which can improve the matching stability between UAV view images and satellite view images under scale changes, view changes and local occlusion conditions.

[0057] 3. In the technical solution of the present invention, an attention residual pooling mechanism is introduced. On the basis of the stability of the regional mean features, a light attention shift is introduced, so that the local regional features can focus on more discriminative patch tokens, while avoiding the training instability problem caused by relying entirely on attention pooling.

[0058] 4. In the technical solution of the present invention, learnable square ring weights are set so that local areas of different spatial ranges can automatically adjust their contribution levels according to training data, thereby enhancing the model's ability to express the relative importance between the central building and the surrounding spatial structure.

[0059] 5. In the technical solution of the present invention, a training method combining global anchoring and local priority is adopted. While maintaining the stability of the global semantic space of CLS, the retrieval capability of local region features is enhanced, which helps to reduce the interference of cross-view local region errors on the training of the model.

[0060] 6. The technical solution of the present invention supports two retrieval directions: UAV to satellite and satellite to UAV. It can output query images, Top-K retrieval results, similarity scores, sample numbers, file names and image paths, which is convenient for application in UAV-assisted positioning, remote sensing image retrieval, urban inspection and cross-view geographic target matching systems. Attached Figure Description

[0061] Figure 1 This is an overall flowchart of the retrieval method for the UAV-satellite cross-view geolocation retrieval system of the present invention.

[0062] Figure 2 This is a diagram of the overall system architecture of the UAV-satellite cross-view geolocation and retrieval system of the present invention.

[0063] Figure 3 This is the feature extraction diagram of the DINOv2 + CLS / patch token of the present invention.

[0064] Figure 4 This is a partial region division diagram of the concentric square ring of the present invention.

[0065] Figure 5 This is for the generation of local priority features and Top-K retrieval graphs in this invention.

[0066] Figure 6 This is a schematic diagram showing the results of satellite view image retrieval from the perspective of an unmanned aerial vehicle (UAV) according to the present invention.

[0067] Figure 7 This is a schematic diagram showing the results of satellite-view image retrieval from a drone's perspective, as presented in this invention. Detailed Implementation

[0068] To further illustrate the cross-view geolocation retrieval method and apparatus for UAVs and satellites according to the present invention, and to achieve the intended purpose of the invention, the following detailed description, in conjunction with preferred embodiments, details the specific implementation, structure, features, and effects of the cross-view geolocation retrieval method and apparatus for UAVs and satellites proposed according to the present invention. In the following description, different "an embodiment" or "an embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0069] The following will provide a more detailed description of the cross-view geolocation retrieval method and device for UAVs and satellites according to the present invention, with reference to specific embodiments:

[0070] This invention relates to the fields of computer vision, remote sensing image processing, deep learning, and cross-view geolocation retrieval, providing a method and apparatus for cross-view geolocation retrieval using UAVs and satellites. The invention uses a deep visual coding network as the basic feature extractor to perform unified preprocessing and deep token representation extraction on UAV and satellite view images. Based on global CLS (Clearly Scaled Substance) marker features, a CLS-anchored square ring local region representation mechanism is introduced. The spatial patch token is divided into multiple square ring regions expanding outward from the image center, and local region aggregation, feature projection, normalization, and weighted combination are performed on each region to obtain a local priority image feature representation suitable for cross-view geolocation. Subsequently, the system sorts the query image and images in the library in descending order based on feature similarity, outputting Top-K retrieval results. This invention supports both Drone→Satellite retrieval of UAV images and Satellite→Drone retrieval of satellite images, and can be combined with a web-based interactive interface to complete model weight selection, query image upload, retrieval direction configuration, experimental metric display, and training log viewing.

[0071] The specific implementation method is as follows:

[0072] Example 1.

[0073] See Figure 1 , 3 -7. This invention provides a cross-view geolocation retrieval method for UAV-view images and satellite-view images. The method mainly includes the following steps: image data acquisition, retrieval direction determination, image preprocessing, basic visual feature extraction, CLS anchored square ring local region representation, local feature generation, similarity calculation, Top-K ranking, and result output.

[0074] The specific operating steps are as follows:

[0075] 101: Acquire the drone-view image set and the satellite-view image set, and determine the query image set and the image library image set according to the preset search direction. Specifically:

[0076] The Drone view image set represents geographic target images acquired by a drone platform from a low-altitude or oblique perspective, while the Satellite view image set represents geographic target images acquired by a satellite platform from a high-altitude, overhead perspective.

[0077] In this embodiment of the invention, the cross-view search direction includes two modes: Drone→Satellite (D2S) and Satellite→Drone (S2D). When the search direction is Drone→Satellite, the drone image is used as the query image and the satellite image is used as the gallery image; when the search direction is Satellite→Drone, the satellite image is used as the query image and the drone image is used as the gallery image.

[0078] 102: Image Preprocessing

[0079] The system performs unified preprocessing on both query images and images from the image library. Preprocessing operations include: image reading, color format conversion, size scaling, pixel normalization, and tensor format conversion.

[0080] In one specific implementation, the input image can be uniformly scaled to... The input image is then normalized according to the mean and standard deviation required by the basic visual coding network. Its preprocessing process can be expressed as: .in, This represents the preprocessing function. and Indicates the input image size. and These represent the normalized mean and standard deviation, respectively. This represents the preprocessed input image of the model.

[0081] 103: Basic Visual Feature Extraction

[0082] Preprocessed image Inputting the basic visual coding network yields fundamental coding features for both the query image and the library images, including global label features and spatial patch token features. Global label features represent the overall semantic information of the entire image, while spatial patch token features represent local visual information at different spatial locations within the image.

[0083] The basic visual coding network can be a visual Transformer network, a visual Transformer network with a register token, a self-supervised pre-trained visual model, or other deep image coding networks.

[0084] In a preferred implementation, combining Figure 3 The basic visual encoding network uses the DINOv2 visual Transformer network. For any input image... The output of the basic visual coding network can be represented as: .in, Represents the basic visual coding network. Represents the global marker features of CLS. Indicates the characteristics of a register token. Indicates the feature channel dimension. Indicates patch token characteristics, Indicates the number of patch tokens.

[0085] Therefore, the spatial patch token feature matrix can be represented as: , Indicates the first Each patch token has characteristics. Indicates the number of space patch tokens, excluding CLS tokens and register tokens; Indicates the feature channel dimension. When the input image size is... The patch size is ,and and All can be When divided by integers, the number of patch tokens is: .

[0086] In one specific implementation, the input image size is... The patch size is The size of the spatial patch grid is: That is, to obtain A patch token.

[0087] 104: Generate global semantic features based on CLS global marker features. Specifically:

[0088] The CLS global label features of the query image and the library image output by the basic visual coding network are normalized to obtain the global semantic features (i.e., global features) of the query image and the global semantic features (i.e., global features, global anchoring features) of the library image. The global semantic features are represented as follows: .in, This indicates a feature normalization operation. This feature is used to represent the overall semantic information and global scene information of the input image. It can characterize global semantic information such as the overall scene, building distribution, and regional environment of the image.

[0089] During training, global semantic features can also serve as global anchors for local feature learning, maintaining the stability of the cross-view feature space and providing a stable global reference for local region feature learning.

[0090] 105: Divide the spatial patch token features into multiple concentric square ring local regions. Specifically:

[0091] To address the issues of perspective differences, scale variations, and inconsistent spatial layouts between drone images and satellite images, this invention divides the spatial patch tokens of the query image and the library image output by the basic visual coding network into multiple square ring regions, expanding outward from the image center. This enhances the model's ability to represent local geographic structures. Specifically:

[0092] Combination Figure 4 Suppose the patch tokens are arranged as follows The spatial grid has a number of square ring regions. , No. The mask corresponding to each square ring region is: Then the first Each annular region can be represented as: .in, Represents a point with the image center as the reference and a side length of . a square area , .

[0093] In one specific implementation, when At that time, the set of square ring dimensions is set as follows: This creates five localized zones expanding outwards from the center, representing the main building, surrounding roads, green spaces, adjacent structures, and the larger spatial environment. Ultimately, this results in a series of localized spatial zones that gradually expand from the central building to the outer roads, green spaces, and adjacent structures.

[0094] 106: Attention residual pooling is performed on the local regions of each square ring to obtain the region context features. Combined with... Figure 5 Specifically:

[0095] For the For each square ring region, first calculate the mean characteristic of the patch tokens within that region: .in, Indicates the first Each ring region contains the number of patch tokens. For the first Each patch token has a unique characteristic.

[0096] Secondly, the attention weights for the patch tokens within this region are calculated using the following formula: .in, and Indicates the first Attention parameters corresponding to each square ring region Presentation layer normalization operation, Indicates the first The patch token at the _ ... Attention weights in a square ring region.

[0097] No. The attention aggregation features of the square ring region are: .

[0098] Based on the mean pooling result, an attention residual is introduced to obtain the first... The contextual features (i.e., local residual features) of each region are represented as follows: .in, Indicates the first The residual intensity coefficients of the square ring region are used. This method uses mean aggregation as a stabilizing benchmark and supplements the discriminative local response with attention residuals.

[0099] In one specific implementation, the residual strength coefficient can be expressed as: .in, Indicates the upper limit of residual strength. This represents the Sigmoid function. This represents the learnable parameters.

[0100] 107: Project and normalize the contextual features of each ring region to obtain local region descriptors. Specifically:

[0101] For the Each square ring region, its context features Input the corresponding projection layer and perform normalization to obtain the local region descriptor, which is represented as: .in, Indicates the first The projection function corresponding to each square ring region Indicates the first A local region descriptor.

[0102] In one specific implementation, the projection function can consist of layer normalization and linear mapping, mapping the feature dimensions of each region to lower-dimensional local descriptors. For example, if the output channel dimension of the basic visual coding network is 768, each square ring region descriptor can be mapped to 96 dimensions.

[0103] 108: Learnable weighted combination of multiple local region descriptors yields local priority features (i.e., local features). Specifically:

[0104] In order to enable square ring regions of different spatial ranges to automatically adjust their contribution level according to training data, this embodiment of the invention sets learnable weights for each square ring region.

[0105] First of all, the The weights of each square ring region can be expressed as: .in, Indicates the first Learnable weight parameters corresponding to each square ring region This indicates the number of square ring regions.

[0106] Subsequently, the descriptors of each region are weighted, concatenated, and normalized to obtain the local priority features (i.e., local features), represented as: .in, Indicates local priority features, This indicates a feature splicing operation.

[0107] The method for obtaining local priority features of query images and library images is the same, both using steps 105-108.

[0108] In one specific implementation, when the number of square ring regions is 5 and the descriptor dimension of each region is 96, the local priority feature dimension is: This local priority feature can comprehensively express cross-perspective geographical clues such as the main building, road structure, green areas, and surrounding spatial layout.

[0109] 109: Introduce a single-region random deactivation mechanism during the training phase to improve the robustness of local features. Specifically:

[0110] To avoid the model becoming overly reliant on a single local square ring region, this embodiment of the invention introduces a single-region random deactivation mechanism for local region descriptors during the training phase. Let... Indicates the first If the preserved states of each region descriptor are given, then the local priority features during the training phase can be represented as: In this process, at most one local region descriptor is randomly discarded for each training sample. This mechanism enables the model to comprehensively utilize geographical cues from multiple spatial regions, reducing the impact of local occlusion or changes in local appearance on the retrieval results.

[0111] 110: The model is trained using a contrastive learning approach that combines global anchoring with local prioritization. Specifically:

[0112] During the model training phase, a bidirectional contrastive learning loss is used to constrain the correspondence between UAV images and satellite images in the same feature space. For the first image in a training batch... Cross-perspective positive samples ,in This indicates the viewpoint image being queried. This represents the corresponding image from another perspective. For any descriptor... The bidirectional contrastive learning loss can be expressed as:

[0113] in, Indicates the number of samples in the batch. Indicates the learnable temperature coefficient. Descriptor The corresponding feature representation.

[0114] In one specific implementation, this embodiment employs both globally anchored features (i.e., global features) and locally prioritized features (i.e., local features) to jointly constrain the cross-view feature space. The total training loss includes a bidirectional contrastive learning loss of locally prioritized features and globally anchored features (i.e., global features), and its formula is as follows: .in, The bidirectional contrastive learning loss represents the local preference features. The bidirectional contrastive learning loss representing global semantic features is calculated using the aforementioned bidirectional contrastive learning loss calculation method. and They represent the first The local loss weights and global loss weights corresponding to each training phase.

[0115] Through the above training method, the model uses local priority features as the main retrieval descriptors, while using global semantic features to constrain the feature space, so that the representation of local regions remains stable even when there are large differences across perspectives.

[0116] 111: Calculate the similarity between the query image and images in the image library based on the feature descriptors selected by the user. Combined with... Figure 5 Specifically:

[0117] During the retrieval phase, the system can determine the query image and library images based on the user's selected search direction, and extract the corresponding search features for each. Search features can be selected using global features, local priority features, or single local region descriptors, as shown below:

[0118]

[0119] In a preferred implementation, the following is adopted: As the main retrieval feature, adopt As global anchoring features and auxiliary retrieval features.

[0120] For query images and the Image library The similarity score can be expressed as: .because and All features have been normalized, and this similarity is equivalent to cosine similarity, which is used to measure the degree of matching between the query image and the image in the library in the same cross-view feature space.

[0121] In one specific implementation, the system can also fuse global features and locally prioritized features at different score levels during the evaluation phase, as follows:

[0122] First, determine the features of the query image (global features + local priority features). Image feature collection .

[0123] Then, the similarity is calculated and fused using a score-based approach combining global and local priority features. The formula is as follows: .in, Represents the global similarity weight. Indicates global feature similarity. Let j represent the local priority feature similarity, j=1,2,…,M.

[0124] 112: Sort the images in the image library in descending order based on similarity scores and output the Top-K search results. Specifically:

[0125] Let the image library be: in, This indicates the number of images in the image library. The system calculates the number of images retrieved. With each gallery image The similarity scores are calculated and sorted from highest to lowest to obtain the Top-K search results, which can be represented as: .in, Indicates query image The corresponding Top-K search result set, Indicates the number of results returned. This indicates that the top results are selected in descending order of similarity scores. Operations on each result.

[0126] The search results include: query image, search direction, search features, result image, sort number, similarity score, sample number, image file name, and image path.

[0127] Search results as follows Figure 6-7 As shown, the data can be visualized through a web interface and can also be used for subsequent experimental analysis or geographic target matching. This visualization output method allows users to intuitively determine the correspondence between geographic targets in UAV-view images and satellite-view images, and can be used for software demonstration, experimental verification, and application integration of cross-view geolocation retrieval systems.

[0128] In summary, this embodiment of the invention, through steps 101 to 112, achieves bidirectional cross-view geolocation retrieval between UAV-view images and satellite-view images. This method utilizes a basic visual coding network to extract global semantic features and spatial patch token features, and generates local priority features through the CLS anchored square ring local region representation mechanism. This enhances the model's ability to express geographical cues such as building structures, road structures, and the surrounding environment, and improves matching stability under complex viewpoint differences.

[0129] Example 2.

[0130] Referring to Figure 2, this embodiment of the invention provides a device for cross-view geolocation retrieval of UAV-satellite. Figure 2 shows the complete processing chain of the two core stages of local priority feature generation and Top-K retrieval ranking in the device. The overall device includes: an image acquisition module, an image preprocessing module, a model weight and configuration management module, a basic visual feature extraction module, a feature extraction module, a retrieval ranking module, a result display module, and an experimental index and log management module.

[0131] The feature extraction module includes a CLS global anchor feature extraction module, a square ring local region representation module, and a local priority feature generation module. The CLS global anchor feature extraction module is used to construct global features based on the global marker features output by the basic visual feature extraction module. The square ring local region representation module is used to divide and represent the square ring local region based on the spatial patch token features. The local priority feature generation module is used to process the output of the square ring local region representation module and generate local priority features based on the regional context features of the square ring local region.

[0132] Specifically:

[0133] (1) Image acquisition module

[0134] The image acquisition module is used to acquire query images and gallery images. Based on the user-selected search direction, this module automatically determines the query image set and gallery image set for the cross-view search task.

[0135] When a user selects a drone view to search for a satellite view, this module uses the drone image as the query image and the satellite image as the gallery image; when a user selects a satellite view to search for a drone view, this module uses the satellite image as the query image and the drone image as the gallery image.

[0136] In one specific implementation, when the search direction is U1652-D2S, the query image directory is query_drone and the gallery image directory is gallery_satellite; when the search direction is U1652-S2D, the query image directory is query_satellite and the gallery image directory is gallery_drone.

[0137] (2) Image preprocessing module

[0138] The image preprocessing module is used to perform unified format processing on query images and library images, including: image reading, color format conversion, size adjustment, pixel normalization, and tensor quantization. For any input image... The preprocessing process can be represented as: .in, This represents the preprocessing function. and Indicates the model input dimensions. and These represent the normalized mean and standard deviation, respectively. This represents the preprocessed image tensor.

[0139] (3) Model weight and configuration management module

[0140] The model weight and configuration management module is used to scan, read, and load the trained model weight files, and build the model structure according to the experimental configuration corresponding to the weight files.

[0141] This module can read information such as model weight path, experiment name, retrieval direction, feature descriptors, run number, and evaluation metrics to ensure consistency between the model structure and weight parameters. After the user selects model weights in the interface, this module loads the corresponding parameters into the deep vision model for subsequent feature extraction and retrieval calculations.

[0142] (4) Basic visual feature extraction module

[0143] The basic visual feature extraction module is used to input the preprocessed query image and library image into the basic visual coding network to obtain the global label features and spatial patch token features of the query image and library image, respectively.

[0144] In one specific implementation, the underlying visual coding network employs the DINOv2 Visual Transformer network. For any input image... Its output can be represented as: .in, Represents the global marker features of CLS. Indicates the characteristics of a register token. Indicates the first Each patch token has characteristics. Indicates the number of patch tokens. Indicates the channel dimension.

[0145] The spatial patch token feature matrix can be represented as: When the image size is The patch size is At that time, the number of patches was: .

[0146] (5) Feature extraction module

[0147] This module is used to construct global features and local priority features based on the CLS global marker features and spatial patch token features output by the basic visual feature extraction module. Specifically: The CLS global anchoring feature extraction module normalizes the CLS global marker features to obtain global features, and uses these global features as global anchor features for local feature learning. The square ring local region representation module divides the spatial patch token into multiple concentric square ring local regions based on the spatial patch token features, and performs region representation on each square ring local region to obtain multiple square ring region context features; the local priority feature generation module projects, normalizes, and learns a weighted combination of the multiple square ring region context features to obtain local priority features.

[0148] ①CLS Global Anchoring Feature Extraction Module

[0149] The CLS global anchoring feature extraction module is used to construct global features based on global labeled features. Global anchoring features can be represented as: .in, This indicates a normalization operation. It represents the overall semantic features of the image.

[0150] ② Local region representation module of square ring

[0151] For the spatial patch token, let it be restored to The two-dimensional grid, the first The mask for a square ring region is represented as follows: .in, Represents a point with the image center as the reference and a side length of . a square area This indicates the number of square ring regions.

[0152] ③ Local Priority Feature Generation Module

[0153] The local priority feature generation module performs region aggregation, attention residual pooling, feature projection, weight combination, and normalization on patch tokens within multiple square ring regions to generate local priority retrieval features. Specifically:

[0154] For the The characteristics of the mean value of the square ring region are as follows: Attention weights are: Attention aggregation features are: The region context features after attention residual pooling are: .in, Indicates the first The residual strength coefficient of each region. The local region descriptors are: .in, Indicates the first The projection function corresponding to each region.

[0155] Set learnable weights for different square ring regions: .in, Indicates the first Learnable weight parameters for each square ring region.

[0156] The final local priority feature is: This module is used to enhance the model's ability to express geographical clues such as building structures, road boundaries, green areas, and surrounding spatial structures.

[0157] (6) Search and sorting module

[0158] The search ranking module is used to perform similarity calculations and ranking based on the search features selected by the user. Users can choose global features, local priority features, or specify local region features as search descriptors.

[0159] For query images and stock images If all features have been normalized, then the similarity score is: .in, Indicates querying image features, Indicates the first Image features of a library.

[0160] The system sorts the results from highest to lowest similarity score to obtain the Top-K search results: .in, Indicates the number of images in the gallery. Indicates the number of results returned.

[0161] (7) Results Display Module

[0162] The results display module shows the query image, search direction, search features, and Top-K result images. It also outputs the sort number, similarity score, sample number, image file name, and image path for each search result.

[0163] This module enables visual interaction through a browser page, allowing users to intuitively determine the correspondence between geographic targets in images from both the drone's and satellite perspectives.

[0164] (8) Experimental Indicators and Log Management Module

[0165] The experimental metrics and log management module is used to read and display evaluation metrics and training logs under different experimental configurations. Evaluation metrics include Recall@1, Recall@5, Recall@10, Recall@top1%, and AP, etc.

[0166] This module can also display the model run number, experiment name, retrieval direction, feature descriptor, weight path, and training log content to assist in model selection, experiment comparison, and system maintenance.

[0167] It should be noted that the device descriptions in the above embodiments correspond to the method descriptions in Embodiment 1.

[0168] Example 3.

[0169] A computer device includes a processor and a memory. The memory stores computer-executable instructions that can be executed by the processor, which executes the computer-executable instructions to implement the cross-view geolocation retrieval method of Embodiment 1.

[0170] It should be noted that the description in this embodiment corresponds to the description of the cross-view geolocation retrieval method in Embodiment 1, and will not be repeated here. The execution entity of the processor and memory mentioned above can be a computer, server, embedded computing device, edge computing device, or other device with computing functions. In specific implementation, this embodiment of the invention does not limit the execution entity.

[0171] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program. The computer program includes program instructions, which, when executed by a processor, cause the processor to perform the steps of the cross-view geolocation retrieval method in Embodiment 1 above. This computer-readable storage medium includes, but is not limited to, flash memory, hard disk, solid-state drive, memory card, server storage medium, or data center storage medium.

[0172] Example 4.

[0173] This invention provides an experimental verification method for validating a local region representation and local priority retrieval mechanism based on CLS anchored square rings. This experiment illustrates the effectiveness and stability of the method described in this invention in cross-view geolocation retrieval tasks using UAV-view images and satellite-view images, as well as the impact of each component on retrieval performance.

[0174] In one specific implementation, the experimental environment may include the following configuration:

[0175] 1. Hardware environment: computer, graphics processor, memory, and storage devices;

[0176] 2. Software environment: Operating system, Python runtime environment, PyTorch deep learning framework, timm visual model library, OpenCV image processing library, Albumentations image preprocessing library, and Streamlit visualization interface program;

[0177] 3. Model environment: DINOv2 basic visual encoding network, CLS global anchoring module, square ring local region segmentation module, attention residual pooling module, learnable square ring weight module, local priority feature generation module, and retrieval ranking module;

[0178] 4. Output environment: The search results can be displayed through the software interface or saved as a structured file. The structured file includes information such as the query image path, the image path in the library, the sample number, the sort number, the similarity score, and the image file name in the library.

[0179] The experimental data used was the University-1652 dataset. This dataset includes a set of images from both UAV (unmanned aerial vehicle) and satellite perspectives. The UAV image set represents geographic target images acquired from low-altitude or oblique perspectives, while the satellite image set represents geographic target images acquired from high-altitude overhead perspectives. A cross-perspective correspondence between UAV and satellite images was established for each geographic target using a sample number.

[0180] Experimental search directions include:

[0181] 1. Retrieve from UAV perspective to satellite perspective, i.e., D2S retrieval direction, using UAV images as query images and satellite images as library images;

[0182] 2. Satellite-view search: The UAV-view search direction uses satellite images as the query images and UAV images as the library images.

[0183] In this embodiment, for any input image The spatial patchtoken output by the basic visual coding network can be represented as: .in, Indicates the first Each patchtoken has certain characteristics. Indicates the number of patch tokens. Indicates the feature channel dimension.

[0184] The local region division of the square ring is represented as follows: .in, Indicates the first Each ring area, Represents a point with the image center as the reference and a side length of . a square area Indicates the number of square ring regions. The attention residual pooling feature of the square ring region is represented as follows: .in, Indicates the first Mean pooling characteristics of the square ring region This indicates the attention aggregation characteristics of the region. This represents the attention residual intensity coefficient.

[0185] The local region descriptor is represented as: .in, Indicates the first The projection function corresponding to each square ring region This represents the normalization operation. The final local preference feature is represented as: .in, Indicates the first The weight of each square ring region This indicates a splicing operation.

[0186] Query Image With the Image library The similarity is expressed as: .in, Indicates querying image features, This represents the image features in the image library. Since the features have been normalized, this similarity is equivalent to cosine similarity. The Top-K search results are represented as follows: .in, Indicates the number of images in the gallery. This indicates the number of search results returned.

[0187] The experiment used the following evaluation indicators:

[0188] 1. Recall@K, used to indicate that the correctly matched image appears in the first position. The proportion of each search result can be expressed as: .in, Indicates the number of images queried. Indicates the first The correct set of image libraries corresponding to each query image. Indicates the first Top-K search results for each query image.

[0189] 2. AP, used to evaluate the ranking accuracy of a single query image, can be represented as: .in, Indicates the preceding Accuracy of each search result Indicates the first Whether each search result is a relevant result.

[0190] 3. mAP, used to evaluate the average ranking performance of all query images, can be expressed as: To verify the effectiveness of each component in the method of this invention, the following component ablation experiments can be set up:

[0191] The first group uses only global features under the matching settings as the baseline method;

[0192] The second group removed the global anchoring constraint and verified the impact of CLS global anchoring on the stability of local priority training.

[0193] The third group replaced the local region division of the square ring with the grid region division to verify the influence of the square ring structure on the cross-view spatial representation.

[0194] The fourth group replaced attention residual pooling with mean pooling to verify the effect of attention residual pooling;

[0195] In the fifth group, the learnable square ring weights were replaced with fixed weights to verify the effect of adaptive region weights.

[0196] The sixth group removed the random deactivation mechanism in local regions and verified the impact of part dropout on the robustness of local features.

[0197] The seventh group incorporates local region-level contrast loss based on the method of this invention. To verify whether explicit local supervision is beneficial to the final retrieval performance;

[0198] The eighth group adopts the complete CA-SRPL-LF method of this invention, which combines the local region partitioning of square rings, attention residual pooling, learnable square ring weights, global anchoring, and local random deactivation mechanism.

[0199] The experimental results are shown in Table 1-2.

[0200] Table 1 Experimental results of different components

[0201]

[0202] Table 2 Experimental results for different components

[0203]

[0204] Based on the experimental results shown in Table 1-2, the analysis can be conducted from aspects such as global baseline, local region division of square rings, attention residual pooling, learnable square ring weights, global anchoring, random deactivation of local regions, and local region-level contrast loss.

[0205] First, based on the global baseline matching results, when using only global features, the Recall@1 for D2S retrieval is 94.81%, and the mAP is 95.60%; while the Recall@1 for S2D retrieval is 96.58%, and the mAP is 95.22%. This result indicates that global semantic features based on DINOv2 can provide strong cross-view image representation capabilities. However, global features primarily focus on the semantic expression at the overall image level, and their ability to depict local geographic cues such as building structures, road boundaries, green areas, and surrounding structures remains limited.

[0206] Secondly, comparing the complete method with the global baseline, the complete method of this invention, Ours / CA-SRPL-LF, achieves a Recall@1 of 95.95% and a mAP of 96.56% in the D2S direction, representing improvements of 1.14 percentage points and 0.96 percentage points respectively compared to the global baseline; and a Recall@1 of 97.57% and a mAP of 95.85% in the S2D direction, representing improvements of 0.99 percentage points and 0.63 percentage points respectively compared to the global baseline. These results demonstrate that introducing CLS anchored square ring local region representation and local priority features on top of the DINOv2 global semantic features helps improve the cross-view matching performance between UAV-view images and satellite-view images.

[0207] Furthermore, comparing the local area division using a square ring with that using a grid, the D2S Recall@1 is 95.03%, D2S mAP is 95.81%, S2D Recall@1 is 97.00%, and S2D mAP is 94.90%. However, when using the local area division using the square ring of this invention, D2S Recall@1 increases to 95.95%, D2S mAP increases to 96.56%, S2D Recall@1 increases to 97.57%, and S2D mAP increases to 95.85%. This result indicates that the square ring region expanding outwards from the center is more suitable for describing the central building and its surrounding spatial structure in cross-perspective geolocation, exhibiting a more stable spatial inductive bias compared to ordinary grid division.

[0208] Furthermore, comparing attention residual pooling and mean pooling, the mean pooling method achieves 95.90% Recall@1 and 96.50% mAP in the D2S direction, while the attention residual pooling method in this invention achieves 95.95% and 96.56% respectively; in the S2D direction, both achieve a Recall@1 of 97.57%. These results demonstrate that attention residual pooling can introduce adaptive responses from locally discriminative tokens while maintaining the stability of mean pooling, thereby improving or maintaining key retrieval metrics.

[0209] Comparing the learnable ring weights and fixed ring weights, the fixed weight method achieves 95.77% Recall@1 and 96.38% mAP in the D2S direction, while the present invention, using learnable ring weights, improves these figures to 95.95% and 96.56%, respectively. In the S2D direction, the present invention achieves 95.85% mAP, higher than the fixed weight method's 95.77%. These results indicate that different ring regions do not contribute entirely to cross-view matching; adaptively adjusting the contributions of the central and peripheral regions through learnable weights helps improve overall retrieval stability.

[0210] Comparing the local region random deactivation mechanisms, when removing part dropout, D2S Recall@1 is 95.82%, D2S mAP is 96.45%, S2D Recall@1 is 97.15%, and S2D mAP is 95.84%. After adopting the part dropout method of this invention, D2S Recall@1 increases to 95.95%, D2S mAP increases to 96.56%, S2D Recall@1 increases to 97.57%, and S2D mAP remains basically stable. These results indicate that randomly deactivating individual local region descriptors during the training phase can reduce the model's dependence on a single region, allowing the model to more fully utilize complementary geographical cues within multiple square ring regions.

[0211] Comparing the results with and without global anchoring constraints, when global anchoring is removed, D2S Recall@1 is 95.70%, D2SmAP is 96.35%, S2D Recall@1 is 97.43%, and S2D mAP is 95.81%. After introducing global anchoring into the complete method, these metrics reach 95.95%, 96.56%, 97.57%, and 95.85%, respectively. These results demonstrate that CLS global anchoring features can provide a stable global semantic reference for learning local priority features, alleviating the problem of unstable cross-view correspondences in local regions.

[0212] Finally, regarding whether to introduce local region-level contrast loss... In comparison, adding Afterwards, D2SRecall@1 was 95.65%, D2S mAP was 96.29%, S2D Recall@1 was 97.43%, and S2D mAP was 96.04%; the complete method was not used. At that time, D2S Recall@1 and mAP improved to 95.95% and 96.56%, respectively, while S2DRecall@1 improved to 97.57%. These results indicate that explicitly constraining each local region to perform cross-view alignment does not necessarily lead to better main retrieval performance and may introduce local region correspondence noise. This invention uses local priority features as the main retrieval descriptor, while not forcing each square ring local region to independently bear the contrast learning objective, thus achieving a more balanced bidirectional retrieval effect.

[0213] In summary, the experimental results demonstrate that the CA-SRPL-LF method described in this invention achieves a stable improvement over matching the global baseline on the University-1652 dataset. The complete method achieves 95.95% Recall@1 and 96.56% mAP in the D2S direction, and 97.57% Recall@1 and 95.85% mAP in the S2D direction. This indicates that the method can effectively utilize global semantic information, local area information of square rings, and local priority feature representation to improve the cross-view geolocation retrieval performance between UAV-view images and satellite-view images.

[0214] The above experiments lead to the following conclusions:

[0215] First, the square ring local region division proposed in this invention is more suitable for UAV-satellite cross-view geolocation tasks than ordinary grid division, and can better describe the relationship between the central target and the surrounding spatial structure.

[0216] Second, the attention residual pooling method used in this invention can enhance the local discriminative response on the basis of the stability of mean pooling, which is beneficial to improving or maintaining cross-view retrieval performance.

[0217] Third, the learnable square ring weights used in this invention can adaptively adjust the contribution of local descriptors according to the importance of different spatial regions, thereby enhancing the expressive power of local priority features.

[0218] Fourth, the global anchoring mechanism adopted in this invention can stabilize the local priority feature training process by utilizing CLS global semantic features, thereby reducing the impact of cross-viewpoint erroneous correspondences in local regions.

[0219] Fifth, the local region random deactivation mechanism adopted in this invention can improve the model's ability to make comprehensive use of multiple local regions and enhance the model's robustness to occlusion, scale changes and local appearance changes.

[0220] Sixth, this invention does not include As a necessary training objective for the complete method, it is shown that since DINOv2 semantic tokens already have strong representational capabilities, it may not be necessary to force explicit cross-view alignment for each local region. The present invention adopts a local-first global descriptor for retrieval, which is more conducive to obtaining stable results.

[0221] Unless otherwise specified, the model numbers of the various devices in this embodiment of the invention are not limited, and any device that can perform the above functions is acceptable.

[0222] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A cross-view geolocation retrieval method for UAVs and satellites, characterized in that, Includes the following steps: S1: Obtain the set of images from the perspective of the UAV and the set of images from the perspective of the satellite, and determine the query images and images in the image library according to the preset search direction; S2: Preprocess the query image and the image library image to obtain the preprocessed query image and image library image; S3: Input the preprocessed query image and library image into the basic visual coding network to obtain the global label features and spatial patch token features of the query image, and the global label features and spatial patch token features of the library image; S4: Normalize the global marker features of the query image and the image library image respectively to obtain the global features of the query image and the global features of the image library image; S5: After dividing the spatial patch token features of the query image and the image library image into multiple concentric square ring local regions, attention residual pooling is performed on each square ring local region to obtain region context features; then the region context features are projected and normalized to obtain local region descriptors; the local region descriptors are learned and weighted to obtain local features of the query image and local features of the image library image. S6: Based on the global and local features of the query image and the image in the library, calculate the similarity between the query image and the image in the library, sort them in descending order of similarity, and output the Top-K search results.

2. The cross-view geolocation retrieval method according to claim 1, characterized in that, In step S1, when the search direction is UAV → satellite, the UAV image is used as the query image and the satellite image is used as the image in the image library; when the search direction is satellite → UAV, the satellite image is used as the query image and the UAV image is used as the image in the image library. In step S2, for any input image The preprocessing process is represented as follows: ;in, This represents the image preprocessing function. and Indicates the model input dimensions. and These represent the normalized mean and standard deviation, respectively. In step S3, the basic visual coding network adopts the DINOv2 visual Transformer network, and its output can be represented as: ; in, Represents the basic visual coding network. Represents global labeling features. Indicates the characteristics of a register token. For spatial patch token features, N represents the number of patch token features, and C represents the feature channel dimension.

3. The cross-view geolocation retrieval method according to claim 1, characterized in that, In step S5, the process of dividing the spatial patch token feature into multiple concentric square ring local regions is as follows: Assume the patch tokens are arranged in a G×G spatial grid, the number of square ring regions is K, and the mask corresponding to the kth square ring is... Then we have: ;in, Represents a point with the image center as the reference and a side length of . a square area ,and .

4. The cross-view geolocation retrieval method according to claim 3, characterized in that, In step S5, the process of obtaining the region context features is as follows: First, calculate the mean characteristic of the patch tokens within the k-th square ring region, using the following formula: ； in, This indicates the number of patch tokens contained in the i-th square ring region; Then, attention residual pooling is performed on the patch tokens within the square ring region. The attention weight is represented by the following formula: ； in, and This represents the attention parameters corresponding to the k-th square ring region. Presentation layer normalization operation; The attention aggregation feature of the k-th square ring region is represented as follows: ; Finally, the regional context features of the k-th square ring region are represented as follows: ;in, This represents the residual intensity coefficient of the k-th square ring region.

5. The cross-view geolocation retrieval method according to claim 1, characterized in that, In step S5, the local features are represented as follows: ; in, Indicates feature concatenation operation; For local region descriptors, the context features of each ring region are used. The result is obtained by projection and normalization; The weight of the k-th square ring region; The aforementioned The calculation formula is: ;in, This represents the learnable weight parameter corresponding to the k-th square ring region, where K represents the number of square ring regions.

6. The cross-view geolocation retrieval method according to claim 1, characterized in that, In step S5, the total training loss includes a bidirectional contrastive learning loss for local and global features, and its formula is as follows: ； in, Bidirectional contrastive learning loss representing local features. The bidirectional contrastive learning loss represents the global features. and These represent the local loss weights and global loss weights corresponding to the e-th training phase, respectively. The formula for the bidirectional contrastive learning loss is: ； Where B represents the batch sample size. Indicates the learnable temperature coefficient. This represents the feature representation corresponding to the descriptor d.

7. The cross-view geolocation retrieval method according to claim 1, characterized in that, In step S6, the formula for calculating the similarity between the query image and the images in the image library is as follows: ； Where q is the query image. For the j-th image in the image library, Represents the global similarity weight. Indicates global feature similarity. Indicates local feature similarity; The formulas used to calculate the global feature similarity and local feature similarity are as follows: .

8. A cross-view geolocation and retrieval device for UAVs and satellites, characterized in that, To implement the cross-view geolocation retrieval method according to any one of claims 1-7, comprising: The image acquisition module is used to acquire query images and images from the image library; The image preprocessing module is used to perform unified preprocessing on the query image and the image library image; The basic visual feature extraction module is used to input the preprocessed query image and library image into the basic visual coding network to obtain the global label features and spatial patch token features of the query image and library image, respectively. The feature extraction module includes a CLS global anchor feature extraction module, a square ring local region representation module, and a local priority feature generation module. The CLS global anchor feature extraction module is used to construct global features based on the global marker features. The square ring local region representation module and the local priority feature generation module are used to construct multiple square ring local features based on the spatial patch token features. The search and ranking module is used to perform similarity calculations and ranking based on the search features selected by the user. The results display module is used to display the results.

9. A computer device, characterized in that, The method includes a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the cross-view geolocation retrieval method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the cross-view geolocation retrieval method as described in any one of claims 1-7.

Images