A UAV view geolocation method with multi-directional cross-image interaction

By employing adaptive channel interaction and multi-directional feature segmentation methods, combined with a cross-image interactive perception mechanism, the problem of cross-view matching between UAV perspective images and satellite perspective images was solved, achieving high-precision UAV geolocation and improving positioning accuracy and robustness.

CN123071869APending Publication Date: 2026-11-10ZHEJIANG WANLI UNIV
0 Cites 0 Cited by

Patent Information

Application Number
CN202611264320.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-20
Publication Date
2026-11-10

Smart Images

  • Figure CN123071869A_ABST
    Figure CN123071869A_ABST
Patent Text Reader

Abstract

This invention discloses a multi-directional cross-image interactive method for UAV view geolocation: It acquires a batch of images including the UAV's view and multiple geotagged satellite view images; through a feature extraction network incorporating a channel interaction mechanism, static landmarks are enhanced and dynamic interference is suppressed to obtain an optimized spatial feature map; regions are divided along multiple spatial directions to generate local region features with location markers, and a cross-image region feature sequence is constructed based on these markers; interference is filtered out through a cross-image interaction module, and cross-image feature association is modeled based on attention to fuse corresponding location information; the final features of each image are obtained through aggregation and normalization; and the geographical location is determined based on the feature similarity between the UAV and satellite images. This invention can achieve accurate cross-view geolocation in complex dynamic scenes and similar geographical environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image processing and management technology, and in particular to a method for multi-directional cross-image interaction in UAV view geolocation. Background Technology

[0002] Cross-view image matching between UAV and satellite perspectives is one of the key technologies for achieving autonomous geolocation of UAVs. Due to significant differences between UAV aerial images and satellite remote sensing images in terms of shooting altitude, viewing angle, lighting conditions, and scene scale, images of the same geographical location exhibit drastically different appearances from different perspectives, making accurate cross-view matching a major challenge.

[0003] In feature extraction, existing methods typically employ convolutional neural networks or Transformer architectures as the backbone network to perform global feature encoding on the input image. However, images from a drone's perspective often contain numerous dynamic foreground objects such as vehicles and pedestrians. The positions and shapes of these dynamic disturbances are not fixed in images taken at different times, while static landmarks such as buildings and roads are the key information with stable geographic identification functions. Existing feature extraction networks lack mechanisms for adaptively distinguishing and differentially responding to static landmarks and dynamic disturbances, resulting in a large amount of unstable dynamic information mixed in with the extracted features, weakening the discriminative and generalization abilities of the features.

[0004] In terms of feature utilization, most existing methods rely solely on a single global feature vector obtained by global pooling of the entire image for similarity matching. While global features can summarize the overall layout of a scene, they inevitably lose fine-grained spatial information of local regions, which are often key to distinguishing locations that are similar in appearance but different in location. Some methods attempt to segment the feature map into regular image blocks or sub-regions to introduce local information, but they usually lack a systematic division of multi-directional spatial structures and fail to establish an effective mechanism for aligning and comparing local regions across different images. This results in low efficiency in utilizing local information and insufficient robustness of the model to similar scenes.

[0005] More significantly, existing cross-image geolocation methods typically extract features from each query image independently and compare them one by one with candidate satellite images, resulting in isolated feature extraction processes between images. When different geographical locations appear highly similar in images due to similar road layouts, architectural styles, or vegetation distributions, this isolated processing method easily leads to perceptual confusion, mismatching satellite images that are similar to the query image but have different actual geographical locations as the location result. Existing technologies lack mechanisms for information exchange and collaborative perception among multiple images within a batch, making it impossible to utilize information from other images in the same batch to enhance the feature representation of the current image, and also making it difficult to effectively distinguish between negative samples with highly similar appearances, thus limiting further improvements in positioning accuracy. Summary of the Invention

[0006] To address the aforementioned shortcomings, this invention proposes a multi-directional cross-image interactive UAV view geolocation method, which achieves accurate cross-view geolocation in complex dynamic scenes and similar geographical environments.

[0007] This invention provides the following technical solution: a multi-directional cross-image interactive UAV view geolocation method, comprising: Acquire image batches, which include images from the drone's perspective to be located and multiple satellite images with geolocation tags; The feature extraction network is used to process each image in the image batch, and a channel interaction mechanism is introduced into the feature extraction network. The channel interaction mechanism is used to adaptively adjust the channel representation of the feature map according to the average statistics and dispersion statistics along the channel dimension of the feature map, so as to enhance the static landmark features and suppress dynamic interference, and obtain an optimized spatial feature map. For each spatial feature map, the regions are divided along at least two spatial directions to generate multiple local region features with location identifiers; For each location identifier, the local region features of all images in the image batch under that location identifier are constructed into a set of region feature sequences; Each set of regional feature sequences is input into the cross-image interaction module. The cross-image interaction module is configured to: filter out interference information in the local regional features in the regional feature sequence, and model the relationship between local regional features of different images under the same location identifier based on the attention mechanism, so that the local regional features of each image are fused with the corresponding location information of other images in the image batch, and output the associated local regional features. For each image, the associated local region features under all location identifiers are aggregated and normalized to obtain the final feature representation of the image; The geographical location of the UAV is determined based on the similarity between the final feature representation of the UAV's view image and the final feature representation of each satellite view image.

[0008] As an improvement, the channel interaction mechanism is specifically as follows: Perform average pooling and standard deviation pooling operations on the input features respectively to obtain average pooling features and standard deviation pooling features; Learnable weight parameters are used to perform weighted fusion of average pooling features and standard deviation pooling features to adjust the channel representation; Among them, the standard deviation pooling operation is used to replace the max pooling operation in the triple attention mechanism, so as to complement the average pooling, enhance static landmark features and suppress dynamic interference.

[0009] As an improvement, the feature extraction network is a Transformer encoder; An adaptive channel interaction adapter is added between the activation layer and the upper and lower projection layers of each Transformer block in the Transformer encoder and inserted in parallel. The adaptive channel interaction adapter contains two sub-branches, which learn channel attention weights interactively from average pooling features and standard deviation pooling features, respectively, to enhance channel responses to static landmarks and suppress channel responses to dynamic objects. The outputs of the two sub-branches are concatenated and merged to form the final output of the adapter.

[0010] As an improvement, the adaptive channel interaction adapter processes input features as follows: The input features are processed by average pooling and standard deviation pooling respectively to obtain average pooling features and standard deviation pooling features; The two sub-branches take average pooling features and standard deviation pooling features as inputs, respectively, and each learns channel attention weights. The outputs of the two sub-branches are concatenated to form the final output of the adaptive channel interaction adapter.

[0011] As an improvement, the region is divided along at least two spatial directions, specifically including: The spatial feature map is divided into multiple grid regions in a non-overlapping manner along the height and width directions. Each grid region is assigned a unique location identifier, so that the same location identifier in different images corresponds to the same spatial relative position.

[0012] As an improvement, the final feature representation of the image is obtained, specifically including: While dividing the spatial feature map into regions, the spatial feature map is globally aggregated through generalized average pooling to generate global features. The local region features of the same image after association under all location identifiers are concatenated, and then concatenated again with the global features. The concatenated global features are then L2 normalized to obtain the final feature representation of the image.

[0013] As an improvement, the cross-image interaction module includes cascaded interaction attention modules and interaction perception modules; The interactive attention module is used to filter out interference information from local region features in the region feature sequence. Its processing steps include: For the local region features of the input, global average pooling and global max pooling are performed along the channel dimension respectively to obtain two compressed feature maps; The two compressed feature maps are morphologically modulated and then fused to generate a spatial mask. Multiply the spatial mask by the input local region features to obtain the weighted region features; The interactive perception module is used to model cross-image relationships based on the same location identifier of weighted region features using a self-attention mechanism, and output the associated local region features.

[0014] As an improvement, gradient scaling factors are set for the compressed feature maps obtained by global average pooling and global max pooling, respectively. Among them, the gradient scaling factor is set for the compressed feature map derived from global average pooling. The gradient scaling factor set for the compressed feature map derived from global max pooling is greater than that set for the feature map. ; Will and These are applied to the corresponding compressed feature maps to adjust their contribution to spatial mask generation.

[0015] As an improvement, the interactive perception module includes at least one Transformer encoder layer for performing self-attention-based context encoding on weighted region features under the same location identifier, so that the local region features of each image are aggregated with corresponding location information from other images in the image batch.

[0016] As an improvement, it also includes the step of training the networks that constitute the feature extraction network and the cross-image interaction module; During training, a loss function based on metric learning is used to update the network parameters. Within the same training batch, the loss function applies higher weights to hard-to-distinguish sample pairs based on the self-similarity of positive sample pairs and the relative similarity of negative sample pairs. The loss function is a multi-similarity loss, which is achieved through hyperparameters. , , The similarity boundaries for positive sample pairs and negative sample pairs are adjusted separately to achieve online mining of difficult-to-distinguish positive and negative samples. The multi-similarity loss is calculated as follows: in, For batch size, and These are the sets of positive sample pairs and the sets of negative sample pairs, respectively. Indicates similarity. , , This is a hyperparameter.

[0017] Compared with the prior art, the advantages of the present invention are as follows: This invention provides a UAV view geolocation method with multi-directional cross-image interaction and designs a UAV view geolocation model on the same platform. This method achieves high-precision geolocation of UAV and satellite view images through the coordinated operation of three core mechanisms: adaptive channel interaction feature extraction, multi-directional feature segmentation, and cross-image interaction perception.

[0018] In the feature extraction stage, the adaptive channel interaction feature extraction method introduces an adaptive channel interaction adapter unit while freezing the backbone network parameters. It replaces the max pooling operation in the traditional triple attention mechanism with a complementary combination of average pooling and standard deviation pooling. This effectively enhances the channel response to static landmarks such as buildings and roads, while suppressing interference from dynamic foreground objects such as vehicles and pedestrians. This allows the network to maintain training efficiency while highlighting static landmarks and suppressing dynamic objects, avoiding catastrophic forgetting. The extracted spatial feature map can maintain attention to stable geographic landmarks even in complex dynamic scenes. Based on this, a multi-directional feature segmentation method divides the spatial feature map into non-overlapping grid regions in the horizontal and vertical directions, assigning a unique location identifier to each grid region. This allows local regions in the same spatial relative position in different images to be accurately indexed and aligned, generating local fine-grained feature branches. This forms a paradigm of complementary enhancement between global and local features, resulting in rich and discriminative feature representations, providing a spatially consistent foundation for subsequent fine-grained cross-image information interaction.

[0019] The cross-image interactive perception method achieves information flow and mutual reference between images within a batch at the region granularity through cascaded interactive attention modules and interactive perception modules. The interactive attention module performs global average pooling and global max pooling along the channel dimension for each local region feature, and sets asymmetric gradient scaling factors for the two pooling results. This ensures that compressed features from average pooling receive a larger gradient response to preserve smooth and robust global context, while compressed features from max pooling receive a smaller gradient response to weaken interfering sampling points in local textures that are detrimental to classification improvement. After morphological modulation and fusion, a spatial mask is generated, which weights the original region features, effectively removing channel redundancy information and improving the information aggregation capability of the mask. The interactive perception module performs self-attention-based context encoding on the weighted region features through at least one Transformer encoder layer, associating features of the same batch of scene images. This allows each image representation to obtain useful information from the feature representations of other images, thereby reducing the influence of similar negative samples and solving the perceptual confusion problem. This mechanism ensures that the local features of each image are no longer extracted in isolation from its own perspective, but are integrated with supplementary information from other images in the same batch at the same spatial location, which greatly alleviates the perceptual confusion caused by the high similarity in appearance between different locations.

[0020] In the feature aggregation stage, the associated local region features of the same image under all location identifiers are concatenated, and then concatenated again with the global features generated by generalized average pooling, followed by L2 normalization to form a final feature representation that is complementary and enhanced by global features and multi-directional local features. Global features provide a macroscopic description of the overall scene layout, while multi-directional local features capture fine-grained geographical details from different directions. The fusion of the two gives the final feature representation both global discriminative power and local discriminative power. During training, a multi-similarity loss function based on metric learning is used. The similarity boundaries of positive and negative sample pairs are adjusted by hyperparameters, and an online hard mining strategy is combined to apply higher weights to difficult-to-distinguish positive and negative samples, enabling the network to learn discriminative feature representations more effectively. The coordinated operation of the above modules enables the present invention to effectively overcome multiple challenges such as appearance differences, dynamic interference, and confusion of similar scenes between UAV and satellite perspectives without the need for explicit view alignment operations. It highlights static landmarks and suppresses dynamic interference through the adaptive channel interaction feature extraction module, achieves global and local feature complementarity through the multi-directional feature segmentation mechanism, and effectively weakens the impact of similar negative sample perception confusion through the cross-image interaction perception mechanism, thus significantly improving the accuracy and robustness of cross-view geolocation. Attached Figure Description

[0021] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Figure 1 A framework diagram of a UAV view geolocation method with multi-directional cross-image interaction; Figure 2 This is a schematic diagram of adaptive pooling. Figure 3 A flowchart for the adaptive channel interaction adapter; Figure 4 A flowchart for the Interactive Attention (IA) module; Figure 5 This is a flowchart of the Interactive Awareness (IP) module. Detailed Implementation

[0022] like Figures 1 to 5 As shown, this invention provides a multi-directional cross-image interactive UAV view geolocation method, which is implemented based on the designed UAV view localization model U-MDCI. The overall architecture of the U-MDCI model is as follows: Figure 1 As shown, the invention comprises three core components: an adaptive channel interaction-based feature extraction module (AFE), a multi-directional feature segmentation module (MDFP) focusing on fine-grained features, and a cross-image interaction perception module (CIIP) that associates different image features. The specific implementation steps of the method are described in detail below with reference to the accompanying drawings.

[0023] Step 1: Image Batch Acquisition and Preprocessing First, a batch of images to be processed is acquired. This batch includes one UAV-view image to be located and multiple satellite-view images with geolocation labels. In practice, both the UAV-view and satellite-view images are adjusted to a uniform input resolution, such as 384 pixels × 384 pixels, to meet the input requirements of the subsequent feature extraction network. The geolocation labels of all satellite images are known information and serve as a candidate matching library for subsequent similarity retrieval.

[0024] Step 2: Feature Extraction Based on Adaptive Channel Interaction This step processes each image in the image batch using a feature extraction network and introduces a channel interaction mechanism into the feature extraction network. In a specific embodiment, the feature extraction network uses a Transformer encoder, and more specifically, a refined DINOv2 backbone network with a ViT-B / 14 architecture.

[0025] To implement the channel interaction mechanism, this invention adds an adaptive channel interaction (ACI) adapter in parallel to each Transformer block of the Transformer encoder, such as... Figure 2 As shown. The ACI adapter is specifically added between the activation layer and the upper and lower projection layers of the Transformer block, i.e., inserted in parallel. The internal structure of the ACI adapter is as follows. Figure 3 As shown, it contains two sub-branches, each of which reorganizes the feature map back to the original dimensions through steps such as rotation transformation, adaptive pooling, attention weight calculation, batch normalization, and Sigmoid activation.

[0026] The ACI adapter processes input features as follows: First, it converts the input features... After performing average pooling and standard deviation pooling operations respectively, average pooling features and standard deviation pooling features are obtained. The average pooling operation is performed along the specified dimension. Execute the calculation of the mean of the feature along that dimension; standard deviation pooling is performed along the same dimension. The process involves calculating the standard deviation of the feature along that dimension. This invention replaces the max pooling operation commonly used in traditional triple attention mechanisms with standard deviation pooling, allowing standard deviation pooling and average pooling to complement each other and work together to adjust channel representation. The above process can be represented as: in, Indicates average pooling. Indicates standard deviation pooling. It is the dimension along which average pooling and standard deviation pooling operations are performed.

[0027] Subsequently, two learnable weight parameters are used. and Weighted fusion of average pooling features and standard deviation pooling features is performed. and All parameters are trainable floating-point parameters with values ​​ranging from 0 to 1. The specific calculation method for weighted fusion is as follows: first, the average pooling feature and the standard deviation pooling feature are added element-wise and multiplied by half to obtain the basic fusion term; then... Perform element-wise multiplication with broadcast on the average pooling feature, and... Element-wise multiplication with broadcast is performed using the standard deviation pooling features; finally, the three features are added element-wise to obtain the fused output features. The specific calculation formula is as follows: In the formula, , ∈[0,1] are two trainable floating-point parameters. This represents element-wise addition. This indicates element-wise multiplication with broadcast.

[0028] Feature map of backbone network Dimensions The input is fed into two sub-branches of the ACI module. In this approach, two sub-branches learn channel attention weights interactively, starting from average pooling features and standard deviation pooling features respectively. This strengthens the channel response for static landmarks and suppresses the channel response for dynamic objects. Each sub-branch generates its own sub-feature map. and The outputs of the two are cascaded and combined to form the final output of the ACI adapter. This process can be represented as: in, and It is generated by combining two sub-feature maps generated by the ACI module. .

[0029] Within each Transformer block, the ACI adapter runs parallel to the main path in a residual connection manner, and its output is multiplied by a scaling factor. Then it is added to the main path features. The computation process for each adaptive Transformer block is as follows: first, the output of the previous layer is processed... Perform layer normalization Then it is fed into the multi-head self-attention MHA module and combined with Residual connections are formed to obtain intermediate features; then, these intermediate features are again subjected to layer normalization and fed into the Multilayer Perceptron (MLP) module and the ACI adapter, respectively. The outputs of the MLP and the ACI adapter are multiplied by a scaling factor. The final output of the Transformer block is obtained by adding the result, intermediate features, and other data. The computation of each adaptive Transformer block can be represented as: In the formula, Indicates ACI adapter, The scaling factor is used. By freezing the backbone network parameters, only the learnable weight parameters in the ACI adapter are updated, allowing the network to maintain training efficiency while highlighting static landmarks, suppressing dynamic objects, and avoiding catastrophic forgetting.

[0030] After processing by the aforementioned feature extraction network, each image in the image batch is encoded into a set of output image tokens, and these tokens are then reshaped into a format with dimensions of [missing information]. Spatial feature map, in which For batch size, and These represent the height and width of the feature map, respectively. This represents the number of channels. The spatial feature map is the result optimized by the channel interaction mechanism, where the feature responses of static landmarks are enhanced and dynamic interference is suppressed.

[0031] Step 3: Global and Local Feature Generation Based on Multi-directional Feature Segmentation For each spatial feature map, this step generates global features through generalized average pooling and local region features through multi-directional feature segmentation.

[0032] Specifically, in the global branch, the spatial feature map is globally aggregated using generalized average pooling to generate global features. Generalized average pooling controls the smoothness of pooling behavior through a learnable pooling parameter. It tends to max pooling when the parameter approaches infinity and degenerates into average pooling when the parameter is 1, thus achieving a flexible balance between preserving details and global smoothness.

[0033] In the local branch, the spatial feature map is divided into regions using the Multi-Directional Feature Plotting (MDFP) method. Specifically, the spatial feature map is divided into three equal parts along its height and width, resulting in a non-overlapping grid of 9 3×3 grid regions. Each grid region is assigned a unique location identifier, for example, numbered sequentially from the top left corner (row or column) as position 1 to position 9, ensuring that the same location identifier in different images corresponds to the same relative spatial position. The first image in the batch of processed images is then... Each region feature is used as a token vector to form a region feature sequence. ,in The value range is from 1 to 9. Therefore, the overall fine-grained feature... It can be represented as a set of 9 regional feature sequences, namely: In the formula, .

[0034] Through the above processing, global features are simultaneously obtained for each image. And fine-grained features consisting of 9 local region features. This has led to a paradigm where global and local features complement and enhance each other.

[0035] Step 4: Region-level feature association based on cross-image interaction perception This step utilizes the Cross-Image Interactive Perception (CIIP) module to perform batch-based cross-image information exchange on local region features. The CIIP module comprises two sub-modules: a cascaded Interactive Attention (IA) module and an Interactive Perception (IP) module, as described below. Figure 4 and Figure 5 As shown.

[0036] First, for each location marker, the local region features of all images within the image batch at that location marker are extracted into a set of region feature sequences. Specifically, for the first... Each location identifier will be within the batch. Local region features of each image at that location. These features are combined to form a set of regional feature sequences corresponding to that location identifier. In this way, the granularity of cross-image interaction is refined from the whole-image level to the region level.

[0037] Then, the feature sequences of each group of regions are processed sequentially through the interactive attention module and the interactive perception module.

[0038] The specific processing steps of the interactive attention module are as follows: For the input local region features... Its dimensions are Global average pooling is performed along the channel dimension. and global max pooling The first compressed feature map is obtained. Second compression feature map Both have the same dimension. Global average pooling imparts global smoothness to feature maps, while global max pooling preserves local texture characteristics. This process can be represented as: Next, and Perform downsampling to reduce its resolution to The downsampled compressed feature map is obtained. and All dimensions , which serves as the main input for weight learning.

[0039] Then, morphological modulation is performed on the two downsampled compressed feature maps. The specific method of morphological modulation is as follows: using a modulation coefficient generator... In gradient scaling factor With the assistance of [unclear], modulation coefficients acting on one compressed feature map are generated from one compressed feature map. The gradient scaling factor is set for the compressed feature map derived from global average pooling. The gradient scaling factor set for the compressed feature map derived from global max pooling is greater than that set for the feature map. In a specific embodiment, The value is set to 0.4. The value is set to 0.1. Specifically, with As parameters, with As input, modulation coefficients are generated by the modulation coefficient generator and then compared with... Perform element-wise multiplication to obtain the modulated features. ;by As parameters, with As input, modulation coefficients are generated by the modulation coefficient generator and then compared with... Perform element-wise multiplication to obtain the modulated features. This morphological modulation process can be represented as: in, This represents the modulation coefficient generator, which is based on the gradient scaling factor. With the assistance of [unclear], based on input features Modulation coefficients are generated. By setting an asymmetric gradient scaling factor, compressed features derived from average pooling obtain larger gradient responses in subsequent fusion to preserve a smooth and robust global context; compressed features derived from max pooling obtain smaller gradient responses to weaken interfering sampling points in local textures that are detrimental to classification improvement.

[0040] The two modulated feature maps and Upsampling Restore to original resolution ,get and The two upsampled feature maps are concatenated and then input into the convolution function. After processing, it is then passed through the Sigmoid function. Generate a spatial mask Its dimensions are The above process can be represented as: Combine the spatial mask with the original local region features of the input. Perform element-wise multiplication to obtain the weighted region features. ,Right now: Through the above interactive attention processing, redundant channel information is effectively removed, sampling points that are not conducive to classification are weakened, and the information aggregation capability of the mask is improved.

[0041] The processing steps of the interactive sensing module are as follows: This module contains at least one Transformer encoder layer; in a specific embodiment, two Transformer encoder layers are used. Each encoder layer also contains a Multilayer Perceptron (MLP) layer and a Layer Normalization (LN) layer. For each location identifier... The weighted region features of all images under this location are identified. The input sequence is fed into the Transformer encoder layer of the interactive perception module. A multi-head self-attention mechanism is used to perform contextual encoding on the weighted region features of different images at the same location marker, modeling the feature dependencies between different images within the image batch at that location marker. This processing allows the local region features of each image to aggregate corresponding location information from other images within the batch, achieving cross-image collaborative perception. The interactive perception module outputs the correlated local region features, i.e., fine-grained correlated features.

[0042] Through the above cross-image interactive perception processing, each image representation can obtain useful information from the feature representations of other images, reducing the influence of similar negative samples and effectively solving the problem of perception confusion.

[0043] Step 5: Generation of the final feature representation For each image, the fine-grained correlation features output by the interactive perception module at all nine location markers are sequentially concatenated, i.e., flattened. Simultaneously, the concatenated local features are concatenated again with the global features generated by generalized average pooling in step three. L2 normalization is then applied to the concatenated global features to obtain the final feature representation of the image. This final feature representation simultaneously contains a macroscopic description of the global scene layout, multi-directional local fine-grained geographic details, and cross-image complementary information, resulting in stronger discriminative power and robustness.

[0044] Step Six: Similarity Matching and Geographic Location Determination The similarity between the final feature representation of the UAV's view image and the final feature representations of each satellite view image is calculated. In practice, cosine similarity is used as the similarity measure. The satellite images are sorted from highest to lowest similarity, and the geographic location label corresponding to the satellite image with the highest similarity is used as the UAV's geographic location result. This method supports both UAV localization tasks that use UAV view images as queries and retrieve matching images from a satellite image database, and navigation tasks that use satellite view images as queries and retrieve matching images from a UAV image database.

[0045] Step 7: Model Training Prior to the localization steps described above, the overall network constituting the feature extraction network and the cross-image interaction module needs to be trained. Training is performed using a fully supervised learning approach. The training dataset is the GSV-Cities dataset, which contains 560,000 images captured at 67,000 locations with highly accurate geolocation labels.

[0046] During training, a metric-learning-based multi-similarity loss function is used to update the network parameters. Within the same training batch, this loss function assigns higher weights to hard-to-distinguish sample pairs based on the self-similarity of positive sample pairs and the relative similarity of negative sample pairs, achieving online hard mining. The multi-similarity loss is implemented through hyperparameters... , and The similarity boundaries for positive and negative sample pairs are adjusted separately to achieve online mining of difficult-to-distinguish positive and negative samples. The specific calculation method for the multi-similarity loss is as follows: for each query image in the batch processing... The formula for calculating the multi-similarity loss is: In the formula, For batch size, It corresponds to The index set of positive samples, It corresponds to The negative sample index set, and It is the cosine similarity between positive and negative sample pairs. , and There are three hyperparameters. The loss function consists of two parts: a positive sample term and a negative sample term. For each positive sample pair, the positive sample term calculates an exponentially decaying similarity difference, while for each negative sample pair, the negative sample term calculates an exponentially increasing similarity difference. The logarithms of these differences are multiplied by the reciprocal of the corresponding hyperparameter, and then summed. Finally, the average is taken over all query images within the batch. By combining this multi-similarity loss with an online hard mining strategy, the network can learn more discriminative feature representations in a more targeted manner.

[0047] To validate the effectiveness of our method, we compared it with other methods on seven datasets: VPAir, Pitts30k, MSLS, Tokyo24 / 7, Nordland, AmsterTime, and SVOX. Nordland, AmsterTime, and SVOX are three challenging datasets used to demonstrate the superior performance of our model. Scene information and image count for each dataset are described in Table 1.

[0048] In this embodiment, the recall rate (R@K) metric is used to evaluate recognition performance. This metric measures whether, among the first K retrieved database images, there is at least one reference image that matches the query image within the true annotation threshold. Following the standard evaluation process, thresholds were set for different datasets in this embodiment: 25 meters and 40 degrees for MSLS, 25 meters for Pitts30k, Tokyo24 / 7, and SVOX, ±10 frames for Nordland, a unique corresponding sample for AmsterdamTime, and ±3 frames for VPAir.

[0049] Table 1. Description of scene information and number of images for different datasets In this embodiment, the proposed U-MDCI is first compared with six mainstream global retrieval-based methods (NetVLAD, SFRS, CosPlace, GCL, MixVPR, and EigenPlaces). Table 2 reports the quantitative results on Pitts30k, Tokyo24 / 7, and MSLS. Further, after reducing the dimensionality of U-MDCI features to 4096 using PCA, R@K decreases slightly, but the overall performance is still superior to other methods. On Pitts30k, which is mainly affected by perspective changes, U-MDCI improves R@1 to 94.8%. On MSLS and Tokyo24 / 7, where conditions vary more drastically, U-MDCI still performs outstandingly, with R@5 reaching 95.5% on MSLS-val and R@1 reaching 94.6% on Tokyo24 / 7, significantly outperforming global retrieval and two-stage methods.

[0050] Table 2 Comparison results with existing competing methods on Pitts30k, MSLS-val, and Tokyo 24 / 7. On the VPAir dataset based on UAV view images, the proposed U-MDCI also achieved excellent localization accuracy. As shown in Table 3, compared with MixVPR and EigenPlaces, U-MDCI achieved an absolute improvement of 32.9% and 36.5% in R@1, respectively.

[0051] Table 3 shows the comparison results with existing competing methods on the VPAir dataset. To further evaluate the performance of the proposed method under extreme scenarios, experiments were conducted on three challenging datasets: the Nordland dataset, which shows seasonal changes; the AmsterdamTime dataset, which contains historical (black and white) and contemporary (color) images; and the SVOX dataset, which shows extreme light and weather variations. As shown in Tables 4 and 5, the results demonstrate that the proposed method has significant advantages over existing competing methods (EigenPlaces), achieving absolute improvements of 20.0%, 14.6%, 28.7%, 7.8%, and 9.7% in R@1 on Nordland, AmsterdamTime, SVOX-Night, SVOX-Rain, and SVOX-Sun, respectively. This further demonstrates the high robustness of the feature representations generated by the U-MDCI network.

[0052] Table 4. Comparison results with existing competing methods on the Nordland and Amsterdam Time datasets. Table 5. Comparison results with existing competing methods on SVOX's Night, Rain, and Sun scenes. In summary, this invention effectively improves upon existing UAV view localization methods on the same platform by employing an adaptive channel interaction feature extraction (AFE) method, a multi-directional feature segmentation (MDFP) method, and a cross-image interactive perception (CIIP) method incorporating interaction concern (IA) and interaction awareness (IP). These methods address issues such as dynamic foreground interference, perceptual confusion, and neglect of fine-grained local features. The invention achieves superior performance compared to existing algorithms on multiple public datasets. This method enhances the robustness and discriminative power of UAV view localization on the same platform.

[0053] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0054] The units described in some embodiments of this disclosure can be implemented in software or in hardware. The described units can also be located in a processor, and the functions described above can be performed at least in part by one or more hardware logic components.

[0055] The above description only illustrates the preferred embodiments of the present invention and should not be construed as limiting the scope of the claims. The present invention is not limited to the above embodiments, and variations in its specific structure are permitted. All modifications made within the scope of the independent claims of this invention are also within the scope of protection of this invention.

Claims

1. A method for UAV view geolocation with multi-directional cross-image interaction, characterized in that, include: Acquire image batches, which include UAV view images to be located and multiple satellite view images with geographic location tags; The image batch is processed using a feature extraction network, and a channel interaction mechanism is introduced into the feature extraction network. The channel interaction mechanism is used to adaptively adjust the channel representation of the feature map according to the average statistic and dispersion statistic along the channel dimension of the feature map, so as to enhance static landmark features and suppress dynamic interference, thereby obtaining an optimized spatial feature map. For each of the aforementioned spatial feature maps, regions are divided along at least two spatial directions to generate multiple local region features with location identifiers; For each location identifier, the local region features of all images in the image batch under that location identifier are constructed into a set of region feature sequences; Each set of region feature sequences is input into the cross-image interaction module, which is configured to: filter out interference information from the local region features in the region feature sequences, and model the relationship between the local region features of different images under the same location identifier based on the attention mechanism, so that the local region features of each image are fused with the corresponding location information of other images in the image batch, and output the associated local region features. For each image, the associated local region features under all location identifiers are aggregated and normalized to obtain the final feature representation of the image; The geographical location of the UAV is determined based on the similarity between the final feature representation of the UAV's view image and the final feature representation of each satellite view image.

2. The UAV view geolocation method with multi-directional cross-image interaction according to claim 1, characterized in that, The channel interaction mechanism is specifically as follows: Perform average pooling and standard deviation pooling operations on the input features respectively to obtain average pooling features and standard deviation pooling features; The average pooling feature and the standard deviation pooling feature are weighted and fused using learnable weight parameters to adjust the channel representation. The standard deviation pooling operation is used to replace the max pooling operation in the triple attention mechanism, complementing the average pooling to enhance static landmark features and suppress dynamic interference.

3. The UAV view geolocation method with multi-directional cross-image interaction according to claim 2, characterized in that, The feature extraction network is a Transformer encoder; An adaptive channel interaction adapter is added between the activation layer and the upper and lower projection layers of each Transformer block in the Transformer encoder and inserted in parallel. The adaptive channel interaction adapter includes two sub-branches, which learn channel attention weights interactively from the average pooling feature and the standard deviation pooling feature, respectively, to enhance the channel response for static landmarks and suppress the channel response for dynamic objects. The outputs of the two sub-branches are fused together to form the final output of the adapter.

4. The UAV view geolocation method with multi-directional cross-image interaction according to claim 3, characterized in that, The processing flow of the adaptive channel interaction adapter for input features is as follows: The input features are processed by average pooling and standard deviation pooling respectively to obtain average pooling features and standard deviation pooling features; The two sub-branches take the average pooling feature and the standard deviation pooling feature as inputs, respectively, and each learns the channel attention weights. The outputs of the two sub-branches are concatenated to form the final output of the adaptive channel interaction adapter.

5. The UAV view geolocation method with multi-directional cross-image interaction according to claim 1, characterized in that, The division of the region along at least two spatial directions specifically includes: The spatial feature map is divided into multiple grid regions in a non-overlapping manner along the height and width directions. Each of the grid regions is assigned a unique location identifier, such that the same location identifier in different images corresponds to the same spatial relative position.

6. The UAV view geolocation method with multi-directional cross-image interaction according to claim 1, characterized in that, The final feature representation of the image obtained specifically includes: While dividing the spatial feature map into regions, global aggregation of the spatial feature map is performed through generalized average pooling to generate global features. The local region features of the same image after association under all location identifiers are concatenated, and then concatenated again with the global features. The concatenated global features are then L2 normalized to obtain the final feature representation of the image.

7. The UAV view geolocation method with multi-directional cross-image interaction according to claim 1, characterized in that, The cross-image interaction module includes a cascaded interaction attention module and an interaction perception module. The interactive attention module is used to filter out interference information from each local region feature in the region feature sequence. Its processing steps include: For the local region features of the input, global average pooling and global max pooling are performed along the channel dimension respectively to obtain two compressed feature maps; The two compressed feature maps are morphologically modulated and then fused to generate a spatial mask. The spatial mask is multiplied by the input local region features to obtain the weighted region features; The interactive perception module is used to model cross-image relationships under the same location identifier based on the self-attention mechanism of the weighted region features, and output the associated local region features.

8. The UAV view geolocation method with multi-directional cross-image interaction according to claim 7, characterized in that, Set gradient scaling factors for the compressed feature maps obtained by global average pooling and global max pooling, respectively; Among them, the gradient scaling factor is set for the compressed feature map derived from global average pooling. The gradient scaling factor set for the compressed feature map derived from global max pooling is greater than that set for the feature map. ; Will and These are applied to the corresponding compressed feature maps to adjust their contribution to spatial mask generation.

9. A UAV view geolocation method with multi-directional cross-image interaction according to claim 7, characterized in that, The interactive perception module includes at least one Transformer encoder layer for performing self-attention-based context encoding on weighted region features under the same location identifier, so that the local region features of each image are aggregated from the corresponding location information of other images in the image batch.

10. A UAV view geolocation method with multi-directional cross-image interaction according to claim 1, characterized in that, It also includes the step of training the networks that constitute the feature extraction network and the cross-image interaction module; During training, a loss function based on metric learning is used to update the network parameters. Within the same training batch, the loss function applies higher weights to hard-to-distinguish sample pairs based on the self-similarity of positive sample pairs and the relative similarity of negative sample pairs. The loss function is a multi-similarity loss, which is achieved through hyperparameters. , , The similarity boundaries of positive sample pairs and negative sample pairs are adjusted separately to achieve online mining of difficult-to-distinguish positive and negative samples. The multi-similarity loss is calculated as follows: in, For batch size, and These are the sets of positive sample pairs and the sets of negative sample pairs, respectively. Indicates similarity. , , This is a hyperparameter.