Cross-view geolocalization oriented frequency enhanced view-aware feature learning method

Abstract

The application provides a frequency enhanced view perception feature learning method for cross-view geolocation, and belongs to the technical field of data processing, and specifically comprises the following steps: obtaining multi-view input data containing ground view, satellite view and intermediate view; inputting the multi-view input data into a visual feature extraction network with shared parameters respectively to obtain local image features and global features of each view, constructing a view perception label for each view, and updating the view perception label; performing frequency domain decomposition on the input images of each view to extract frequency domain features, performing cross-modal interaction between the updated view perception label and the frequency domain features to obtain enhanced view perception features; generating final discriminative feature representations of each view based on the enhanced view perception features and the global features, and performing geolocation of cross-view images by calculating the similarity between the discriminative feature representations under different views. The scheme improves the positioning accuracy and robustness.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a frequency-enhanced perspective-aware feature learning method for cross-view geolocation. Background Technology

[0002] Currently, to improve cross-view matching performance, some studies have introduced attention mechanisms, feature weighting modules, and multi-scale feature fusion structures into deep networks to enhance the modeling ability of key regions and discriminative information. Meanwhile, researchers have also subdivided cross-view geolocation tasks into two categories based on the imaging characteristics of different platforms: ground-to-satellite cross-view geolocation and air-to-satellite cross-view geolocation. The former mainly addresses the matching problem between ground-view images and overhead satellite images, while the latter addresses the matching needs between oblique or overhead images acquired by aerial platforms such as UAVs and satellite images. Although the two types of tasks differ in imaging perspective, their core objective is to improve positioning accuracy and real-time performance under cross-view conditions.

[0003] Furthermore, cross-view geolocation tasks present significant technical challenges. On the one hand, images from different viewpoints exhibit substantial differences in imaging geometry, scale distribution, and appearance, leading to difficulties in feature alignment. On the other hand, real-world applications demand high levels of positioning accuracy and real-time performance, while existing methods still fall short in terms of model complexity, inference efficiency, and system robustness.

[0004] It is evident that there is an urgent need for a frequency-enhanced perspective-aware feature learning method for cross-perspective geolocation that offers high positioning accuracy and robustness. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a frequency-enhanced viewpoint-aware feature learning method for cross-view geolocation, which at least partially solves the problems of poor positioning accuracy and robustness in the prior art.

[0006] This invention provides a frequency-enhanced viewpoint-aware feature learning method for cross-view geolocation, comprising:

[0007] Step 1: Acquire ground view images and satellite view images, and generate intermediate view images based on ground view images to obtain multi-view input data containing ground view, satellite view and intermediate view.

[0008] Step 2: Input the multi-view input data into the visual feature extraction network with shared parameters to obtain the local image features and global features of each view. Construct a view perception label for each view and perform interactive modeling between the view perception label and the local image features of the corresponding view to update the view perception label.

[0009] Step 3: Decompose the input images from each viewpoint into frequency domain, extract frequency domain features, and use the updated viewpoint perception markers and frequency domain features to perform cross-modal interaction, integrating frequency information into the viewpoint perception markers to obtain enhanced viewpoint perception features.

[0010] Step 4: Based on the enhanced viewpoint perception features and global features, generate the final discriminative feature representations for each viewpoint. By calculating the similarity between the discriminative feature representations under different viewpoints, perform geolocation of cross-viewpoint images.

[0011] According to a specific implementation of an embodiment of the present invention, the step of generating an intermediate view image based on a ground view image includes:

[0012] The ground view image is processed by inverse perspective mapping to generate a bird's-eye view image as an intermediate view image.

[0013] According to a specific implementation of an embodiment of the present invention, the step of constructing a view-aware marker for each view includes:

[0014] Initialize a base tag shared by all viewpoints:

[0015] ;

[0016] in, Indicates a shared base tag. Representing tensor space, Indicates the size of the feature dimension;

[0017] Define a learnable viewpoint bias vector for each viewpoint:

[0018] ;

[0019] in, Represents the viewpoint bias vector. Indicates the number of viewpoint types;

[0020] Add the base label to the view bias vector of the viewpoint to which the current input image belongs to generate the view-aware label for that viewpoint:

[0021] ;

[0022] in, express Perceptual markers of perspective Indicates a shared base tag. Indicates the viewpoint identifier.

[0023] According to a specific implementation of an embodiment of the present invention, the step of interactively modeling the viewpoint-aware marker and the local image features of the corresponding viewpoint includes:

[0024] The viewpoint-aware markers are concatenated with the local image features of the corresponding viewpoints to construct an initial feature sequence:

[0025] ;

[0026] in, Represents the initial feature sequence. This represents the sequence of image patch features from this perspective;

[0027] The initial feature sequence is input into a multi-layer self-attention network for iterative updates, allowing the viewpoint-aware marker to perform attention calculations with local image features in each layer, aggregating discriminative region information in the image. The expression for the multi-layer self-attention network is:

[0028] ;

[0029] in, For normalization function, , , These represent the query vector, key vector, and value vector, respectively.

[0030] According to a specific implementation of an embodiment of the present invention, the step of performing frequency domain decomposition on the input images from each viewpoint and extracting frequency domain features includes:

[0031] Perform frequency transformation on the input image to convert it to the frequency domain;

[0032] Low-frequency and high-frequency components can be separated from the frequency spectrum using frequency masks;

[0033] The low-frequency components and high-frequency components are respectively encoded to obtain low-frequency features and high-frequency features;

[0034] Low-frequency and high-frequency features are processed into frequency feature sequences consistent with the dimension of the viewpoint-aware label.

[0035] According to a specific implementation of an embodiment of the present invention, the frequency transformation employs a Fast Fourier Transform:

[0036] ;

[0037] in, Indicates Fourier transform, express Input data after viewpoint normalization;

[0038] The frequency mask includes a low-pass mask and a high-pass mask:

[0039] ;

[0040] ;

[0041] in, Indicates low-frequency input. For high frequency input, , For frequency mask, Represents the dot product;

[0042] The feature encoding is processed using a lightweight convolutional network:

[0043] ;

[0044] ;

[0045] in, Indicates low-frequency characteristics. Indicates high-frequency characteristics, This represents pointwise convolution. This represents a smooth convolution with a kernel of 3. Indicates the activation function;

[0046] The encoded low-frequency and high-frequency features are dimensionally aligned, flattened, and concatenated to obtain a frequency feature sequence:

[0047] ;

[0048] in, Represents a frequency characteristic sequence. This represents the concatenation function. This represents the flattening function.

[0049] According to a specific implementation of an embodiment of the present invention, the step of performing cross-modal interaction using the updated view-aware marker and the frequency domain features includes:

[0050] Perceive markers from the updated perspective As a query term, use the frequency feature sequence as both key and value:

[0051] ;

[0052] ;

[0053] ;

[0054] in, , , These represent the query vector, key vector, and value vector, respectively. For layer normalization function;

[0055] Perform cross-attention computation:

[0056] ;

[0057] Residual connectivity and nonlinear enhancement are performed to obtain frequency-enhanced viewpoint perception features, wherein the expressions for residual connectivity and nonlinear enhancement are:

[0058] ;

[0059] ;

[0060] in, This indicates a fully connected layer.

[0061] According to a specific implementation of an embodiment of the present invention, the step of generating the final discriminative feature representation of each viewpoint includes:

[0062] The enhanced view-aware features are fused with global features using a residual connection method to obtain fused features;

[0063] The fused features are input into a multilayer perceptron for mapping to obtain the final discriminative feature representation.

[0064] According to one specific implementation of the present invention, the visual feature extraction network is a pre-trained model based on the visual Transformer structure.

[0065] According to a specific implementation of an embodiment of the present invention, the method further includes:

[0066] During the training phase, a triple contrastive learning loss function based on ground view, intermediate view and satellite view is constructed. By constraining the similarity between features from different viewpoints of the same geographical location and the differences between features from different geographical locations, the model parameters of the pre-trained model based on the visual Transformer structure are optimized.

[0067] The frequency-enhanced view perception feature learning scheme for cross-view geolocation in this embodiment of the invention includes: Step 1, acquiring ground view images and satellite view images, and generating intermediate view images based on the ground view images to obtain multi-view input data containing ground view, satellite view, and intermediate view; Step 2, inputting the multi-view input data into a visual feature extraction network with shared parameters to obtain local image features and global features for each view, constructing view perception labels for each view, and performing interactive modeling between the view perception labels and the local image features of the corresponding view to update the view perception labels; Step 3, performing frequency domain decomposition on the input images of each view, extracting frequency domain features, and using the updated view perception labels and frequency domain features to perform cross-modal interaction, integrating frequency information into the view perception labels to obtain enhanced view perception features; Step 4, generating final discriminative feature representations for each view based on the enhanced view perception features and global features, and performing cross-view image geolocation by calculating the similarity between discriminative feature representations under different views.

[0068] The beneficial effects of this invention are as follows: Based on the traditional ground-satellite dual-view framework, this invention introduces an intermediate view representation based on Inverse Perspective Mapping (IPM) to construct a joint modeling mechanism for three perspectives: ground view, satellite view, and bird's-eye view (BEV). By introducing the bird's-eye view as a transitional perspective, the geometric differences between different imaging perspectives are effectively alleviated, improving the stability of cross-view feature alignment. DINOv2, based on a visual Transformer structure, is used as the feature extraction network to perform unified feature encoding on images from different perspectives. In the feature extraction process, in addition to conventional patch tokens and global tokens, this invention further introduces additional learnable global guide tokens (view tokens). For different perspectives, independent learnable initialization parameters are set for each global guide token to introduce view-related prior biases, enabling the model to adaptively focus on discriminative image regions under different perspectives during subsequent attention calculations. Furthermore, a multi-layer self-attention mechanism is designed in the feature encoding stage, inputting the global guide token and patch tokens together into the self-attention module for joint modeling. Through multi-layer self-attention computation, the global guiding marker can progressively aggregate and update important information from local regions at different levels, thereby forming a global representation with high discriminative power for the current viewpoint image. A frequency feature extraction module is introduced. This module performs frequency domain transformation on input images from different viewpoints, decomposing the image into high-frequency and low-frequency components to characterize the structural contour information and overall semantic information in the image. Subsequently, a lightweight convolutional neural network is used to encode the high-frequency and low-frequency features, and through feature mapping alignment operations, the frequency features are mapped to a feature dimension space consistent with the global guiding marker. After multi-layer self-attention modeling, a cross-attention mechanism is introduced, using the updated global guiding marker as the query term and high-frequency and low-frequency features as keys and values, to achieve selective aggregation of frequency information by global semantic features. Through this cross-attention mechanism, the model can adaptively adjust the degree of attention to high-frequency and low-frequency information according to different viewpoints and scenarios. The updated global guiding marker is fused with the original global features through residual connections, and the fused features are mapped through a multi-layer perceptron to obtain the final cross-viewpoint discriminative feature representation. This feature is used for subsequent similarity calculation and contrastive learning training. During the training phase, this invention constructs a contrastive learning mechanism based on three-view input. By jointly constraining the similarity relationship between features from the ground view, bird's-eye view, and satellite view, the same geographical location under different views maintains higher consistency in the feature space, while the features of different geographical locations maintain sufficient distinguishability, thereby effectively improving the overall performance of cross-view geolocation. Attached Figure Description

[0069] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a flowchart illustrating a frequency-enhanced viewpoint-aware feature learning method for cross-view geolocation provided in an embodiment of the present invention. Detailed Implementation

[0071] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0072] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0073] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this invention, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0074] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The illustrations only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0075] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0076] This invention provides a frequency-enhanced perspective-aware feature learning method for cross-view geolocation, which can be applied to the cross-view geolocation process in navigation and positioning scenarios.

[0077] See Figure 1 This is a flowchart illustrating a frequency-enhanced viewpoint-aware feature learning method for cross-view geolocation provided by an embodiment of the present invention. Figure 1 As shown, the method mainly includes the following steps:

[0078] Step 1: Acquire ground view images and satellite view images, and generate intermediate view images based on ground view images to obtain multi-view input data containing ground view, satellite view and intermediate view.

[0079] Step 2: Input the multi-view input data into the visual feature extraction network with shared parameters to obtain the local image features and global features of each view. Construct a view perception label for each view and perform interactive modeling between the view perception label and the local image features of the corresponding view to update the view perception label.

[0080] Step 3: Decompose the input images from each viewpoint into frequency domain, extract frequency domain features, and use the updated viewpoint perception markers and frequency domain features to perform cross-modal interaction, integrating frequency information into the viewpoint perception markers to obtain enhanced viewpoint perception features.

[0081] Step 4: Based on the enhanced viewpoint perception features and global features, generate the final discriminative feature representations for each viewpoint. By calculating the similarity between the discriminative feature representations under different viewpoints, perform geolocation of cross-viewpoint images.

[0082] The method of the present invention will now be described in detail with reference to a specific embodiment. The overall process is as follows:

[0083] Step 1, Input Preprocessing and Multi-view Construction: Normalize and resize the ground view image and satellite view image, and construct the bird's-eye view (BEV) image based on inverse perspective mapping to form three-view input data.

[0084] Step 2, Multi-view feature encoding: The dinov2 visual feature extraction backbone network with shared parameters is used to encode the features of the three viewpoint images to obtain image patch features and global feature representations.

[0085] Step 3, Viewpoint-aware global label initialization: Introduce independent learnable global guiding labels for each viewpoint and use them as bias injection feature encoding process.

[0086] Step 4, Multi-layer self-attention feature modeling: Through a multi-layer self-attention mechanism, a deep interaction between global guiding markers and image patch features is achieved, and the global feature representation is updated layer by layer.

[0087] Step 5, Frequency Domain Feature Extraction: Perform frequency transformation on the input images from different viewpoints to extract high-frequency and low-frequency features, and encode them through a lightweight convolutional network.

[0088] Step 6, Frequency Feature Dimension Alignment and Serialization: Map the frequency domain features to the embedding dimension consistent with the global label and construct the frequency feature sequence.

[0089] Step 7, Cross-attention frequency enhancement: Selective fusion of frequency information by global semantic features is achieved through a cross-attention mechanism.

[0090] Step 8, Global Feature Fusion and Discriminative Feature Generation: The final cross-view discriminative features are generated through residual connections and multilayer perceptrons.

[0091] Step 9, Triple contrastive learning optimization: Construct a triple contrastive learning loss function based on ground view, bird's-eye view and satellite view to complete model training.

[0092] Step 1 includes the following sub-steps:

[0093] Step 1.1 Raw Data Analysis: Let the remote sensing image be... The corresponding satellite view image is .

[0094] Step 1.2 Bird's-eye View (BEV) Generation: Based on camera intrinsic and extrinsic parameters, generate ground-view images. Apply inverse perspective mapping (IPM) to generate a bird's-eye view image:

[0095] ;

[0096] Step 1.3 Standardization Processing: Normalize, pad, and crop the images from the three viewpoints to obtain the model input format:

[0097] ;

[0098] in:

[0099] Indicates image input, Indicates the length and width of the output. , , These represent the ground view, satellite view, and BEV view, respectively.

[0100] Step 2: Multi-view Feature Encoding: Assume the visual feature extraction network is a Transformer backbone network with shared parameters (DINOv2), and perform forward encoding for each view:

[0101] ;

[0102] in:

[0103] Represents image patch features (Patch Tokens). Represents a global token;

[0104] Furthermore, the specific implementation of step 3 includes the following sub-steps:

[0105] Step 3.1 Unify global tag definition:

[0106] Initialize a shared, learnable global token (View Token):

[0107] ;

[0108] in, This is the feature embedding dimension. This global tag is shared across all perspectives and is used to carry consistent global semantic information across perspectives.

[0109] Step 3.2 Define the viewpoint offset parameter, i.e.:

[0110] To characterize the differences between different viewpoints, a viewpoint offset parameter table is introduced:

[0111] ;

[0112] in, The number of viewpoint types in this invention =3, corresponding to ground view, bird's-eye view, and satellite view, respectively. Each In order to be with the first The learnable viewpoint bias vector corresponding to each viewpoint.

[0113] Step 3.3, View-Aware Global Marker Generation:

[0114] During the training and inference phases, view-aware global guidance markers are generated based on the viewpoint identifiers of the input samples in the following manner:

[0115] ;

[0116] Different values ​​correspond to different viewpoint types. This operation superimposes the corresponding viewpoint bias term onto the shared global label, so that the generated global label can adaptively introduce viewpoint-related information while maintaining a unified semantic space. This allows the model to supplement domain-invariant features for different viewpoints in the subsequent attention mechanism.

[0117] Step 3.4, Feature sequence construction:

[0118] The generated view-aware global markers are concatenated with the image patch features of the corresponding viewpoints to construct an initial feature sequence:

[0119] ;

[0120] in, This represents the sequence of image patch features from this perspective.

[0121] The specific process of step 4 is as follows:

[0122] Step 4.1, Feature Normalization:

[0123] ;

[0124] in, Represents the normalization function;

[0125] Step 4.2, Multi-head self-attention calculation:

[0126] ;

[0127] in:

[0128] ;

[0129] After residual linking, the final attention output is:

[0130] ;

[0131] Step 4.3 Feedforward Network (FFN) Enhancement:

[0132] ;

[0133] Where GELU is the activation function:

[0134] ;

[0135] Step 4.5 Residual Update:

[0136] After stacking LLL layers of self-attention, the final globally guided feature is obtained:

[0137] ;

[0138] Step 5: Frequency domain feature extraction:

[0139] To fully explore the structural and texture information in images from different perspectives, this invention introduces a frequency domain feature enhancement module in addition to spatial domain feature modeling.

[0140] Step 5.1 Frequency Conversion:

[0141] ;

[0142] ;

[0143] ;

[0144] in, Indicates Fourier transform, express Input data after viewpoint normalization , For frequency mask.

[0145] Step 5.2 Lightweight frequency feature convolutional coding:

[0146] ;

[0147] ;

[0148] in, Indicates low-frequency characteristics. Indicates high-frequency characteristics, This represents pointwise convolution. This represents a smooth convolution with a kernel of 3.

[0149] The specific process of frequency feature dimension alignment and serialization in step 6 is as follows:

[0150] Frequency features are mapped to the embedding dimension through linear mapping and normalization operations:

[0151] ;

[0152] ;

[0153] Step 5.4 Frequency Feature Flattening and Stitching:

[0154] ;

[0155] Step 7: Enhance cross-attention frequency:

[0156] To achieve deep interaction between global semantics and frequency information, this invention introduces a cross-attention fusion module.

[0157] Step 7.1 Query, key, and value construction:

[0158] ;

[0159] ;

[0160] ;

[0161] Step 7.2 Cross-attention computation:

[0162] ;

[0163] Step 7.3 Residual and Nonlinear Enhancement:

[0164] ;

[0165] ;

[0166] MLP stands for fully connected layer, i.e.:

[0167] .

[0168] This embodiment provides a frequency-enhanced view-aware feature learning method for cross-view geolocation. Based on the traditional ground-satellite dual-view framework, it introduces an intermediate view representation based on inverse perspective mapping (IPM) to construct a joint modeling mechanism for three perspectives: ground view, satellite view, and bird's-eye view (BEV). By introducing the bird's-eye view as a transitional perspective, it effectively alleviates the geometric differences between different imaging perspectives and improves the stability of cross-view feature alignment. DINOv2, based on a visual Transformer structure, is used as the feature extraction network to perform unified feature encoding on images from different perspectives. In the feature extraction process, in addition to conventional image patch tokens and global tokens, this invention further introduces an additional learnable global guide token. For different perspectives, independent learnable initialization parameters are set for this global guide token to introduce view-related prior biases, enabling the model to adaptively focus on discriminative image regions under different perspectives during subsequent attention calculations. A multi-layer self-attention mechanism is designed in the feature encoding stage, inputting the global guide token and image patch features together into the self-attention module for joint modeling. Through multi-layer self-attention computation, the global guiding marker can progressively aggregate and update important information from local regions at different levels, thereby forming a global representation with high discriminative power for the current viewpoint image. A frequency feature extraction module is introduced. This module performs frequency domain transformation on input images from different viewpoints, decomposing the image into high-frequency and low-frequency components to characterize the structural contour information and overall semantic information in the image. Subsequently, a lightweight convolutional neural network is used to encode the high-frequency and low-frequency features, and through feature mapping alignment operations, the frequency features are mapped to a feature dimension space consistent with the global guiding marker. After multi-layer self-attention modeling, a cross-attention mechanism is introduced, using the updated global guiding marker as the query term and high-frequency and low-frequency features as keys and values, to achieve selective aggregation of frequency information by global semantic features. Through this cross-attention mechanism, the model can adaptively adjust the degree of attention to high-frequency and low-frequency information according to different viewpoints and scenarios. The updated global guiding marker is fused with the original global features through residual connections, and the fused features are mapped through a multi-layer perceptron to obtain the final cross-viewpoint discriminative feature representation. This feature is used for subsequent similarity calculations and comparative learning training.During the training phase, this invention constructs a contrastive learning mechanism based on three-view input. By jointly constraining the similarity relationship between features from the ground view, bird's-eye view, and satellite view, the same geographical location under different views maintains higher consistency in the feature space, while the features of different geographical locations maintain sufficient distinguishability, thereby effectively improving the overall performance of cross-view geolocation.

[0169] The publicly available datasets CVACT-val and CVACT-test were used as test data. The CVACT dataset is a large-scale, cross-view dataset. This dataset uses buildings in a specific region, including most urban buildings, with ground images sourced from Google Street View panoramas and matching satellite images from Google Maps. To support fine-grained urban geolocation testing, an additional test set containing 92,802 images is also provided, as shown in Table 1. Compared to several existing state-of-the-art methods, the method of this invention achieves the highest accuracy.

[0170] Table 1

[0171]

[0172] Compared with existing technologies, the frequency-enhanced viewpoint-aware feature learning method for cross-view geolocation proposed in this invention has at least the following significant technical advantages:

[0173] Advantage 1: Compared with single-view feature modeling methods, this invention introduces a multi-view collaborative representation mechanism, significantly enhancing cross-view discrimination capabilities. Existing cross-view geolocation methods typically rely solely on dual-view feature matching between ground and satellite views, making it difficult to effectively model geometric distortions and appearance differences caused by viewpoint changes. This invention, based on ground and satellite views, introduces a bird's-eye view representation based on inverse perspective mapping (IPM), constructing a three-view collaborative input system of ground, satellite, and bird's-eye views. By jointly modeling and contrastive learning constraints on the features of the three views, the model can simultaneously perceive view-invariant semantic information and view-related geometric information, effectively alleviating matching difficulties caused by changes in shooting height, viewpoint tilt, and scale, thereby significantly improving the robustness and accuracy of cross-view geolocation.

[0174] Advantage 2: Compared with unified global feature modeling methods, this invention proposes a viewpoint-aware global token initialization mechanism to achieve explicit viewpoint differentiation. Existing Transformer-based methods typically use shared global tokens to uniformly model features from different viewpoints, making it difficult to explicitly distinguish the focus of attention from different viewpoints, resulting in inaccurate attention distribution. This invention introduces a viewpoint-aware global token initialization mechanism, configuring independent viewpoint bias vectors for different viewpoints based on shared learnable global tokens, and uniformly injecting viewpoint prior information during the training and testing phases. This allows the global tokens to adaptively focus on discriminative regions based on viewpoint type during subsequent self-attention and cross-attention calculations. This mechanism can explicitly model viewpoint differences without introducing additional complex structures, enhancing feature alignment capabilities and exhibiting good generalization and engineering practicality.

[0175] Advantage 3: Compared to pure spatial domain feature learning methods, this invention introduces a frequency domain enhancement module to improve structure and texture discrimination capabilities. Existing cross-view geolocation methods mainly rely on spatial domain features for feature matching, which underutilizes the frequency information contained in images. However, road structures, building edges, and feature outlines have more stable and view-insensitive representational characteristics in the frequency domain. This invention performs frequency transformation on images from different viewpoints, dividing the spectral information into high-frequency and low-frequency components, and extracts frequency domain features through a lightweight convolutional network, enabling the model to simultaneously perceive global structural information and local detail information. Through the frequency enhancement mechanism, the insufficient discrimination capability of single spatial domain features in cross-view scenarios is effectively compensated for, thereby improving the stability and accuracy of cross-view feature matching.

[0176] Advantage 4: Compared to simple feature concatenation or weighted fusion methods, this invention employs a cross-attention mechanism to achieve deep interaction between global semantic and frequency information. Existing technologies, when fusing spatial and frequency domain features, often use channel concatenation or linear weighting, lacking the ability to adaptively model the importance of different frequency components. This invention uses a viewpoint-aware global marker as a query vector, guiding it to perform cross-attention computation with high-frequency and low-frequency features, enabling global semantic features to adaptively select the frequency information most discriminative for the localization task. Through residual connections and nonlinear mapping, feature representation capabilities are further enhanced, achieving deep fusion of spatial and frequency domain features without significantly increasing computational overhead.

[0177] Advantage 5: Compared with high-parameter backbone network methods, this invention significantly reduces model complexity while maintaining performance. Some existing cross-view geolocation methods rely on large-scale convolutional or hybrid architecture backbone networks (such as high-parameter convolutional networks). While these can improve feature representation capabilities, they result in a large number of model parameters and high computational resource consumption, making them difficult to deploy in resource-constrained real-world applications. This invention is based on an efficient feature modeling framework. Through shared backbone feature extraction, lightweight frequency branches, and a compact attention interaction module, it maintains or even improves positioning performance while reducing the overall model parameter count to approximately 80% of existing mainstream convolutional methods. This design offers significant advantages in computational efficiency, storage overhead, and inference speed, making it particularly suitable for large-scale map retrieval and real-time positioning applications.

[0178] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof.

[0179] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A frequency-enhanced viewpoint-aware feature learning method for cross-viewpoint geolocation, characterized in that, include: Step 1: Acquire ground view images and satellite view images, and generate intermediate view images based on ground view images to obtain multi-view input data containing ground view, satellite view and intermediate view. Step 2: Input the multi-view input data into the visual feature extraction network with shared parameters to obtain the local image features and global features of each view. Construct a view perception label for each view and perform interactive modeling between the view perception label and the local image features of the corresponding view to update the view perception label. Step 3: Decompose the input images from each viewpoint into frequency domain, extract frequency domain features, and use the updated viewpoint perception markers and frequency domain features to perform cross-modal interaction, integrating frequency information into the viewpoint perception markers to obtain enhanced viewpoint perception features. The step of using updated view-aware markers and frequency domain features for cross-modal interaction includes: Perceive markers from an updated perspective As a query term, use the frequency feature sequence as both key and value: in, , , These represent the query vector, key vector, and value vector, respectively. Represents a frequency characteristic sequence. The layer normalization function is represented. , , The weight matrix is ​​a learnable matrix; Perform cross-attention computation: ； Residual connectivity and nonlinear enhancement are performed to obtain frequency-enhanced viewpoint perception features, wherein the expressions for residual connectivity and nonlinear enhancement are: in, It is a fully connected layer; Step 4: Based on the enhanced viewpoint perception features and global features, generate the final discriminative feature representations for each viewpoint. By calculating the similarity between the discriminative feature representations under different viewpoints, perform geolocation of cross-viewpoint images.

2. The method according to claim 1, characterized in that, The step of generating an intermediate-view image based on a ground-view image includes: The ground view image is processed by inverse perspective mapping to generate a bird's-eye view image as an intermediate view image.

3. The method according to claim 1, characterized in that, The step of constructing view-aware labels for each view includes: Initialize a base tag shared by all viewpoints: in, Indicates a shared base tag. Representing tensor space, Indicates the size of the feature dimension; Define a learnable viewpoint bias vector for each viewpoint: in, Represents the viewpoint bias vector. Indicates the number of viewpoint types; Add the base label to the viewpoint bias vector of the viewpoint to which the current input image belongs to generate the viewpoint-aware label for that viewpoint: in, express Perceptual markers of perspective Indicates a shared base tag. Indicates the viewpoint identifier.

4. The method according to claim 3, characterized in that, The step of interactively modeling the viewpoint-aware markers with the corresponding local image features includes: The viewpoint-aware markers are concatenated with the local image features of the corresponding viewpoints to construct an initial feature sequence: in, Represents the initial feature sequence. This represents the sequence of image patch features from this perspective; The initial feature sequence is input into a multi-layer self-attention network for iterative updates, allowing the viewpoint-aware marker to perform attention calculations with local image features in each layer, aggregating discriminative region information in the image. The expression for the multi-layer self-attention network is: in, For normalization function, , , These represent the query vector, key vector, and value vector, respectively.

5. The method according to claim 4, characterized in that, The step of performing frequency domain decomposition on the input images from each viewpoint and extracting frequency domain features includes: Perform frequency transformation on the input image to convert it to the frequency domain; Low-frequency and high-frequency components can be separated from the frequency spectrum using frequency masks; The low-frequency components and high-frequency components are respectively encoded to obtain low-frequency features and high-frequency features; Low-frequency and high-frequency features are processed into frequency feature sequences consistent with the dimension of the viewpoint-aware label.

6. The method according to claim 5, characterized in that, The frequency transformation employs a Fast Fourier Transform: in, Indicates Fourier transform, express Input data after viewpoint normalization; The frequency mask includes a low-pass mask and a high-pass mask: in, Indicates low-frequency input. For high frequency input, , For frequency mask, Represents the dot product; The feature encoding is processed using a lightweight convolutional network: in, Indicates low-frequency characteristics. Indicates high-frequency characteristics, This represents pointwise convolution. This represents a smooth convolution with a kernel of 3. Indicates the activation function; The encoded low-frequency and high-frequency features are dimensionally aligned, flattened, and concatenated to obtain a frequency feature sequence: in, Represents a frequency characteristic sequence. This represents the concatenation function. This represents the flattening function.

7. The method according to claim 6, characterized in that, The step of generating the final discriminative feature representation for each viewpoint includes: The enhanced view-aware features are fused with global features using a residual connection method to obtain fused features; The fused features are input into a multilayer perceptron for mapping to obtain the final discriminative feature representation.

8. The method according to claim 1, characterized in that, The visual feature extraction network is a pre-trained model based on the visual Transformer structure.

9. The method according to claim 8, characterized in that, The method further includes: During the training phase, a triple contrastive learning loss function based on ground view, intermediate view and satellite view is constructed. By constraining the similarity between features from different viewpoints of the same geographical location and the differences between features from different geographical locations, the model parameters of the pre-trained model based on the visual Transformer structure are optimized.

Images