Cross-view image retrieval method based on xgate-t module
By using the cross-view image retrieval method of the xgate-t module, the problems of feature alignment and similarity measurement between aerial images and satellite images in complex environments are solved, and high-precision UAV positioning and remote sensing image retrieval are achieved.
Patent Information
- Application Number
- CN202511569416.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-30
- Publication Date
- 2026-02-06
AI Technical Summary
Existing technologies struggle to achieve cross-view matching between aerial and satellite images in complex environments. Feature alignment is difficult and similarity measurement is inaccurate, affecting the positioning accuracy and robustness of drones.
We adopt a cross-view image retrieval method based on the xgate-t module. Through feature mapping, cross-modal attention and geometric perception fine-tuning, we achieve feature alignment and dynamic similarity adjustment. Combined with gating fusion and temperature scaling mechanisms, we improve feature consistency and model generalization ability.
It effectively alleviates the difficulty of cross-view feature alignment, improves the positioning accuracy of UAVs and the robustness of the model in complex environments, and is suitable for remote sensing image retrieval tasks.
Smart Images
Figure CN121479005A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of remote sensing image processing technology, specifically a method for retrieving satellite tile images based on aerial images. Background Technology
[0002] The autonomous navigation of unmanned aerial vehicles (UAVs) relies on Global Navigation Satellite Systems (GNSS) and Inertial Navigation Systems (INS). However, GNSS is susceptible to signal interference and may fail, while INS suffers from cumulative errors that cannot be eliminated. Neither of these systems can meet the high-precision positioning requirements in complex environments.
[0003] Therefore, cross-viewpoint matching based on aerial and satellite images has become a key autonomous localization technology. However, existing methods still face two major technical bottlenecks when dealing with the huge differences in viewpoint, illumination, and scale between the two types of images: first, it is difficult to learn stable cross-viewpoint invariant features, leading to difficulties in feature alignment; second, similarity measurement methods use fixed parameters and cannot adaptively adjust according to the difficulty of sample matching, limiting the optimization space and generalization ability of the model. Therefore, there is an urgent need in this field for a new technology that can simultaneously solve the problems of feature alignment and similarity measurement to improve the accuracy and robustness of UAV visual localization. Summary of the Invention
[0004] (a) Technical problems to be solved
[0005] To address the shortcomings of existing technologies, this invention provides a cross-view image retrieval method based on the xgate-t module, which solves the problems mentioned in the background section.
[0006] (II) Technical Solution
[0007] This invention provides a method for cross-view remote sensing image retrieval, which can solve the problems of difficult feature alignment and inaccurate similarity measurement, while achieving high-precision retrieval.
[0008] The cross-view image retrieval method of the present invention includes the following steps:
[0009] Step S1: Feature extraction: Acquire satellite images of the target flight area, and perform grid-based segmentation on the satellite images to obtain several satellite tile images; input the aerial images acquired by the UAV and the satellite tile images into the backbone network to extract surface features.
[0010] Step S2: Feature Mapping: The surface features are uniformly mapped to a 256-dimensional vector space through linear mapping, and the mapped features are batch normalized to stabilize the feature distribution.
[0011] Step S3: Cross-modal attention: Project the mapped features to obtain query vector Q, key vector K, and value vector V; perform attention scoring on the query vector Q and key vector K, and introduce a globally learnable temperature parameter; simultaneously, input the concatenated features into a lightweight multilayer perceptron to generate learnable gating parameters and bias terms, output gated fused features, and update aerial and satellite features in the dual dimension respectively.
[0012] Step S4: Geometrically Aware Fine-Tuning: Based on the enhanced features, predict the six-dimensional affine transformation parameter theta, and decompose the affine transformation parameter into a scaling factor and an offset factor. Fine-tune the original features by limiting the magnitude to 0.1 through the scaling factor and the offset factor to achieve geometrically aligned feature output.
[0013] Step S5: Similarity Calculation: Perform cosine similarity calculation on the geometrically aligned features to obtain the target satellite tile image with the highest similarity to the real-time aerial image. Determine the UAV's location information based on the target satellite tile image to complete UAV localization. Further, scale the satellite image according to the aerial image resolution and then cut it into satellite tile images of the same scale as the aerial image, maintaining a 60% overlap during cutting.
[0014] Furthermore, the surface features are uniformly mapped to a 256-dimensional vector space through linear mapping, and then each feature dimension within a batch is normalized individually, as shown in the formula:
[0015]
[0016] Where: u B and These are the mean and variance of the current batch; γ and β are learnable scaling and offset parameters (initially 1 and 0) to improve the stability of feature representation.
[0017] Furthermore, the aerial photographic features are denoted as f. a The satellite's characteristic is f s First, stack the dual dimensions and add position encoding:
[0018] f c = stack([f a ,f s ])+P
[0019] Where P is the position code. Then, for f... c Performing a linear projection yields the query, key, and value:
[0020] Q = f c W Q K = f c W K V=fc W V
[0021] Furthermore, the attention scoring formula after incorporating globally learnable temperature parameters is as follows:
[0022]
[0023] Among them: Q h and K h Q and K are rearranged according to the number of heads H; τ is a temperature parameter. When 0 < τ < 1, the attention distribution is more concentrated, which helps the model focus on areas that are difficult to match. When τ > 1, the attention is smoother, which helps to stabilize training.
[0024] Furthermore, the spliced features are input into the MLP to obtain the gate g and the bias b, which are expressed as:
[0025] [gate_raw,bias_raw]=MLP(concat([f a ,f s ]))
[0026] g=σ(gate_raw), b=tanh(bias_raw)
[0027] Where: f a Indicates the features of an aerial photograph, f s Indicates the features of aerial photographs. This represents the sigmoid activation function. The hyperbolic tangent function is a non-linear activation function that introduces non-linear expressive power into neural networks.
[0028] Furthermore, to balance robustness and controllability, the output employs gated fusion, which can be expressed as:
[0029] O=(0.5+0.5g)⊙Y+(0.5-0.5g)⊙f c
[0030] Furthermore, the predicted six-dimensional affine transformation parameter θ is expressed as:
[0031] θ=[s,t_x,t_y,r,sh_x,sh_y]
[0032] This parameter is input into the film layer to extract γ and β. These parameters are then used to fine-tune the aerial and satellite image features to enhance cross-modal alignment consistency. The fine-tuning formula is as follows:
[0033] f″ a =f a ·(1+0.1(tanh(γ)))+0.1(tanh(β))
[0034] f″ s =f s ·(1+0.1(tanh(γ)))-0.1(tanh(β))
[0035] Furthermore, the processed features are projected, and the satellite tile image with the highest similarity is obtained through similarity calculation. The formula is as follows:
[0036] Sim = f′ a ·f s ' T
[0037] (III) Beneficial Effects
[0038] Compared with existing technologies, this invention provides a coupled gating and temperature-based image retrieval algorithm, which has the following advantages:
[0039] This invention introduces a coupled gating and temperature scaling mechanism to achieve adaptive fusion and dynamic similarity adjustment of cross-view features, effectively alleviating the difficulty of feature alignment between aerial and satellite images under different conditions such as viewpoint, scale, and lighting. Combined with geometric perception fine-tuning and amplitude limiting strategies, it further improves the spatial consistency and robustness of cross-modal features. The overall method enhances the model's generalization ability while ensuring positioning accuracy, making it suitable for UAV visual positioning and remote sensing retrieval tasks in complex environments. Attached Figure Description
[0040] Figure 1 This is a flowchart of a cross-view image retrieval method according to the present invention;
[0041] Figure 2 This is a schematic diagram of the overall algorithm of the present invention;
[0042] Figure 3 This is the flowchart for the cross-modal attention module; Detailed Implementation
[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0044] Example
[0045] like Figure 1 , 2 As shown in Figure 3, an embodiment of the present invention proposes a cross-view image retrieval method based on the xgate-t module, which includes the following steps.
[0046] Step S1: Feature Extraction: Acquire satellite images of the target flight area, and perform grid-based segmentation on the satellite images to obtain several satellite tile images; input the aerial images acquired by the UAV and the satellite tile images into the backbone network to extract surface features. Specifically, let R be the resolution of the aerial images. a The resolution of the satellite image is R. s The satellite image zoom level is:
[0047]
[0048] Enlarge the original satellite image by this factor:
[0049] W s '=W s ·k,H s '=H s ·k
[0050] Then, the same dimensions (W) as the aerial photograph were adopted. tile =W a H tile =H a The scaled satellite image is then segmented. To ensure the continuity of spatial features between adjacent sub-images, the segmentation process sets an overlap ratio α = 0.6, with a step size of:
[0051] Δ x =W a (1-α),Δ y =H a (1-α)
[0052] Step S2: Feature Mapping: The surface features are uniformly mapped to a 256-dimensional vector space using linear mapping, and batch normalization is performed on the mapped features to stabilize the feature distribution. Specifically, each feature dimension within a batch is normalized individually, using the following formula:
[0053]
[0054] Where: u B and These are the mean and variance of the current batch; γ and β are learnable scaling and offset parameters (initially 1 and 0) to improve the stability of feature representation.
[0055] Step S3: Cross-modal attention: Project the mapped features to obtain query vector Q, key vector K, and value vector V; perform attention scoring on the query vector Q and key vector K, and introduce a globally learnable temperature parameter; simultaneously, input the concatenated features into a lightweight multilayer perceptron to generate learnable gating parameters and bias terms, output gated fused features, and update aerial and satellite features in the dual dimension respectively.
[0056] Denote the aerial photographic features as f a The satellite's characteristic is f s First, stack the dual dimensions and add position encoding:
[0057] f c = stack([f a ,f s ])+P
[0058] Where P is the position code. Then, for f... c Performing a linear projection yields the query, key, and value:
[0059] Q = f c W Q K = f c W K V=f c W V
[0060] Furthermore, the attention scoring formula after incorporating globally learnable temperature parameters is as follows:
[0061]
[0062] Among them: Q h and K h Q and K are rearranged according to the number of heads H; τ is a temperature parameter. When 0 < τ < 1, the attention distribution is more concentrated, which helps the model focus on areas that are difficult to match. When τ > 1, the attention is smoother, which helps to stabilize training.
[0063] Simultaneously, the concatenated features are input into the MLP to obtain the gate g and the bias b, represented as:
[0064] [gate_raw,bias_raw]=MLP(concat([f a ,f s ]))
[0065] g=σ(gate_raw), b=tanh(bias_raw)
[0066] Where: f a Indicates the features of an aerial photograph, f s Indicates the features of aerial photographs. This represents the sigmoid activation function. The hyperbolic tangent function is a non-linear activation function that introduces non-linear expressive power into neural networks.
[0067] Adding the bias to logits and normalizing it yields a new attention distribution:
[0068] Z'=Z+b
[0069] A = softmax(Z')
[0070] Weighted aggregation and output projection of values within multiple heads:
[0071] U = AV h
[0072] Output projection matrix, through Projecting back to the original feature dimension
[0073] Y = UW o
[0074] To balance robustness and controllability, the output employs gated fusion, which can be represented as:
[0075] O=(0.5+0.5g)⊙Y+(0.5-0.5g)⊙f c
[0076] The updates are split into two paths on the dual dimension: aerial photography and satellite photography.
[0077] f a '=O[:,0,:],f s =O[:,1,:]
[0078] In this module, the parameters (W1, W2, b1, b2) of the MLP are learned through data-driven learning of the gating factor g and the attention bias b, thereby suppressing cross-view background interference and highlighting the discrimination region during feature interaction. A single temperature parameter τ specifically acts on the logits from aerial imagery to satellite imagery, adjusting the sharpness of the attention distribution and avoiding overfitting to easy samples or under-optimization for difficult samples. The two work together to make cross-view feature alignment more stable and accurate.
[0079] Step S4: Geometrically Aware Fine-Tuning: Based on the enhanced features, predict the six-dimensional affine transformation parameters θ, and decompose the affine transformation parameters into scaling factors and offset factors. Fine-tune the original features by limiting the magnitude to 0.1 through the scaling factors and offset factors to achieve geometrically aligned feature output.
[0080] The concatenated features are input into theta_pred network, which consists of a first linear transformation layer, a ReLU activation function, and a second linear transformation layer. This network predicts the six-dimensional affine transformation parameters θ, as follows:
[0081] θ=[s,t_x,t_y,r,sh_x,sh_y]
[0082] This parameter is input into the film layer to extract γ and β. These parameters are then used to fine-tune the aerial and satellite image features to enhance cross-modal alignment consistency. The fine-tuning formula is as follows:
[0083] f a =f a ·(1+0.1(tanh(γ)))+0.1(tanh(β))
[0084] f s =f s ·(1+0.1(tanh(γ)))-0.1(tanh(β))
[0085] Step S5: Similarity Calculation: Perform cosine similarity calculation on the geometrically aligned features to obtain the target satellite tile image with the highest similarity to the real-time aerial image, and determine the UAV's location information based on the target satellite tile image to complete the UAV positioning.
[0086] The processed features are then projected onto a linear layer, and the satellite tile image with the highest similarity is calculated using cosine similarity. The formula is as follows:
[0087] Sim=f a '·f s 'T
[0088] This allows for the precise selection of the corresponding satellite tile image from real-time aerial photographs, and then the acquisition of the actual location information based on the satellite tile image, thus enabling the drone's positioning.
[0089] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A cross-view image retrieval method, characterized in that, Includes the following steps: Step S1: Feature extraction: Acquire satellite images of the target flight area, and perform grid-based segmentation on the satellite images to obtain several satellite tile images; input the aerial images acquired by the UAV and the satellite tile images into the backbone network to extract surface features. Step S2 Feature mapping: The surface features are uniformly mapped to a 256-dimensional vector space through linear mapping, and the mapped features are batch normalized to stabilize the feature distribution. Step S3: Cross-modal attention: Project the mapped features to obtain query vector Q, key vector K, and value vector V; perform attention scoring on the query vector Q and key vector K, and introduce a globally learnable temperature parameter; simultaneously, input the concatenated features into a lightweight multilayer perceptron to generate learnable gating parameters and bias terms, output gated fused features, and update aerial and satellite features in the dual dimension respectively. Step S4: Geometrically Aware Fine-Tuning: Based on the enhanced features, predict the six-dimensional affine transformation parameters, and decompose the affine transformation parameters into scaling factors and offset factors. Fine-tune the original features by limiting the magnitude to 0.1 times using the scaling factors and offset factors to achieve geometrically aligned feature output. Step S5: Similarity Calculation: Perform cosine similarity calculation on the geometrically aligned features to obtain the target satellite tile image with the highest similarity to the real-time aerial image, and determine the UAV's location information based on the target satellite tile image to complete the UAV positioning.
2. The cross-view image retrieval system according to claim 1, characterized in that, In step S1, the satellite image is scaled up according to the resolution of the aerial image and then cut into satellite tile images of the same scale as the aerial image, while maintaining a 60% overlap rate during cutting.
3. The cross-view image retrieval system according to claim 1, characterized in that, In step S2, the surface features are uniformly mapped to a 256-dimensional vector space through linear mapping, and then each feature dimension within the batch is normalized individually, using the following formula: Where: u B and These are the mean and variance of the current batch; γ and β are learnable scaling and offset parameters (initially 1 and 0) to improve the stability of feature representation.
4. The cross-view image retrieval system according to claim 1, characterized in that, In step S3, the aerial photographic feature is denoted as f. a The satellite's characteristic is f s First, stack the dual dimensions and add position encoding: f c =stack([f a ,f s ])+P Where P is the position code. Then, for f... c Performing a linear projection yields the query, key, and value: Q=f c W Q ,K=f c W K ,V=f c W V 5. The cross-view image retrieval system according to claim 1, characterized in that, In step S3, the attention scoring formula after incorporating the globally learnable temperature parameter is as follows: Among them: Q h and K h Q and K are rearranged according to the number of heads H; τ is a temperature parameter. When 0 < τ < 1, the attention distribution is more concentrated, which helps the model focus on areas that are difficult to match. When τ > 1, the attention is smoother, which helps stabilize training.
6. The cross-view image retrieval system according to claim 1, characterized in that, In step S3, the concatenated features are input into the MLP to obtain the gate g and the bias b, which are expressed as: [gate_raw,bias_raw]=MLP(concat([f a ,f s ])) g=σ(gate_raw), b=tanh(bias_raw) Where: f a Indicates the features of an aerial photograph, f s Indicates the features of aerial photographs. This represents the sigmoid activation function. The hyperbolic tangent function is a non-linear activation function that introduces non-linear expressive power into neural networks.
7. The cross-view image retrieval system according to claim 1, characterized in that, In step S3, to balance robustness and controllability, the output employs gated fusion, which can be expressed as: O=(0.5+0.5g)⊙Y+(0.5-0.5g)⊙f c 8. The cross-view image retrieval system according to claim 1, characterized in that, In step S4, the predicted six-dimensional affine transformation parameter θ is expressed as: θ=[s,t_x,t_y,r,sh_x,sh_y] This parameter is input into the film layer to extract γ and β. These parameters are then used to fine-tune the aerial and satellite image features to enhance cross-modal alignment consistency. The fine-tuning formula is as follows: f a ”=f a ·(1+0.1(tanh(γ)))+0.1(tanh(β)) f s ”=f s ·(1+0.1(tanh(γ)))-0.1(tanh(β)) 9. A cross-view image retrieval system according to claim 1, characterized in that, In step S4, the processed features are first projected, and then the satellite tile image with the highest similarity is obtained through similarity calculation. The formula is as follows: Sim=f′ a ·f s 'T 。