Adaptive contrast learning method for cross-view matching of uav-satellite images based on three-view cooperation

Abstract

This invention discloses an adaptive contrastive learning method for UAV-satellite image cross-view matching based on three-view collaboration. First, an adaptive weighted contrastive learning model based on three-view collaboration is constructed. Then, through the geometric correction module, the UAV tilted image is converted into a geometrically corrected bird's-eye view image with a unified viewpoint. Next, through the shared-weight visual Transformer module, the feature space of the three views is unified. A mutually exclusive sampling strategy is used to construct training batches. The UAV tilted image to be located and its corrected bird's-eye view image are input into the trained model, and feature matching and collaborative retrieval are performed with a satellite image database. The orthorectified satellite image with the highest matching degree to the UAV tilted view image is output, achieving cross-view geolocation. This method collaboratively utilizes global layout and local detail information to achieve high-precision cross-view geolocation.

Description

Technical Field

[0001] This invention relates to the field of photogrammetry and remote sensing technology, specifically to an adaptive contrastive learning method for cross-view matching of UAV-satellite images based on three-view collaboration. Background Technology

[0002] Cross-view geolocation aims to determine the geographical location of a target by matching images acquired from different platforms and perspectives. It is one of the core technologies in fields such as UAV autonomous navigation, precision agriculture, and disaster emergency response. Among them, matching oblique images captured by low-altitude UAVs with orthophotos acquired by high-altitude satellites is a key approach to achieving wide-area, high-precision positioning.

[0003] However, there is a significant perspective difference between UAV oblique imagery and satellite orthophotos. Satellite imagery is a top-view orthophoto projection, while UAV imagery is a low-altitude oblique perspective projection. There are severe perspective distortions and geometric deformations between the two, resulting in huge differences in the appearance of the same ground feature in different views, making direct feature matching extremely difficult.

[0004] Existing cross-view image matching methods are mainly divided into two categories: The first category is traditional methods based on handcrafted features (such as SIFT and SURF). These methods are effective when the viewpoint changes gradually, but their feature repeatability and robustness drop sharply under extreme viewpoint differences, such as those between drones and satellites. The second category is deep learning-based methods, which typically use convolutional neural networks (CNNs) or visual transformers (ViTs) to extract global or local features and use strategies such as contrastive learning to narrow the feature distance between matching image pairs.

[0005] Despite the progress made by deep learning-based methods, significant limitations remain: First, most methods directly process raw, tilted UAV imagery, failing to explicitly eliminate the geometric distortion between it and satellite imagery. This forces feature extraction networks to expend considerable resources learning this viewpoint change, rather than focusing on the content itself. Second, existing methods are mostly dual-view matching frameworks of "UAV-satellite" or "ground-satellite," failing to fully utilize the corrected bird's-eye view as a bridge to synergistically leverage the advantages of global layout and local detail information. Finally, existing contrastive learning strategies typically employ fixed loss functions and static weight allocations, lacking dynamic adaptability to different view feature quality, varying sample matching difficulty, and loss stability during training. This results in limited generalization performance and localization accuracy in complex scenarios such as varying lighting and terrain undulations.

[0006] Therefore, there is an urgent need for a cross-view geolocation method that can effectively overcome geometric distortion, deeply integrate multi-view information, and improve the robustness and accuracy of matching in complex environments through an adaptive optimization mechanism. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies and propose an adaptive contrastive learning method for UAV-satellite image cross-view matching based on three-view collaboration. By constructing a three-view collaborative framework of UAV oblique image, geometrically corrected bird's-eye view image, and satellite orthophoto image, the view gap is explicitly reduced through geometric correction. Combined with an adaptive weighted contrastive learning mechanism based on feature quality and loss stability, the method collaboratively utilizes global layout and local detail information to achieve high-precision cross-view geolocation.

[0008] To achieve the above objectives, the technical solution specifically adopted by the present invention is as follows: An adaptive contrastive learning method for UAV-satellite image cross-view matching based on three-view collaboration includes the following steps: Step 1: Construct an adaptive weighted contrast learning model for three-view collaboration; the model adopts a visual Transformer structure with three branches sharing weights, integrating a geometric correction module, a unified feature extraction module, and an adaptive weight fusion contrast loss module; Step 2: Obtain the image dataset, which includes UAV oblique images and satellite orthophotos. Through the geometric correction module, based on the UAV's pitch angle, roll angle, camera focal length, and sensor size, the UAV oblique images are converted into geometrically corrected bird's-eye view images with a unified perspective, thereby obtaining a three-view image composed of UAV oblique images, geometrically corrected bird's-eye view images, and satellite orthophotos. Step 3: Using the shared weighted visual Transformer module, extract the depth features from the UAV tilted image, geometrically corrected bird's-eye view image, and satellite orthophoto image respectively to achieve spatial unification of the three-view features; Step 4: Construct training batches using a mutual exclusion sampling strategy, that is: treat all images to be trained as graph nodes, and the matching relationship between images as edges. After sampling each node, remove that node and all its adjacent nodes, repeating until the batch sample quantity requirement is met, so that there are no direct connections between sample nodes in the batch. Step 5: The adaptive weight fusion contrast loss module calculates the matching loss of the UAV-satellite branch and the bird's-eye view-satellite branch based on the adaptive multi-scale weighted contrast loss function, and dynamically allocates the weights of the two branches according to the feature quality and loss stability. After fusing to obtain the total loss, the model is trained end-to-end. Step 6: Input the tilted image of the UAV to be located and its corrected bird's-eye view image into the trained model, perform feature matching and collaborative retrieval with the satellite image database, and output the orthophoto satellite image with the highest matching degree with the UAV tilted view image to achieve cross-view geolocation.

[0009] Preferably, the geometric correction module in step 2 is implemented as follows: Based on the pitch angle, roll angle, camera focal length, and the horizontal and vertical dimensions of the sensor, the pitch angle and roll angle are first converted from degrees to radians; then, the horizontal field of view is calculated based on the sensor's horizontal dimensions and focal length, and the vertical field of view is calculated based on the sensor's vertical dimensions and focal length; next, a roll rotation matrix around the X-axis and a pitch rotation matrix around the Y-axis are constructed, and the two are multiplied to obtain a joint rotation matrix; then, a random sampling consistency algorithm is used to estimate the homography matrix from the image grid point transformation pairs; finally, the perspective transformation function is called to generate the corrected bird's-eye view image based on the homography matrix.

[0010] Preferably, the specific process for extracting depth features in step 3 is as follows: the input UAV images, bird's-eye view images and satellite images are uniformly scaled to a size of 384×384 pixels, and then input into the pre-trained visual Transformer model respectively, and the corresponding depth feature vectors are output.

[0011] Preferably, the adaptive multi-scale weighted contrast loss function in step 5 is calculated as follows: for each branch, two different temperature parameters are introduced to construct two scales; for each scale, the weight coefficient of that scale is first calculated, that is, the exponent of the temperature parameter of that scale is divided by the sum of the exponents of the temperature parameters of the two scales; then the total loss of that scale is calculated, that is, the average of the adaptive losses of all samples in the batch; finally, the total losses of the two scales of that branch are weighted and summed according to the corresponding scale weight coefficients to obtain the matching loss of that branch.

[0012] Preferably, the adaptive loss of the sample is calculated as follows: For each sample in the batch, firstly calculate the basic positive sample loss, hard example mining loss, and label smoothing loss; then calculate the label smoothing coefficient, which is equal to 1 minus 1 divided by 1 plus the sum of the exponential function obtained by multiplying the negative hyperparameter by the matching quality weight; finally, calculate the adaptive loss of the sample as: the difference obtained by subtracting the label smoothing coefficient from 1 multiplied by the sum of the basic positive sample loss of 0.7 and the hard example mining loss of 0.3, plus the label smoothing coefficient multiplied by the label smoothing loss.

[0013] Preferably, the specific calculation methods for the basic positive sample loss, hard example mining loss, and label smoothing loss are as follows: First, calculate the product of the learnable scale parameter and the cosine similarity, then multiply it by the temperature parameter of the current scale to obtain the scaled similarity; The base positive sample loss is equal to: the negative self-scaling similarity plus the logarithm of the sum of the scaling similarity indices of all key features within the batch; The hard example mining loss is equal to: the mean scaled similarity of the negative hard example negative samples plus the logarithm of the sum of the scaled similarity indices of the hard example negative samples; where, a hard example negative sample refers to a predetermined number of negative samples with the highest similarity to the query sample. The label smoothing loss is equal to the mean of all scaled similarities within a negative batch plus the logarithm of the sum of all scaled similarity indices.

[0014] As a preferred embodiment, the specific method for obtaining the total loss in step five is as follows: First, calculate the proportion of drone feature quality, which is the mean standard deviation of the drone feature vector divided by the sum of the mean standard deviations of the drone and the bird's-eye view feature vectors; Then calculate the loss stability index, which is the sum of 1 divided by 1 plus the absolute value of the difference between the UAV-satellite branch loss and the bird's-eye view-satellite branch loss; Next, the adaptive weight of the UAV-satellite branch is calculated. This weight is equal to 0.5 plus 0.1 multiplied by the proportion of UAV feature quality and then multiplied by the loss stability index. Finally, the total loss is calculated as: the adaptive weight multiplied by the UAV-satellite branch loss plus 1 minus the adaptive weight, multiplied by the bird's-eye view-satellite branch loss.

[0015] Preferably, gradient clipping is also used during the model training process in step 5 to prevent gradient explosion. Specifically, the clipped gradient is equal to the smaller of the original gradient multiplied by the ratio of the clipping threshold to the original gradient norm and 1.

[0016] Preferably, the feature matching and collaborative retrieval process in step 6 is as follows: First, for the tilted image of the UAV to be located and its corrected bird's-eye view image, the corresponding depth feature vectors are extracted respectively; Then, the cosine similarity between the UAV feature vector and each satellite feature vector in the satellite image database, as well as the cosine similarity between the bird's-eye view feature vector and each satellite feature vector, are calculated separately. The two cosine similarities of the same satellite image are then added together to obtain the final fusion similarity. Finally, the satellite images are sorted from high to low according to the final fusion similarity, and the top K satellite images are output as candidate positioning results. The positioning performance of the method is evaluated using recall and distance error, where recall is the proportion of test samples with a correct match ranking less than or equal to K, and distance error is the average distance between the top-ranked candidate satellite image and the query sample's geographic coordinates.

[0017] This invention has the following characteristics and beneficial effects: This invention innovatively constructs a three-view collaborative framework consisting of "UAV oblique imagery - geometrically corrected bird's-eye view imagery - satellite orthophoto imagery". The UAV oblique imagery preserves the global layout and contextual information of the scene, while the corrected bird's-eye view imagery maintains the same perspective as the satellite orthophoto imagery, enhancing the alignment of local details. Through feature fusion and collaborative retrieval of the three views, the invention effectively overcomes the shortcomings of incomplete information from a single view, significantly improving matching robustness and localization recall in complex scenes.

[0018] Unlike existing methods that implicitly learn perspective transformations, this invention pioneers an explicit geometric correction module based on UAV attitude parameters (pitch angle, roll angle) and camera intrinsic parameters before feature extraction. This module accurately converts tilted images into orthophoto bird's-eye views through perspective projection transformation, significantly reducing the perspective difference with satellite orthophotos from the source and eliminating the interference of perspective distortion on subsequent feature learning. This allows the shared-weight feature extractor to focus on learning semantic content associations between views, rather than redundant geometric deformations, thereby improving the efficiency and accuracy of feature alignment.

[0019] This invention designs a full-process adaptive weighted contrastive learning mechanism from the sample level to the branch level. Specifically: Sample-level adaptation: The label smoothing coefficient is dynamically adjusted according to the matching quality, which enhances the mining of difficult examples on simple samples and increases smoothing on difficult samples, thereby improving the stability and effectiveness of training.

[0020] Scale-level adaptation: Based on learnable temperature parameters, multi-scale contrast loss is automatically fused to optimize the discriminative power of the feature space.

[0021] Branch-level Adaptation: This innovative approach integrates feature quality metrics (feature standard deviation) with loss stability metrics to dynamically calculate the weights of the UAV-satellite branch and the bird's-eye view-satellite branch in the total loss. This mechanism automatically adjusts the optimization focus based on the current learning state and feature performance of each branch, avoiding suboptimal solutions caused by static weights. This significantly improves the model's generalization ability and final positioning accuracy in complex environments such as lighting changes and ground occlusion (significantly reducing the distance error@1 metric).

[0022] In summary, this invention effectively solves the core technical challenges of severe geometric distortion, low utilization of multi-view information, and rigid optimization strategies in cross-view matching of UAVs and satellite imagery by organically combining "explicit geometric correction + three-view collaboration + full-process adaptive optimization". It has achieved breakthrough progress in both positioning recall and accuracy and has important engineering application value. Attached Figure Description

[0023] Figure 1 This is an overall method flowchart of an embodiment of the present invention; Figure 2 This is the overall framework diagram of the three-view collaborative adaptive contrastive learning model described in this invention; Figure 3 This is a comparative schematic diagram of geometric correction before and after in an embodiment of the present invention; Figure 4 This is a schematic diagram of three-view collaborative retrieval in an embodiment of the present invention. Detailed Implementation

[0024] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0025] An adaptive contrastive learning method for UAV-satellite image cross-view matching based on three-view collaboration, such as... Figure 1 As shown, it includes the following steps: Step 1: Construct an adaptive weighted contrast learning model for three-view collaboration; the model adopts a visual Transformer structure with three branches sharing weights, such as... Figure 2 As shown, the system integrates a geometric correction module, a unified feature extraction module, and an adaptive weight fusion contrast loss module to achieve multi-view features and dynamic optimization.

[0026] Step 2: Obtain an image dataset, which includes UAV oblique images and satellite orthophotos. Using the geometric correction module, based on the UAV's pitch angle, roll angle, camera focal length, and sensor size, convert the UAV oblique images into geometrically corrected bird's-eye view images with a unified perspective, thereby obtaining a three-view image composed of UAV oblique images, geometrically corrected bird's-eye view images, and satellite orthophotos.

[0027] Understandably, oblique images from drones and orthophotos from satellites can be acquired directly.

[0028] Specifically, in this embodiment, the geometric correction module is implemented as follows: Based on the pitch angle, roll angle, camera focal length, and the horizontal and vertical dimensions of the sensor, the pitch angle and roll angle are first converted from degrees to radians; then, the horizontal field of view is calculated based on the sensor's horizontal dimensions and focal length, and the vertical field of view is calculated based on the sensor's vertical dimensions and focal length; next, a roll rotation matrix around the X-axis and a pitch rotation matrix around the Y-axis are constructed, and the two are multiplied to obtain a joint rotation matrix; then, a random sampling consistency algorithm is used to estimate the homography matrix from the image grid point transformation pairs; finally, the perspective transformation function is called to generate the corrected bird's-eye view image based on the homography matrix.

[0029] In this embodiment, based on the UAV attitude parameters (pitch angle) Roll angle Camera focal length and sensor size ( For the sensor's horizontal dimensions, (For the sensor's vertical dimension), the viewpoint conversion is achieved through the following steps: Angle conversion: ， ; in, The pitch angle is in radians. This is the value in radians for the roll angle; Field of view calculation: ， ; in For horizontal field of view, This refers to the vertical field of view. Rotation matrix construction: , Combination matrix ; in For the roll rotation matrix, For pitch and rotation matrices, For the joint rotation matrix; The homography matrix is ​​estimated using the RANSAC algorithm. To achieve perspective projection transformation of pixels; The perspective transformation function is called to generate a corrected bird's-eye view based on the homography matrix. .

[0030] Step 3: Using the shared weighted visual Transformer module, extract the depth features of the UAV tilted image, geometrically corrected bird's-eye view image, and satellite orthophoto image respectively to achieve spatial unification of the three-view features.

[0031] Specifically, the process of extracting depth features is as follows: the input drone images, bird's-eye view images and satellite images are uniformly scaled to a size of 384×384 pixels, and then input into the pre-trained visual Transformer model to output the corresponding depth feature vectors.

[0032] In this embodiment, a pre-trained Vision Transformer model is used as the feature extractor. First, the input UAV imagery is processed... Bird's-eye view Satellite imagery The data is uniformly scaled to 384×384 pixels before being input into the model to extract deep feature embeddings. The feature extraction formula is: , , ; in, , , These are, respectively, oblique imagery from drones, geometrically corrected bird's-eye view imagery, and orthophoto imagery from satellites. The feature vector of the UAV image. The feature vectors of the bird's-eye view image. The feature vector of the satellite image. (.) is the pre-trained visual Transformer feature extraction function (input is an image, output is a depth feature vector).

[0033] Step 4: Construct training batches using a mutual exclusion sampling strategy, that is: treat all images to be trained as graph nodes, and the matching relationship between images as edges. After sampling each node, remove that node and all its adjacent nodes, repeating until the batch sample quantity requirement is met, so that there are no direct connections between sample nodes in the batch.

[0034] Specifically, the implementation process of the mutual exclusion sampling strategy is as follows: First, all images to be trained are treated as nodes in a graph structure. The matching relationships between images (correspondence within the same geographic region) are considered as edges in the graph. ; Secondly, when constructing training batches, a mutually exclusive sampling rule is adopted: after sampling each node (image), the node and all its neighboring nodes (i.e., nodes that are connected to the node by an edge) are removed. Finally, repeat the above sampling process until the batch sample quantity requirement is met, ensuring that all sample nodes in the batch are not directly connected (distance between nodes > 1), so as to avoid related samples interfering with the stability of the contrastive learning.

[0035] Step 5: Based on the adaptive multi-scale weighted contrastive loss function, calculate the matching loss of the UAV-satellite branch and the bird's-eye view-satellite branch respectively, and dynamically allocate the weights of the two branches according to the feature quality and loss stability. After fusing to obtain the total loss, perform end-to-end training on the model.

[0036] Specifically, the adaptive multi-scale weighted contrastive loss function is implemented in a hierarchical calculation and layer-by-layer aggregation manner: First, the sample-level adaptive loss is calculated using a single sample as the basic unit; then, two different temperature parameters are set for each branch to form two independent calculation scales; subsequently, for each scale, the corresponding scale weight coefficient is calculated based on the temperature parameter, and the adaptive loss of all samples at that scale is averaged to obtain the total scale loss; finally, the total losses of the two scales under the same branch are weighted and fused according to the corresponding scale weight coefficients to obtain the final matching loss of that branch.

[0037] First, calculate the sample adaptive loss, which serves as the lowest-level basic unit for the entire loss calculation: First, the scaling similarity is obtained using the learnable scale parameter, cosine similarity, and scale temperature parameter: in, This is a learnable scale parameter used to adjust the similarity distribution; To query feature vectors; Key feature vectors; For the first Temperature parameters at various scales are used to control feature discrimination.

[0038] Based on scaled similarity, the basic positive sample loss, hard example mining loss, and label smoothing loss are calculated separately: in, For the first Scale, First The base positive sample loss of each sample is used to narrow the feature distance of the correct matching pair; For the first The first scale The query features and the first Scaling similarity of key features; This represents the total number of samples within the batch.

[0039] in, For the first Scale, First The hard example mining loss for each sample is used to improve the model's ability to distinguish similar negative samples; The number of negative samples in difficult cases; For the first The set of indexes of difficult negative samples corresponding to each sample; For the first The first scale The query features and the first Scaling similarity of features of difficult negative samples.

[0040] in, For the first Scale, First The label smoothing loss for each sample is used to alleviate model overfitting and improve generalization ability.

[0041] Then, calculate the adaptive label smoothing coefficient based on the sample matching quality: in, For the first The label smoothing coefficient for each sample; Preset scale hyperparameters; For the first The matching quality weights of each sample pair are used to dynamically adjust the smoothing intensity.

[0042] By weighting and fusing the above three types of losses, we obtain the sample adaptive loss: in, For the first Scale, First The final adaptive loss for each sample; , These are fixed weighting coefficients, used to balance the positive sample constraint and the hard sample discrimination constraint, respectively.

[0043] Secondly, the branch matching loss is calculated by aggregating the sample adaptive loss upwards: a dual-scale structure is used for each matching branch. First, the adaptive loss of all samples at the same scale is averaged to obtain the total scale loss. in, For the first Total loss of scale; This refers to the batch sample size. For the above-mentioned number Scale, First The adaptive loss for each sample.

[0044] Calculate the scale-adaptive weights based on dual-scale temperature parameters: in, For the first The weighting coefficients of the scale; For the first The temperature parameters at the scale are consistent with the parameters used for scaling similarity at the underlying level; For scale indexing.

[0045] The branch matching loss for this branch is obtained by summing the two-scale losses according to their corresponding weights: in, The final loss for a single matching branch.

[0046] This method contains two independent matching branches, which yield: Drone-Satellite Branch Loss Bird's-eye view - satellite branch loss Both branches use the same calculation method described above.

[0047] Finally, the losses from the two branches are further fused to obtain the global total loss: First, calculate the feature quality ratio and the loss stability index: in, The percentage of characteristic quality of drones; The standard deviation of the characteristics of the drone is the mean. The mean standard deviation of the aerial view features; Minimum value (default) ), used to avoid the denominator being zero.

[0048] in, This is a loss stability metric used to balance the training progress of the two branches; For drone-satellite branch loss; For bird's-eye view - satellite branch loss.

[0049] Calculate the global adaptive weights based on the above indicators: in, The adaptive weights for the UAV-satellite branch have values ​​ranging from... between; Basic weights; This is the adjustment coefficient.

[0050] The two branch losses are weighted and summed using global adaptive weights to obtain the final total loss of the optimized model: in, This represents the model's global total loss, used for backpropagation to update network parameters.

[0051] Furthermore, in this embodiment, the Adam optimizer, cosine annealing learning rate scheduling, and mixed precision training techniques are used during training to perform end-to-end training on the model, thereby enhancing the cross-view matching capability of the three-view features.

[0052] Step 6: Input the tilted image of the UAV to be located and its corrected bird's-eye view image into the trained model, perform feature matching and collaborative retrieval with the satellite image database GTA-UAV, and output the satellite orthophoto image with the highest matching degree with the UAV tilted view image to achieve cross-view geolocation.

[0053] It should be noted that the GTA-UAV satellite imagery database can be obtained directly.

[0054] First, feature matching: The matching degree between the feature to be located and the features in the satellite database is calculated based on cosine similarity, using the following formula: ; in Features to be located (UAV features) or aerial view features ), Features in satellite image database ( (For database indexes) ( ) is the cosine similarity calculation function; Secondly, similarity fusion: the weighted sum of the similarities of the two-branch matching is used to obtain the final fused similarity, as shown in the formula. ; in For the image to be located and the first The final similarity of the satellite images, For the characteristics of drones and the first Similarity of satellite features For bird's-eye view features and the first Similarity of satellite features; Finally, results retrieval and evaluation: by Sort in descending order and return the Top-K (default K=5) most similar satellite images as candidate results; Evaluation indicators include: Recall rate @K: ;in The total number of test samples, For the first Rank of correct match results for each query This is an indicator function (1 if the condition is met, 0 otherwise); Distance error @1: ;in For the first Geographic coordinates (in meters) of each query sample. The geographic coordinates (in meters) of the top-ranked candidate satellite image. This is a function for calculating geographic coordinate distance (using the Haversine formula).

[0055] Based on the above technical solution, this embodiment uses the GTA-UAV geolocation dataset to present the actual performance of the trained model. Table 1 shows the results of finding satellite remote sensing image tiles that match UAV oblique photogrammetric images.

[0056] Table 1: Performance comparison of different training methods on the GTA-UAV dataset (mutually exclusive sampling) As shown in Table 1, the recall and average precision of the network learning using the method of the present invention have been improved.

[0057] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. An adaptive contrast learning UAV-satellite image cross-view matching method based on three-view cooperation, characterized in that, Includes the following steps: Step 1: Construct an adaptive weighted contrast learning model for three-view collaboration; the adaptive weighted contrast learning model adopts a visual Transformer structure with three branches sharing weights, integrating a geometric correction module, a unified feature extraction module, and an adaptive weight fusion contrast loss module; Step 2: Obtain the image dataset, which includes UAV oblique images and satellite orthophotos. Through the geometric correction module, based on the UAV's pitch angle, roll angle, camera focal length, and sensor size, the UAV oblique images are converted into geometrically corrected bird's-eye view images with a unified perspective, thereby obtaining a three-view image composed of UAV oblique images, geometrically corrected bird's-eye view images, and satellite orthophotos. Step 3: Using the shared weighted visual Transformer module, extract the depth features from the UAV tilted image, geometrically corrected bird's-eye view image, and satellite orthophoto image respectively to achieve spatial unification of the three-view features; Step 4: Construct training batches using a mutual exclusion sampling strategy, that is, treat all images to be trained as graph nodes and the matching relationship between images as edges. After sampling each node, remove that node and all its adjacent nodes, repeating until the batch sample quantity requirement is met, so that there are no direct connections between sample nodes in the batch. Step 5: The adaptive weight fusion contrast loss module calculates the matching loss of the UAV-satellite branch and the bird's-eye view-satellite branch based on the adaptive multi-scale weighted contrast loss function, and dynamically allocates the weights of the two branches according to the feature quality and loss stability. After fusing to obtain the total loss, the model is trained end-to-end. Step 6: Input the tilted image of the UAV to be located and its corrected bird's-eye view image into the trained model, perform feature matching and collaborative retrieval with the satellite image database GTA-UAV, and output the satellite view with the highest matching degree with the UAV tilted view image to achieve cross-view geolocation.

2. The adaptive contrast learning UAV-satellite image cross-view matching method based on three-view cooperation according to claim 1, characterized in that, The specific implementation process of the geometric correction module in step 2 is as follows: Based on the pitch angle, roll angle, camera focal length, and the horizontal and vertical dimensions of the sensor of the UAV, the pitch angle and roll angle are first converted from degrees to radians; then, the horizontal field of view is calculated based on the horizontal dimension and focal length of the sensor, and the vertical field of view is calculated based on the vertical dimension and focal length of the sensor; next, the roll rotation matrix around the X-axis and the pitch rotation matrix around the Y-axis are constructed, and the two are multiplied to obtain the joint rotation matrix; then, the homography matrix is ​​estimated from the image grid point transformation pairs using the random sampling consistency algorithm; finally, the perspective transformation function is called to generate the corrected bird's-eye view image based on the homography matrix.

3. The adaptive contrast learning UAV-satellite image cross-view matching method based on three-view cooperation according to claim 1, characterized in that, The specific process for extracting depth features in step 3 is as follows: the input UAV images, bird's-eye view images and satellite images are uniformly scaled to a size of 384×384 pixels, and then input into the pre-trained visual Transformer model respectively, and the corresponding depth feature vectors are output.

4. The adaptive contrast learning UAV-satellite image cross-view matching method based on three-view cooperation according to claim 1, characterized in that, In step 5, the adaptive multi-scale weighted contrast loss function is calculated as follows: for each branch, two different temperature parameters are introduced to construct two scales; for each scale, the weight coefficient of that scale is first calculated, which is the exponent of the temperature parameter of that scale divided by the sum of the exponents of the temperature parameters of the two scales; then the total loss of that scale is calculated, which is the average of the adaptive loss of all samples in the batch; finally, the total loss of the two scales of that branch is summed according to the corresponding scale weight coefficients to obtain the matching loss of that branch.

5. The adaptive contrastive learning UAV-satellite image cross-view matching method based on three-view collaboration according to claim 4, characterized in that, The calculation process of the sample adaptive loss is as follows: For each sample in the batch, firstly calculate the basic positive sample loss, hard example mining loss, and label smoothing loss; then calculate the label smoothing coefficient, which is equal to 1 minus 1 divided by 1 plus the sum of the exponential function obtained by multiplying the negative hyperparameter by the matching quality weight; finally, calculate the adaptive loss of the sample as: the difference obtained by subtracting the label smoothing coefficient from 1 multiplied by the sum of the basic positive sample loss of 0.7 and the hard example mining loss of 0.3, plus the label smoothing coefficient multiplied by the label smoothing loss.

6. The adaptive contrastive learning UAV-satellite image cross-view matching method based on three-view collaboration according to claim 5, characterized in that, The specific calculation methods for the basic positive sample loss, hard example mining loss, and label smoothing loss are as follows: First, calculate the product of the learnable scale parameter and the cosine similarity, then multiply it by the temperature parameter of the current scale to obtain the scaled similarity; The base positive sample loss is equal to: the negative self-scaling similarity plus the logarithm of the sum of the scaling similarity indices of all key features within the batch; The hard example mining loss is equal to: the mean scaled similarity of the negative hard example negative samples plus the logarithm of the sum of the scaled similarity indices of the hard example negative samples; where, a hard example negative sample refers to a predetermined number of negative samples with the highest similarity to the query sample. The label smoothing loss is equal to the mean of all scaled similarities within a negative batch plus the logarithm of the sum of all scaled similarity indices.

7. The adaptive contrastive learning UAV-satellite image cross-view matching method based on three-view collaboration according to claim 1, characterized in that, The specific method for obtaining the total loss in step five is as follows: First, calculate the proportion of drone feature quality, which is the mean standard deviation of the drone feature vector divided by the sum of the mean standard deviations of the drone and the bird's-eye view feature vectors; Then calculate the loss stability index, which is the sum of 1 divided by 1 plus the absolute value of the difference between the UAV-satellite branch loss and the bird's-eye view-satellite branch loss; Next, the adaptive weight of the UAV-satellite branch is calculated. This weight is equal to 0.5 plus 0.1 multiplied by the proportion of UAV feature quality and then multiplied by the loss stability index. Finally, the total loss is calculated as: the adaptive weight multiplied by the UAV-satellite branch loss plus 1 minus the adaptive weight, multiplied by the bird's-eye view-satellite branch loss.

8. The adaptive contrastive learning UAV-satellite image cross-view matching method based on three-view collaboration according to claim 1, characterized in that, In the model training process of step 5, gradient clipping is also used to prevent gradient explosion. Specifically, the clipped gradient is equal to the smaller of the original gradient multiplied by the ratio of the clipping threshold to the original gradient norm and 1.

9. The adaptive contrastive learning UAV-satellite image cross-view matching method based on three-view collaboration according to claim 1, characterized in that, The specific process of feature matching and collaborative retrieval in step 6 is as follows: First, for the tilted image of the UAV to be located and its corrected bird's-eye view image, the corresponding depth feature vectors are extracted respectively; Then, the cosine similarity between the UAV feature vector and each satellite feature vector in the satellite image database, as well as the cosine similarity between the bird's-eye view feature vector and each satellite feature vector, are calculated separately. The two cosine similarities of the same satellite image are then added together to obtain the final fusion similarity. Finally, the satellite images are sorted from high to low according to the final fusion similarity, and the top K satellite images are output as candidate positioning results. The positioning performance of the method is evaluated using recall and distance error, where recall is the proportion of test samples with a correct match ranking less than or equal to K, and distance error is the average distance between the top-ranked candidate satellite image and the query sample's geographic coordinates.

Images