Multi-scale cross-view geographic position positioning method, system, equipment and medium

Abstract

The invention relates to a multi-scale cross-view geographic position positioning method, system and device and a medium. The method comprises the following steps: acquiring a cross-view remote sensing image to be processed and a street view image at the same place; constructing a multi-scale cross-view geographic position positioning model, wherein the model is composed of two feature extraction branches and an output network, wherein the two feature extraction branches comprise feature extractors of the same structure; the feature extractor is used for extracting multi-scale features of an input image by adopting a multi-stage model based on a Swindow-transformer; the output network is used for determining a geographic position positioning result according to the multi-scale features output by the two feature extraction branches; and training the model by adopting a comparative learning mode according to a pre-prepared image sample, and inputting the remote sensing image to be processed and the streetscape image into the trained multi-scale cross-view geographic position positioning model to obtain a geographic position positioning result. The method can improve the positioning precision of the geographic position, and greatly reduces the calculation cost.

Description

Technical Field

[0001] The present invention belongs to the technical field of position positioning, and relates to a multi-scale cross-viewing angle geographic location positioning method, system, device and medium. Background Art

[0002] Cross-view geo-localization is a hot research topic in the field of computer vision. It aims to locate street view images by retrieving the most similar images to the street view images to be retrieved in the remote sensing image dataset with GPS tags. This task can be regarded as an image retrieval task. With the development of deep learning technology, multiple cross-view geo-localization methods have been proposed, but cross-view geo-localization based on deep learning still has difficulties and challenges.

[0003] On the one hand, street view images and remote sensing images have huge differences in perspective and field of view. Street view images are panoramic images centered on the shooting location, while remote sensing images are orthographic and have a wider field of view, such as Figure 1 As shown in the figure, (a) is a street view image and (b) is a remote sensing image. This results in the same object having different features and positions in street view images and remote sensing images, so how to effectively align these features becomes a challenging problem. On the other hand, street view images and remote sensing images may be taken at different times, places and seasons. This results in visual differences and environmental changes between them, increasing the difficulty of the cross-view geolocation task. The model needs to be able to adapt to these differences to ensure that the geographic location of street view images is accurately determined under different conditions, which involves solving the problem of inconsistency in visual features caused by time, weather and seasonal changes. In addition, the purpose of cross-view geolocation is to locate street view images globally, which needs to be applied and tested on large-scale datasets. Therefore, it is also important to reduce the computational cost and retrieval time associated with this process. Summary of the invention

[0004] Based on this, it is necessary to address the above-mentioned technical problems. The present invention proposes a multi-scale cross-view geolocation positioning method, a multi-scale cross-view geolocation positioning system, a computer device and a computer-readable storage medium, which can greatly improve the cross-view geolocation positioning performance.

[0005] In order to achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0006] In one aspect, a multi-scale cross-view geolocation method is provided, the method comprising:

[0007] Obtain remote sensing images and street view images to be processed at the same location across perspectives.

[0008] A multi-scale cross-view geolocation positioning model is constructed, which includes feature extraction branches and output networks for remote sensing images and street view images; the two feature extraction branches include feature extractors with the same structure; the feature extractor is used to extract multi-scale features of the input image using a multi-stage model based on Swin-transformer; the output network is used to determine the geolocation positioning result based on the multi-scale features output by the two feature extraction branches.

[0009] The multi-scale cross-view geolocation positioning model is trained by contrastive learning according to the pre-prepared image samples to obtain a trained multi-scale cross-view geolocation positioning model;

[0010] The remote sensing images and street view images to be processed are input into the trained multi-scale cross-view geolocation positioning model to obtain the geolocation positioning results.

[0011] In one of the embodiments, the image samples include multiple groups of remote sensing images and street view images across perspectives; during the training process, a feature extractor that shares weights with the feature extractor is added to the feature extraction branch of the remote sensing image, a strong enhancement module and a weak enhancement module are respectively added to the two feature extractors, and a mapping head is added after each of the two feature extractors.

[0012] The multi-scale cross-view geolocation model in the testing process is obtained by removing the added strong enhancement, weak enhancement modules, feature extractor and two mapping heads from the model in the training process.

[0013] The multi-scale cross-view geolocation positioning model is trained by contrastive learning according to the pre-prepared image samples to obtain a trained multi-scale cross-view geolocation positioning model, including:

[0014] The pre-prepared image samples are divided into training samples and test samples.

[0015] The remote sensing images in the training samples are processed by the strong enhancement module and the weak enhancement module respectively to obtain the strongly enhanced image and the weakly enhanced image.

[0016] The strongly enhanced image and the weakly enhanced image are respectively input into two feature extractors of the feature extraction branch of the remote sensing image to obtain the multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image.

[0017] The multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image are respectively input into two mapping heads with shared parameters to obtain strongly enhanced mapping features and weakly enhanced mapping features.

[0018] The street view images in the training samples are input into the feature extractor of the feature extraction branch of the street view images to obtain multi-scale features of the street view images.

[0019] Construct a total loss function; the total loss function includes two retrieval losses and one contrast loss.

[0020] Determine the total loss according to the strong enhancement mapping features, the weak enhancement mapping features, the multi-scale features of the street view image, and the total loss function;

[0021] The multi-scale cross-view geolocation positioning model is directionally trained according to the total loss to obtain a trained multi-scale cross-view geolocation positioning model.

[0022] The multi-scale cross-view geolocation positioning model input into the test process is tested using the test samples to obtain a trained multi-scale cross-view geolocation positioning model.

[0023] In one embodiment, the feature extractor includes a 4-stage module, two maximum pooling layers, and two multi-layer perceptrons, wherein each stage module includes several Swin Transformer modules.

[0024] The strongly enhanced image and the weakly enhanced image are respectively input into two feature extractors of the feature extraction branch of the remote sensing image to obtain multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image, including:

[0025] The strongly enhanced image is input into the 4-stage module in the first feature extractor of the feature extraction branch of the remote sensing image to obtain 4 features with different dimensions.

[0026] The features of the third dimension are processed successively by the first maximum pooling layer and the first multi-layer perceptron and subjected to a matrix inner product operation with the features of the third dimension to obtain the first inner product features.

[0027] The features of the fourth dimension are processed successively by the second maximum pooling layer and the second multi-layer perceptron and a matrix inner product operation is performed with the features of the fourth dimension to obtain the second inner product features.

[0028] The first inner product feature and the second inner product feature are concatenated to obtain the multi-scale features of the strongly enhanced remote sensing image.

[0029] The weakly enhanced image is input into the second feature extractor of the feature extraction branch of the remote sensing image to obtain the multi-scale features of the weakly enhanced remote sensing image.

[0030] In one embodiment, the multilayer perceptron includes: two fully connected layers.

[0031] In one embodiment, the mapping head includes: a first linearization layer, a RELU activation function, a batch normalization layer, and a second linearization layer.

[0032] The multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image are respectively input into two mapping heads with shared parameters to obtain the strongly enhanced mapping features and the weakly enhanced mapping features, including:

[0033] The multi-scale features of the strongly enhanced remote sensing image are input into the first mapping head, and are processed by the first linearization layer, RELU activation function, batch normalization layer and the second linearization layer in sequence to obtain the strongly enhanced mapping features.

[0034] The multi-scale features of the strongly enhanced remote sensing image are input into the second mapping head to obtain the weakly enhanced mapping features.

[0035] In one embodiment, constructing a total loss function includes:

[0036] According to the strong enhancement mapping features and weak enhancement mapping features obtained from the strong enhancement and weak enhancement pictures of the same remote sensing image, the contrast loss function is constructed as follows:

[0037]

[0038] in, is the contrast loss, d pos-con With d neg-con They represent the Euclidean distance between the first matching pair and the first unmatched pair, respectively, where the first matching pair is the strongly enhanced and weakly enhanced images from the same remote sensing image, and the first unmatched pair is the strongly enhanced and weakly enhanced images from different remote sensing images.

[0039] According to the strong enhancement mapping features and weak enhancement mapping features, as well as the multi-scale features of street view images, two loss functions are constructed as follows:

[0040]

[0041]

[0042] in, is the first retrieval loss, is the second retrieval loss, d pos-ret1 With d neg-ret1 Represent the Euclidean distance between the second matching pair and the second unmatching pair, respectively, where the second matching pair is a street view image from the same location and a remote sensing image after strong enhancement, and the unmatching pair is a street view image from different locations and a remote sensing image after strong enhancement; d pos-ret2 With d neg-ret2They represent the Euclidean distance between the third matching pair and the third unmatched pair, respectively. The third matching pair is a street view image from the same location and a weakly enhanced remote sensing image, and the unmatched pair is a street view image from different locations and a weakly enhanced remote sensing image.

[0043] According to the contrast loss function and two retrieval functions, the total loss function is constructed as:

[0044]

[0045] in, is the total loss.

[0046] In one embodiment, the remote sensing image to be processed and the street view image are input into a trained multi-scale cross-view geolocation positioning model to obtain a geolocation positioning result, including:

[0047] The remote sensing image and street view image to be processed are respectively input into the feature extractors of the feature extraction branches of the remote sensing image and the street view image of the trained multi-scale cross-view geolocation positioning model to obtain multi-scale features of the remote sensing image and the street view image.

[0048] The cosine similarity between the multi-scale features of the street view image and the multi-scale features of the remote sensing image is calculated, and the remote sensing images are sorted according to the size of the cosine similarity to obtain the geographic location positioning result.

[0049] On the other hand, a multi-scale cross-viewing geographical location positioning device is also provided, the device comprising:

[0050] The multi-view image acquisition module is used to acquire the remote sensing images and street view images to be processed at the same location across view angles.

[0051] A multi-scale cross-view geolocation model construction module is used to construct a multi-scale cross-view geolocation model. The multi-scale cross-view geolocation model includes feature extraction branches and output networks for remote sensing images and street view images; the two feature extraction branches include feature extractors with the same structure; the feature extractor is used to extract multi-scale features of the input image using a multi-stage model based on Swin-transformer; the output network is used to determine the geolocation result based on the multi-scale features output by the two feature extraction branches.

[0052] The multi-scale cross-view geolocation positioning model training module is used to train the multi-scale cross-view geolocation positioning model by contrastive learning based on pre-prepared image samples to obtain a trained multi-scale cross-view geolocation positioning model.

[0053] The multi-scale cross-view geolocation module is used to input the remote sensing image and street view image to be processed into the trained multi-scale cross-view geolocation model to obtain the geolocation results of the remote sensing image and street view image to be processed.

[0054] On the other hand, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned multi-scale cross-view geolocation positioning method when executing the computer program.

[0055] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned multi-scale cross-viewing geographic location positioning method are implemented.

[0056] The above-mentioned multi-scale cross-viewpoint geographic location positioning method, system, device and medium, the method includes obtaining a remote sensing image and a street view image to be processed at the same place across perspectives; constructing a multi-scale cross-viewpoint geographic location positioning model, the multi-scale cross-viewpoint geographic location positioning model includes feature extraction branches and output networks for remote sensing images and street view images; the two feature extraction branches include feature extractors with the same structure; the feature extractor is used to extract multi-scale features of the input image using a multi-stage model based on Swin-transformer; the output network is used to determine the geographic location positioning result according to the multi-scale features output by the two feature extraction branches; the multi-scale cross-viewpoint geographic location positioning model is trained by contrastive learning according to the pre-prepared image samples to obtain a trained multi-scale cross-viewpoint geographic location positioning model; the remote sensing image and the street view image to be processed are input into the trained multi-scale cross-viewpoint geographic location positioning model to obtain the geographic location positioning result. This method can improve the accuracy of geographic location positioning and greatly reduce the calculation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The difference in viewing angle and field of view between street view image and remote sensing image, where (a) is street view image and (b) is remote sensing image;

[0058] Figure 2 Schematic diagram of a process of a multi-scale cross-viewing geographical location positioning method in one embodiment;

[0059] Figure 3 A structural diagram of a multi-scale and cross-viewing geographic location positioning model in one embodiment;

[0060] Figure 4 A multi-scale cross-view geolocation model framework during training in another embodiment;

[0061] Figure 5 is a structural block diagram of a multi-scale cross-viewing geographical location positioning device in one embodiment;

[0062] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] In one embodiment, Figure 2 As shown, a multi-scale cross-viewing geographic location positioning method is provided, and the method comprises the following steps:

[0065] Step 100: Obtain a remote sensing image to be processed and a street view image of the same location across viewing angles.

[0066] Step 102: construct a multi-scale cross-view geolocation positioning model, which includes feature extraction branches and an output network for remote sensing images and street view images; the two feature extraction branches include feature extractors with the same structure; the feature extractor is used to extract multi-scale features of the input image using a multi-stage model based on Swin-transformer; the output network is used to determine the geolocation positioning result based on the multi-scale features output by the two feature extraction branches.

[0067] Specifically, in order to solve the problem of feature alignment caused by perspective and field of view, how to achieve feature alignment between street view images and remote sensing images has become a hot topic in the field of cross-perspective geolocation positioning. The current mainstream methods include CNN (Convolutional Neural Network) based methods, GAN (Generative Adversarial Network) based methods and Tranformer based methods. CNN-based methods rely on polar coordinate transformation (Polar Transform), which applies polar transformation to remote sensing images, transforming remote sensing images into images with similar geometric distribution to street view images. Polar transformation is a mathematical transformation, not a learning parameter, and requires the alignment of the orientation and center points of street view images and remote sensing images, which lacks generalization. When the alignment relationship is not satisfied, the method may fail. GAN can generate realistic images. GAN-based methods take advantage of this advantage to generate realistic street view images from remote sensing images that have undergone polar coordinate transformation, and use the generated street view images for retrieval. However, GAN-based methods also rely on the alignment of the orientation and center points of street view images and remote sensing images. At the same time, GAN-based models require huge computational losses and face the problem of model collapse. CNN-based and GAN-based models rely on the alignment of street view images and remote sensing images, making it difficult to learn the geometric position relationship between the two. In order to solve this problem, a Transformer-based method was proposed. The Transformer-based method uses position encoding technology to learn the position relationship between street view images and remote sensing images, and uses multi-head attention (Multi-head Self-attention) technology is used to extract important features from images, and feature alignment between street view images and remote sensing images is achieved. The method does not rely on the alignment relationship between the two types of images. Although the existing Transformer-based methods can learn the geometric position relationship between remote sensing images and street view images to a certain extent, they still have some problems. On the one hand, the current Transformer-based methods adopt a single-stage downsampling strategy to generate a single-resolution feature map, which may lead to information loss. On the other hand, the current Transformer-based methods have not yet considered the influence of changing factors such as time, weather, and season. Finally, the existing Transformer-based methods perform multi-head attention mechanism calculations on a global scale, which has large computational losses and long retrieval times.

[0068] A multi-stage model based on Swin-transformer is used as the backbone network to extract multi-scale features. On the one hand, the network can generate feature maps of different resolutions in multiple stages. Based on the feature maps, the network can extract multi-scale features. This feature can contain more detailed features. On the other hand, the network adopts the window-based attention mechanism and the shifted window-based attention mechanism to perform multi-head attention calculations locally, reducing computational loss and retrieval time. The structure of the multi-scale cross-view geolocation positioning model is shown in the figure. Figure 3 shown.

[0069] This method generates feature maps of different resolutions through a multi-stage feature extractor based on Swin-transformer to extract multi-scale features. Multi-scale features contain more detailed information, which enhances the information extraction ability of this method. On the other hand, this method only performs multi-head attention calculations locally, which greatly reduces the computational cost and improves the usability of this method.

[0070] Step 104: training the multi-scale cross-view geographical location positioning model by contrastive learning according to the pre-prepared image samples to obtain a trained multi-scale cross-view geographical location positioning model.

[0071] Specifically, the contrastive learning method refers to performing strong enhancement and weak enhancement on the input remote sensing image respectively in the feature extraction branch of the remote sensing image, and then inputting them into two feature extractors with shared weights respectively, and then the two mapping heads with shared parameters obtain two multi-scale mapping features of the remote sensing images, and inputting the input street view image into the feature extractor in the feature extraction branch of the street view image to obtain the multi-scale features of the street view image, determining the total loss according to the multi-scale mapping features of the two remote sensing images and the multi-scale features of the street view image, and reversely training the model to obtain a trained multi-scale cross-view geolocation positioning model.

[0072] The idea of ​​contrastive learning is introduced in the network training process. By introducing contrastive learning, the model will be able to cope with the problem of inconsistency of visual features caused by changing factors such as time, weather and season.

[0073] Contrastive learning is a self-supervised learning method that aims to generate meaningful representations by learning the similarities and differences between samples. This method usually involves dividing samples into positive sample pairs (similar samples) and negative sample pairs (dissimilar samples), and then optimizing the model so that similar samples are closer in the embedding space and dissimilar samples are farther away. By introducing contrastive learning, this method can generate similar feature representations from strongly enhanced and weakly enhanced images, increasing the robustness of the features and enhancing the ability of this method to cope with images caused by time, weather and seasonal changes.

[0074] Step 106: Input the remote sensing image to be processed and the street view image into the trained multi-scale cross-view geolocation positioning model to obtain the geolocation positioning result.

[0075] In the above multi-scale cross-viewpoint geographic location positioning method, the method includes obtaining a remote sensing image and a street view image to be processed at the same location across perspectives; constructing a multi-scale cross-viewpoint geographic location positioning model, the multi-scale cross-viewpoint geographic location positioning model includes feature extraction branches and output networks for remote sensing images and street view images; the two feature extraction branches include feature extractors with the same structure; the feature extractor is used to extract multi-scale features of the input image using a multi-stage model based on Swin-transformer; the output network is used to determine the geographic location positioning result based on the multi-scale features output by the two feature extraction branches; the multi-scale cross-viewpoint geographic location positioning model is trained by contrastive learning based on pre-prepared image samples to obtain a trained multi-scale cross-viewpoint geographic location positioning model; the remote sensing image and the street view image to be processed are input into the trained multi-scale cross-viewpoint geographic location positioning model to obtain the geographic location positioning result. This method can improve the accuracy of geographic location positioning and greatly reduce the computational cost.

[0076] In one embodiment, the image samples include multiple groups of cross-view remote sensing images and street view images; in the training process, a feature extractor sharing weights with the feature extractor is added to the feature extraction branch of the remote sensing image, a strong enhancement module and a weak enhancement module are added to the two feature extractors respectively, and a mapping head is added after each of the two feature extractors; the multi-scale cross-view geographic location positioning model in the testing process is obtained by removing the added strong enhancement, weak enhancement modules, feature extractor and two mapping heads in the model in the training process; step 104 includes the following steps:

[0077] Step 200: Divide the pre-prepared image samples into training samples and test samples;

[0078] Step 202: The remote sensing images in the training samples are processed using a strong enhancement module and a weak enhancement module to obtain a strongly enhanced image and a weakly enhanced image;

[0079] Step 204: inputting the strongly enhanced image and the weakly enhanced image into two feature extractors of the feature extraction branch of the remote sensing image respectively, to obtain multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image;

[0080] Step 206: inputting the multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image into two mapping heads with shared parameters respectively, to obtain strongly enhanced mapping features and weakly enhanced mapping features;

[0081] Step 208: inputting the street view image in the training sample into the feature extractor of the feature extraction branch of the street view image to obtain multi-scale features of the street view image;

[0082] Step 210: construct a total loss function; the total loss function includes two retrieval losses and one contrast loss;

[0083] Step 212: determining a total loss according to the strong enhancement mapping features, the weak enhancement mapping features, the multi-scale features of the street view image, and the total loss function;

[0084] Step 214: performing direction training on the multi-scale cross-view geolocation positioning model according to the total loss to obtain a trained multi-scale cross-view geolocation positioning model;

[0085] Step 216: Use the test sample to test the multi-scale cross-view geolocation positioning model input into the test process to obtain a trained multi-scale cross-view geolocation positioning model.

[0086] Specifically, Figure 4 Shown is the multi-scale cross-view geolocation positioning model (GeoViewMatch framework) during training, including a feature extraction branch for remote sensing images (including two feature extractors), two mapping heads with shared parameters, and a feature extraction branch for street view images.

[0087] During the training phase, the network input includes: remote sensing images and street view images.

[0088] For remote sensing images, the network performs strong augmentation and weak augmentation to obtain two enhanced images. The two enhanced images will be input into two feature extractors with shared weights. For strongly enhanced or weakly enhanced images, the feature extractor will generate four features f1, f2, f3 and f4 with different dimensions. The model selects features f3 and f4 for maximum pooling along the channel dimension, and then uses a multi-layer perceptron to perform matrix inner product with f3 and f4 to obtain feature f s3 With f s4 , the feature f s3With f s4 Splice to get the final multi-scale feature f multi The strongly enhanced remote sensing images and the weakly enhanced remote sensing images will obtain multi-scale features f multi-s With f multi-w .f multi-w With f multi-s The feature f will be obtained through two mapping heads with shared weights head-w With f head-s .

[0089] For street view images, the model does not perform strong or weak enhancement, but inputs the original image into feature extraction, and also obtains four features f1, f2, f3, and f4. Then, through the multi-layer perceptron, the obtained features are matrix-inner-producted with f3 and f4 to obtain feature f s3 With f s4 , the feature f s3 With f s4 Splice to get the final multi-scale feature f multi-p The multi-scale features do not pass through the mapping head.

[0090] During the training process, the model performs strong periodic and weak enhancement on remote sensing images and introduces a contrastive learning framework, so that the model can cope with the impact of changes in remote sensing images due to time, weather, and seasons. At the same time, the model extracts different features at different stages to obtain multi-scale features, which will help the model capture more information.

[0091] During the test, neither the street view image nor the remote sensing image was strongly or weakly enhanced. The street view image and the remote sensing image were multi-scaled through the feature extractor, and the multi-scale features were not passed through the mapping head. The cosine similarity between the multi-scale features of the street view image and the multi-scale features of the remote sensing image was calculated, and the remote sensing images were sorted according to the cosine similarity to obtain the final retrieval results.

[0092] In one embodiment, the feature extractor includes a 4-stage module, two maximum pooling layers, and two multi-layer perceptrons, wherein each stage module includes a plurality of Swin Transformer modules; step 204 includes:

[0093] The strongly enhanced image is input into the 4-stage module of the first feature extractor of the feature extraction branch of the remote sensing image to obtain 4 features of different dimensions;

[0094] The features of the third dimension are processed by the first maximum pooling layer and the first multi-layer perceptron successively and the matrix inner product operation is performed with the features of the third dimension to obtain the first inner product features;

[0095] The features of the fourth dimension are processed by the second maximum pooling layer and the second multi-layer perceptron successively and the matrix inner product operation is performed with the features of the fourth dimension to obtain the second inner product features;

[0096] The first inner product feature and the second inner product feature are concatenated to obtain a multi-scale feature of a strongly enhanced remote sensing image;

[0097] The weakly enhanced image is input into the second feature extractor of the feature extraction branch of the remote sensing image to obtain the multi-scale features of the weakly enhanced remote sensing image.

[0098] In one embodiment, the multilayer perceptron includes: two fully connected layers.

[0099] In one embodiment, the mapping head includes: a first linearization layer, a RELU activation function, a batch normalization layer, and a second linearization layer; step 206 includes: inputting the multi-scale features of the strongly enhanced remote sensing image into the first mapping head, and obtaining the strongly enhanced mapping features after being processed by the first linearization layer, the RELU activation function, the batch normalization layer, and the second linearization layer in sequence; inputting the multi-scale features of the strongly enhanced remote sensing image into the second mapping head to obtain the weakly enhanced mapping features.

[0100] In one embodiment, step 210 includes: constructing a contrast loss function based on the strong enhancement mapping features and the weak enhancement mapping features obtained from the strong enhancement and weak enhancement pictures of the same remote sensing image:

[0101]

[0102] in, is the contrast loss, d pos-con With d neg-con Respectively represent the Euclidean distance between the first matching pair and the first unmatched pair, where the first matching pair is the strongly enhanced and weakly enhanced images from the same remote sensing image, and the first unmatched pair is the strongly enhanced and weakly enhanced images from different remote sensing images;

[0103] According to the strong enhancement mapping features and weak enhancement mapping features, as well as the multi-scale features of street view images, two loss functions are constructed as follows:

[0104]

[0105]

[0106] in, is the first retrieval loss, is the second retrieval loss, d pos-ret1 With d neg-ret1Represent the Euclidean distance between the second matching pair and the second unmatching pair, respectively, where the second matching pair is a street view image from the same location and a remote sensing image after strong enhancement, and the unmatching pair is a street view image from different locations and a remote sensing image after strong enhancement; d pos-ret2 With d neg-ret2 Respectively represent the Euclidean distance between the third matching pair and the third unmatched pair, where the third matching pair is a street view image from the same location and a remote sensing image after weak enhancement, and the unmatched pair is a street view image from different locations and a remote sensing image after weak enhancement;

[0107] According to the contrast loss function and two retrieval functions, the total loss function is constructed as:

[0108]

[0109] in, is the total loss.

[0110] Specifically, the method proposed in this application mainly includes three losses: retrieval loss With contrast loss Total loss Defined as:

[0111]

[0112] All three losses use soft-margin triplet loss:

[0113]

[0114] Where α is the weight, which is set to 10 in this paper. pos With d neg Represents the Euclidean distance between matched pairs and unmatched pairs.

[0115] (1) Retrieval loss In the retrieval loss, the matching pairs are street view images and remote sensing images from the same location after strong enhancement, and the mismatched pairs are street view images and remote sensing images from different locations after strong enhancement. In the above figure, the matching pairs are street view images from the same location and remote sensing images after weak enhancement, and the unmatched pairs are street view images from different locations and remote sensing images after weak enhancement. The retrieval loss can drive matching pairs to generate similar features, while unmatched pairs to generate dissimilar features.

[0116] (2) In contrast loss In , the matching pairs are strong and weak enhanced images from the same remote sensing image, and the unmatched pairs are strong and weak enhanced images from different remote sensing images. The contrast loss requires that the strong and weak enhanced images from the same image generate similar features, which will help the model cope with the impact of time, weather, and seasonal changes.

[0117] In one embodiment, step 106 includes: inputting the remote sensing image to be processed and the street view image into the feature extractors of the feature extraction branches of the remote sensing image and the street view image of the trained multi-scale cross-view geolocation positioning model, respectively, to obtain multi-scale features of the remote sensing image and multi-scale features of the street view image; calculating the cosine similarity between the multi-scale features of the street view image and the multi-scale features of the remote sensing image, sorting the remote sensing images according to the size of the cosine similarity, and obtaining the geolocation positioning result.

[0118] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0119] In a verification example, experiments were conducted on two datasets, CVUSA and CVACT. CVUSA dataset: images were collected in the United States, covering commercial, residential, suburban and rural areas. It includes 35532 pairs of training images and 8884 pairs of test images. The original size of the street view images is 224×1232, and the original size of the remote sensing images is 800×800. For street view images, we scale them to 128×640. For remote sensing images, we scale them to 256×256. CVACT: images were collected in Australia, covering cities and suburbs. Similarly. It includes 35532 pairs of training images, 8884 pairs of test images, and provides the location information of the images. The original size of the street view images is 832×1664, and the original size of the remote sensing images is 1200×1200. For street view images, we first crop them to 416×1664 and then scale them to 128×640. For remote sensing images, we scale them to 256×256.

[0120] (1) Experimental parameters

[0121] The experiments were conducted on two NVIDIA RTX 4090 GPUs using the PyTorch framework. We used AdamW as the parameter optimization rate. The weight decay of the AdamW optimizer was set to 1×10-4. In addition, we used Sharpness-Aware Minimization (ASAM) for model optimization to prevent the model from falling into the local minimum. The total number of training epochs was 200. The batch size was set to 64. For the feature extractor in the model, we used the pre-trained weights on ImageNet-1K to initialize the Patch Embedding module and the ST Block module. The learning rates of these modules were set to 1×10 -4 The learning rate of the multilayer perceptron in the feature extractor and the mapping head is set to 2×10 -4 We use random color jitter, random grayscale, and Gaussian blur as data augmentation.

[0122] (2) Evaluation indicators and comparison models

[0123] In the experiment, I took Top-K recall and two meter-level evaluations as evaluation indicators.

[0124] Top-K recall rate: The evaluation indicator of the Top-K recall rate retrieval problem. The calculation process is as follows: sort the retrieval results by cosine similarity, and take out the top K remote sensing images with the largest similarity. If these K images contain the correct retrieval results, it is considered a correct retrieval. Divide the number of correct retrievals by the total number of retrievals to get the retrieval accuracy, recorded as R@K.

[0125] Meter-level evaluation 1: Select a threshold T and use the remote sensing image with the largest cosine similarity as the retrieval result. If the distance between the retrieval result and the street view image to be retrieved is less than or equal to the threshold, it is considered a correct retrieval. Divide the number of correct retrievals by the total number of retrievals to obtain the retrieval accuracy, recorded as M@T.

[0126] Meter-level evaluation 2: The search results are sorted by cosine similarity, and the first K remote sensing images with the largest similarity are taken out. The average distance between the street view image to be retrieved and these K remote sensing images is calculated. The smaller the average distance, the better the performance.

[0127] To demonstrate the effectiveness of our approach, we use several advanced cross-view geolocation models for quantitative comparison: LOCG, CVTF, SAFA, CDTE, and L2LTR.

[0128] (3) Experimental results

[0129] In order to verify the effectiveness of this method in generating maps from remote sensing images, we compared the proposed method with the above methods. Table 1 lists the R@K results of each method on the dataset.

[0130] Table 1 R@K results of various methods on CVUSA and CVACT

[0131]

[0132] Experimental results show that the proposed model (GeoViewMatch) has the best performance in two datasets and four R@K results. On CUVAShujuji, GeoViewMatch achieved 1.42%, 0.86%, 0.63% and 0.06% improvement. On the CVACT dataset, GeoViewMatch achieved 4.76%, 1.79%, 1.14% and 0.11% improvement. This proves the effectiveness of our model.

[0133] Tables 2 and 3 show the results of two meter-level evaluations. Since the CVUSA dataset does not provide image location information, we only perform this evaluation on the CVACT dataset.

[0134] Table 2 Meter-level evaluation 1

[0135]

[0136]

[0137] Table 3 Meter-level evaluation 2

[0138]

[0139] Experimental results show that our model achieves the best performance in both meter-level evaluations, which indicates that GeoViewMatch can limit the retrieval results to a smaller range and reduce the error of geographic location positioning.

[0140] (4) Calculate loss

[0141] In order to compare the computational costs of each model, Table 4 shows the floating-point operations (GFLOPS) and GPU memory usage during model training, as well as the inference time required for the test process.

[0142] Table 4 Calculation cost

[0143]

[0144] The experimental results show that the number of floating point operations and inference time of GeoViewMatch have been significantly reduced while maintaining a relatively low GPU memory usage. Among the compared models, the SAFA model has the lowest number of floating point operations, which is 42.24×10 9 , while GeoViewMatch’s floating point operations are only 23% of SAFA’s. Among the comparison models, L2LTR has the shortest inference time of 82ms, while GeoViewMatch’s inference time is only 51% of L2LTR’s. Compared with other models, GeoViewMatch’s computational cost is significantly reduced, which means that the model has a greater application in large-scale geolocation tasks in real-world scenarios.

[0145] In one embodiment, Figure 5 As shown, a multi-scale cross-viewing geographic location positioning device is provided, including: a multi-viewing image acquisition module, a multi-scale cross-viewing geographic location positioning model construction module, a multi-scale cross-viewing geographic location positioning model training module and a multi-scale cross-viewing geographic location positioning module, wherein:

[0146] A multi-view image acquisition module is used to acquire remote sensing images and street view images to be processed at the same location across multiple viewpoints;

[0147] A multi-scale cross-view geolocation model construction module is used to construct a multi-scale cross-view geolocation model, which includes feature extraction branches and output networks for remote sensing images and street view images; the two feature extraction branches include feature extractors with the same structure; the feature extractor is used to extract multi-scale features of the input image using a multi-stage model based on Swin-transformer; the output network is used to determine the geolocation result based on the multi-scale features output by the two feature extraction branches;

[0148] The multi-scale cross-view geolocation positioning model training module is used to train the multi-scale cross-view geolocation positioning model by contrastive learning based on pre-prepared image samples to obtain a trained multi-scale cross-view geolocation positioning model.

[0149] The multi-scale cross-view geolocation module is used to input the remote sensing image and street view image to be processed into the trained multi-scale cross-view geolocation model to obtain the geolocation results of the remote sensing image and street view image to be processed.

[0150] In one of the embodiments, the image samples include multiple groups of cross-view remote sensing images and street view images; during the training process, a feature extractor sharing weights with the feature extractor is added to the feature extraction branch of the remote sensing image, a strong enhancement module and a weak enhancement module are added to the two feature extractors respectively, and a mapping head is added after each of the two feature extractors; the multi-scale cross-view geolocation positioning model in the testing process is obtained by removing the added strong enhancement, weak enhancement modules, feature extractor and two mapping heads in the model in the training process; the multi-scale cross-view geolocation positioning model training module is also used to divide the pre-prepared image samples into training samples and test samples; the remote sensing images in the training samples are processed by the strong enhancement module and the weak enhancement module respectively, to obtain the strongly enhanced image and the weak enhanced image; the strongly enhanced image and the weak enhanced image are respectively input into the two feature extraction branches of the remote sensing image. The multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image are obtained in the extractor; the multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image are respectively input into two mapping heads with shared parameters to obtain the strongly enhanced mapping features and the weakly enhanced mapping features; the street view image in the training sample is input into the feature extractor of the feature extraction branch of the street view image to obtain the multi-scale features of the street view image; a total loss function is constructed; the total loss function includes two retrieval losses and one contrast loss; the total loss is determined according to the strongly enhanced mapping features, the weakly enhanced mapping features, the multi-scale features of the street view image and the total loss function; the multi-scale cross-view geolocation positioning model is directional trained according to the total loss to obtain the trained multi-scale cross-view geolocation positioning model; the multi-scale cross-view geolocation positioning model input into the test process is tested using a test sample to obtain a trained multi-scale cross-view geolocation positioning model.

[0151] In one embodiment, the feature extractor includes a 4-stage module, two maximum pooling layers and two multi-layer perceptrons, wherein each stage module includes several Swin Transformer modules; the multi-scale cross-view geolocation positioning model training module is also used to input the strongly enhanced image into the 4-stage module in the first feature extractor of the feature extraction branch of the remote sensing image to obtain 4 features of different dimensions; the features of the third dimension are processed by the first maximum pooling layer and the first multi-layer perceptron successively and the matrix inner product operation is performed with the features of the third dimension to obtain the first inner product feature; the features of the fourth dimension are processed by the second maximum pooling layer and the second multi-layer perceptron successively and the matrix inner product operation is performed with the features of the fourth dimension to obtain the second inner product feature; the first inner product feature and the second inner product feature are spliced ​​to obtain the multi-scale features of the strongly enhanced remote sensing image; the weakly enhanced image is input into the second feature extractor of the feature extraction branch of the remote sensing image to obtain the multi-scale features of the weakly enhanced remote sensing image.

[0152] In one of the embodiments, the multi-layer perceptron in the multi-scale cross-view geolocation positioning model training module includes: two fully connected layers.

[0153] In one of the embodiments, the mapping head includes: a first linearization layer, a RELU activation function, a batch normalization layer and a second linearization layer; a multi-scale cross-view geolocation positioning model training module, which is also used to input the multi-scale features of the strongly enhanced remote sensing image into the first mapping head, and after being processed by the first linearization layer, the RELU activation function, the batch normalization layer and the second linearization layer in sequence, the strongly enhanced mapping features are obtained; the multi-scale features of the strongly enhanced remote sensing image are input into the second mapping head to obtain the weakly enhanced mapping features.

[0154] In one embodiment, the multi-scale cross-view geolocation positioning model training module is also used to construct a contrast loss function based on the strong enhancement mapping features and weak enhancement mapping features obtained from the strong enhancement and weak enhancement pictures of the same remote sensing image:

[0155]

[0156] in, is the contrast loss, d pos-con With d neg-con Respectively represent the Euclidean distance between the first matching pair and the first unmatched pair, where the first matching pair is the strongly enhanced and weakly enhanced images from the same remote sensing image, and the first unmatched pair is the strongly enhanced and weakly enhanced images from different remote sensing images;

[0157] According to the strong enhancement mapping features and weak enhancement mapping features, as well as the multi-scale features of street view images, two loss functions are constructed as follows:

[0158]

[0159]

[0160] in, is the first retrieval loss, is the second retrieval loss, d pos-ret1 With d neg-ret1 Represent the Euclidean distance between the second matching pair and the second unmatching pair, respectively, where the second matching pair is a street view image from the same location and a remote sensing image after strong enhancement, and the unmatching pair is a street view image from different locations and a remote sensing image after strong enhancement; d pos-ret2 With d neg-ret2 Respectively represent the Euclidean distance between the third matching pair and the third unmatched pair, where the third matching pair is a street view image from the same location and a remote sensing image after weak enhancement, and the unmatched pair is a street view image from different locations and a remote sensing image after weak enhancement;

[0161] According to the contrast loss function and two retrieval functions, the total loss function is constructed as:

[0162]

[0163] in, is the total loss.

[0164] In one of the embodiments, the multi-scale cross-view geolocation module is further used to input the remote sensing image and street view image to be processed into the feature extractors of the feature extraction branches of the remote sensing image and the street view image of the trained multi-scale cross-view geolocation model, respectively, to obtain the multi-scale features of the remote sensing image and the multi-scale features of the street view image; calculate the cosine similarity between the multi-scale features of the street view image and the multi-scale features of the remote sensing image, sort the remote sensing images according to the size of the cosine similarity, and obtain the geolocation result.

[0165] For the specific definition of the multi-scale cross-viewing geographical location positioning device, please refer to the definition of the multi-scale cross-viewing geographical location positioning method above, which will not be repeated here. Each module in the above multi-scale cross-viewing geographical location positioning device can be implemented in whole or in part by software, hardware and their combination. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the above modules.

[0166] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a multi-scale cross-viewing geographical location positioning method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0167] Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0168] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above method embodiment when executing the computer program.

[0169] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are performed.

[0170] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0171] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0172] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A multi-scale cross-viewing geographic location positioning method, characterized in that: The method comprises: Obtain remote sensing images and street view images to be processed at the same location across perspectives; Constructing a multi-scale cross-view geolocation positioning model, the multi-scale cross-view geolocation positioning model includes feature extraction branches and output networks for remote sensing images and street view images; the two feature extraction branches include feature extractors with the same structure; the feature extractors are used to extract multi-scale features of the input image using a multi-stage model based on Swin-transformer; the output network is used to determine the geolocation positioning result according to the multi-scale features output by the two feature extraction branches; The multi-scale cross-view geographical location positioning model is trained by contrastive learning according to the pre-prepared image samples to obtain a trained multi-scale cross-view geographical location positioning model; The remote sensing images and street view images to be processed are input into the trained multi-scale cross-view geolocation positioning model to obtain the geolocation positioning results.

2. The method according to claim 1, characterized in that The image samples include multiple groups of remote sensing images and street view images across viewing angles; During the training process, a feature extractor sharing weights with the feature extractor is added to the feature extraction branch of the remote sensing image, a strong enhancement module and a weak enhancement module are added to the two feature extractors respectively, and a mapping head is added after each of the two feature extractors; The multi-scale cross-view geolocation model in the testing process is obtained by removing the added strong enhancement, weak enhancement modules, feature extractor and two mapping heads from the model in the training process; The multi-scale cross-view geographical location positioning model is trained by contrastive learning according to the pre-prepared image samples to obtain a trained multi-scale cross-view geographical location positioning model, including: Divide the pre-prepared image samples into training samples and test samples; The remote sensing images in the training samples are processed by the strong enhancement module and the weak enhancement module respectively to obtain the strong enhancement image and the weak enhancement image; Inputting the strongly enhanced image and the weakly enhanced image into two feature extractors of a feature extraction branch of a remote sensing image respectively, to obtain multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image; The multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image are respectively input into two mapping heads with shared parameters to obtain strongly enhanced mapping features and weakly enhanced mapping features; Input the street view images in the training samples into the feature extractor of the feature extraction branch of the street view images to obtain multi-scale features of the street view images; Constructing a total loss function; the total loss function includes two retrieval losses and one contrast loss; Determining a total loss according to the strong enhancement mapping feature, the weak enhancement mapping feature, the multi-scale feature of the street view image and a total loss function; Directionally training the multi-scale cross-view geographical location positioning model according to the total loss to obtain a trained multi-scale cross-view geographical location positioning model; The multi-scale cross-view geolocation positioning model input into the test process is tested using the test samples to obtain a trained multi-scale cross-view geolocation positioning model.

3. The method according to claim 2, characterized in that The feature extractor includes a 4-stage module, two maximum pooling layers and two multi-layer perceptrons, wherein each stage module includes several Swin Transformer modules; The strongly enhanced image and the weakly enhanced image are respectively input into two feature extractors of a feature extraction branch of a remote sensing image to obtain multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image, including: Inputting the strongly enhanced image into a 4-stage module in a first feature extractor of a feature extraction branch of a remote sensing image to obtain 4 features of different dimensions; The features of the third dimension are processed by the first maximum pooling layer and the first multi-layer perceptron successively and the matrix inner product operation is performed with the features of the third dimension to obtain the first inner product features; The features of the fourth dimension are processed by the second maximum pooling layer and the second multi-layer perceptron successively and the matrix inner product operation is performed with the features of the fourth dimension to obtain the second inner product features; The first inner product feature and the second inner product feature are concatenated to obtain a multi-scale feature of a strongly enhanced remote sensing image; The weakly enhanced image is input into the second feature extractor of the feature extraction branch of the remote sensing image to obtain the multi-scale features of the weakly enhanced remote sensing image.

4. The method according to claim 3, characterized in that The multi-layer perceptron includes two fully connected layers.

5. The method according to claim 3, characterized in that: The mapping head includes: a first linearization layer, a RELU activation function, a batch normalization layer and a second linearization layer; The multi-scale features of the strongly enhanced remote sensing image and the weakly enhanced remote sensing image are respectively input into two mapping heads with shared parameters to obtain the strongly enhanced mapping features and the weakly enhanced mapping features, including: The multi-scale features of the strongly enhanced remote sensing image are input into the first mapping head, and are processed by the first linearization layer, the RELU activation function, the batch normalization layer, and the second linearization layer in sequence to obtain the strongly enhanced mapping features; The multi-scale features of the strongly enhanced remote sensing image are input into the second mapping head to obtain the weakly enhanced mapping features.

6. The method according to claim 2, characterized in that Construct the total loss function, including: According to the strong enhancement mapping features and weak enhancement mapping features obtained from the strong enhancement and weak enhancement pictures of the same remote sensing image, the contrast loss function is constructed as follows: in, is the contrast loss, d pos-con With d neg-con Respectively represent the Euclidean distance between the first matching pair and the first unmatched pair, where the first matching pair is the strongly enhanced and weakly enhanced images from the same remote sensing image, and the first unmatched pair is the strongly enhanced and weakly enhanced images from different remote sensing images; According to the strong enhancement mapping feature and the weak enhancement mapping feature, and the multi-scale feature of the street view image, two index loss functions are constructed as follows: in, is the first retrieval loss, is the second retrieval loss, d pos-ret1 With d neg-ret1 Represent the Euclidean distance between the second matching pair and the second unmatching pair, respectively, where the second matching pair is a street view image from the same location and a remote sensing image after strong enhancement, and the unmatching pair is a street view image from different locations and a remote sensing image after strong enhancement; d pos-ret2 With d neg-ret2 Respectively represent the Euclidean distance between the third matching pair and the third unmatched pair, where the third matching pair is a street view image from the same location and a remote sensing image after weak enhancement, and the unmatched pair is a street view image from different locations and a remote sensing image after weak enhancement; According to the contrast loss function and the two retrieval functions, the total loss function is constructed as: in, is the total loss.

7. The method according to claim 1, characterized in that Input the remote sensing image and street view image to be processed into the trained multi-scale cross-view geolocation positioning model to obtain the geolocation positioning results, including: The remote sensing image to be processed and the street view image are respectively input into the feature extractors of the feature extraction branches of the remote sensing image and the street view image of the trained multi-scale cross-view geolocation positioning model to obtain multi-scale features of the remote sensing image and multi-scale features of the street view image; The cosine similarity between the multi-scale features of the street view image and the multi-scale features of the remote sensing image is calculated, and the remote sensing images are sorted according to the cosine similarity to obtain the geographic location positioning result.

8. A multi-scale, cross-viewing geographical location positioning device, characterized in that: The device comprises: A multi-view image acquisition module is used to acquire remote sensing images and street view images to be processed at the same location across multiple viewpoints; A multi-scale cross-view geolocation model construction module is used to construct a multi-scale cross-view geolocation model, wherein the multi-scale cross-view geolocation model includes feature extraction branches and an output network for remote sensing images and street view images; the two feature extraction branches include feature extractors with the same structure; the feature extractor is used to extract multi-scale features of the input image using a multi-stage model based on Swin-transformer; the output network is used to determine the geolocation result according to the multi-scale features output by the two feature extraction branches; A multi-scale cross-view geolocation positioning model training module is used to train the multi-scale cross-view geolocation positioning model by contrastive learning based on pre-prepared image samples to obtain a trained multi-scale cross-view geolocation positioning model; The multi-scale cross-view geolocation module is used to input the remote sensing image and street view image to be processed into the trained multi-scale cross-view geolocation model to obtain the geolocation results of the remote sensing image and street view image to be processed.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images