Geolocation Method and System Based on Cross-Perspective Matching

Abstract

The present invention discloses a geolocation method and system based on cross-view matching, including: performing geolocation on street view images by constructing a siamese neural network including a Transformer network and a hybrid loss function; when constructing the positioning framework, a framework from coarse to fine is designed, improving the accuracy of geolocation, which can be better than the ground resolution of map data in the dataset; a hybrid function of triplet loss and semi-positive sample loss metrics is used for image similarity measurement. The present invention improves the accuracy of geolocation, improves the utilization rate of effective information between non-aligned image pairs and the effectiveness of feature matching; extracts features between image pairs through a siamese neural network with hierarchical Transformer, enriches the feature description, improves the accuracy of matching, and uses cross-view image geolocation in various scenarios with fewer features, solving the problem that existing algorithms rely on artificial ground object information.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous planetary navigation, and particularly relates to a geolocation method and system based on cross-view matching. Background Art

[0002] Planetary surface exploration will be one of the most core capabilities and technologies in future deep space exploration. Among them, autonomous positioning in an environment lacking artificial features is one of the key technologies for exploration. Currently, the communication delay from the lunar surface to the ground is at the second level, and the communication delay from Mars to the ground even reaches more than 10 minutes, which cannot meet the requirements of autonomous exploration for real-time high-precision navigation. And the environmental adaptability of the navigation method is one of the keys to determining the success or failure of the exploration mission.

[0003] Currently, the research focus is mainly on achieving positioning by using image retrieval. Ground data is collected for datasets in different cities, feature learning and extraction are carried out, features are matched, and the closest image is selected according to similarity. However, there is a very large perspective difference between ground street view images and top-down maps. Therefore, the expression of ground object information in images is not very similar, and the imaging method of street view images will result in the characteristic that the ground resolution is higher the closer it is to the camera. Therefore, in addition to perspective differences, the influence of resolution differences also needs to be overcome. The existing technologies have problems of low positioning accuracy and inaccuracy and over-reliance on ground object features. Summary of the Invention

[0004] The purpose of the present invention is to provide a geolocation method and system based on cross-view matching. This method aims to solve the problems of low positioning accuracy and inaccuracy and over-reliance on ground object features existing in traditional methods.

[0005] To achieve the above purpose, on the one hand, the present invention provides a geolocation method based on cross-view matching, including:

[0006] Step S1: Preprocess the database images, perform image transformation on the obtained top-down map using polar coordinate transformation, and perform image stitching on the obtained street view images to be located according to different perspectives;

[0007] Step S2: Construct a Siamese neural network based on Transformer. The Siamese neural network includes two identical network branches. The two network branches share weights and parameters. Each network branch constitutes a Transformer neural network. Each network branch includes: a Transformer encoder, a two-dimensional convolution function, a ReLU activation function, and a block normalization function;

[0008] Step S3: Based on the top-down map and panoramic image after the polar coordinate transformation in Step S1, construct a data set, train the Transformer neural network according to the data set to obtain a training model, use the training model to test the street snapshot image to be recognized after the image stitching in Step S1 to obtain a first sorting result, perform block downsampling on the candidate map according to the first sorting result to obtain multiple top-down sample images, and record the spatial relationship between the blocks;

[0009] Step S4: Use the training model to test the multiple top-down sample images again to obtain a second sorting result, weight the second sorting result according to the spatial relationship, and use Gaussian fitting to obtain the positioning coordinates of the street scene image to be recognized.

[0010] Preferably, for the polar coordinate transformation of the top-down map in Step S1, its calculation expression is:

[0011]

[0012] In the formula: represents the pixel coordinates in the original image, represents the pixel coordinates in the transformed image, (W s , H s ) represents the size of the original image, (W t , H t ) represents the size of the transformed image.

[0013] Preferably, in Step S1, for image stitching of the street scene image to be located obtained from different perspectives, its specific calculation includes:

[0014] Define the two-dimensional coordinates of a certain pixel point in the image as (x, y), and its corresponding world coordinates as (X, Y, Z), and its expression is:

[0015]

[0016] In the formula: R is the rotation matrix, K is the internal parameter matrix of the camera, -1 represents matrix inversion, define (u, v) as the mapped two-dimensional coordinates, then the coordinates after spherical projection are:

[0017]

[0018] In the formula, s represents the scale, which is proportional to the camera focal length,

[0019] Perform inverse mapping on the image to obtain the world coordinates after inverse mapping as:

[0020]

[0021] Preferably, a Transformer-based siamese neural network is constructed.

[0022] The two network branches are input into a Transformer network with 4-layer fused positional encoding after downsampling using a convolution function, obtaining feature vectors at 4 scales. The feature vectors at each scale are subjected to difference and normalization operations and then concatenated into a global feature vector.

[0023] The 4 scales are respectively (H / 2, W / 2), (H / 4, W / 4), (H / 8, W / 8), (H /

[0024] 16, W / 16), where H and W are the length and width of the input image. Then the area of the patches is the product of the above four scales, and the numbers of patches are: 4, 16, 64, 256 respectively. Therefore, the dimension of the unified feature vector is 257*1*1. The concatenated global feature vector is 1028*1*1 dimensional.

[0025] Preferably, training the Transformer neural network according to the dataset in step S3 specifically includes:

[0026] The Transformer neural network is iteratively trained using the dataset, and a hybrid loss function is shared during the iterative training; during each training, a triplet loss function and a semi-hard positive loss function based on dynamic sample selection are used to train the image retrieval model. The ground feature map and the aerial feature map are obtained through siamese network inference. After rough angle alignment through similarity measurement, the effectiveness of the triplet (A, P, N) is then measured. The expression of the hybrid loss function is:

[0027]

[0028] where A is the anchor sample, P is the positive sample and semi-hard positive sample, N is the negative sample, α is the set distance interval between the same-class samples and different-class samples, d(·,·) is the L2 distance metric between samples, S semi and S pos represent the cosine similarity of the positive sample pairs and semi-hard positive sample pairs in the feature space, and IOU semi and IOU pos represent the overlap rates of the positive sample and semi-hard positive sample pairs respectively.

[0029] Preferably, testing the street view image to be recognized after image stitching in step S1 using the training model in step S3 to obtain the first sorting result specifically includes:

[0030] Input the captured street view image to be located and the map part in the dataset into the trained network; the network will automatically encode the features of the two cross-view images into vectors of 1028*1*1 dimensions. After performing similarity measurement, the map will be sorted according to the similarity.

[0031] The step of performing block downsampling on the candidate map according to the first sorting result in step S3 specifically includes:

[0032] Select the top 1 candidate result in the sorting and record their position information {T}. The size of each map is (w map , map ). Divide the original map data into blocks according to displacements of 0.1*w map and 0.1* map . The new map size is (0.5*w map , 0.5* map ). Then stretch the image size to w map , map ) to obtain 100 candidate maps, and their position information {T exp}. Assign weights h according to the distance information from {T}.

[0033] Preferably, the step of re-testing multiple top-down sample images using the training model in step S4 specifically includes:

[0034] Downsample the street view image to be located and input 100 candidate maps into the trained network; the network will automatically encode the features of the two cross-view images into vectors of 1028*1*1 dimensions. After performing similarity measurement, combine the similarity measurement result and the weight h to obtain a new measurement value, establish a 10*10 two-dimensional matrix, fill the new measurement value into the two-dimensional matrix according to the position of {T exp}, perform Gaussian fitting on the two-dimensional matrix, and obtain the peak position (x fit , y fit ), which is the geographical coordinate of the street view image to be located.

[0035] On the other hand, the present invention also provides a geographical positioning system based on cross-view matching, which is used to implement the above-mentioned geographical positioning method based on cross-view matching, and specifically includes:

[0036] The first module is used to transform the map and street view image in the dataset into a form available for matching;

[0037] The second module is used to construct a twin neural network based on Transformer, and the twin neural network includes: a Transformer encoder, a two-dimensional convolution, a ReLU activation function, and a block normalization function;

[0038] A third module is used to train the siamese neural network model in a set of planetary surface datasets; for a street view image to be located, use the trained model to test in the dataset to obtain a first sorting result; perform block division, resolution processing, and weighting on the map of the first sorting result.

[0039] A fourth module is used to retest in the new candidate map data after block division with the trained model, combine the similarity measurement result and the weight to obtain a two-dimensional matrix, perform two-dimensional Gaussian fitting on it, and take the peak coordinates to realize the positioning of the street view image to be located.

[0040] The present invention also provides an electronic device, including a processor and a memory. The memory is used to store data, models, and program instructions. The processor is used to call the stored instructions in the memory to execute the planetary surface geolocation method based on cross-view image matching described above.

[0041] The present invention also provides a readable storage medium, on which a model and a computer program are stored. When the computer program is executed, the planetary surface geolocation method based on cross-view image matching described above is realized.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention constructs a siamese neural network including a Transformer network and a hybrid loss function to perform geolocation on street view images; the present invention improves the accuracy of geolocation, which is better than the ground resolution of the map data in the dataset; when performing similarity measurement between images, the present invention adopts a hybrid function of triplet loss and semi-positive sample loss measurement, which improves the utilization rate of effective information between misaligned image pairs, can adapt to street view images from 90° to 360°, and improves the effectiveness of feature matching; the present invention also extracts features between image pairs through a siamese neural network with hierarchical Transformer, enriches the feature description, and improves the matching accuracy; and uses the similarity measurement fitting of the distance weighting factor to achieve high-precision positioning. The research on this problem can be used for autonomous navigation in planetary exploration, as well as autonomous navigation in GPS-denied environments, etc. The fine positioning module fully considers the resolution difference between cross-view images and solves the positioning problem of images through training and weighting. The present invention can perform cross-view image geolocation in various scenarios with fewer features and solves the problem that existing algorithms rely on artificial ground object information. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] To more clearly illustrate the technical solution of the present invention, the accompanying drawings required in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description are an embodiment of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings:

[0045] Figure 1 It is a schematic flowchart of a geographic positioning method based on cross-view matching provided by an embodiment of the present invention. Specific embodiments

[0046] The following combines the attached Figure 1 drawings and specific embodiments to further elaborate on a geographic positioning method and system based on cross-view matching proposed by the present invention. According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the attached drawings are in a very simplified form and all use non-precise scales, only for conveniently and clearly assisting in explaining the purpose of the embodiments of the present invention. In order to make the purpose, features, and advantages of the present invention more obvious and understandable, please refer to the attached drawings. It should be known that the structures, ratios, sizes, etc. shown in the drawings of this specification are only used to cooperate with the content disclosed in the specification for those familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the present invention. Therefore, they do not have technical substance significance. Any modification of the structure, change of the proportional relationship, or adjustment of the size, without affecting the effects that the present invention can produce and the purposes that can be achieved, should still fall within the scope covered by the technical content disclosed by the present invention.

[0047] In view of the problems of low positioning accuracy and accuracy and excessive dependence on ground object features in traditional methods. On the one hand, this embodiment provides a geographic positioning method based on cross-view matching, including:

[0048] Step S1: Preprocess the database image, use polar coordinate transformation to perform image transformation on the obtained top-down map, and perform image stitching on the obtained street view images to be located according to different perspectives.

[0049] The polar coordinate transformation of the top-down map in step S1 has the following calculation expression:

[0050]

[0051] In the formula: represents the pixel coordinates in the original image, represents the pixel coordinates in the transformed image, (W s , H s ) represents the size of the original image, (W t , H t ) represents the size of the transformed image.

[0052] In step S1, the obtained street view images to be located are stitched according to different perspectives, and the specific calculations include:

[0053] Define the two-dimensional coordinates of a pixel point in the image as (x, y), and its corresponding world coordinates as (X, Y, Z). The expression is:

[0054]

[0055] In the formula: R is the rotation matrix, K is the internal parameter matrix of the camera, -1 represents matrix inversion. Define (u, v) as the mapped two-dimensional coordinates, then the coordinates after spherical projection are:

[0056]

[0057] In the formula, s represents the scale, which is proportional to the camera focal length.

[0058] Perform inverse mapping on the image to obtain the world coordinates after inverse mapping as:

[0059]

[0060] Step S2: Construct a Siamese neural network based on Transformer. The Siamese neural network includes two identical network branches. The two network branches share weights and parameters. Each network branch constitutes a Transformer neural network. Each network branch includes: a Transformer encoder, a two-dimensional convolution function, a ReLU activation function, and a block normalization function.

[0061] Construct a Siamese neural network based on Transformer, which specifically includes:

[0062] For the two network branches, use the convolution function to obtain downsampled inputs to the Transformer network with 4-layer fused positional encoding, obtaining 4-scale feature vectors. After performing difference and normalization operations on the feature vectors at each scale, they are concatenated into a global feature vector.

[0063] The four scales are (H / 2, W / 2), (H / 4, W / 4), (H / 8, W / 8), (H / 16, W / 16), where H and W are the length and width of the input image. Then the area of the patches is the product of the above four scales, and the number of patches is: 4, 16, 64, 256. Therefore, the dimension of the unified feature vector is 257*1*1. The concatenated global feature vector is 1028*1*1 dimensions.

[0064] Step S3: Based on the top-down map and panoramic image after the polar coordinate transformation in Step S1, construct a dataset. Train the Transformer neural network according to the dataset to obtain a training model. Use the training model to test the street snapshot image to be recognized after the image stitching in Step S1 to obtain a first sorting result. Perform block downsampling on the candidate map according to the first sorting result to obtain multiple top-down sample images, and record the spatial relationship between the blocks.

[0065] In this embodiment, training the Transformer neural network according to the dataset specifically includes: performing iterative training on the Transformer neural network using the dataset, and sharing a hybrid loss function during the iterative training; using a triplet loss function and a semi-hard positive loss function based on dynamic sample selection for training the image retrieval model during each training. After obtaining the ground feature map and the top-down feature map through the inference of the Siamese network and performing rough angle alignment through similarity measurement, then measure the effectiveness of the triplet (A, P, N). The expression of the hybrid loss function is:

[0066]

[0067] where A is the anchor sample, P is the positive sample and semi-hard positive sample, N is the negative sample, α is the set distance interval between the same-class samples and different-class samples, d(·,·) is the L2 distance metric between samples, S semi and S pos represent the cosine similarity of the positive sample pair and the semi-hard positive sample pair in the feature space, and IOU semi and IOU pos represent the overlap rates of the positive sample and the semi-hard positive sample pair respectively.

[0068] Using the training model to test the street snapshot image to be recognized after the image stitching in Step S1 to obtain a first sorting result specifically includes:

[0069] Input the captured street view image to be located and the map part in the dataset into the trained network; the network will automatically encode the features of the two cross-view images into vectors of 1028*1*1 dimensions, and after performing similarity measurement, sort the maps according to the similarity.

[0070] Performing block downsampling on the candidate map according to the first sorting result specifically includes: taking the top 1 candidate results of the sorting and recording their position information {T}. The size of each map is w map , map ), and according to 0.1*w map and 0.1* mapPerform block division on the original map data according to the displacement, and the new map size is (0.5*w map , 0.5* map ), and then stretch the image size to w map , map ), to obtain 100 candidate maps, and their position information {T exp}. Assign weights h according to the distance information from {T}.

[0071] Step S4: Retest multiple top-down sample images using the training model to obtain a second sorting result, weight the second sorting result according to the spatial relationship, and use Gaussian fitting to obtain the positioning coordinates of the street view image to be recognized.

[0072] Retest multiple top-down sample images using the training model, which specifically includes: performing downsampling on the street view image to be positioned and inputting 100 candidate maps into the trained network; the network will automatically encode the features of two cross-view images into a 1028*1*1-dimensional vector, and after similarity measurement, combine the similarity measurement result with the weight h to obtain a new measurement value, establish a 10*10 two-dimensional matrix, and fill the new measurement value into the two-dimensional matrix according to the position of {T exp}, perform Gaussian fitting on the two-dimensional matrix to obtain the peak position (x fit , y fit ), which is the geographical coordinates of the street view image to be positioned.

[0073] On the other hand, this embodiment also provides an image geographical positioning system based on cross-view matching, which is used to implement the geographical positioning method based on cross-view matching as described above, and specifically includes: a first module for transforming the maps and street view images in the dataset into a form available for matching;

[0074] A second module for constructing a Siamese neural network based on Transformer, the Siamese neural network includes: a Transformer encoder, a two-dimensional convolution, a ReLU activation function, and a block normalization function; a third module for training the Siamese neural network model in the set planetary surface dataset; for the street view image to be positioned, use the trained model to test in the dataset to obtain a first sorting result; perform block division, resolution, and weight assignment processing on the maps of the first sorting result; a fourth module for retesting in the new candidate map data after block division using the trained model, combining the similarity measurement result with the weight to obtain a two-dimensional matrix, and taking the peak coordinates after performing two-dimensional Gaussian fitting on it to achieve the positioning of the street view image to be positioned.

[0075] This embodiment also provides an electronic device, including a processor and a memory. The memory is used to store data, models, and program instructions, and the processor is used to call the stored instructions in the memory to execute the geolocation method based on cross-perspective matching described above.

[0076] This embodiment also provides a readable storage medium, on which a model and a computer program are stored. When the computer program is executed, the geolocation method based on cross-perspective matching described above is implemented.

[0077] In summary, this embodiment constructs a siamese neural network including a Transformer network and a hybrid loss function to perform geolocation on street view images. This embodiment improves the accuracy of geolocation, which is better than the ground resolution of map data in the dataset. When performing image similarity measurement, this embodiment uses a hybrid function of triplet loss and semi-positive sample loss measurement, which improves the utilization rate of effective information between misaligned image pairs and can adapt to street view images from 90° to 360°, enhancing the effectiveness of feature matching. This embodiment also extracts features between image pairs through a siamese neural network with hierarchical Transformer, enriching feature descriptions and improving the accuracy of matching. And high-precision positioning is achieved by fitting the similarity measurement with a distance weighting factor. The research on this problem can be used for autonomous navigation in planetary exploration and autonomous navigation in GPS-denied environments, etc. The fine positioning module fully considers the resolution differences between cross-perspective images and solves the image positioning problem through training and weighting. This embodiment can perform geolocation on cross-perspective images in various scenarios with fewer features, solving the problem that existing algorithms rely on artificial ground object information.

[0078] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0079] It should be noted that the devices and methods disclosed in the embodiments of this article can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of this article. In this regard, each block in the flowchart or block diagram may represent a module, program, or part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0080] Although the content of the present invention has been introduced in detail through the above preferred embodiments, it should be recognized that the above description should not be considered as a limitation of the present invention. After those skilled in the art have read the above content, various modifications and alternatives to the present invention will be obvious. Therefore, the protection scope of the present invention should be defined by the appended claims.

Claims

1. A geolocation method based on cross-viewpoint matching, characterized in that, Including: Step S1: Preprocess the database images. Use polar coordinate transformation to perform image transformation on the obtained top-down map, and perform image stitching on the obtained street view images to be located according to different perspectives. Step S2: Construct a Siamese neural network based on Transformer. The Siamese neural network includes two identical network branches. The two network branches share weights and parameters. Each network branch constitutes a Transformer neural network. Each network branch includes: a Transformer encoder, a two-dimensional convolution function, a ReLU activation function, and a block normalization function. Step S3: According to the top-down map and panoramic image after the polar coordinate transformation in Step S1, construct a dataset. Train the Transformer neural network according to the dataset to obtain a training model. Use the training model to test the street view pictures to be recognized after the image stitching in Step S1 to obtain a first sorting result. Perform block downsampling on the candidate map according to the first sorting result to obtain multiple top-down sample images, and record the spatial relationship between the blocks. Step S4: Use the training model to test the multiple top-down sample images again to obtain a second sorting result. Weight the second sorting result according to the spatial relationship, and use Gaussian fitting to obtain the positioning coordinates of the street view image to be located.

2. The geolocation method based on cross-perspective matching according to claim 1, characterized in that For the polar coordinate transformation of the top-down map in Step S1, its calculation expression is: In the formula: represents the pixel coordinates in the original image, represents the pixel coordinates in the transformed image, (W s , H s ) represents the size of the original image, (W t , H t ) represents the size of the transformed image.

3. The geolocation method based on cross-perspective matching according to claim 2, wherein In Step S1, for the image stitching of the obtained street view images to be located according to different perspectives, its specific calculation includes: Define the two-dimensional coordinates of a certain pixel point in the image as (x, y), and its corresponding world coordinates as (X, Y, Z). Its expression is: In the formula: R is the rotation matrix, K is the internal parameter matrix of the camera, -1 represents matrix inversion. Define (u, v) as the mapped two-dimensional coordinates, then the coordinates after spherical projection are: In the formula, s represents the scale, which is proportional to the camera focal length. Perform inverse mapping on the image to obtain the world coordinates after inverse mapping as: 。 4. The geolocation method based on cross-perspective matching according to claim 3, wherein In Step S2, construct a Siamese neural network based on Transformer. For the two network branches, use a convolution function to obtain downsampled input to a Transformer network with 4-layer fused positional encoding to obtain feature vectors of 4 scales. Connect the feature vectors at each scale after differential and normalization operations to form a global feature vector.

5. The geolocation method based on cross-perspective matching according to claim 4, wherein, For the training of the Transformer neural network according to the dataset in Step S3, it specifically includes: The Transformer neural network is iteratively trained using the said dataset, and a hybrid loss function is shared during the iterative training; during each training, a triplet loss function and a semi-hard positive loss function based on dynamic sample selection are used to train the image retrieval model. After obtaining the ground feature map and the aerial view feature map through siamese network inference and performing rough angle alignment through similarity measurement, the effectiveness of the triplet (A, P, N) is then measured. The expression of the hybrid loss function is as follows: Where A is the anchor sample, P is the positive and semi-positive samples, N is the negative sample, α is the set distance interval between the same-class samples and different-class samples, d(·,·) is the L2 distance metric between samples, S semi and S pos represent the cosine similarity of positive sample pairs and semi-positive sample pairs in the feature space, IOU semi and IOU pos represent the overlap rates of positive sample pairs and semi-positive sample pairs respectively.

6. The geolocation method based on cross-viewpoint matching according to claim 5, wherein In step S3, the to-be-recognized street snapshot image after image stitching in step S1 is tested using the training model to obtain a first sorting result, which specifically includes: The captured street view image to be located and the map part in the dataset are input into the trained network; the network will automatically encode the features of the two cross-perspective images into vectors of 1028*1*1 dimensions. After performing similarity measurement, the map is sorted according to the similarity. In step S3, the first sorting result is used to perform a block downsampling process on the candidate map, which specifically includes: Select the top 1 sorted candidate results and record their position information {T}; The size of each map is (w map , h map ). Chunk the original map data according to displacements of 0.1*w map and 0.1*h map . The new map size is (0.5*w map , 0.5*h map ). Then stretch the image to size (w map , h map ) to obtain 100 candidate maps, and their position information {T exp}; Assign weights h according to the distance information from {T}.

7. The geolocation method based on cross-perspective matching according to claim 6, characterized in that In step S4, multiple aerial view sample images are retested using the training model, which specifically includes: Downsample the to-be-localized street view image and input it along with 100 candidate maps into the trained network; the network will automatically encode the features of two cross-perspective images into vectors of dimension 1028*1*1. After performing similarity measurement, the similarity measurement result is combined with the weight h to obtain a new measurement value. A 10*10 two-dimensional matrix is established, and the new measurement value is filled into the two-dimensional matrix according to the positions of {T exp}. Gaussian fitting is performed on the two-dimensional matrix to obtain the peak position (x fit , y fit ), which is the geographical coordinate of the to-be-localized street view image.

8. A geolocation system based on cross-perspective matching, characterized in that, It is used to implement the cross-perspective matching-based geolocation method according to any one of claims 1-7, and specifically includes: The first module is used to transform the map and the street view image in the dataset into a form available for matching; The second module is used to construct a siamese neural network based on Transformer. The siamese neural network includes: a Transformer encoder, a two-dimensional convolution, a ReLU activation function, and a block normalization function; The third module is used to train the siamese neural network model in the set planetary surface dataset; for the street view image to be located, the trained model is used to perform tests in the dataset to obtain a first sorting result; the map of the first sorting result is divided into blocks for resolution and weighting processing; The fourth module is used to perform retesting on the newly candidate map data after block division using the trained model, combine the similarity measurement result and the weight to obtain a two-dimensional matrix, perform two-dimensional Gaussian fitting on it, and take the peak coordinates to achieve the location of the street view image to be located.

9. An electronic device, characterized in that, It includes a processor and a memory. The memory is used to store data, models, and program instructions. The processor is used to call the stored instructions in the memory to execute the cross-perspective matching-based geolocation method according to any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a model and a computer program. When the computer program is executed, it implements the cross-perspective matching-based geolocation method according to any one of claims 1-7.

Images