Geographic positioning method and system for multi-source quadrilateral area

Abstract

The invention discloses a multi-source quadrilateral region geographic positioning method and system. The method comprises the following steps: firstly, unifying geographic reference for a multi-source base map library, and extracting and storing domain self-adaptive global features by adopting a ViT-adapter; performing coarse retrieval on the query image to obtain a candidate area and cutting the candidate area into candidate quadrilateral slices; and carrying out local feature matching, mapping point pairs into a tetrad (x, y, X, Y), screening key areas through a 3 * 3 grid, completing coarse positioning by using RANSAC fitting affine, and carrying out sub-pixel fine registration to output a result and an optional confidence / residual error. According to the method, a more complex geometrical relationship among multi-source, multi-resolution and multi-temporal images can be processed, the multi-source domain deviation is reduced, the robustness under the multi-scene condition is improved, the operation speed and high-precision positioning requirements are considered, and the method has obvious advantages in retrieval precision and migration performance and is more suitable for engineering deployment.

Description

Technical Field

[0001] This invention belongs to the field of electronic data processing technology, and further relates to a geolocation method and system for multi-source quadrilateral regions in high-resolution optical images, applicable to the field of visual location recognition technology. This invention can be used in multi-source quadrilateral region positioning systems for high-resolution optical images to achieve rapid and accurate positioning of optical remote sensing images. Background Technology

[0002] The first step in visual place recognition is to transform image content into a mathematical representation, or feature descriptor, that can be compared and retrieved by a computer. In recent years, deep learning-based feature extraction methods, using convolutional neural networks or visual ViT (Transformer), have enabled systems to automatically learn more robust and advanced semantic features for place recognition tasks from massive amounts of data. These features are no longer limited to low-level corners and edges, but encompass abstract concepts of texture, structure, and even objects, providing a more powerful representation for addressing the complex variations in visual place recognition (VPR). However, general-purpose visual models are mostly trained on natural scenes, exhibiting significant domain bias when dealing with multi-source remote sensing imagery; direct transfer of these domain biases can easily lead to a decline in retrieval performance.

[0003] The Shenzhen Institutes of Advanced Technology (SIAT) disclosed a method for georegistration of UAV images using semantic segmentation and feature matching in its patent application, "A Method and Related Equipment for Georegistration of UAV Images Based on Semantic Segmentation" (Patent Application No. 202310483317.4, Authorization Announcement No.: CN 116485856 B). This method first acquires UAV image information and obtains an orthorectified image through attitude calculation. Then, the orthorectified image is input into a pre-trained semantic segmentation model to extract road semantic regions, and road feature points are extracted based on these regions. Next, the method acquires the corresponding reference image, crops and annotates it to obtain road semantic constraint results, and extracts feature points from it and stores them in a feature point database. Finally, a feature correspondence is established between the UAV image and the reference image using a feature point matching algorithm. The geographic coordinates of the UAV image are calculated using the geographic coordinate information of the reference image, thereby generating a registered image with geographic coordinates. However, this patented solution still has the following shortcomings: First, its registration accuracy is highly dependent on the semantic segmentation model's recognition performance of road areas, and it is prone to semantic missegmentation in scenes with vegetation occlusion or complex surface textures, leading to unstable feature point extraction. Second, the method mainly focuses on road area features and lacks effective constraints on non-road areas (such as buildings, rivers, landmarks, etc.), limiting its applicability in diverse terrains. Third, the solution uses a feature extraction and matching method based on the difference of Gaussian pyramid, which has high computational complexity and is difficult to run in real time on embedded systems or drones. Therefore, this method still has room for further improvement in high-precision registration scenarios involving complex terrain, low-power platforms, and multimodal feature fusion.

[0004] Wuhan University, Yangtze Space Information Technology Engineering Co., Ltd. (Wuhan), and Tianjin Surveying and Mapping Institute Co., Ltd. disclosed a fully automatic lake positioning method for domestic optical remote sensing satellite imagery in their patent application, "A Fully Automatic Lake Positioning Method and System for Domestic Optical Remote Sensing Satellite Imagery" (Patent Application No.: 202311421608.7, Authorization Announcement No.: CN 117409078 B). This method first establishes a lake image baseline database, collecting publicly available or self-built time-series lake datasets. Then, it uses the normalized water index to extract lakes from the image to be corrected, obtaining stable lake boundaries through contour smoothing and noise removal, and cropping the lake areas to calculate their areas. Next, the lake vector images in the database are converted into raster images, and the sample with the closest area is searched as the reference image, followed by spatial resolution resampling and coordinate system unification. Finally, using the reference image as a reference, the translation parameters are calculated using the maximum overlap area method to achieve automatic image correction and lake positioning. However, this patented solution still has certain limitations. First, its registration accuracy depends on the accuracy of water body index extraction. When there are cloud shadows, algae, or snow cover at the lake edge, the normalized water body index extraction is easily disturbed, leading to lake outline shift. Second, the scheme uses area similarity and translation matching for positioning, ignoring rotation and scale changes, making it difficult to adapt to complex geometric differences between different sensors or multi-temporal images. Third, this method focuses on matching single water body elements and lacks adaptability to remote sensing images containing multiple types of land cover features (such as urban, river network, or mountain scenes). Therefore, there is still room for improvement in complex surface environments, multi-source image fusion, and high-precision georegistration tasks. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a geolocation method for multi-source quadrilateral regions based on visual ViT and adapter for multi-source remote sensing image feature extraction and aggregation. This method aims to solve problems such as strong semantic dependence, single feature type, insufficient computational efficiency, and weak adaptability to complex environments and multiple geometric transformations.

[0006] The idea behind achieving the objective of this invention is: This invention proposes a multi-source substrate... Figure 4 A phased geolocation method for tiling polygonal regions. First, based on a pre-trained Visual Transformer (ViT) backbone, lightweight adapter modules are inserted into at least some TransformerBlocks. Only the adapters and a few normalization parameters are updated to achieve domain-adaptive feature alignment for multi-source remote sensing images with different satellites / platforms, resolutions, imaging angles, and temporal phases, thereby obtaining a global descriptor suitable for retrieval. Second, a coarse visual location recognition retrieval is performed based on the global descriptor to obtain the coarse visual location of the query image, and a multi-source base map is constructed. Figure 4The visual location candidate set for quadrilateral region slices: The multi-source base map regions hit by the coarse search are cropped into quadrilateral region slices according to their geographical boundaries, which serve as objects for subsequent local geometric verification. Then, for each pair of "query image patch - candidate base map", Figure 4 The "border region slice" is used for local feature matching to obtain matching point pairs (x, y) and (u, v), and these are then mapped and expanded into quadruplets (x, y, X, Y), where (X, Y) are candidate base pairs. Figure 4 The coordinates of the polygonal region slices are mapped to a unified geographic coordinate system / unified grid coordinate system. Further filtering based on a 3×3 grid overlay using triple metrics identifies key connected regions, retaining only quadruplets within these key regions for geometric estimation. Finally, a robust estimation algorithm, such as random sample consistency, is used to fit an affine transformation on the set of key region quadruplets to obtain coarse geographic locations. Subpixel-level fine-tuning registration is then performed at these coarse geographic locations, outputting the final geolocation result and optional confidence / residual indices.

[0007] The specific implementation steps of the method of the present invention include the following:

[0008] Step 1: Use the multi-source remote sensing base map library to pre-extract the multi-source remote sensing domain adaptive adapter features;

[0009] Step 2: Perform scale normalization and block division on the queried image and the base image library images in sequence;

[0010] Step 3: Based on the ViT-adapter network, the visual location of the query image is coarsely retrieved, and a multi-source background model is constructed. Figure 4 A candidate set of visual locations for polygonal region slices;

[0011] Step 4: Perform local feature matching on each query image patch and its corresponding visual location candidate set, and perform key region screening by quadruple mapping and triple measurement based on the matching points.

[0012] Step 5: Perform affine transformation estimation on the selected quadruples to fit the coarse geolocation location; perform sub-pixel level fine registration at the coarse geolocation location to obtain the final geolocation output.

[0013] Furthermore, the step of pre-extracting the multi-source remote sensing domain adaptive adapter features using a multi-source remote sensing base map library is as follows:

[0014] The first step is to acquire multi-source remote sensing base map images from different satellites or platforms, with different resolutions, imaging angles, or time phases, and to establish a one-to-one correspondence between each base map and its geographic coordinates to build a multi-source base map library.

[0015] The second step involves inputting the base map image into a feature extraction and aggregation network to obtain a fixed-length global descriptor vector, and then writing the global descriptor vector and its corresponding geographic coordinates into a feature library. The feature extraction and aggregation network uses a pre-trained visual Transformer as its backbone, inserting adapter modules within at least a portion of the Transformer Block to achieve directional adaptation of multi-source remote sensing domain bias. The adapter modules are positioned after the self-attention sublayer and / or feedforward network sublayer of the Transformer Block. Each adapter module includes at least: a first linear mapping layer that reduces the dimensionality of the input token features, an activation layer that performs a nonlinear transformation on the dimensionality reduction result, and a second linear mapping layer that increases the dimensionality of the nonlinear transformation result back to the original feature dimension. The output of the adapter module is added to the input token features as a residual to form the adapted token features. The first and second linear mapping layers constitute a bottleneck structure, with the bottleneck dimension being smaller than the token feature dimension.

[0016] Furthermore, the steps of performing scale normalization and block division on the queried image and the base image library image in sequence are as follows:

[0017] The first step is to scale the image to be queried proportionally according to a preset query resolution to balance the stability of feature representation and computational efficiency; the preset query resolution is 1m per pixel.

[0018] The second step is to divide the query image into several fixed-size query image blocks when the size of the scale-normalized query image exceeds a preset threshold; the preset threshold is that the image size is less than 224 pixels.

[0019] The third step is to generate a rotated version of each query image patch that contains at least 90°, 180° and 270°, and combine the original image patch with the rotated version to form the subsequent coarse search input set.

[0020] Furthermore, the ViT-adapter network includes: receiving input query image patches or background images. Figure 4The polygonal region is sliced ​​into several image patches and linearly embedded to obtain patch tokens. These are then superimposed with positional encodings and input into a pre-trained visual Transformer backbone formed by stacking multiple Transformer Blocks, outputting token-level features. Each Transformer Block contains a normalization layer, a multi-head self-attention layer, and a feedforward network layer, and at least some Transformer... An adapter module is inserted into the self-attention layer of the Block or its adjacent position. The adapter module is a bottleneck structure of "dimensional reduction linear mapping - nonlinear activation - dimensional increase linear mapping", and is added to the corresponding token in a residual manner to achieve multi-source remote sensing domain adaptation. The token-level features are rearranged into a two-dimensional feature map according to the spatial layout and then input into the attention pooling module. The attention pooling module generates a spatial attention score map through "1×1 convolution - ReLU - 1×1 convolution - Sigmoid", and performs element-wise weighting on the two-dimensional feature map, then performs weighted summation in the spatial dimension, and normalizes it with the attention score sum to obtain the image-level global descriptor vector. The image-level global descriptor vector is input into the cross-image encoder for context relationship modeling. The cross-image encoder is composed of a stacked structure of "normalization - multi-head self-attention - normalization - feedforward network" to output a global descriptor vector for coarse retrieval of visual position recognition.

[0021] Furthermore, the visual location of the query image includes: performing a similarity search between the global descriptor vector of the query image patch and the global descriptor vector of the base map in the feature library, so as to output the candidate base map slice sequence that is most similar to the query image patch at the scene level, thereby obtaining the coarse search visual location of the query image.

[0022] Furthermore, the construction of the multi-source substrate Figure 4 The visual location candidate set of the quadrilateral region slice refers to the visual location candidate set output by performing similarity retrieval on the query features and the base map library features; the multi-source base map regions corresponding to the visual location candidate set are cropped into quadrilateral region slices according to geographical boundaries to form a visual location candidate set for subsequent local feature matching and geometric verification.

[0023] Furthermore, the step of performing local feature matching on each query image patch and its corresponding visual location candidate set is as follows:

[0024] The first step is to process each pair of "query image patch - candidate base" in the visual location candidate set. Figure 4 "Rectangular region slicing" refers to slicing the query image patch and the candidate base. Figure 4 Key point detection is performed on the polygonal region slice to obtain a set of local feature points. A corresponding local descriptor is calculated for each local feature point to form a query descriptor set and a candidate descriptor set.

[0025] The second step is to perform nearest neighbor matching based on the query descriptor subset and the candidate descriptor subset to obtain an initial set of matching pairs.

[0026] The third step outputs the filtered set of matching point pairs as input for subsequent quadruple expansion, where each matching point pair contains the pixel coordinates (x, y) of the query image patch and the candidate base. Figure 4 The pixel coordinates (u, v) of the polygonal region slice.

[0027] Furthermore, the steps for initial screening of key regions based on the matching points through sequential mapping of quadruples and triple metrics are as follows:

[0028] The first step is to match the pixel coordinates (x, y) of each pair of query image patches with the candidate base... Figure 4 The pixel coordinates (u, v) of a polygonal region slice are expanded into a quadruple (x, y, X, Y), where (x, y) represents the pixel coordinates on the query image patch, and (X, Y) represents the candidate base. Figure 4 The corresponding coordinates of the polygonal region slices in a unified geographic coordinate system or a unified grid coordinate system are used as the sample set for geometric estimation by the quadruples.

[0029] The second step is to establish a regular grid in the candidate base map coordinate system and perform statistics based on the grid cells into which the quadruples fall; construct multiple 3×3 grid candidate regions based on the coarse search prediction position, the single grid position with the largest number of quadruples, and the region with the largest total number of quadruples in the 3×3 grid block.

[0030] The third step involves identifying the overlapping areas of multiple candidate regions as key regions, retaining only the quadruplets falling within these key regions for subsequent geometric estimation; when the number of quadruplets within a key region falls below a preset threshold, the current candidate region is skipped. Figure 4 Slice the polygonal region; otherwise, proceed to affine transformation estimation.

[0031] Furthermore, the step of performing sub-pixel-level fine registration at the coarse-positioned geographic location is as follows:

[0032] The first step involves robustly fitting the query image patch to the candidate base using a random sample consensus algorithm on the set of key region quadruples. Figure 4 The affine transformation matrix between the slices of the polygonal region is used to iteratively remove outliers and retain the set of inliers.

[0033] The second step is to use an affine transformation matrix to map the query image patch to the candidate base map coordinate system to obtain the approximate location result.

[0034] The third step is to use the approximate position result after affine alignment as the starting state, and perform sub-pixel level fine registration in the local area corresponding to the coarsely located geographical location to further correct residual translation, small rotation and local nonlinear deformation.

[0035] The fourth step, sub-pixel level fine registration, employs one or more of the following methods: phase correlation, pyramid optical flow, or mutual information-based registration.

[0036] The fifth step is to output the precise location of the query image in the geographic coordinate system, and simultaneously output the registration residual or confidence index for quality assessment.

[0037] The geolocation system for multi-source quadrilateral regions of the present invention is implemented based on a geolocation method for multi-source quadrilateral regions; the positioning system includes the following modules:

[0038] The multi-source remote sensing base map library construction and feature pre-computation module is used to integrate remote sensing image data from multiple sensors, multiple time phases, and multiple resolutions to construct a base map library with a unified geographic reference. It also utilizes a feature extraction network based on ViT and adapters for offline feature extraction and storage, providing domain-adaptive global descriptors and local features for subsequent retrieval.

[0039] The geographic location coarse localization module based on ViT-adaptor is used to receive query images, extract their global descriptors through the same ViT-adaptor network, and perform fast similarity matching and retrieval in the base map library using Euclidean distance to achieve preliminary location recognition in a large-scale scene, and output the best matching candidate reference image and its coarse-grained geographic range.

[0040] The quadruple mapping and key region segmentation module is used to extract and match local feature points of the query image and the reference image within the candidate region obtained by coarse localization, and bind the matching point pairs into quadruples (query pixel coordinates, reference image geographic coordinates); then, based on the superposition and filtering mechanism of triple metrics (prediction center, quadruple density, 3×3 grid matching number), high-confidence connected key regions are segmented from the candidate region for subsequent fine geometric modeling.

[0041] The affine transformation fitting module is used to fit the optimal affine transformation model between the query image and the base map based on the four-tuple matching point pairs within the selected key areas using robust estimation algorithms such as random sample consistency. This allows for the preliminary solution of the geometric projection parameters of the query image and a more accurate geolocation bounding box.

[0042] The subpixel fine registration module is used to perform local fine-tuning based on the initial alignment provided by affine transformation using subpixel-level registration algorithms (such as phase correlation, iterative optimization, etc.) to further improve the registration accuracy and finally output high-precision geolocation results at the pixel level or even subpixel level of the query image.

[0043] Compared with the prior art, the present invention has the following advantages:

[0044] First, this invention employs a multi-source remote sensing feature extraction network with a ViT-adaptor, forming a "VPR coarse retrieval visual location recognition—oriented towards multi-source background" model. Figure 4 The unified process of "candidate set of polygonal region slices - local geometric verification" overcomes the shortcomings of existing technologies that feature extraction relies on a single semantic region and leads to insufficient feature points in scenarios of occlusion / missegmentation / semantic sparsity. This invention enables the simultaneous use of multiple types of land features such as buildings, roads, and water bodies, improving robustness and candidate recall stability under multiple scenario conditions.

[0045] Second, the present invention in the candidate substrate Figure 4 Quadruple mapping and 3×3 grid triple measurement are performed on the polygonal region slices to divide the key regions. Explicit modeling of rotation, scale changes and multi-view geometric differences overcomes the problem that existing technologies based solely on area similarity and translation matching have weak geometric modeling capabilities and are difficult to adapt to complex deformations. As a result, it can handle more complex geometric relationships between multi-source, multi-resolution and multi-temporal images and has a wider range of applications.

[0046] Third, the present invention adopts a ViT backbone combined with an adapter structure, which overcomes the problems in the prior art that directly use a general pre-trained model, resulting in significant domain bias, or that comprehensive fine-tuning to adapt to new sources brings huge parameter numbers and computational overhead. This invention enables multi-source remote sensing domain adaptation without significantly increasing the number of model parameters, and has obvious advantages in retrieval accuracy and transfer performance. It is more suitable for deployment on resource-constrained UAVs, airborne and embedded platforms.

[0047] Fourth, the "VPR coarse retrieval visual position recognition + multi-source background" proposed in this invention Figure 4 The phased architecture of "key region division of the polygonal region slice + affine transformation + sub-pixel fine registration" realizes the gradual convergence from large-scale candidate locking to local sub-pixel registration, taking into account both computing speed and high-precision positioning requirements. It is suitable for engineering deployment and has better overall performance. Attached Figure Description

[0048] Figure 1 This is a flowchart of the method of the present invention;

[0049] Figure 2 This is a schematic diagram of the adapter of the present invention;

[0050] Figure 3 This is a schematic diagram of the feature aggregation network of the present invention;

[0051] Figure 4 This is a schematic diagram of the attention pooling module of the present invention;

[0052] Figure 5 This is a schematic diagram illustrating the principle of key area division in this invention;

[0053] Figure 6 This is a schematic diagram of the quadruple mapping principle of the present invention. Detailed Implementation

[0054] The present invention will now be described in more detail with reference to the accompanying drawings and embodiments.

[0055] Reference Figure 1 The specific implementation steps of the method embodiments of the present invention will be further described below.

[0056] Step 1: Construction of multi-source remote sensing base map library and feature pre-extraction of multi-source remote sensing domain adaptive adapter.

[0057] Step 1.1: Obtain multi-source remote sensing base map images (from different satellites / platforms, different resolutions, and different imaging angles / time phases), and establish a one-to-one correspondence between each base map and its geographic coordinates; the geographic coordinates can be latitude and longitude, or planar coordinates under a unified projection coordinate system.

[0058] Step 1.2 involves inputting the labeled base map into the feature extraction and aggregation network to obtain a fixed-length global descriptor vector, which is then written into the feature library along with the base map's geographic coordinates. (Refer to...) Figure 2 The network uses a pre-trained Visual Transformer (ViT) as its backbone and inserts lightweight adapter modules into at least part of the Transformer Block. The adapter adopts a bottleneck structure of "dimensionality reduction linear layer - nonlinear activation - dimension increase linear layer" and adds it to the corresponding token features in a residual manner, thereby achieving directional adaptation to the bias of multi-source remote sensing domain while freezing the backbone parameters as much as possible.

[0059] Reference Figure 3 The patch token-level features output by the ViT backbone are rearranged into a two-dimensional feature map representation according to their spatial layout, and then input into the feature aggregation module for image-level representation construction. The feature aggregation module preferably includes an attention pooling module and a cross-image encoder to improve the distinguishability and cross-source robustness of the global descriptor.

[0060] Reference Figure 4The attention pooling module performs adaptive weighted aggregation on the two-dimensional feature map: first, it generates a spatial attention score map through "1×1 convolution—ReLU—1×1 convolution—Sigmoid" to represent the contribution of each spatial location to the localization judgment; then, it weights the attention score with the two-dimensional feature map element by element to obtain weighted features, and performs weighted summation in the spatial dimension to obtain the pooled feature vector. At the same time, it sums the attention score in the spatial dimension to obtain a normalization factor. After adding a small constant to prevent division by zero, it normalizes the pooled feature vector to obtain the image-level global descriptor vector. Further, the image-level global descriptor vector is input into the cross-image encoder for context relationship modeling. The cross-image encoder is composed of a stacked structure of "normalization—multi-head self-attention—normalization—feedforward network" to enhance relationships and suppress bias among features in the same batch, so as to output a global descriptor vector for subsequent large-scale similarity retrieval.

[0061] Step 2: Query image scale normalization and block division.

[0062] Step 2.1: Scale the image to be queried proportionally according to the preset query resolution (constrained by the long side or the short side) to balance the stability of feature representation and computational efficiency.

[0063] Step 2.2: When the size of the query image is still too large, it is divided into several fixed-size image blocks. To enhance robustness to differences in viewpoint, 90° / 180° / 270° rotated versions can be generated for each image block, and the original image block and the rotated version can be used together as input for subsequent searches.

[0064] Step 3: Coarse-grained visual location recognition and candidate set construction based on ViT-adapter networks.

[0065] Step 3.1: Input each query image patch (including optional rotated version) into the feature extraction and aggregation network with the same structure as in Step 1.2 to obtain the corresponding global descriptor vector; since the network learns multi-source domain alignment capability through the adapter, it can improve the coarse retrieval recall and ranking stability.

[0066] Step 3.2: Perform similarity retrieval (e.g., Euclidean distance or cosine similarity) on the query features and the base map library features, and output the most similar visual location candidate base map tile sequence as the subsequent fine-fit candidate set, thereby quickly narrowing the search space from the entire library to a small number of candidates.

[0067] Step 3.3: For the multi-source base map region covered by the visual location candidate base map tile output in Step 3.2, determine the quadrilateral geographic boundary (defined by the coordinates of the four corner points) based on its corresponding georeferenced information, and then crop the candidate base map according to this geographic boundary. Figure 4 Slicing of polygonal regions; from all candidate bases Figure 4The polygonal region slices constitute a visual location candidate set for subsequent local feature matching and geometric verification.

[0068] Step 4: For each pair of "query image patch - candidate base" Figure 4 Local feature matching, quadruple mapping and triple metric screening of key regions for “border region slices”.

[0069] Step 4.1, for each pair of "query image patch - candidate base" in the visual location candidate set... Figure 4 "Slice the polygonal region" to extract local key points and local descriptors respectively; key point detection and description can be performed using methods such as SuperPoint, ORB or R2D2.

[0070] Step 4.2: Based on the query-side descriptor set and candidate-side descriptor set obtained in Step 4.1, perform nearest neighbor matching and optionally perform ratio testing / cross-validation to obtain a set of matched point pairs that pass the screening; wherein each matched point pair contains the pixel coordinates (x, y) of the query image patch and the candidate bottom edge descriptor set. Figure 4 The pixel coordinates (u, v) of the polygonal region slice.

[0071] Based on this, each pair of matching points is expanded into a quadruple (x, y, X, Y): where (x, y) are the pixel coordinates of the query image patch; (X, Y) are the coordinates of the candidate base. Figure 4 The pixel coordinates (u, v) of the polygonal region slice are converted by combining its geographic reference parameters, representing the corresponding coordinates of the point in a unified geographic coordinate system or a unified grid coordinate system; all four tuples constitute the sample set for subsequent key area screening and geometric estimation.

[0072] Step 5: Optimization of key regions and estimation of affine transformation based on 3×3 grid overlay verification.

[0073] Step 5.1: Establish a regular grid in the candidate base map coordinate system and perform statistics on the quadruplets according to the grid cells they fall into.

[0074] To nominate high-confidence candidate regions from discrete matching, refer to Figure 5 Three complementary region selection metrics are used:

[0075] Step 5.1.1: 3×3 grid area centered on the predicted location given by VPR coarse positioning (prior constraint);

[0076] Step 5.1.2: 3×3 grid region centered on the grid with the largest number of quadruplets in a single grid (local evidence strength).

[0077] Step 5.1.3: Select the region with the largest number of quadruplets within the 3×3 grid block as the candidate (introduce connectivity to suppress scattered spikes).

[0078] The overlapping areas of the above three types of regions are defined as key regions, and only quadruplets falling within the key regions are retained for subsequent geometric fitting.

[0079] Step 5.2: If the number of quadruplets in the key area exceeds the preset threshold, proceed to step 5.3; otherwise, skip the current candidate base map and continue processing the next candidate to avoid unstable mapping when there is insufficient evidence.

[0080] Step 5.3, refer to Figure 6 RANSAC is used to robustly estimate the affine transformation matrix between the query image and the candidate base map on the set of quadruples in the key region. Outliers are iteratively removed and the set of inliers is retained. The estimated affine transformation is used to map the query image into the base map coordinate system to obtain the approximate position result, which provides the initial alignment state for the next step of subpixel fine registration.

[0081] Step 6: Subpixel-level fine-grained registration and final geolocation output.

[0082] Step 6.1: Starting with the affine alignment result obtained in Step 5.3, perform sub-pixel-level fine registration within a local area to further correct residual translations, minor rotations, and local nonlinear deformations. Phase correlation, pyramid optical flow, or mutual information-based registration methods can be used. The final output is the precise location of the query image in the geographic coordinate system, and registration residuals / confidence scores can be output simultaneously for quality assessment.

Claims

1. A geolocation method for a multi-source quadrilateral region, characterized in that, The steps of this positioning method include the following: Step 1: Use the multi-source remote sensing base map library to pre-extract the multi-source remote sensing domain adaptive adapter features; Step 2: Perform scale normalization and block division on the queried image and the base image library images in sequence; Step 3: Based on the ViT-adapter network, the visual location of the query image is coarsely retrieved, and a candidate set of visual locations for quadrilateral region slices of multi-source base map is constructed. Step 4: Perform local feature matching on each query image patch and its corresponding visual location candidate set, and perform key region screening by quadruple mapping and triple measurement based on the matching points. Step 5: Perform affine transformation estimation on the selected quadruples to fit the coarse geolocation location; perform sub-pixel level fine registration at the coarse geolocation location to obtain the final geolocation output.

2. The geographic positioning method for a multi-source quadrilateral region according to claim 1, characterized in that, The steps described in step 1 for pre-extracting multi-source remote sensing domain adaptive adapter features using a multi-source remote sensing base map library are as follows: The first step is to acquire multi-source remote sensing base map images from different satellites or platforms, with different resolutions, imaging angles, or time phases, and to establish a one-to-one correspondence between each base map and its geographic coordinates to build a multi-source base map library. The second step involves inputting the base map image into a feature extraction and aggregation network to obtain a fixed-length global descriptor vector, and then writing the global descriptor vector and its corresponding geographic coordinates into a feature library. The feature extraction and aggregation network uses a pre-trained visual Transformer as its backbone, inserting adapter modules within at least a portion of the Transformer Block to achieve directional adaptation of multi-source remote sensing domain bias. The adapter modules are positioned after the self-attention sublayer and / or feedforward network sublayer of the Transformer Block. Each adapter module includes at least: a first linear mapping layer that reduces the dimensionality of the input token features, an activation layer that performs a nonlinear transformation on the dimensionality reduction result, and a second linear mapping layer that increases the dimensionality of the nonlinear transformation result back to the original feature dimension. The output of the adapter module is added to the input token features as a residual to form the adapted token features. The first and second linear mapping layers constitute a bottleneck structure, with the bottleneck dimension being smaller than the token feature dimension.

3. The geographic positioning method for a multi-source quadrilateral region according to claim 1, characterized in that, The steps in step 2, which involve scaling the queried image and dividing it into blocks, are as follows: The first step is to scale the image to be queried proportionally according to a preset query resolution to balance the stability of feature representation and computational efficiency; the preset query resolution is 1m per pixel. The second step is to divide the query image into several fixed-size query image blocks when the size of the scale-normalized query image exceeds a preset threshold; the preset threshold is that the image size is less than 224 pixels. The third step is to generate a rotated version of each query image patch that contains at least 90°, 180° and 270°, and combine the original image patch with the rotated version to form the subsequent coarse search input set.

4. The geographic positioning method for a multi-source quadrilateral region according to claim 3, characterized in that, Step 3 of the ViT-Adapter network includes: dividing the input query image patch or base image quadrilateral region slice into several image patches and performing linear embedding to obtain patch tokens; superimposing positional encoding and inputting it into a pre-trained visual Transformer backbone formed by stacking multiple Transformer Blocks; and outputting token-level features; wherein each Transformer Block contains a normalization layer, a multi-head self-attention layer, and a feedforward network layer, and at least some Transformer... An adapter module is inserted into the self-attention layer of the Block or its adjacent position. The adapter module is a bottleneck structure of "dimensional reduction linear mapping - nonlinear activation - dimensional increase linear mapping", and is added to the corresponding token in a residual manner to achieve multi-source remote sensing domain adaptation. The token-level features are rearranged into a two-dimensional feature map according to the spatial layout and then input into the attention pooling module. The attention pooling module generates a spatial attention score map through "1×1 convolution - ReLU - 1×1 convolution - Sigmoid", and performs element-wise weighting on the two-dimensional feature map, then performs weighted summation in the spatial dimension, and normalizes it with the attention score sum to obtain the image-level global descriptor vector. The image-level global descriptor vector is input into the cross-image encoder for context relationship modeling. The cross-image encoder is composed of a stacked structure of "normalization - multi-head self-attention - normalization - feedforward network" to output a global descriptor vector for coarse retrieval of visual position recognition.

5. The geographic positioning method for a multi-source quadrilateral region according to claim 1, characterized in that, Step 3, which involves determining the visual location of the query image, includes performing a similarity search between the global descriptor vector of the query image patch and the global descriptor vector of the base map in the feature library. This search outputs a sequence of candidate base map slices that are most similar to the query image patch at the scene level, thereby obtaining the coarse visual location of the query image.

6. The geographic positioning method for a multi-source quadrilateral region according to claim 1, characterized in that, The construction of the visual location candidate set for quadrilateral region slices of multi-source base map in step 3 refers to performing similarity retrieval on the query features and base map library features, outputting the visual location candidate set; and cropping the multi-source base map regions corresponding to the visual location candidate set into quadrilateral region slices according to geographical boundaries to form the visual location candidate set for subsequent local feature matching and geometric verification.

7. The geographic positioning method for a multi-source quadrilateral region according to claim 1, characterized in that, The steps in step 4, which involve performing local feature matching between each query image patch and its corresponding visual location candidate set, are as follows: The first step is to perform key point detection on each pair of "query image patch - candidate base image quadrilateral region slice" in the visual location candidate set, respectively, to obtain a set of local feature points. Then, calculate the corresponding local descriptor for each local feature point to form a query-side descriptor set and a candidate-side descriptor set. The second step is to perform nearest neighbor matching based on the query descriptor subset and the candidate descriptor subset to obtain an initial set of matching pairs. The third step outputs the set of matched point pairs that have been filtered as input for subsequent quadruple expansions, where each matched point pair contains the pixel coordinates (x, y) of the query image patch and the pixel coordinates (u, v) of the candidate base map quadrilateral region slice.

8. The geographic positioning method for a multi-source quadrilateral region according to claim 7, characterized in that, The steps in step 4 for initial screening of key regions based on matching points through quadruple mapping and triple measurement are as follows: The first step is to map and expand the pixel coordinates (x, y) of each query image patch to the pixel coordinates (u, v) of the candidate base map quadrilateral region slice into a quadruple (x, y, X, Y), where (x, y) represents the pixel coordinates on the query image patch and (X, Y) represents the corresponding coordinates of the candidate base map quadrilateral region slice in a unified geographic coordinate system or a unified grid coordinate system. The quadruple is used as the sample set for geometric estimation. The second step is to establish a regular grid in the candidate base map coordinate system and perform statistics based on the grid cells into which the quadruples fall; construct multiple 3×3 grid candidate regions based on the coarse search prediction position, the single grid position with the largest number of quadruples, and the region with the largest total number of quadruples in the 3×3 grid block. The third step is to identify the overlapping parts of multiple candidate regions as key regions and retain only the quadruplets that fall within the key regions for subsequent geometric estimation. When the number of quadruplets in the key regions is less than a preset threshold, the current candidate base map quadrilateral region slice is skipped; otherwise, affine transformation estimation is entered.

9. The geographic positioning method for a multi-source quadrilateral region according to claim 1, characterized in that, The steps for performing sub-pixel-level fine-tuning registration at the coarse-positioned geographic location described in step 5 are as follows: The first step is to use a random sample consensus algorithm to robustly fit the set of quadruples in the key regions to obtain the affine transformation matrix between the query image patch and the candidate base image quadrilateral region slice, and iteratively remove outliers and retain the set of inliers. The second step is to use an affine transformation matrix to map the query image patch to the candidate base map coordinate system to obtain the approximate location result. The third step is to use the approximate position result after affine alignment as the starting state, and perform sub-pixel level fine registration in the local area corresponding to the coarsely located geographical location to further correct residual translation, small rotation and local nonlinear deformation. The fourth step, sub-pixel level fine registration, employs one or more of the following methods: phase correlation, pyramid optical flow, or mutual information-based registration. The fifth step is to output the precise location of the query image in the geographic coordinate system, and simultaneously output the registration residual or confidence index for quality assessment.

10. A geographic positioning system for a multi-source quadrilateral region, characterized in that, This is achieved based on the geolocation method for a multi-source quadrilateral region as described in any one of claims 1-9; the positioning system includes the following modules: The multi-source remote sensing base map library construction and feature pre-computation module is used to integrate remote sensing image data from multiple sensors, multiple time phases, and multiple resolutions to construct a base map library with a unified geographic reference. It also utilizes a feature extraction network based on ViT and adapters for offline feature extraction and storage, providing domain-adaptive global descriptors and local features for subsequent retrieval. The geographic location coarse localization module based on ViT-adaptor is used to receive query images, extract their global descriptors through the same ViT-adaptor network, and perform fast similarity matching and retrieval in the base map library using Euclidean distance to achieve preliminary location recognition in a large-scale scene, and output the best matching candidate reference image and its coarse-grained geographic range. The quadruple mapping and key region segmentation module is used to extract and match local feature points of the query image and the reference image within the candidate region obtained by coarse localization, and bind the matching point pairs into quadruples (query pixel coordinates, reference image geographic coordinates); then, based on the superposition and filtering mechanism of triple metrics (prediction center, quadruple density, 3×3 grid matching number), high-confidence connected key regions are segmented from the candidate region for subsequent fine geometric modeling. The affine transformation fitting module is used to fit the optimal affine transformation model between the query image and the base map based on the four-tuple matching point pairs within the selected key areas using robust estimation algorithms such as random sample consistency. This allows for the preliminary solution of the geometric projection parameters of the query image and a more accurate geolocation bounding box. The subpixel fine registration module is used to perform local fine-tuning based on the initial alignment provided by affine transformation using subpixel-level registration algorithms (such as phase correlation, iterative optimization, etc.) to further improve the registration accuracy and finally output high-precision geolocation results at the pixel level or even subpixel level of the query image.

Images