Cross-visual-angle geographic positioning method based on biological visual perception

Abstract

The invention provides a cross-view geographic positioning method based on biological visual perception, and belongs to the technical field of computer vision. Comprising the following steps: acquiring a ground view image and an aviation view image, and performing feature extraction; carrying out local feature modeling and global feature modeling on the ground image features and the aerial image features respectively, splicing the local features and the global features, and carrying out weighted fusion to obtain enhanced ground image feature representation and enhanced aerial image feature representation; further processing the enhanced ground image feature representation and the enhanced aerial image feature representation through a self-adaptive attention enhancement module to generate a self-adaptive attention enhanced ground image feature representation and a self-adaptive attention enhanced aerial image feature representation, and projecting the self-adaptive attention enhanced ground image feature representation and the self-adaptive attention enhanced aerial image feature representation into a shared feature space; and cross-view image matching and positioning are carried out by calculating the cosine similarity between the images. According to the method, a fusion strategy based on a bionic vision mechanism is adopted, and the feature expression ability and the matching precision of the cross-view-angle image are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to the technical field of cross-view geolocation, and more particularly to a cross-view geolocation method based on biological visual perception. BACKGROUND

[0002] With the development of remote sensing technology and unmanned aerial vehicle technology, cross-view geolocation has become an important application scenario in computer vision. In many applications, ground view images and aerial view images have significant view angle differences. How to efficiently extract useful information from these images and match them has always been a research focus.

[0003] In the prior art, many methods attempt to solve this problem by using traditional image matching methods, but these methods usually cannot effectively capture the view angle differences between ground view images and aerial view images, resulting in low accuracy of the matching results. Especially under different viewing angles, local information and global context information of images often cannot be effectively modeled at the same time, so an efficient method is needed that can comprehensively process local features and global features of images. SUMMARY

[0004] In view of the technical problem of low accuracy of cross-view geolocation based on existing methods, a cross-view geolocation method based on biological visual perception is provided. The present application uses bionic visual mechanism, synchronously combines local detail information and global structure information, and introduces adaptive attention enhancement to query semantic enhancement and spatial structure modeling of the attention mechanism, which can improve the matching accuracy of cross-view geolocation.

[0005] The technical means adopted by the present application are as follows: A cross-view geolocation method based on biological visual perception, comprising the following steps: Obtain ground view images and aerial view images, use a transformer encoder network with a twin structure to extract image features of the ground view images and the aerial view images, and then gradually extract multi-level feature representations of the ground view images and the aerial view images through a multi-layer transformer structure, so as to obtain ground image features and aerial image features; Model the local features and global features of the ground image features and the aerial image features, respectively, wherein the local features extract regional information in the image block through a sliding window mechanism, and the global features extract context information of the entire image through a pooling operation, the local features and the global features are spliced and weighted fused to obtain enhanced ground image feature representation and enhanced aerial image feature representation; The adaptive attention enhancement module is further used for processing the enhanced ground image feature representation and the enhanced aerial image feature representation, to generate an adaptive attention enhanced ground image feature representation and an adaptive attention enhanced aerial image feature representation, wherein the focusing ability on the image region is enhanced by introducing a learnable query vector, and the spatial structure perception ability is enhanced by introducing a position identifier to generate a spatial bias. The adaptive attention enhanced ground image feature representation and the adaptive attention enhanced aerial image feature representation are projected into a shared feature space, and the matching and positioning between the images are performed by calculating the cosine similarity between the images.

[0006] Further, the local feature extracts the region information in the image block through a sliding window mechanism, including: performing local feature extraction on the image input, dividing each image into blocks through a sliding window, and extracting the local feature in each image block through a window with a size of Finally, the local feature representation covering the image is obtained by moving the window step by step.

[0007] Further, the global feature extracts the context information of the whole image through a pooling operation, including: performing global feature extraction on the image input, downsampling the feature map of the image through average pooling to extract the global structure feature of the image, and further processing the global structure feature obtained by the pooling operation through projection and activation to compress and convert it into a global feature representation for feature fusion.

[0008] Further, the local feature and the global feature are spliced and weightedly fused, including calculating the importance of the similarity of the query keyword for the local feature representation and the global feature representation through the same softmax, to realize the coupling of the two paths.

[0009] Further, the adaptive attention enhancement module processes the image feature through the following steps: A learnable query vector is introduced, and a learnable query is added in all query tokens; a position perception mechanism is constructed, a set of learnable tokens is used to replace the key in the sliding window path, the learnable token interacts with the query to generate a dynamic position bias, the focus of the attention mechanism is adjusted according to the input feature, and the understanding of the spatial structure of the image is optimized.

[0010] Further, the adaptive attention enhancement module is further used for processing the enhanced ground image feature representation and the enhanced aerial image feature representation, including using a transformer-based network architecture, and combining local-global fusion and adaptive enhancement, and the overall network can be divided into four stages: In the first stage, the image is input, a convolution kernel with a size of 7 and a step of 4 is used for image blocking, the attention head is 24, the number of attention layers is 5 layers, the number of channels changes from 3 to 96, and the size of the output feature map changes to one fourth of the input; In the second stage, a convolution kernel with a size of 3 and a step of 2 is used for feature map blocking, the attention head is 24, the number of attention layers is 5 layers, the number of channels changes from 96 to 192, and the size of the output feature map changes to one eighth of the input; In the third stage, a convolution kernel with a size of 3 and a step of 2 is used for feature map blocking, the attention head is 24, the number of attention layers is 23 layers, the number of channels changes from 192 to 384, and the size of the output feature map changes to one sixteenth of the input; In the fourth stage, a convolution kernel with a size of 3 and a step of 2 is used for feature map blocking, the attention head is 24, the number of attention layers is 5 layers, the number of channels changes from 384 to 768, and the size of the output feature map changes to one thirty-second of the input.

[0011] Further, the adaptive attention enhanced ground image feature representation and the adaptive attention enhanced aerial image feature representation are projected into a shared feature space, and the matching and positioning of cross-view images are performed by calculating the cosine similarity between images, including measuring the similarity between images by calculating the cosine similarity between images, and matching and sorting according to the similarity of image features.

[0012] Compared with the prior art, the present application has the following advantages: The present application provides a cross-view geographic positioning method based on biological visual perception, which comprises: in the feature extraction process, a local-global attention fusion is constructed based on a local sliding window and a global pooling, a biological visual perception attention is combined, and each label on the feature map has global perception by simulating biological foveal vision and continuous eye movement, so that cross-view matching is realized in a way of mainly matching the center and supplementing the global structural information, and the position deviation of the reference object caused by the view angle difference is compensated. The adaptive attention enhancement is introduced to enhance the semantic focus ability and structure adaptation ability of the attention mechanism. The local-global attention fusion is spliced and normalized after modeling the local and global attention respectively. The adaptive attention enhancement enhances the semantic focus ability and structure adaptation ability of the attention mechanism by introducing a learnable query vector and a position identifier. The present application adopts a fusion strategy based on bionic visual mechanism, effectively improves the feature expression ability and matching precision of cross-view images. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to make the technical solution of the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0014] Figure 1 The model network architecture applied to the cross-view geographical positioning method based on biological visual perception in the embodiments of the present application.

[0015] Figure 2 The polar coordinate transformation method used in the embodiments of the present application. DETAILED DESCRIPTION

[0016] In order to make the technical solution of the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned accompanying drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0018] As Figure 1 shown, the present application provides a cross-view geographical positioning method based on biological visual perception, comprising: S1, acquisition of input images and pre-processing of aerial images.

[0019] In the present embodiment, the input images include ground view images and aerial view images, both of which are images with significant view angle difference. The ground view images are usually obtained by ground shooting devices, while the aerial view images are obtained by satellite, unmanned aerial vehicle and the like. In order to reduce the influence of view angle difference, the input aerial images are first subjected to polar coordinate transformation. As Figure 2As shown, this transformation converts the image from the traditional Cartesian coordinate system to the polar coordinate system, thereby reducing the distortion caused by the change of viewpoint and making the extraction of image features more robust.

[0020] S2. Perform feature extraction on the ground image to obtain ground features, and simultaneously perform feature extraction on the aerial image to obtain aerial features.

[0021] This invention employs a Siamese network architecture, using two identical branches to extract features from ground and aerial images respectively. Taking the ground image branch as an example, this method, based on self-attention, adds local-global feature extraction and adaptive attention enhancement to each layer of self-attention. The overall network can be divided into four stages. In the first stage, the input ground and aerial images are segmented using convolutional kernels of size 7 and stride 4, with 24 attention heads, 5 attention layers, and the number of channels increasing from 3 to 96. The output feature map size becomes one-quarter of the input. In the second stage, feature map segmentation is performed using convolutional kernels of size 3 and stride 2, with 24 attention heads, 5 attention layers, and the number of channels increasing from 96 to 192. The output feature map size becomes one-eighth of the input. In the third stage, feature map segmentation is performed using convolutional kernels of size 3 and stride 2, with 24 attention heads, 23 attention layers, and the number of channels increasing from 192 to 384. The output feature map size becomes one-sixteenth of the input. In the fourth stage, a convolutional kernel with a size of 3 and a stride of 2 is used to divide the feature map into blocks. The attention head is 24, the number of attention layers is 5, the number of channels increases from 384 to 768, and the size of the output feature map becomes 1 / 32 of the input. The exact same architecture is used in the aerial image branch.

[0022] S3. Perform local-global feature modeling on the ground image and simultaneously perform local-global feature modeling on the aerial image.

[0023] A local-global feature extraction network is used to extract features for each modality separately. Taking a ground-based input image as an example, the extracted image features will be subjected to local and global feature extraction respectively. In the local feature extraction part, this invention uses a sliding window mechanism to divide the input feature map into several small blocks. A sliding window of varying size scans the feature map, performing self-attention calculations on local feature maps within the window to extract information from local regions. By traversing the feature map through the sliding window, local features are extracted at each location. For global feature extraction, this invention employs parameter-free adaptive average pooling to extract global structural information from the feature map. Average pooling averages the pixel values ​​of the feature map, focusing on the global structural information of the image. To prevent significant information loss during average pooling, a single-layer neural network is used for projection and activation before pooling, pre-compressing and extracting useful information, improving the information compression rate after downsampling. After extracting local and global features, this invention combines them through weighted fusion to obtain an enhanced image feature representation. The local and global features are then concatenated. Local features are image patch features extracted through the sliding window mechanism, while global features are contextual information extracted through average pooling. These two feature sets are concatenated into a long vector, preserving a complete representation of both local and global information. Based on the concatenated features, softmax is used to further integrate the information from both. The importance of query keyword similarity results for two paths is calculated within the same softmax function, achieving coupling between local and global features after concatenation. The purpose of softmax is to enhance the expressive power of features by fostering competition between fine-grained and coarse-grained features. The features fused by softmax can comprehensively capture both local details and global structural information in the image.

[0024] S4. Adaptive attention enhancement for the extracted features.

[0025] To further enhance the representation of image features, this invention employs adaptive attention enhancement. Specifically, this can be divided into two aspects: a query enhancement mechanism and a position-aware mechanism. Building upon the original query-key-value attention mechanism, learnable vectors are added to enhance the focusing ability and spatial awareness of image regions. Specifically, a learnable query vector is introduced, adding a learnable query to all query tokens. The model's applicability to cross-view tasks is optimized by training task-specific query vectors. A position-aware mechanism is constructed. In the sliding window path, in addition to the original transformer's key-value pairs, a set of learnable tokens is used to replace the keys. These learnable tokens interact with the query, generating a dynamic positional bias based on the input image. The focus of the attention mechanism is adjusted according to the input features, optimizing the understanding of the image's spatial structure.

[0026] S5. Project the enhanced ground image features and aerial image features into a shared feature space, and calculate the similarity between the images using cosine similarity. After local-global feature modeling and adaptive attention enhancement, we obtain the enhanced feature representation of each image. These features contain local details and spatial structure information in the image. To make image features from different viewpoints comparable, we map the features of ground view images and aerial view images into a shared feature space, so that image features from different viewpoints have the same dimension and feature representation. Specifically, after obtaining the ground and aerial features, we first pass them through two fully connected layers and a GLUE activation function, and then through a fully connected layer to obtain the final descriptor. For the descriptors Fground and Fsatlite of the ground view image and aerial view image, we calculate the cosine similarity between them. A matching score is generated based on the calculated cosine similarity. A high similarity score indicates a high matching degree between the two images, and a low similarity score indicates a low matching degree. We can sort the images according to the similarity score to determine the best matching image pair.

[0027] This invention addresses the limitations of existing technologies by proposing an image feature enhancement method based on a local-global attention fusion mechanism and an adaptive attention enhancement module. Through local and global fusion, this invention can simultaneously model both local and global contextual information of an image, resolving the data spatial misalignment between ground and aerial images caused by viewpoint differences and ensuring high matching accuracy across various perspectives. Furthermore, the adaptive enhancement method improves the information aggregation capability of features. Compared to traditional methods, this invention not only improves the accuracy of cross-viewpoint image matching but also exhibits strong robustness, adapting to the challenges of different environments and viewpoint variations, thus demonstrating broad application prospects.

[0028] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cross-perspective geolocation method based on biological visual perception, characterized in that, Includes the following steps: Ground view images and aerial view images are acquired. Image features are extracted from the ground view images and aerial view images using a twin-structure transformer coding network. Then, multi-level feature representations of the ground view images and aerial view images are extracted step by step through a multi-layer transformer structure, thereby obtaining ground image features and aerial image features. Local and global features are modeled for ground image features and aerial image features, respectively. Local features are extracted from the region information in the image patch through the sliding window mechanism, while global features are extracted from the context information of the entire image through pooling operation. The local and global features are concatenated and weighted to obtain the enhanced ground image feature representation and the enhanced aerial image feature representation. The enhanced ground image feature representation and enhanced aerial image feature representation are further processed by the adaptive attention enhancement module to generate adaptive attention enhanced ground image feature representation and adaptive attention enhanced aerial image feature representation. The ability to focus on image regions is enhanced by introducing learnable query vectors, and the ability to perceive the spatial structure of the image is enhanced by introducing location identifiers to generate spatial bias. The adaptive attention-enhanced ground image feature representation and the adaptive attention-enhanced aerial image feature representation are projected into a shared feature space, and cross-view image matching and localization are performed by calculating the cosine similarity between images.

2. The cross-view geolocation method based on biological visual perception according to claim 1, characterized in that, Local feature extraction utilizes a sliding window mechanism to extract region information from image patches. This includes: extracting local features from the image input; dividing each image into patches using a sliding window with a size of [missing information]. The window extracts local features within each image patch. By progressively moving the window to extract multiple regions, a local feature representation of the covered image is finally obtained.

3. The cross-view geolocation method based on biological visual perception according to claim 2, characterized in that, Global features are extracted from the contextual information of the entire image through pooling operations, including: extracting global features from the image input, downsampling the image feature map through average pooling, extracting the global structural features of the image, further processing the global structural features obtained by pooling by projection and activation, compressing and transforming them into a global feature representation for feature fusion.

4. The cross-view geolocation method based on biological visual perception according to claim 3, characterized in that, The local and global features are concatenated and weighted, including using the same softmax to calculate the importance of the similarity of query keywords for both the local and global feature representations, thereby coupling the two paths.

5. The cross-view geolocation method based on biological visual perception according to claim 1, characterized in that, The adaptive attention enhancement module processes image features through the following steps: Introduce learnable query vectors and add a learnable query to all query tokens; A position-aware mechanism is constructed. In the sliding window path, a set of learnable tokens is used instead of keys. The learnable tokens interact with the query to generate a dynamic position bias. The focus of the attention mechanism is adjusted according to the input features to optimize the understanding of the image spatial structure.

6. The cross-view geolocation method based on biological visual perception according to claim 1, characterized in that, Further processing of enhanced ground image feature representations and enhanced aerial image feature representations is performed through an adaptive attention enhancement module. This includes employing a transformer-based network architecture and combining local-global fusion and adaptive enhancement. The overall network can be divided into four stages: In the first stage, the image input is divided into blocks using a convolutional kernel with a size of 7 and a stride of 4. The attention head is 24, the number of attention layers is 5, the number of channels is changed from 3 to 96, and the size of the output feature map is reduced to one-quarter of the input. In the second stage, a convolutional kernel with a size of 3 and a stride of 2 is used to divide the feature map into blocks. The attention head is 24, the number of attention layers is 5, the number of channels is changed from 96 to 192, and the size of the output feature map is reduced to one-eighth of the input. In the third stage, a convolutional kernel with a size of 3 and a stride of 2 is used to divide the feature map into blocks. The attention head is 24, the number of attention layers is 23, the number of channels is changed from 192 to 384, and the size of the output feature map is reduced to one-sixteenth of the input. In the fourth stage, a convolutional kernel with a size of 3 and a stride of 2 is used to divide the feature map into blocks. The attention head is 24, the number of attention layers is 5, the number of channels is changed from 384 to 768, and the size of the output feature map is reduced to one-thirty-second of the input.

7. The cross-view geolocation method based on biological visual perception according to claim 1, characterized in that, The adaptive attention-enhanced ground image feature representation and the adaptive attention-enhanced aerial image feature representation are projected into a shared feature space. Cross-view image matching and localization are performed by calculating the cosine similarity between images. This includes measuring the similarity between images by calculating the cosine similarity between images and ranking the matching based on the similarity of image features.

Images