Feature extraction method for cross-view image positioning and related equipment

Abstract

The invention provides a cross-view image positioning feature extraction method and related equipment, and relates to the technical field of cyberspace surveying and mapping. The method comprises the following steps: 1, converting a satellite image into a ground view angle image; 2, inputting the ground view angle image into a feature extractor, and extracting local features; step 3, inputting the local features into a global spatial relationship perception attention module, learning spatial positions and structural information of the features, and generating attention weighted features; and step 4, aggregating and representing the attention weighted features as global features representing the whole image by using a Sinkhorn algorithm of a local aggregation vector. According to the invention, the robustness and generalization of the cross-view image positioning feature extraction model can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cyberspace mapping, and in particular to a feature extraction method for cross-viewing image positioning and related equipment. Background Art

[0002] Image-based geolocation refers to the technology of retrieving matching images from a pre-built image database containing location information for a given query image, thereby determining the geographic location of the query image. It is usually expressed as an instance-level image retrieval problem, which has important application value in autonomous driving and navigation, cyberspace situational awareness, and ubiquitous geographic information perception on the Internet. Traditional methods usually use ground street view images with geographic coordinates to form a database to achieve image positioning, but street view images are mainly concentrated in urban areas. For areas such as suburbs and rural areas that lack sufficient reference images, it is impossible to build a large-scale reference image database, and the scope of application is limited. Satellite images have the advantages of wide coverage and simple acquisition, and are an ideal data source for image geolocation. However, due to the huge differences in perspective and scale between ground-based images and satellite images, the research and application of cross-perspective image positioning based on satellite images are huge challenges.

[0003] Early methods used manual features to achieve cross-view image positioning, but the inherent limitations of manual features led to poor positioning results. With the rapid development of deep learning technology, various cross-view positioning methods represented by convolutional neural networks and Vision Transformer have become the mainstream of current research. Deep learning-based methods mainly focus on learning discriminative image representations to optimize the distance of image feature vectors in feature space. The core lies in a robust and efficient feature representation method. The image features obtained by this method can effectively deal with the difficulty of cross-view positioning caused by perspective differences and scale differences. In recent years, scholars at home and abroad have conducted extensive research on the perspective difference problem and feature robustness problem in cross-view positioning.

[0004] Existing deep learning methods mainly improve the effect of cross-view image localization by mining and utilizing the global context information of the image, carefully designing the backbone network for feature extraction, introducing feature aggregation modules and regional alignment strategies, etc. Attention mechanisms and multi-scale feature fusion methods are also widely used in cross-view image localization. CVM-Net combines the twin network architecture with NetVLAD to achieve the purpose of learning viewpoint invariant features in cross-view localization. CVFT establishes feature transformation between different image domains by introducing optimal transmission to increase the reliability of cross-view localization. GeoCapsNet uses capsule networks to learn the spatial hierarchical relationship between different objects in the image, thereby obtaining feature expressions of geometric invariance and viewpoint invariance. GeoDTR separates geometric information from the original features and proposes an enhanced geometric layout extractor module to learn the spatial correlation between visual features from ground-air image pairs. TransGeo takes advantage of Transformer's advantages in global information modeling and explicit position information encoding to model the global correlation of image features.

[0005] A lot of work has shown that in cross-view image localization, a robust and efficient feature expression method is effective and necessary. However, the existing methods mentioned above do not adequately mine the spatial position structure information in images and do not fully consider the positional relationship information of the features and the information redundancy of the features themselves. Summary of the invention

[0006] For cross-view image positioning, in order to improve the robustness and generalization of feature expression methods, the present invention provides a feature extraction method and related equipment for cross-view image positioning, which utilizes global spatial relationship attention to learn the spatial position structure information of features, and utilizes the Sinkhorn algorithm of local aggregation vectors to selectively discard useless information in the process of feature clustering and allocation, thereby improving the generalization and robustness of descriptors.

[0007] In a first aspect, the present invention provides a feature extraction method for cross-view image positioning, comprising:

[0008] Step 1: Convert satellite images to ground-level images;

[0009] Step 2: Input the ground view image into a feature extractor to extract local features;

[0010] Step 3: Input the local features into the global spatial relationship perception attention module, learn the spatial position and structural information of the features, and generate attention weighted features;

[0011] Step 4: Use the Sinkhorn algorithm of the local aggregation vector to aggregate the attention-weighted features into global features representing the entire image.

[0012] Further, step 1 specifically includes: converting the satellite image into a ground view image using polar coordinates according to the following formula;

[0013]

[0014] in, and Represent the pixel coordinates of the satellite image and the transformed image, respectively, s ×H s and W ps ×H ps Represent the sizes of satellite images and converted images respectively.

[0015] Furthermore, in step 2, the process of constructing the feature extractor includes:

[0016] The last convolution layer and classification layer in the ConvNeXt network pre-trained on ImageNet are deleted to obtain the feature extractor.

[0017] Further, in step 3, the global spatial relationship perception attention module specifically includes a grouped convolutional layer, a spatial position attention module and a structural relationship attention module;

[0018] Correspondingly, the generation process of the attention weighted feature specifically includes: using the grouped convolutional layer to divide the local feature into multiple sub-features in the channel dimension; for each sub-feature, using the spatial position attention module to generate the attention scores of the sub-feature in the horizontal and vertical directions respectively; for each sub-feature, using the structural relationship attention module to generate the attention score of the relative spatial relationship information of the sub-feature; for each sub-feature, weighting it using all its corresponding attention scores to obtain the corresponding grouped attention weighted feature; splicing all the grouped attention weighted features in the channel dimension to obtain the attention weighted feature.

[0019] Furthermore, for each sub-feature, the spatial position attention module is used to generate the attention scores of the sub-feature in the horizontal direction and the vertical direction, specifically including:

[0020] Pair feature F i Average pooling is performed in the horizontal and vertical directions respectively, the two average pooling results are concatenated along the height direction, and then the concatenated feature vector is input into a 1×1 convolution. The output of the 1×1 convolution is then separated into feature vectors in the horizontal and vertical directions. Finally, the Sigmoid function is used to obtain the attention scores in the horizontal and vertical directions respectively.

[0021] Furthermore, for each sub-feature, the structural relationship attention module is used to generate an attention score of the relative spatial relationship information of the sub-feature, specifically including:

[0022] Generate the spatial relationship matrix R between all feature elements in the sub-feature s ; Wherein, each element in the spatial relationship matrix is ​​used to represent the spatial relationship between two corresponding feature elements in the sub-feature, and the calculation formula of the spatial relationship is:

[0023]

[0024] Among them, x i and x j Represents two feature elements in the sub-feature; θ s and Represents two spatial information encoding functions consisting of 1×1 convolution, batch normalization and ReLU nonlinear activation functions;

[0025] Generate a global spatial relationship vector for each feature element in the sub-feature based on the spatial relationship matrix; wherein the feature element x i The global spatial relationship vector r i for:

[0026] r i =(r i,j ,r j,i )=[R s (i,:),R s (:,i)]

[0027] Among them, [] indicates splicing in the channel dimension;

[0028] The global spatial relationship vector of each feature element is input into a 1×1 convolution, and then the Sigmoid function is used to obtain the attention score of the relative spatial relationship information of each feature element.

[0029] Furthermore, step 4 specifically includes:

[0030] The attention weighted feature is equally divided into several local features according to the set length, each local feature is reduced in dimension using a learnable dimensionality reduction function, an initial score matrix of each reduced local feature is calculated using a learnable score function, and the initial score matrix is ​​expanded in columns to obtain an expanded score matrix Among them, the expanded columns are used to allocate useless information features using a learnable parameter z;

[0031] Set μ = 1 n , The Sinkhorn algorithm is used to iteratively Regularize the rows and columns in , find the optimal transmission allocation between μ and κ, and finally delete κ to obtain the optimal allocation matrix in, n represents the number of local features, m represents the hyperparameter of the number of cluster categories C; 1 n Represents a column vector of length n and value 1; Represents a row vector with a length of m and a value of 1;

[0032] According to the allocation relationship in the optimal allocation matrix P, the features are directly aggregated into the clusters to which they are assigned to obtain an aggregated vector matrix, which is then L2 regularized and flattened to obtain global features representing the entire image.

[0033] In a second aspect, the present invention provides a feature extraction system for cross-view image positioning, comprising:

[0034] An image perspective conversion module, used to convert satellite images into ground perspective images;

[0035] A feature extractor, used to extract local features of the input ground perspective image;

[0036] A global spatial relationship-aware attention module, used to learn the spatial position and structural information of features from the local features to generate attention-weighted features;

[0037] The feature aggregation module is used to use the Sinkhorn algorithm of the local aggregation vector to aggregate the attention-weighted features into global features representing the entire image.

[0038] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0039] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect.

[0040] Beneficial effects of the present invention:

[0041] The present invention utilizes the spatial position structure information of the learning feature using global spatial relationship attention, which can fully enhance the perception and discrimination capabilities of the model; and, due to the use of grouped convolution, attention has almost no effect on the number of parameters and complexity of the model. In addition, the present invention represents the aggregation of local features as an optimal transmission problem, and uses the Sinkhorn algorithm of local aggregation vectors to selectively discard useless information during the clustering and allocation of features, thereby improving the generalization and robustness of the descriptor. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic flow chart of a feature extraction method for cross-view image positioning provided by an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of converting a satellite image into a ground-view image using polar coordinates provided by an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of the basic principle of the Global Spatial Relationship Awareness Attention (GSRA) module provided in an embodiment of the present invention;

[0045] Figure 4 A schematic diagram of the structure of a feature extraction system for cross-view image positioning provided by an embodiment of the present invention;

[0046] Figure 5 A structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] like Figure 1 As shown, an embodiment of the present invention provides a feature extraction method for cross-view image positioning, comprising the following steps:

[0049] S101: converting the satellite image into a ground-view image;

[0050] Specifically, in cross-view image positioning, the query image is usually a ground panoramic image, while the database image is usually a satellite image (including satellite aerial images or satellite remote sensing images). There are huge differences in perspective and scale between the two. In order to reduce this difference, it is necessary to convert the satellite image into a ground perspective image.

[0051] S102: Inputting the ground view image into a feature extractor to extract local features;

[0052] Specifically, the feature extractor in this embodiment (i.e., the backbone network in the figure) is constructed based on the ConvNeXt network, and the construction process mainly includes: deleting the last convolution layer and classification layer in the ConvNeXt network pre-trained on ImageNet to obtain the feature extractor. The feature extractor can retain the performance of the original model to the greatest extent while greatly reducing the number of parameters and the amount of calculation of the model.

[0053] For a given input image After the ConvNeXt extractor, a local feature F∈R will be obtained. c×h×w , where h×w is the spatial resolution and c is the number of channels.

[0054] S103: Input the local features into the global spatial relationship perception attention module, learn the spatial position and structural information of the features, and generate the attention weighted feature F a ;

[0055] S104: Using the Sinkhorn Algorithm for Locally Aggregated Descriptors (SALAD) to aggregate the attention weighted features into a global feature F representing the entire image g .

[0056] The feature extraction method provided by the embodiment of the present invention utilizes the spatial position structure information of the feature learned by global spatial relationship attention, which can fully enhance the perception and discrimination capabilities of the model; in addition, the aggregation of local features is represented as an optimal transmission problem, and the Sinkhorn algorithm of the local aggregation vector is used to selectively discard useless information in the process of feature clustering and allocation, thereby improving the generalization and robustness of the descriptor.

[0057] In one embodiment, the present invention uses polar coordinates to achieve the conversion of satellite image perspective to ground image perspective. Figure 2 As shown in the figure, the horizontal direction in the ground panoramic image has an approximate depth, which corresponds to the concentric circles with the image center as the center in the satellite image (the red dotted line in the figure). The depth in the ground image also increases along the vertical line upward, which corresponds to the radial direction with the image center as the origin in the aerial image (the blue solid line with an arrow in the figure). Due to this correspondence in spatial layout, the satellite image can be transformed into polar coordinates to obtain an image that is roughly aligned with the spatial layout of the ground perspective.

[0058] Specifically, corresponding to the image pixel position, the center of the satellite aerial image is set as the polar coordinate origin; the north direction (upward direction) of the aerial image is used as the polar coordinate 0° angle direction. The height of the image after polar coordinate conversion is set to the height of the ground image, and each column of the image after polar coordinate conversion is arranged horizontally as the ground image, so that its width is consistent with the ground image. A uniform sampling strategy is used along the radial lines in the satellite image so that the innermost and outermost layers of the aerial image are mapped to the bottom and top lines of the converted image, respectively. Polar coordinate conversion can be expressed by the following formula:

[0059]

[0060] In the formula, and Represent the pixel coordinates of the original satellite image and the image after polar coordinate conversion, W s ×H s and W ps ×H ps Represent the sizes of the original satellite image and the converted image respectively.

[0061] Through the calculation of the above formula, the circular lines in the satellite remote sensing image are converted into horizontal lines in the ground perspective image, the radial lines are converted into vertical lines, and the north direction line (i.e., the radial line with the center of the satellite image as the origin upward) is converted to the horizontal direction W in the converted image. ps / 2 corresponds to the vertical line.

[0062] In one embodiment, in order to enable the network model to more centrally learn the viewpoint invariant features in the image, and further utilize the feature position information and the relative position relationship between the features to improve the robustness of the model, the present invention proposes a lightweight global spatial relation attention module (GSRA) based on coordinate attention. The GSRA module adopts grouped convolution, which fully enhances the perception and discrimination capabilities of the model while having almost no effect on the number of parameters and complexity of the model.

[0063] The basic structure of GSRA is as follows Figure 3 As shown, for the feature F∈R obtained by the feature extractor c×h×w , the embodiment of the present invention first divides F into g sub-features in the channel dimension, F = [F1, F i ,…,F g ],F i ∈R c / / g×h×w , where “ / / ” means rounding down; then, for the sub-feature F i The spatial dimension of F is averaged horizontally and vertically. iThe encoding vector z of each feature x at height h h It can be expressed as:

[0064]

[0065] Similarly, we can get the encoding vector z of feature x at width w w for:

[0066]

[0067] By averaging pooling in the horizontal and vertical directions in space, GSRA can capture the spatial position information of features in these two directions. Then, the present invention converts the feature vector z h and z w The image is spliced ​​along the height direction, and the spliced ​​feature vector is input into a 1×1 convolution, so that the horizontal position information and the vertical position information can interact, and then F is obtained. i Finally, the output of the 1×1 convolution needs to be separated and restored into feature vectors in the horizontal and vertical directions, and the Sigmoid function is used to obtain the attention score of the spatial position information.

[0068] Although the above operation can obtain the spatial position information of the feature, it cannot learn the global structural relationship between image features and lacks the ability to reason about the global context. Therefore, another part of GSRA obtains the global structural relationship between features by calculating the relationship matrix. Specifically, for the sub-feature F i ∈R c / / g×h×w Any two features x i and x j The spatial relationship between i,j It can be expressed by the following formula:

[0069]

[0070] In the formula, θ s and represents two spatial information encoding functions consisting of 1×1 convolution, batch normalization and ReLU nonlinear activation function, sub-feature F i The spatial relationship matrix between all features in R can be used s ∈R hw×hw At the same time, due to the existence of the spatial information encoding function, the feature x i The global spatial relationship vector r i It should be expressed as x i and x j A bidirectional relationship between:

[0071]

[0072] The specific operation is to R s (i,:) and R s (:,i) is concatenated in the channel dimension, and then a 1×1 convolution and Sigmoid function are used to obtain the attention score f about the relative spatial relationship information of the input features. r .

[0073] Broadcast all attention scores obtained by GSRA to sub-features F i The grouped attention weighted feature F can be obtained ia , input feature F = [F1,F i ,…,F g ]’s attention-weighted feature F a ∈R c×h×w That is [F 1a ,F ia ,…,F ga ]Re-stitching and combination in the channel dimension.

[0074] Based on the above embodiment, this embodiment provides a specific aggregation process. Specifically, first, for the feature descriptor F obtained in the previous step, a , which is regarded as a set of local features {t1,…,t i ,…,t n}, where n = h × w, t i represents a vector of length c; for local features, this embodiment uses a learnable dimensionality reduction function F(t) to reduce the dimension of the input feature. After the dimensionality reduction function F(t) is processed, the local feature t i The dimension of R c Change to R d , d is a hyperparameter for controlling the length of the vector after clustering, and in the present invention, the value of d is 64. The expression of the dimension reduction function F(t) is as follows:

[0075] F(t)=W2(σ(W1(t)+b1))+b2 (6)

[0076] Where W1 and W2 represent 1×1 convolution, σ represents the ReLU nonlinear activation function, and b1 and b2 represent the inductive bias.

[0077] Then, this embodiment uses a score function S(t) to calculate the local feature t i The initial score matrix S∈R n×m , m represents the hyperparameter of the number of cluster categories C, which is 128 in this paper, and the element s in S i,j Represents the feature t i Assigned to each cluster C jThe score function S(t) consists of a set of learnable convolutional layers, and its expression is as follows:

[0078] S(t)=W2(σ(W1(t)+b1'))+b'2 (7)

[0079] Where b1' and b'2 represent the inductive bias.

[0080] Next, the embodiment of the present invention uses a learnable parameter z to allocate useless information features, so that useless information is allocated to z during the learning process of the model. To this end, the score matrix S needs to be expanded to Specifically:

[0081]

[0082] In the formula, 1 n Represents a column vector of length n and value 1.

[0083] The embodiment of the present invention reformulates the allocation of features as an optimal transmission problem, where the allocation of features μ=1 n Must be effectively distributed across clusters or collections of useless information In the example, the Sinkhorn algorithm is used to obtain the allocation matrix

[0084]

[0085] The Sinkhorn algorithm works by iteratively Regularize the rows and columns in the equation, find the optimal transmission allocation between μ and κ, and finally delete the useless information set to obtain the optimal allocation matrix

[0086] Next, according to the distribution relationship in the optimal distribution matrix P, the features are directly aggregated into the clusters to which they are assigned:

[0087]

[0088] In the formula, k represents the vector t i The kth dimension of .

[0089] Finally, the aggregated vector matrix V∈R m×d L2 regularization is performed and flattened to obtain a global descriptor representing the entire image.

[0090] like Figure 4 As shown, an embodiment of the present invention provides a feature extraction system for cross-view image positioning, including an image view conversion module, a feature extractor, a global spatial relationship perception attention module and a feature aggregation module.

[0091] Among them, the image perspective conversion module is used to convert satellite images into ground perspective images; the feature extractor is used to extract local features of the input ground perspective image; the global spatial relationship perception attention module is used to learn the spatial position and structural information of features from the local features to generate attention weighted features; the feature aggregation module is used to use the Sinkhorn algorithm of local aggregation vectors to aggregate the attention weighted features into global features representing the entire image.

[0092] It should be noted that the system provided in the embodiment of the present invention is for implementing the above method, and its specific functions can be referred to the above method embodiments, which will not be described again here.

[0093] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 communicate with each other through the communication bus 504. The processor 501 may call the logic instructions in the memory 503 to execute the feature extraction method for cross-view image positioning, the method comprising: step 1: converting the satellite image into a ground view image; step 2: inputting the ground view image into a feature extractor to extract local features; step 3: inputting the local features into a global spatial relationship perception attention module to learn the spatial position and structural information of the features and generate attention weighted features; step 4: using the Sinkhorn algorithm of the local aggregation vector to aggregate the attention weighted features into global features representing the entire image.

[0094] In addition, when the logic instructions in the above-mentioned memory 503 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0095] An embodiment of the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example, including: Step 1: Converting a satellite image into a ground perspective image; Step 2: Inputting the ground perspective image into a feature extractor to extract local features; Step 3: Inputting the local features into a global spatial relationship perception attention module to learn the spatial position and structural information of the features, and generate attention weighted features; Step 4: Using the Sinkhorn algorithm of the local aggregation vector, the attention weighted features are aggregated to represent the global features of the entire image.

[0096] An embodiment of the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the methods provided by the above-mentioned method embodiments are implemented, for example, including: Step 1: Converting a satellite image into a ground perspective image; Step 2: Inputting the ground perspective image into a feature extractor to extract local features; Step 3: Inputting the local features into a global spatial relationship perception attention module to learn the spatial position and structural information of the features and generate attention weighted features; Step 4: Using the Sinkhorn algorithm of the local aggregation vector, the attention weighted features are aggregated to represent the global features of the entire image.

[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A feature extraction method for cross-view image positioning, characterized in that: include: Step 1: Convert satellite images to ground-level images; Step 2: Input the ground view image into a feature extractor to extract local features; Step 3: Input the local features into the global spatial relationship perception attention module, learn the spatial position and structural information of the features, and generate attention weighted features; Step 4: Use the Sinkhorn algorithm of the local aggregation vector to aggregate the attention-weighted features into global features representing the entire image.

2. The feature extraction method for cross-view image positioning according to claim 1, characterized in that: Step 1 specifically includes: converting the satellite image into a ground view image using polar coordinates according to the following formula; in, and Represent the pixel coordinates of the satellite image and the transformed image, respectively, s ×H s and W ps ×H ps Represent the sizes of satellite images and converted images respectively.

3. The feature extraction method for cross-view image positioning according to claim 1, characterized in that: In step 2, the construction process of the feature extractor includes: The last convolution layer and classification layer in the ConvNeXt network pre-trained on ImageNet are deleted to obtain the feature extractor.

4. The feature extraction method for cross-view image positioning according to claim 1, characterized in that: In step 3, the global spatial relationship perception attention module specifically includes a grouped convolutional layer, a spatial position attention module and a structural relationship attention module; Correspondingly, the generation process of the attention weighted feature specifically includes: using the grouped convolutional layer to divide the local feature into multiple sub-features in the channel dimension; for each sub-feature, using the spatial position attention module to generate the attention scores of the sub-feature in the horizontal and vertical directions respectively; for each sub-feature, using the structural relationship attention module to generate the attention score of the relative spatial relationship information of the sub-feature; for each sub-feature, weighting it using all its corresponding attention scores to obtain the corresponding grouped attention weighted feature; splicing all the grouped attention weighted features in the channel dimension to obtain the attention weighted feature.

5. The feature extraction method for cross-view image positioning according to claim 4, characterized in that: For each sub-feature, the spatial position attention module is used to generate the attention scores of the sub-feature in the horizontal and vertical directions, specifically including: Pair feature F i Average pooling is performed in the horizontal and vertical directions respectively, the two average pooling results are concatenated along the height direction, and then the concatenated feature vector is input into a 1×1 convolution. The output of the 1×1 convolution is then separated into feature vectors in the horizontal and vertical directions. Finally, the Sigmoid function is used to obtain the attention scores in the horizontal and vertical directions respectively.

6. The feature extraction method for cross-view image positioning according to claim 4, characterized in that: The structure relationship attention module is used to generate the attention score of the relative spatial relationship information of each sub-feature, specifically including: Generate the spatial relationship matrix R between all feature elements in the sub-feature s ; Wherein, each element in the spatial relationship matrix is ​​used to represent the spatial relationship between two corresponding feature elements in the sub-feature, and the calculation formula of the spatial relationship is: Among them, x i and x j Represents two feature elements in the sub-feature; θ s and Represents two spatial information encoding functions consisting of 1×1 convolution, batch normalization and ReLU nonlinear activation function; Generate a global spatial relationship vector for each feature element in the sub-feature based on the spatial relationship matrix; wherein the feature element x i The global spatial relationship vector r i for: r i =(r i,j ,r j,i )=[R s (i,:),R s (:,i)] Among them, [] indicates splicing in the channel dimension; The global spatial relationship vector of each feature element is input into a 1×1 convolution, and then the Sigmoid function is used to obtain the attention score of the relative spatial relationship information of each feature element.

7. The feature extraction method for cross-view image positioning according to claim 1, characterized in that: Step 4 specifically includes: The attention weighted feature is equally divided into several local features according to the set length, each local feature is reduced in dimension using a learnable dimensionality reduction function, an initial score matrix of each reduced local feature is calculated using a learnable score function, and the initial score matrix is ​​expanded in columns to obtain an expanded score matrix Among them, the expanded columns are used to allocate useless information features using a learnable parameter z; Set μ = 1 n , The Sinkhorn algorithm is used to iteratively Regularize the rows and columns in , find the optimal transmission allocation between μ and κ, and finally delete κ to obtain the optimal allocation matrix in, n represents the number of local features, m represents the hyperparameter of the number of cluster categories C; 1 n Represents a column vector of length n and value 1; Represents a row vector with a length of m and a value of 1; According to the allocation relationship in the optimal allocation matrix P, the features are directly aggregated into the clusters to which they are assigned to obtain an aggregated vector matrix, which is then L2 regularized and flattened to obtain global features representing the entire image.

8. A feature extraction system for cross-view image positioning, characterized in that: include: An image perspective conversion module, used to convert satellite images into ground perspective images; A feature extractor, used to extract local features of the input ground perspective image; A global spatial relationship-aware attention module, used to learn the spatial position and structural information of features from the local features to generate attention-weighted features; The feature aggregation module is used to use the Sinkhorn algorithm of the local aggregation vector to aggregate the attention-weighted features into global features representing the entire image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images