A Ground-Based Visual Localization Method Based on Location Encoding Attention Mechanism and Statistical Descriptors
By employing a location-encoded attention mechanism and a statistical descriptor-based aerial visual localization method, the problems of low positioning accuracy in complex backgrounds and errors caused by changes in illumination and viewing angle were solved, achieving high-precision and interference-resistant visual localization results.
Patent Information
- Application Number
- CN202410667290.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-28
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-05-28
AI Technical Summary
Existing air-to-ground visual positioning methods have low positioning accuracy in complex and dynamic terrain backgrounds, and changes in lighting and viewing angles lead to large feature differences, resulting in mismatches and inaccurate positioning.
We employ a location-encoding attention mechanism and statistical descriptor-based approach. By enhancing the features of typical regions of an image, we fuse location encoding and appearance encoding using complex representations, combine a multi-head attention mechanism to learn the positional relationships between regions, and utilize statistical descriptors of GIST and LBP features to address the feature differences under varying illumination and viewpoints.
It improves positioning accuracy, enhances anti-interference capabilities, reduces measurement errors between features in the same scene, and improves the accuracy and robustness of visual positioning.
Smart Images

Figure CN118537598B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer technology, and more specifically relates to an aerial-to-ground visual localization method based on a positional encoding attention mechanism and statistical descriptors within the field of computer vision application technology. This invention is used to locate aerial targets within scenes captured in aerial ground images. Background Technology
[0002] Air-to-ground visual positioning serves as the foundation for tasks such as geographic mapping, target detection, building location, and disaster relief, playing an indispensable role in remote sensing fields such as UAVs and satellites. Currently, air-to-ground visual positioning methods transform the positioning problem into an image matching problem. That is, using existing remote sensing images as a reference, the image of the area to be located is matched with the remote sensing image, and the location corresponding to the area with the highest image similarity is the desired location.
[0003] Image-based visual localization can be divided into traditional methods and deep learning-based methods. Traditional image matching methods typically involve two steps: feature extraction and image matching using the extracted feature descriptors. Higher similarity between the feature descriptors of two images increases the matching probability. However, this method relies solely on pixel-level features, leading to severe mismatches and localization errors in complex backgrounds. Deep learning methods, on the other hand, do not rely on pre-designed feature extraction operators but are trained based on the inherent characteristics of the data. Convolutional neural network-based deep learning uses multiple templates with updatable parameters to perform convolution operations on the original image, obtaining multi-layer features, extracting deep semantic information from the image, and using this information for image matching. Attention-based deep learning methods directly segment the image into fixed-size image blocks, encode each block, and use an attention mechanism to measure the similarity of features in each block, obtaining salient regions of the overall image. Finally, matching is performed based on the similarity of features between different images to obtain the matching result.
[0004] The Information Engineering University of the Strategic Support Force of the Chinese People's Liberation Army disclosed a cross-view image matching localization method in its patent application, "A Cross-View Image Matching and Localization Method and System Guided by Feature Relationships" (Patent Application No.: CN202311349747.3, Publication No.: CN 117495956 A). This method constructs a cross-view image matching model based on Siamese neural networks, uses convolutional networks to extract features from the entire image, and trains the cross-view image matching model based on sample data and a preset loss function. It then performs image matching on the preprocessed target aerial remote sensing image and ground-view image, using the trained cross-view image matching model to achieve the localization task of querying panoramic ground images. This method has two shortcomings: First, it extracts features from the entire image, while local areas contain spatial topological relationships. Therefore, it only utilizes global information of the ground scene and does not perform positional correlation analysis on typical local features, resulting in insufficient localization accuracy. Second, the method does not perform saliency analysis on the extracted features, making it susceptible to clutter and mislocalization in complex backgrounds.
[0005] The 54th Research Institute of China Electronics Technology Group Corporation disclosed a method for UAV localization using aerial images and an IMU in its patent application "A Navigation and Positioning Method for Unmanned Aerial Vehicles" (Patent Application No.: CN202210142961.0, Authorization Announcement No.: CN 114509070 B). This method segments remote sensing images based on semantic segmentation to obtain the adaptation area and the topological relationships between various objects; it then uses aerial images and an IMU to perform UAV localization via a SLAM algorithm; it determines whether the UAV is within the adaptation area, corrects the aerial images, and performs semantic segmentation; it identifies the matching area and performs semantic matching; and upon successful matching, the localization result is obtained. The shortcomings of this method are: because semantic segmentation and viewpoint correction methods are greatly affected by lighting and scene conditions, the visual localization effect is poor when there are changes in lighting and viewpoint distortion in the area to be matched; and the use of an IMU sensor introduces inertial navigation errors, affecting the localization result. Summary of the Invention
[0006] The purpose of this invention is to address the shortcomings of existing technologies by proposing an air-ground visual positioning method based on a location-encoded attention mechanism and statistical descriptors. This method aims to solve the problems of low positioning accuracy caused by clutter interference and unutilized local features in complex dynamic terrain backgrounds, as well as low feature similarity in the same scene due to large feature differences under varying illumination and viewing angles.
[0007] The approach to achieving the objectives of this invention is as follows: This invention utilizes a positional encoding attention mechanism to enhance features in typical image regions. By encoding the appearance of each image region and fusing the positional encoding with complex representations, the attention mechanism performs appearance correlation analysis between each region's features and other regions, learning the positional relationships between different regions. This solves the problem of low positioning accuracy caused by clutter interference and unutilized topological relationships between local features in complex dynamic terrain backgrounds. Furthermore, this invention utilizes statistical descriptors of texture and edge features, employing feature extraction functions at different scales to obtain distribution functions of features at different scales within the scene. Based on these distribution functions, statistical features of the features are obtained. These statistical features are insensitive to feature changes caused by variations in illumination intensity distribution and viewing angle, thus addressing the problem of excessively large differences in image features for the same scene under varying viewing angles and illumination in existing methods.
[0008] The specific steps for implementing this invention include the following:
[0009] Step 1: At least 10 remote sensing images containing GPS information of the observation area acquired simultaneously and 1 image acquired by the airborne camera are combined to form an image set. After normalizing each image in the image set, image segmentation is performed, and all segmented images are combined into an image set.
[0010] Step 2: Extract multi-scale GIST and LBP features from each image in the image set, and perform statistical description and post-processing on the obtained GIST and LBP features to obtain statistical descriptors;
[0011] Step 3: Combine the image set with the statistical descriptors to form a training set;
[0012] Step 4: Construct a dual-branch feature extraction network that fuses positional encoding multi-head attention with statistical descriptors;
[0013] Step 5: Input each image in the training set into the first branch of the network, perform position encoding on the output vector of the multi-head self-attention layer in the first branch, input the corresponding statistical descriptor of the image into the second branch of the network, and output the feature vector of the image through the fusion module; use stochastic gradient descent to iteratively update the network parameters, and use cosine similarity as the metric for feature vectors of images with the same label, until the triplet loss function converges, and obtain the trained fused dual-branch feature extraction network;
[0014] Step 6: Using the same method as Step 1 and Step 2, obtain statistical descriptors for the image to be located and the remote sensing image containing GPS information of the observation area. Input the image to be located, the remote sensing image, and the statistical descriptors corresponding to each image into the trained feature extraction network, output a one-dimensional feature vector of the image, and compare the similarity between the feature vectors of the image to be located and the remote sensing image.
[0015] Step 7: Match the image to be located with the one whose feature vectors have the highest similarity with the remote sensing image. The location information contained in the matched remote sensing image is used as the visual location result.
[0016] Compared with existing technologies, the present invention has the following advantages:
[0017] First, because the present invention uses a location-encoded attention mechanism to enhance features in typical areas of an image and uses location coding expressed in complex numbers, it overcomes the shortcomings of low positioning accuracy caused by clutter interference and unutilized topological relationships between local features in complex dynamic terrain backgrounds. This makes the present invention have the advantages of learnable topological relationships between local features and strong anti-interference ability in dynamic and complex scenes, as well as high visual positioning accuracy.
[0018] Secondly, since the present invention utilizes statistical descriptors of GIST features and LBP features, it overcomes the shortcomings of existing methods that cause excessive differences in image features of the same scene under changes in viewpoint and illumination. This makes the present invention reduce the differences in features of the same scene and has the advantage of low measurement error between the same features. Attached Figure Description
[0019] Figure 1 This is a flowchart of the present invention;
[0020] Figure 2 This is a flowchart of the image segmentation method of the present invention;
[0021] Figure 3 This is a schematic diagram illustrating the GIST and LBP feature extraction principles of this invention. Figure 3 (a) is a schematic diagram of the multi-scale GIST feature extraction principle. Figure 3 (b) is a schematic diagram of LBP feature extraction principle;
[0022] Figure 4 This is a schematic diagram illustrating the working principle of the network fusion module of the present invention;
[0023] Figure 5 This is a schematic diagram illustrating the positional encoding attention operation principle of the present invention.
[0024] Figure 6 This is a flowchart of the attention mechanism after encoding in this invention. Detailed Implementation
[0025] The present invention will now be further described with reference to the accompanying drawings and embodiments.
[0026] Reference Figure 1 The implementation steps of the embodiments of the present invention will be further described below.
[0027] Figure 1As shown, the device of the present invention consists of a pair of twin networks with mutually exclusive parameters. Each network is composed of a first branch and a second branch connected in parallel and then connected in series with a fusion module.
[0028] Step 1: At least 10 remote sensing images containing GPS information of the observation area acquired simultaneously and 1 image acquired by the airborne camera are combined to form an image set. After normalizing each image in the image set, image segmentation is performed, and all segmented images are combined into an image set.
[0029] The following reference Figure 2 The image segmentation steps of the present invention will be described in further detail below.
[0030] The image segmentation described in the embodiments of the present invention is based on Figure 2 As shown, for each image acquired by optical remote sensing with GPS information at the image center and each image captured by an airborne camera, the original resolution of the image is 640×640. Each image is divided into 16 pictures according to the length and width being four equal parts, and the resolution of each picture is 160×160.
[0031] Step 2: Extract multi-scale GIST and LBP features from each image in the image set, and perform statistical description and post-processing on the obtained GIST and LBP features to obtain statistical descriptors.
[0032] The multi-scale GIST features are extracted by extracting the feature value of each element of each image according to the following formula, and then forming the feature matrix of the image by combining all the feature values:
[0033]
[0034] in, Let represent the feature value at (x, y) in the coordinate system with the top-left corner of the i-th image as the origin. `concat(·)` represents the concatenation operation. `I(x, y)` represents the grayscale value at (x, y) in the coordinate system with the top-left corner of the i-th image as the origin. `*` represents the convolution operation. -m The scale expansion factor is represented by σ, where a is an empirical constant (a>1), m represents the expansion value when the scale number is 6, π represents pi, and σ represents the circumference of a circle. x σ y Let x and y represent the standard deviations of the Gaussian function in the x and y coordinate directions, respectively; exp represents the exponential operation with the natural exponent e as the base; and x′ and y′ represent the coordinates of x and y after rotation transformation, respectively. This represents the k-th rotation angle, where k represents the index of the rotation angle, k = 0, 1, 2, ..., n-1, and the value of n is equal to the number of rotation directions. sin represents the sine operation, and cos represents the cosine operation. P represents the initial phase of the cosine function. iLet represent the set of all pixel positions in the i-th image.
[0035] The following reference Figure 3 (a) provides a further detailed description of the GIST feature extraction steps of the present invention.
[0036] Figure 3 In (a), each square on the left represents a pixel in the image. The square corresponding to the tail of the left arrow is the location of the GIST feature value to be determined. The number of the square arrows represents the rotation direction, and the arrows point in the positive direction of x′. The kernel function image in the middle represents the function. The left image multiplies the two segmentation scales with the three scaling scales of the intermediate kernel function to form an inflation value with m scale number 6. The right image shows the extracted multi-scale GIST features corresponding to the pixel location.
[0037] The LBP feature is extracted by extracting the feature value of each pixel in each image according to the following formula, and then forming the feature matrix of the image by combining all the feature values:
[0038]
[0039] Among them, LBP i (x c ,y c ) represents the center pixel (x) within the square window in the i-th image. c ,y c The feature value at position ) represents the 8 neighborhoods of the center pixel within the square window, where p represents the neighborhood index, and s(·) represents the sign function. I h I represents the grayscale value of the pixel in the h-th neighborhood within the square window. c Represents the center pixel (x) c ,y c The grayscale value at ().
[0040] The following reference Figure 3 (b) A further detailed description of the LBP feature extraction steps of the present invention is provided.
[0041] Figure 3 In (b), each square represents a pixel, and the number in the upper left square represents the gray value of the corresponding pixel. In this embodiment, the gray levels of the center pixel are compared with those of the neighboring pixels, and the binary LBP feature value corresponding to the pixel position is generated in the direction shown by the broken line arrow.
[0042] The statistical description and post-processing mentioned above refer to calculating the statistical descriptor of each image according to the following formula, thereby obtaining a statistical descriptor subset for each image:
[0043]
[0044] in, Let r represent the statistical descriptor of the r-th feature of the i-th image, where i = 0, 1, 2, ..., 15. Let R represent the statistical moments, where n represents the order of the moments, and R is the order of the moments. r U represents smoothness. r Indicating consistency, SD b This represents the statistical descriptor of the b-th image. Let represent the direct sum operation, and q represent the total number of features. Let μ represent the response value of the r-th feature in the i-th image, μ represent the mean feature response of the i-th image, and p(·) represent... The frequency of occurrence of a feature in the total number of features, μ2 represents the variance of the feature response.
[0045] Step 3: Combine the image set with the statistical descriptors to form a training set.
[0046] The training set refers to a sample set composed of Earth observation images collected by remote sensing satellites and airborne cameras in at least 200 different latitude and longitude regions. The statistical descriptor generated for each image in the sample set is labeled with the same latitude and longitude label for the images in the sample set and their corresponding statistical descriptors. All images in the sample set, their corresponding statistical descriptors and their labels are combined to form the training set.
[0047] The training set of this invention consists of 1652 satellite images and drone aerial images at different latitudes and longitudes obtained from the publicly available dataset University1652. Satellite images and drone images corresponding to the same location are labeled with the same label. 80% of the total number of scenes in the dataset are randomly selected as training set images, and the remaining 20% are used as test set images.
[0048] Step 4: Construct a dual-branch feature extraction network that fuses positional encoding multi-head attention with statistical descriptors.
[0049] The aforementioned fusion dual-branch feature extraction network is composed of a first branch and a second branch connected in parallel and then connected in series with a fusion module;
[0050] The first branch consists of 12 encoders with identical structures connected in series. Each encoder is composed of a first regularization layer, a multi-head attention layer, a dropout layer, a first residual connection layer, a second regularization layer, a multilayer perceptron layer, a dropout layer, and a second residual connection layer connected in series. The number of attention heads in the multi-head attention layer is set to 8. The deactivation ratio of the dropout layer is set to 0.2.
[0051] The second branch structure consists of a first convolutional layer, a batch normalization layer, and a second convolutional layer connected in series. The number of convolutional kernels in the first and second convolutional layers are set to 16 and 8, respectively, and the padding number is set to 1 and 0, respectively. The stride of the first convolutional layer is set to 1, and the output dimension of the second convolutional layer is 196×768.
[0052] The fusion module is composed of a splicing layer, a convolutional layer, a first fully connected layer, a Dropout layer, and a second fully connected layer connected in series; the output dimension of the first and second fully connected layers is set to 1024; and the deactivation ratio of the Dropout layer is set to 0.2.
[0053] The following reference Figure 4 The working principle of the fusion module of the present invention will be described in further detail below.
[0054] First, the statistical descriptor of the image GIST feature in the second branch of the module is concatenated with the LBP feature descriptor, and the output feature height and width are 196×768. Then, the concatenated feature is concatenated with the attention mechanism result of the first branch in the channel dimension. Subsequently, the concatenated feature is passed through a convolutional layer, a fully connected layer, a Dropout layer and a fully connected layer in sequence, and the fully connected layer outputs a 1×1024 feature vector.
[0055] Step 5: Input each image in the training set into the first branch of the network. For each image region, perform position encoding operation through the output vector of the multi-head self-attention layer in the first branch. Input the corresponding statistical descriptor of the image into the second branch of the network. Output the feature vector of the image through the fusion module. Iteratively update the network parameters using the stochastic gradient descent method, using cosine similarity as the metric for feature vectors of images with the same label, until the triplet loss function converges, and obtain the trained fused dual-branch feature extraction network.
[0056] The following reference Figure 5 The positional encoding attention workflow of the present invention will be described in further detail below.
[0057] Figure 5 Using the top-left corner of the image as the origin, and each square as an image region, positional encoding is added to the attention vector of each image region. First, appearance attention vectors Q, K, and V are obtained for each image region X. Then, the Q and K vectors are encoded using... Figure 5 Position encoding is performed in the coordinate system shown, and the encoded vector at each position differs from the vector before encoding only by a constant c, i.e.
[0058]
[0059] in, This represents the encoded attention vector at any position.
[0060] The location encoding operation is obtained according to the following formula:
[0061]
[0062] in, Let (m1, n1) and (m2, n2) represent the positional encoding attention vectors output by the multi-head attention layer of the first branch in the network, respectively, located at coordinates (m1, n1) and (m2, n2). W represents the vectorization operations of the image regions at coordinates (m1, n1) and (m2, n2), respectively. q W k Let represent the learnable parameter matrix, j represent the imaginary unit sign, and arctan(·) represent the arctangent operation.
[0063] The following reference Figure 6 The process of the post-encoding attention mechanism is described in further detail.
[0064] The first step is to obtain the location-encoded... Vector, where Positional Embedding in the diagram represents the positional encoding operation, and V represents the AND operation. Value vectors with the same coordinates.
[0065] The second step involves using the softmax function to perform pairwise operations on each image region. The output of any attention mechanism is as follows:
[0066]
[0067] d k This represents the normalization parameter, which is set to 64 in this embodiment to avoid gradient explosion during network training due to an excessively large softmax function.
[0068] The triplet loss function described in the embodiments of the present invention is as follows:
[0069]
[0070] Wherein, Triplet loss represents the loss value of the true labels in the network training sample set, max represents the maximum value operation, (·) represents the inner product operation, ‖·‖ represents the Euclidean norm, a represents the feature vector of the image to be localized, p represents the positive sample feature vector, n represents the negative sample feature vector, and margin represents the feature separation distance. In this embodiment, margin = 0.5 is taken. During the network training process, the parameters of each layer of the network are updated by backpropagation by optimizing the distance between the a vector and the positive sample feature vector to be less than the cosine distance of the negative sample feature vector.
[0071] Step 6: Using the same method as Step 1 and Step 2, obtain statistical descriptors for the image to be located and the remote sensing image containing GPS information of the observation area. Input the image to be located, the remote sensing image, and the statistical descriptors corresponding to each image into the trained feature extraction network, output a one-dimensional feature vector of the image, and compare the similarity between the feature vectors of the image to be located and the remote sensing image.
[0072] Step 7: Match the image to be located with the one whose feature vectors have the highest similarity with the remote sensing image. The location information contained in the matched remote sensing image is used as the visual location result.
[0073] The effectiveness of this invention can be further demonstrated through the following simulation.
[0074] 1. Simulation experimental conditions.
[0075] The hardware experimental platform of this invention is: processor: Intel i7-13700KF, graphics card: NVIDIA GeForce RTX 4090.
[0076] The software platform for the simulation experiment of this invention is: Ubuntu 20.04 operating system and PyCharm 23.02.
[0077] The simulation experiments of this invention were implemented on a twin image feature extraction network built in PyTorch using Python 3.11. The main software versions used were Python 3.11, CUDA 12.2, and PyTorch 1.11.0.
[0078] 2. Simulation content and result analysis.
[0079] The simulation experiment of this invention involves inputting the dataset generated in the step embodiment of this invention into networks trained using this invention and two existing technologies for testing.
[0080] The two existing technologies used in the simulation experiment are:
[0081] The VGG and ResNet-50 methods used by Zheng et al. in their paper "University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localization[J].arXivpreprint arXiv:2002.12186,2020"
[0082] The CBAM, DenseNet, and ViT methods used by Zhu et al. in their paper "SUES-200: A Multi-Height Multi-Scene Cross-View Image Benchmark Across Drone and Satellite, IEEE Transactions on Circuits and Systems for Video Technology, vol.33, no.9, pp.4825-4839, Sept.2023"
[0083] To verify the simulation effect of this invention, two evaluation metrics (matching accuracy R@1 and accuracy AP) were used to evaluate the classification results of the four methods. Using the following formulas, matching accuracy R@1, accuracy AP, and classification accuracy for 16 land cover classes, all calculation results are plotted in Table 1:
[0084] Matching accuracy
[0085] accuracy
[0086] Table 1. Comparison of positioning accuracy results from different methods
[0087]
[0088] As can be seen from Table 1, compared with the baseline method ResNet-50, the present invention can significantly improve the matching accuracy and precision. Moreover, compared with the ViT method, the present invention can improve the matching accuracy while reducing the number of model parameters, indicating that the method has achieved good results.
[0089] The simulation experiments above demonstrate that the proposed air-to-ground visual localization method based on location encoding attention mechanism and statistical descriptors achieves higher matching accuracy and lower metric error while having lower computational cost. This is the advantage of this invention over other current methods.
Claims
1. A spatial visual localization method based on location encoding attention mechanism and statistical descriptors, characterized in that, The design incorporates a dual-branch feature network to extract image feature vectors, enhances image features through a complex positional encoding attention mechanism, and introduces statistical descriptors of GIST and LBP features to extract image features. The steps of this localization method are as follows: Step 1: At least 10 remote sensing images containing GPS information of the observation area acquired simultaneously and 1 image acquired by the airborne camera are combined to form an image set. After normalizing each image in the image set, image segmentation is performed, and all segmented images are combined into an image set. Step 2: Extract multi-scale GIST and LBP features from each image in the image set, and perform statistical description and post-processing on the obtained GIST and LBP features to obtain statistical descriptors; Step 3: Combine the image set with the statistical descriptors to form a training set; Step 4: Construct a fusion dual-branch feature extraction network consisting of position-encoded multi-head attention and statistical descriptors; Step 5: Input each image in the training set into the first branch of the network. Perform position encoding operation on the output vector of the multi-head self-attention layer in the first branch for each image. Input the corresponding statistical descriptor of the image into the second branch of the network. Output the feature vector of the image through the fusion module. Iteratively update the network parameters using the stochastic gradient descent method, using cosine similarity as the metric for feature vectors of images with the same label, until the triplet loss function converges, and obtain the trained fused dual-branch feature extraction network. Step 6: Using the same method as Step 1 and Step 2, obtain statistical descriptors for the image to be located and the remote sensing image containing GPS information of the observation area. Input the image to be located, the remote sensing image, and the statistical descriptors corresponding to each image into the trained feature extraction network, output a one-dimensional feature vector of the image, and compare the similarity between the feature vectors of the image to be located and the remote sensing image. Step 7: Match the image to be located with the one whose feature vectors have the highest similarity with the remote sensing image. The location information contained in the matched remote sensing image is used as the visual location result.
2. The aerial visual localization method based on location encoding attention mechanism and statistical descriptors according to claim 1, characterized in that, The image segmentation mentioned in step 1 refers to dividing each image into 16 pictures by dividing the optical remote sensing image with GPS information at the image center and each image captured by the airborne camera into four equal parts in length and width.
3. The aerial visual localization method based on location encoding attention mechanism and statistical descriptors according to claim 1, characterized in that, The multi-scale GIST features described in step 2 are extracted by using the following formula to extract the feature value of each element in each image, and then combining all the feature values to form the feature matrix of the image: ; ; ; in, Indicates the first In the coordinate system formed by the top left corner of the image, it is located at... eigenvalues at that location This indicates a splicing operation. Indicates the first In the coordinate system formed by the top left corner of the image, it is located at... grayscale value at that location This represents the convolution operation. Indicates the scale expansion factor. Represents an empirical constant. , This represents the expansion value with a scale number of 6. Represents pi (π). These represent the standard deviations of the Gaussian function in the x and y coordinate directions, respectively. Indicated by the natural index Exponential operations with base 0. They represent Coordinates after rotation transformation Indicates the first One rotation angle, The number indicating the rotation angle. , The value of is equal to the number of rotation directions. Represents the sine operation. Represents the cosine operation. This represents the initial phase of the cosine function. Indicates the first The set of all pixel locations in an image.
4. The aerial visual localization method based on location encoding attention mechanism and statistical descriptors according to claim 3, characterized in that, The LBP feature mentioned in step 2 is to extract the feature value of each pixel in each image according to the following formula, and then form the feature matrix of the image by combining all the feature values: ; in, Indicates the first The center pixel of the square window in the image eigenvalues at that location This represents the 8 neighborhoods of the center pixel within a square window. Indicates the index of the neighborhood. Symbolic functions: , Indicates the number of elements within the square window. The grayscale value of a pixel in a neighborhood. Represents the center pixel The grayscale value at that location.
5. The aerial visual localization method based on location encoding attention mechanism and statistical descriptors according to claim 4, characterized in that, The statistical description and post-processing mentioned in step 2 refers to calculating the statistical descriptor of each image according to the following formula, thereby obtaining a statistical descriptor subset for each image: ; ; ; ; ; in, Indicates the first The first picture Statistical descriptors of bit features =0,1,2,…,15 Represents statistical moments, Indicates the order of the moment. Indicates smoothness, Indicates consistency. Indicates the first Statistical descriptors of images, Indicates direct and operation, This represents the total number of features. Indicates the first Image number 1 The response value of the bit feature, Indicates the first The mean feature response of the images. express Frequency of occurrence in the total number of features This represents the variance of the characteristic response.
6. The aerial visual localization method based on location encoding attention mechanism and statistical descriptors according to claim 5, characterized in that, The training set mentioned in step 3 refers to a sample set composed of Earth observation images collected by remote sensing satellites and airborne cameras in at least 200 different latitude and longitude regions. The corresponding statistical descriptor generated for each image in the sample set is labeled with the same latitude and longitude label for the images in the sample set and their corresponding statistical descriptors. All images in the sample set, their corresponding statistical descriptors and their labels are combined to form a training set.
7. The aerial visual localization method based on location encoding attention mechanism and statistical descriptors according to claim 1, characterized in that, The fusion dual-branch feature extraction network described in step 4 is composed of a first branch and a second branch connected in parallel and then connected in series with the fusion module; The first branch consists of 12 encoders with identical structures connected in series. Each encoder is composed of a first regularization layer, a multi-head attention layer, a dropout layer, a first residual connection layer, a second regularization layer, a multilayer perceptron layer, a dropout layer, and a second residual connection layer connected in series. The number of attention heads in the multi-head attention layer is set to 8. The deactivation ratio of the dropout layer is set to 0.
2. The second branch structure consists of a first convolutional layer, a batch normalization layer, and a second convolutional layer connected in series. The number of convolutional kernels in the first and second convolutional layers are set to 16 and 8, respectively; the stride is set to 1 and 0, respectively; and the padding is set to 1 and 0, respectively. The output dimension of the second convolutional layer is 196×768. The fusion module is composed of a splicing layer, a convolutional layer, a first fully connected layer, a Dropout layer, and a second fully connected layer connected in series; the output dimension of the first and second fully connected layers is set to 1024; and the deactivation ratio of the Dropout layer is set to 0.
2.
8. The spatial visual localization method based on location encoding attention mechanism and statistical descriptors according to claim 5, characterized in that, The position encoding operation described in step 5 is obtained according to the following formula: ; ; in, , These represent the appearance attention vectors output by the multi-head attention layer of the first branch in the network, located at coordinates... and Location code at the location. Representing coordinates Image vectorization operations This represents the learnable parameter matrix. The symbol representing the imaginary unit. This indicates the arctangent operation.
Citation Information
Patent Citations
A method for navigation and positioning of unmanned aerial vehicle
CN114509070B
Cross-view-angle image matching and positioning method and system based on feature relation guidance
CN117495956A
Multi-target detection method based on context information
CN106446933A
An optical remote sensing image ship detection method based on a depth residual dense network
CN109740460A
