Unmanned aerial vehicle training-free layered visual positioning method facing GNSS denial environment

Abstract

The invention relates to an unmanned aerial vehicle training-free layered visual positioning method oriented to a GNSS denial environment. The method comprises the following steps: performing feature extraction on a satellite image and a query image by using a pre-trained large model to obtain a feature map; the method comprises the following steps: modeling an unmanned aerial vehicle pose estimation problem into a task of searching a maximum response value in a discretized four-dimensional pose likelihood body, applying affine transformation to a feature map corresponding to a query image, generating a corresponding query template for each pose hypothesis, and performing efficient matching with a feature map of a satellite image to obtain a query result; a four-dimensional pose likelihood body of four dimensions is constructed; and positioning the pose with the strongest response in the four-dimensional pose likelihood, taking the strongest pose as the center of finer search of the next level, and gradually converging to an accurate solution according to an iteration mode to obtain the final pose of the unmanned aerial vehicle. The method can improve the visual positioning efficiency and precision of the unmanned aerial vehicle.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of visual positioning, in particular to a GNSS-denied environment-oriented unmanned aerial vehicle (UAV) training-free hierarchical visual positioning method. BACKGROUND

[0002] UAV visual positioning based on an overhead image refers to estimating the 4 degrees of freedom (latitude, longitude, altitude and heading angle) of an unmanned aerial vehicle (UAV) through visual information only under the guidance of a satellite map without relying on a global navigation satellite system (GNSS). This capability provides a reliable and flexible solution for UAV navigation, environmental monitoring and many other scenarios. Current mainstream methods usually adopt a “search-cascade matching” framework to complete UAV visual positioning. First, candidate satellite tiles are searched, and then a fine matching is performed to estimate the pose. However, the great difference in scale and rotation between satellite images and query images poses a severe challenge to the robustness of existing algorithms. A direct solution is to construct an image database containing multi-scale and multi-rotation information, but this undoubtedly incurs huge preprocessing and storage costs. Although some people have recently begun to explore template matching methods based on learning models, they usually need to be trained for specific tasks and lack cross-scene generalization ability, resulting in low positioning efficiency and accuracy. SUMMARY

[0003] Therefore, it is necessary to provide a GNSS-denied environment-oriented UAV training-free hierarchical visual positioning method capable of improving the positioning efficiency and accuracy of an unmanned aerial vehicle (UAV).

[0004] A GNSS-denied environment-oriented UAV training-free hierarchical visual positioning method, the method comprising: obtaining a satellite image and a query image; performing feature extraction on the satellite image and the query image by using a pre-trained large model to obtain a feature map; modeling the UAV pose estimation problem as a task of finding the maximum response value in a discretized four-dimensional likelihood body, applying an affine transformation to the feature map corresponding to the query image, generating a corresponding query template for each pose hypothesis, and constructing a four-dimensional pose likelihood body of four dimensions by performing efficient matching with the feature map of the satellite image; finding the strongest pose in the four-dimensional pose likelihood body by positioning, taking the strongest pose as the center of a more fine search in the next level, and gradually converging to an accurate solution according to an iterative manner to obtain the final UAV pose.

[0005] The aforementioned training-free hierarchical visual localization method for UAVs in GNSS-denied environments first extracts feature maps from satellite and query images based on a pre-trained large model. This fully utilizes the rich semantic and detailed features learned by the model to enhance feature matching capabilities between images, laying the foundation for accurate matching and reducing matching errors caused by insufficient feature extraction, thereby improving positioning accuracy. Secondly, the UAV pose estimation problem is modeled as a task of finding the maximum response value in a discretized four-dimensional likelihood volume. Combined with a coarse-to-fine hierarchical pose search framework, an efficient search strategy is implemented. This framework first discretizes pose hypotheses within a coarse prior range of altitude and heading angle, avoiding an exhaustive search of all possible poses, significantly reducing the computational load of the initial search and improving efficiency. Simultaneously, by generating query templates for each pose hypothesis and efficiently matching them with satellite image feature maps to construct a four-dimensional pose likelihood volume, it comprehensively and accurately reflects the matching degree of different pose hypotheses, providing a reliable basis for subsequent localization. Using the pose with the strongest response in the cost body as the center of the next level of more refined search, the search is gradually converged to the exact solution in an iterative manner, so that the search range is gradually reduced from coarse to fine. While ensuring the comprehensiveness of the search, it can quickly focus on the vicinity of the exact solution. This avoids the insufficient accuracy that may be caused by the initial coarse search, and also prevents the inefficiency caused by the excessively large range during the fine search. It achieves a balance between computational efficiency and positioning accuracy, and finally obtains the accurate UAV pose efficiently. Attached Figure Description

[0006] Figure 1 This is a flowchart illustrating a training-free hierarchical visual positioning method for UAVs in GNSS denied environments, as shown in one embodiment. Figure 2 This is a schematic diagram of the flowchart of a training-free hierarchical visual localization method for UAVs in one embodiment; Figure 3 This is a schematic diagram of a hierarchical search algorithm in one embodiment; Figure 4 This is a visualization of the localization process in one embodiment. Detailed Implementation

[0007] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0008] In one embodiment, such as Figure 1 and Figure 2 As shown, a training-free hierarchical visual localization method for UAVs in GNSS-denied environments is provided, including the following steps: Step 102, acquire satellite images and query images; use a pre-trained large model to extract features from the satellite images and query images to obtain feature maps.

[0009] The satellite images and query images are extracted by the middle layer of the DINOv2 model. However, these high-dimensional features will result in huge computational and storage overheads. Therefore, the application proposes a "static reference principal component analysis (SR-PCA) based on global statistics" strategy, that is, the feature subspace is defined by the inherent statistical properties of the entire satellite image. Specifically, given an input query image and a satellite image The first layer network of the DINOv2 model is selected for feature extraction, and the output feature map has C channels and .

[0010] .

[0011] where C is the original feature dimension. Then, the high-dimensional feature map is processed by the SR-PCA method. First, the global mean vector and the covariance matrix of all feature images are calculated.

[0012] Subsequently, by performing eigenvalue decomposition on the covariance matrix Σ, the eigenvectors corresponding to the top k largest eigenvalues are selected to form a fixed projection matrix P (k = 8 in the specific embodiment). For any new input image, the pre-computed fixed projection matrix P is used for dimensionality reduction. The resulting feature map and retains the key information required for matching.

[0013] To further explore the performance of different pre-trained models in the cross-view visual positioning task, DepthAnything V2 is introduced as a feature extractor. The intermediate features are extracted from the decoder head before generating the final depth map. For query images and satellite images , the original feature map is obtained as follows: .

[0014] Step 104, model the UAV pose estimation problem as a task of finding the maximum response value in a discretized four-dimensional likelihood body, apply an affine transformation to the feature map corresponding to the query image, generate a corresponding query template for each pose hypothesis, and construct a four-dimensional pose likelihood body with four dimensions by efficiently matching the feature map of the satellite image.

[0015] ​To solve the huge computational burden of exhaustive search when jointly estimating scale and rotation, a hierarchical search strategy is proposed. The problem of UAV pose estimation is modeled as finding the maximum response value in a discretized four-dimensional likelihood volume L. The input of the method is the UAV aerial image features and satellite image features , and the output is the estimated four-degree-of-freedom pose.

[0016] Four-degree-of-freedom template generation is performed at each level of the hierarchical structure, and the poses are uniformly sampled in the neighborhood of the optimal estimate value of the previous level. For a given scale s and yaw angle θ, an affine transformation is applied to the reduced-dimension query feature map to generate the corresponding template. The transformation aims to transform the query features to the view that matches the satellite image under the assumed pose.

[0017]

[0018] wherein, denotes the transformed feature template, denotes a binary mask that identifies the valid region, denotes the affine transformation, denotes the feature map corresponding to the query image.

[0019] Likelihood volume computation The computation of the cross-correlation between the template and the satellite map at all translation positions is computationally intensive. The cross-correlation theorem provides a highly efficient alternative by converting the problem to the frequency domain. For two two-dimensional signals, their cross-correlation is the inverse Fourier transform of the product of their respective Fourier transforms, one of which is complex conjugated.

[0020] For multi-channel features, the initial correlation score map S' is obtained by summing the results of the computation for each channel c. To eliminate the effects of template size and feature amplitude variations, mask-normalized cross-correlation is used for correction, and the final likelihood value, i.e., the four-dimensional four-dimensional pose likelihood volume, is obtained.

[0021] Step 106, by locating the pose with the strongest response in the four-dimensional pose likelihood volume, and taking the strongest pose as the center of the next level of more refined search, the final UAV pose is obtained by gradually converging to the exact solution according to the iterative method.

[0022] The hierarchical search strategy integrates the above steps into an efficient multi-level algorithm. The algorithm first performs extensive coarse search, and then gradually refines the pose estimate by constructing smaller and higher-resolution likelihood volumes around the best estimate value of the previous level. In this embodiment, the number of iterations is three rounds. The process is shown in Figure 3 .

[0023] The unmanned aerial vehicle training-free hierarchical visual positioning method for GNSS denial environment described above first extracts feature maps of satellite images and query images according to a pre-trained large model, fully utilizes rich semantic and detailed features learned by the model, enhances the feature matching capability between images, lays a foundation for subsequent accurate matching, reduces matching errors caused by insufficient feature extraction, and thus improves positioning accuracy. Secondly, the unmanned aerial vehicle pose estimation problem is modeled as a task of finding the maximum response value in a discretized four-dimensional likelihood body, and a hierarchical pose search framework from coarse to fine is combined to realize an efficient search strategy. The search range of the framework is set to a prior range of height variation ±75 meters and heading angle variation ±15 degrees, and the discretely sampled pose hypotheses are avoided to exhaustively search all possible poses, which greatly reduces the calculation amount of the initial search and improves the efficiency. At the same time, the four-dimensional pose likelihood body is constructed by generating a query template for each pose hypothesis and efficiently matching it with the satellite image feature map, which can comprehensively and accurately reflect the matching degree of different pose hypotheses and provide a reliable basis for subsequent positioning. The pose with the strongest response in the cost body is taken as the center of the next level of more precise search, and the search range is gradually reduced from coarse to fine in an iterative manner, which can quickly focus on the accurate solution while ensuring the comprehensiveness of the search, avoiding the insufficient accuracy caused by the initial rough search, and preventing the low efficiency caused by the too large range in the fine search, achieving a balance between calculation efficiency and positioning accuracy, and finally obtaining the accurate unmanned aerial vehicle pose efficiently.

[0024] In one of the embodiments, the pre-trained large model includes a DINOv2 model and a DepthAnything V2 model; the pre-trained large model is used to extract features of the satellite image and the query image to obtain the feature map, including: The DINOv2 model is used to extract features of the satellite image and the query image to obtain high-dimensional features. The satellite image is divided into blocks, and the blocks are overlapped during the division process; the global mean vector and the covariance matrix of all feature images of the high-dimensional features of each block of the satellite image are counted. The covariance matrix is subjected to eigenvalue decomposition, the eigenvectors corresponding to the first k largest eigenvalues are selected to form a fixed projection matrix, and the high-dimensional features are subjected to dimension reduction processing according to the projection matrix to obtain the feature map.

[0025] In one of the embodiments, the global mean vector and the covariance matrix of all feature images of the high-dimensional features of each block of the satellite image are counted, including: The global mean vector and the covariance matrix of all feature images of the high-dimensional features of each block of the satellite image are counted as

[0026]

[0027] wherein, denotes the global mean vector, denotes the total number of feature vectors in the satellite map, denotes the feature vector index, denotes the high-dimensional feature, denotes the covariance matrix, denotes the transpose operation.

[0028] In one of the embodiments, the high-dimensional feature is reduced in dimension according to the projection matrix to obtain the feature map, comprising: the high-dimensional feature is reduced in dimension according to the projection matrix to obtain the feature map is

[0029] wherein, denotes the feature map corresponding to the query image, denotes the feature map corresponding to the satellite image, denotes the projection matrix.

[0030] In one of the embodiments, the DepthAnything V2 model is used to extract features of the satellite image and the query image to obtain the feature map corresponding to the query image and the feature map corresponding to the satellite image.

[0031] In one of the embodiments, an affine transformation is applied to the feature map corresponding to the query image, comprising: the affine transformation is applied to the feature map corresponding to the query image is

[0032] wherein, denotes the transformed feature template, denotes a binary mask identifying the valid region, denotes the affine transformation parameterized by the rotation angle and the scaling factor, denotes the feature map corresponding to the query image.

[0033] In one of the embodiments, a four-dimensional four-dimensional pose likelihood volume is constructed by efficiently matching the feature map of the satellite image, comprising: the sum of the calculation results of each channel c of the feature map of the satellite image is summed to obtain the initial correlation score map is

[0034] wherein, denotes the number of channels of the feature map, denotes the channel index, denotes the first channel of the feature map of the satellite image. multidimensional feature, denotes a fast Fourier transform, denotes the feature map of the UAV image under the c-th channel after an affine transformation with scale s and yaw θ, denotes the feature map of the UAV image under the c-th channel after an affine transformation with scale s and yaw θ, denotes the template feature after an affine transformation with scale s and yaw θ, denotes a pose combination, denotes a scale factor, denotes a Hadamard product.

[0035] In one embodiment, a four-dimensional four-dimensional pose likelihood volume is constructed from the initial correlation score map as

[0036] wherein, denotes the initial correlation score map, denotes the feature map of the satellite image under the c-th channel, denotes the valid region mask obtained by performing an affine transformation on the UAV feature map with the assumed scale s and yaw angle θ, denotes a small constant to ensure numerical stability.

[0037] In one embodiment, the strongest responding pose in the four-dimensional pose likelihood volume is located, including: locating the strongest responding pose in the four-dimensional pose likelihood volume, i.e., the pose combination with the highest initial correlation score map.

[0038] In specific embodiments, extensive experiments performed by the present application on the refined and corrected UAV-VisLoc and UAV-GeoLoc benchmark datasets show that, as shown in Figure 4 , the visualization of the positioning process. From left to right, in order: the input UAV aerial image, the satellite reference map, the feature maps extracted from the two images, and the final likelihood score map. The obvious peak in the score map indicates that the position and direction of the UAV are successfully estimated, verifying that the present application achieves the most advanced positioning accuracy.

[0039] It should be understood that, although Figure 1 the steps in the flowchart of the present application are shown in sequence according to the direction of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 ​At least one of the steps in the above-mentioned embodiments can include a plurality of sub-steps or a plurality of stages, which are not necessarily executed at the same time, but can be executed at different times, and the order of execution of the sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least one of the other steps or sub-steps or stages of the other steps.

[0040] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0041] Each technical feature of the above-mentioned embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of each technical feature in the above-mentioned embodiments are not described, but as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0042] The above-mentioned embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A UAV training-free hierarchical visual positioning method for GNSS-denied environments, characterized in that, The method comprises: acquiring a satellite image and a query image; extracting features of the satellite image and the query image by using a pre-trained large model to obtain feature maps; modeling the unmanned aerial vehicle pose estimation problem as a task of finding the maximum response value in a discretized four-dimensional pose likelihood body, applying an affine transformation to the feature map corresponding to the query image, generating a corresponding query template for each pose hypothesis, and constructing a four-dimensional four-dimensional pose likelihood body by efficiently matching the feature map of the satellite image; locating the strongest response pose in the four-dimensional pose likelihood body, and taking the strongest pose as the center of the next level of more detailed search, gradually converging to the exact solution according to the iterative method to obtain the final unmanned aerial vehicle pose.

2. The method of claim 1, wherein, The pre-trained large model comprises a DINOv2 model and a DepthAnything V2 model; the features of the satellite image and the query image are extracted by using the pre-trained large model to obtain the feature maps, which comprises: extracting the features of the satellite image and the query image by using the DINOv2 model to obtain high-dimensional features; blocking the satellite image, and ensuring overlap during the blocking process; and calculating the global mean vector and the covariance matrix of all feature images of the high-dimensional features of each block of the satellite image. performing eigenvalue decomposition on the covariance matrix, selecting the eigenvectors corresponding to the first k largest eigenvalues to form a fixed projection matrix, and performing dimension reduction processing on the high-dimensional features according to the projection matrix to obtain the feature maps.

3. The method of claim 2, wherein, The global mean vector and the covariance matrix of all feature images of the high-dimensional features of each block of the satellite image are calculated, which comprises: The global mean vector and the covariance matrix of all feature images of the high-dimensional features of each block of the satellite image are calculated as wherein, represents a global mean vector, represents the total number of feature vectors in the satellite map, represents a feature vector index, represents a high-dimensional feature, represents a covariance matrix, represents a transpose operation.

4. The method of claim 3, wherein, The high-dimensional features are processed by the projection matrix to obtain the feature maps, which comprises: The high-dimensional features are processed by the projection matrix to obtain the feature maps as wherein, denotes a feature map corresponding to the query image, denotes a feature map corresponding to the satellite image, denotes a projection matrix.

5. The method of claim 2, wherein, The method further comprises: extracting the features of the satellite image and the query image by using the DepthAnything V2 model to obtain the feature map corresponding to the query image and the feature map corresponding to the satellite image.

6. The method of claim 1, wherein, An affine transformation is applied to the feature map corresponding to the query image, which comprises: An affine transformation is applied to the feature map corresponding to the query image as wherein, denotes the transformed feature template, denotes a binary mask identifying the valid region, denotes an affine transformation parameterized by a rotation angle and a scale factor, denotes the feature map corresponding to the query image.

7. The method of claim 1, wherein, An efficient matching is performed between the feature map of the satellite image to construct a four-dimensional four-dimensional pose likelihood body, which comprises: The calculation results of each channel c of the feature map of the satellite image are summed to obtain an initial correlation score map as wherein, represents the number of channels of a feature map, represents a channel number, represents the first multi-dimensional feature of a satellite image, represents a fast Fourier transform, represents a feature of a UAV image after an affine transformation is performed on the UAV image according to a scale s and a yaw, to obtain a template feature, represents a pose composition, represents a scale factor, represents a Hadamard product.

8. The method of claim 7, wherein, The method further comprises: A four-dimensional four-dimensional pose likelihood body is constructed according to the initial correlation score map as wherein, represents an initial correlation score map, represents a feature map of the satellite image on the c-th channel, represents an effective region mask obtained by performing an affine transformation on the UAV feature map by a hypothetical scale s and a yaw angle Θ, represents a small constant that ensures numerical stability.

9. The method of claim 1, wherein, The strongest response pose in the four-dimensional pose likelihood body is located, which comprises: The strongest response pose in the four-dimensional pose likelihood body is located, that is, the highest pose combination in the initial correlation score map.

Images