A UAV visual positioning method based on air-space cross-domain collaboration

Abstract

The present application belongs to the field of UAV visual positioning technology. The present application provides a UAV visual positioning method based on cross-domain collaboration between air and space. The disclosed embodiment combines a feature point extraction network and a feature matching network to achieve robust feature point extraction and matching in complex scenarios; through dynamic prediction of the position range, the computational complexity of matching is reduced and efficiency is improved; the fusion positioning of remote sensing images and UAV images can adapt to scenarios in non-GNSS environments; from data collection to positioning result output, the method has a high degree of automation.

Description

Technical Field

[0001] The disclosed embodiments relate to the technical field of drone visual positioning, and in particular to a drone visual positioning method based on air-space cross-domain collaboration. Background Art

[0002] Drone positioning is a key issue in modern drone technology, typically relying on navigation systems. In recent years, visual navigation technology has garnered increasing attention, effectively improving the reliability and accuracy of drone positioning. Visual navigation technology is playing an increasingly important role in fields such as disaster relief and agriculture. Computer vision processes and analyzes visual information through computer systems, enabling target detection, identification, tracking, and location. It possesses strong versatility and scalability. Therefore, drone visual positioning can provide more accurate and efficient support for related tasks.

[0003] UAV visual positioning methods can generally be divided into map-free positioning, positioning based on constructed maps, and positioning based on existing maps. For positioning based on existing maps, such as image matching methods, matching is performed using a pre-recorded georeferenced image library, aiming to identify and align content or structures with similar attributes in two images at the pixel level. Generally speaking, the images to be matched are usually taken from the same or similar scenes or targets, or other types of image pairs with the same shape or semantic information. This ensures a certain degree of compatibility, can provide absolute position information, and does not accumulate errors, thus having good application value.

[0004] However, traditional image matching methods use hand-crafted descriptors to extract features and match remote sensing images based on descriptor similarity and spatial geometric relationships, such as algorithms like SIFT and ORB. These methods are robust to noise, complex geometric deformations, and radiometric differences, but are limited in handling high-resolution and large-scale data. Among deep learning-based image matching methods, with the availability of large amounts of labeled datasets, deep learning methods, particularly convolutional neural networks (CNNs), attention mechanisms, and graph neural networks (GNNs), have demonstrated outstanding performance in the field of image matching. These methods automatically learn features, including both low-level spatial and high-level semantic information, autonomously acquire important image information, and process relationships between nodes, resulting in a dominant position in matching accuracy. However, these methods face challenges such as the heterogeneity of geographic information, insufficient labeled data for diverse application scenarios, significant ground feature variations between drone imagery and remote sensing imagery, differences in viewpoints, and the high real-time requirements of drone platforms.

[0005] Therefore, it is necessary to improve one or more problems existing in the above-mentioned related technical solutions.

[0006] It should be noted that this section is intended to provide background or context for the technical solutions of the present disclosure stated in the claims. The description herein is not admitted to be prior art by virtue of being included in this section. Summary of the Invention

[0007] The purpose of the embodiments of the present disclosure is to provide a UAV visual positioning method based on air-space cross-domain collaboration, thereby overcoming one or more problems caused by the limitations and defects of related technologies, at least to a certain extent.

[0008] According to an embodiment of the present disclosure, a method for visual positioning of a UAV based on air-space cross-domain collaboration is provided, the method comprising:

[0009] Get the drone image of the current view;

[0010] The feature point extraction network is used to extract features from UAV images to obtain fine-grained feature maps and coarse-grained feature maps, and the fine-grained feature maps and coarse-grained feature maps are reconstructed to obtain fusion features;

[0011] The feature point extraction network is used to extract feature points and generate descriptors on the fused feature map to obtain target feature points and their corresponding target descriptors;

[0012] According to the historical location information of the UAV, several remote sensing images are obtained, and feature extraction is performed on each remote sensing image using a feature point extraction network to obtain several groups of remote sensing feature points and their corresponding remote sensing descriptors;

[0013] Based on all remote sensing feature points and their corresponding remote sensing descriptors, a feature matching network is used to traverse and match the target feature points and target descriptors, and the remote sensing image with the most matching feature points is selected as the optimal remote sensing image;

[0014] The coordinate transformation of the center point of the optimal remote sensing image is performed and mapped to the actual geographic coordinate system to obtain the visual positioning result of the current position of the UAV.

[0015] Furthermore, the method further comprises:

[0016] Before the first takeoff, obtain the corresponding satellite map based on the drone's location information;

[0017] The satellite map is divided into remote sensing images of N×M pixels at fixed pixel intervals through a sliding window.

[0018] Furthermore, the feature point extraction network includes a feature extraction module and a feature point-descriptor generation module; wherein,

[0019] The feature extraction module includes a fine-grained feature extractor, a coarse-grained feature extractor, and a feature reconstruction module. The fine-grained feature extractor includes a convolutional network, which includes a first convolutional block. The first convolutional block (i.e., convolutional block-1) includes three layers of maximum pooling and six layers of first convolution. The feature reconstruction module includes a second convolutional block (i.e., convolutional block-2), which includes three layers of second convolution.

[0020] The feature point-descriptor generation module includes a third convolution block and a third convolution with M channels.

[0021] Furthermore, the feature extraction network is used to extract features from the UAV image to obtain a fine-grained feature map and a coarse-grained feature map, and the fine-grained feature map and the coarse-grained feature map are reconstructed to obtain a fusion feature, including:

[0022] In the coarse-grained feature extractor, the drone image is divided into several non-overlapping image blocks;

[0023] Flatten each image block and map it to a multidimensional space through a linear layer to generate the first feature matrix;

[0024] Add position encoding to the first feature matrix and process it through an 8-layer Transformer encoder to generate the second feature matrix;

[0025] Perform a Reshape operation on the second feature matrix to generate a coarse-grained feature map;

[0026] In the fine-grained feature extractor, a convolutional network is used to extract local features to obtain a fine-grained feature map;

[0027] After upsampling the coarse-grained feature map, it is concatenated with the fine-grained feature map along the channel dimension to obtain a concatenated feature map.

[0028] The concatenated feature map is passed through the second convolution block for feature reconstruction to generate a fused feature map.

[0029] Furthermore, the step of extracting feature points and generating descriptors from the fused feature map using a feature point extraction network to obtain target feature points and their corresponding target descriptors includes:

[0030] The fused feature map is input into the third convolution block, and a feature point heat map is generated through a convolution layer with 65 channels;

[0031] Input the fused feature map into the third convolution with a channel number of M to generate a descriptor feature map;

[0032] Perform Softmax operation on the feature point heat map and restore the original image resolution through Reshape to obtain the target feature points;

[0033] The descriptor feature map is input into the descriptor incremental mapping module for optimization to obtain the target descriptor.

[0034] Furthermore, the feature matching network includes a graph neural attention network consisting of MLP, a 9-layer multi-head self-attention alternating with cross-attention, and an optimization module.

[0035] Furthermore, based on all remote sensing feature points and their corresponding remote sensing descriptors, the step of using a feature matching network to traverse and match the target feature points and the target descriptors, and selecting the remote sensing image with the most matching feature points as the optimal remote sensing image, includes:

[0036] After the target feature points are position-encoded using the MLP network, they are feature-encoded with the target descriptor to obtain a target encoding map.

[0037] For a pair of remote sensing feature points and their corresponding descriptors, the remote sensing feature points are position-encoded using the MLP network, and then feature-encoded with the remote sensing descriptors to obtain a remote sensing coding map.

[0038] The target coding map and all remote sensing coding maps are respectively input into the graph neural attention network to update the enhanced target coding map and enhanced remote sensing coding map;

[0039] An extended confidence matrix is ​​constructed based on the enhanced target coding map and the enhanced remote sensing coding map, and the final matching matrix is ​​solved iteratively through the Sinkhorn algorithm.

[0040] According to the final matching matrix, the remote sensing image with the most matching feature points is selected as the optimal remote sensing image.

[0041] Furthermore, the step of performing coordinate transformation on the center point of the optimal remote sensing image and mapping it to the actual geographic coordinate system to obtain the visual positioning result of the current position of the UAV includes:

[0042] Based on the optimal remote sensing image, calculate its homography matrix HM;

[0043] Based on the homography matrix HM, the optimal remote sensing image is transformed into a perspective to obtain the absolute position;

[0044] Map the absolute position to the actual geographic coordinate system to obtain the visual positioning result of the drone's current position.

[0045] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0046] In the embodiments of the present disclosure, the above-mentioned UAV visual positioning method based on air-space cross-domain collaboration is integrated with a multi-scale feature point extraction network and a feature matching network to address positioning challenges in complex environments. This method enables highly robust feature point extraction and precise matching in changing and complex scenarios, significantly improving the stability and reliability of UAV positioning. The collaborative work of the feature point extraction network and the feature matching network enables the system to accurately capture key features in a dynamically changing environment, thereby improving the accuracy and efficiency of feature matching.

[0047] Furthermore, by introducing a dynamic prediction mechanism for position ranges, this method effectively reduces the computational complexity of feature point matching, thereby improving computational efficiency while maintaining high accuracy. This method is particularly suitable for resource-constrained UAV platforms. By integrating remote sensing images with real-time UAV imagery, the method's adaptability is further expanded to environments without GNSS, enabling stable high-precision positioning.

[0048] This method features a high degree of automation from data collection to final positioning output, simplifying the manual intervention in traditional positioning processes while improving the overall efficiency of the system. This method enables precise autonomous positioning of drones in complex and dynamic environments, providing reliable technical support for their intelligent navigation and mission execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0050] Figure 1 A diagram showing the steps of a UAV visual positioning method based on air-space cross-domain collaboration in an exemplary embodiment of the present disclosure;

[0051] Figure 2 A specific flow chart of a UAV visual positioning method based on air-space cross-domain collaboration in an exemplary embodiment of the present disclosure is shown;

[0052] Figure 3 A schematic diagram showing the structure of a feature point extraction network in an exemplary embodiment of the present disclosure is shown;

[0053] Figure 4 A schematic diagram showing the structure of a feature matching network in an exemplary embodiment of the present disclosure is shown;

[0054] Figure 5A diagram showing the visual positioning results of a drone in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0055] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0056] In addition, the accompanying drawings are merely schematic illustrations of embodiments of the present disclosure and are not necessarily drawn to scale. Like reference numerals in the figures represent like or similar parts, and thus repeated descriptions thereof will be omitted. Some of the blocks shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically separate entities.

[0057] This example implementation provides a UAV visual positioning method based on air-space cross-domain collaboration. Figure 1 As shown in , the UAV visual positioning method based on air-space cross-domain collaboration may include:

[0058] Step S101: Acquire the drone image of the current viewing angle;

[0059] Step S102: extracting features from the drone image using a feature point extraction network to obtain a fine-grained feature map and a coarse-grained feature map, and reconstructing the fine-grained feature map and the coarse-grained feature map to obtain a fusion feature;

[0060] Step S103: extracting feature points and generating descriptors from the fused feature map using a feature point extraction network to obtain target feature points and their corresponding target descriptors;

[0061] Step S104: acquiring a plurality of remote sensing images based on the historical location information of the UAV, and performing feature extraction on each remote sensing image using a feature point extraction network to obtain a plurality of groups of remote sensing feature points and their corresponding remote sensing descriptors;

[0062] Step S105: Based on all remote sensing feature points and their corresponding remote sensing descriptors, a feature matching network is used to perform traversal matching on the target feature points and the target descriptors, and the remote sensing image with the most matching feature points is selected as the optimal remote sensing image;

[0063] Step S106: performing coordinate transformation on the center point of the optimal remote sensing image and mapping it to the actual geographic coordinate system to obtain the visual positioning result of the current position of the UAV.

[0064] Through the above-mentioned UAV visual positioning method based on cross-domain collaboration between air and space, combined with the feature point extraction network and feature matching network, robust feature point extraction and matching can still be achieved in complex scenarios; through dynamic prediction of the position range, the computational complexity of matching is reduced and efficiency is improved; the fusion positioning of remote sensing images and UAV images can adapt to scenarios in non-GNSS environments; from data collection to positioning result output, the method has a high degree of automation.

[0065] Below, we will refer to Figures 1 to 5 Each step of the above-mentioned UAV visual positioning method based on air-space cross-domain collaboration in this example implementation is described in more detail.

[0066] In steps S101 to S103, a drone image of the current perspective is obtained; a feature point extraction network is used to extract features from the drone image to obtain a fine-grained feature map and a coarse-grained feature map, and the fine-grained feature map and the coarse-grained feature map are reconstructed to obtain a fused feature; a feature point extraction network is used to extract feature points and generate descriptors on the fused feature map to obtain target feature points and their corresponding target descriptors.

[0067] Specifically, such as Figure 2 Figure 2 shows a detailed flowchart of a UAV visual positioning method based on cross-domain collaboration between air and space. Before the first takeoff, a satellite map is acquired based on the UAV's location information. The satellite map is then divided into N×M pixel remote sensing images using a sliding window at a fixed pixel interval.

[0068] In the coarse-grained feature extractor, the drone image is divided into several non-overlapping image blocks; each image block is flattened and mapped to a multidimensional space through a linear layer to generate a first feature matrix; position encoding is added to the first feature matrix and processed through an 8-layer Transformer encoder to generate a second feature matrix; the second feature matrix is ​​reshaped to generate a coarse-grained feature map; in the fine-grained feature extractor, a convolutional network is used to extract local features to obtain a fine-grained feature map; the coarse-grained feature map is upsampled and then spliced ​​with the fine-grained feature map along the channel dimension to obtain a spliced ​​feature map; the spliced ​​feature map is reconstructed through a second convolutional block to generate a fused feature map.

[0069] In one embodiment, Figure 3 As shown in Figure 1, it is a structural diagram of the feature point extraction network.

[0070] The feature point extraction network includes a feature extraction module and a feature point-descriptor generation module; wherein, the feature extraction module includes a fine-grained feature extractor, a coarse-grained feature extractor and a feature reconstruction module, the fine-grained feature extractor includes a convolutional network, the convolutional network includes a first convolutional block, the first convolutional block includes 3 layers of maximum pooling and 6 layers of first convolution, the feature reconstruction module includes a second convolutional block, and the second convolutional block includes 3 layers of second convolution; the feature point-descriptor generation module includes a third convolutional block and a third convolution with a channel number of M.

[0071] In one embodiment, before the first takeoff, relevant parameters of the drone's takeoff position (such as latitude and longitude, altitude, azimuth, etc.) are collected and input into the algorithm framework to provide an initial position reference for subsequent positioning; a large-area satellite map (i.e., a satellite map) is preprocessed into a 512×512 small-area remote sensing map slice (i.e., a remote sensing image) at a fixed pixel interval of 100pix using a sliding window method.

[0072] During the actual positioning process, the drone captures the image of the current view in real time during flight (i.e., the drone image). Feature points are extracted from the drone image to form a set of N feature points and corresponding descriptors. , and then calculate the confidence, discard the feature points below the threshold (difficult to match), and use the result express.

[0073] Specifically, the encoder operation is first performed on the drone image with an input of (H×W×C); then the fine-grained first convolutional block feature extractor (i.e., fine-grained feature extractor) and the coarse-grained vit / s feature extractor (i.e., coarse-grained feature extractor) are used to obtain the fine-grained feature map and coarse-grained feature map of the drone image; then, the fine-grained feature map and the coarse-grained feature map are reconstructed through the feature fusion module to obtain a fused feature map containing semantic information, and the output is (W / 8×H / 8×128).

[0074] The third convolutional block extracts feature points from the fused feature map. The features of dimension (W / 8×H / 8×128) are first processed to size (W / 8×H / 8×65). Here, 65 means that each pixel in the feature map represents a local region of the original image (8×8), plus a Dustbin channel for output when no feature points exist in the local region. Through softmax and reshape operations, the features are restored to the original image size to obtain the target feature points. The feature descriptor generation network first processes the fused feature map of dimension (W / 8×H / 8×128) to size (W / 8×H / 8×M), where M is the dimension of the feature to be output. After channel-wise normalization, the descriptor feature map is obtained through bilinear interpolation based on the location of the feature points. This descriptor feature map is then enhanced in the descriptor incremental mapping module to obtain the target descriptor.

[0075] In a specific embodiment, the input is a 3×H×W RGB image (i.e., a drone image). The process is divided into two stages: feature extraction and feature point extraction-descriptor generation.

[0076] During the feature extraction phase, a feature point extraction network based on the ViT / S architecture is used. The input image (i.e., drone image) is first divided into 28×28 non-overlapping image blocks, each with dimensions of 3×(H / 28)×(W / 28). Each block is flattened into a vector of 1×3×(H / 28)×(W / 28) and then mapped to a 768-dimensional space using a linear layer. Positional encoding is applied to the 784×768 feature matrix generated by the 784 image blocks to obtain the first feature matrix. This matrix is ​​then processed through an 8-layer Transformer encoder, outputting a 784×768 feature matrix (i.e., the second feature matrix). This matrix is ​​finally reshaped into a coarse-grained feature map of 28×28×384.

[0077] A convolutional network is used to extract local features. Specifically, a first convolutional block structure consisting of 3 layers of max pooling and 6 layers of convolution is used. The convolution kernel size is 3, the number of channels varies from [64, 64, 64, 128, 128], and some convolution layers are downsampled with a stride of 2. The final output is a fine-grained feature map of size W / 8 × H / 8 × 128.

[0078] During the feature point extraction and descriptor generation phase, the coarse-grained feature map is first upsampled to a resolution of W / 8 × H / 8 × 128 and then concatenated with the fine-grained feature map along the channel dimension. The concatenated feature map is reconstructed using a second convolutional block consisting of three convolution layers, outputting a fused feature map of W / 8 × H / 8 × 128.

[0079] In the third convolutional block, a 65-channel convolutional layer generates a W / 8 × H / 8 × 65 feature point heatmap. A W / 8 × H / 8 × M descriptor feature map is generated through a M-channel convolutional layer. A Softmax operation is performed on the feature point heatmap, and the original image resolution is restored through reshaping to obtain the target feature points. Finally, each 1 × M descriptor feature map is optimized and output by the descriptor incremental mapping module to obtain the target descriptor.

[0080] The final output is N feature points and corresponding descriptor sets , used for subsequent matching.

[0081] In step S104, a plurality of remote sensing images are acquired based on the historical position information of the UAV, and feature extraction is performed on each remote sensing image using a feature point extraction network to obtain a plurality of groups of remote sensing feature points and their corresponding remote sensing descriptors.

[0082] Specifically, according to the position of the UAV at the last moment, a remote sensing map with a distance of less than 50m is selected, and the remote sensing feature points and corresponding remote sensing descriptors are extracted from the remote sensing images within the range to obtain .in are the remote sensing feature points and remote sensing descriptors of the i-th remote sensing image. The generation method of remote sensing feature points and corresponding remote sensing descriptors is the same as that of target feature points and target descriptors, which will not be described in detail here.

[0083] In step S105 , based on all remote sensing feature points and their corresponding remote sensing descriptors, a feature matching network is used to perform traversal matching on the target feature points and the target descriptors, and the remote sensing image with the most matching feature points is selected as the optimal remote sensing image.

[0084] Specifically, the target feature points are position-encoded using the MLP network and then feature-encoded with the target descriptor to obtain a target encoding map.

[0085] For a pair of remote sensing feature points and their corresponding descriptors, the remote sensing feature points are position-encoded using the MLP network, and then feature-encoded with the remote sensing descriptors to obtain a remote sensing coding map.

[0086] The target coding map and all remote sensing coding maps are respectively input into the graph neural attention network to update the enhanced target coding map and enhanced remote sensing coding map;

[0087] An extended confidence matrix is ​​constructed based on the enhanced target coding map and the enhanced remote sensing coding map, and the final matching matrix is ​​solved iteratively through the Sinkhorn algorithm.

[0088] According to the final matching matrix, the remote sensing image with the most matching feature points is selected as the optimal remote sensing image.

[0089] In one embodiment, Figure 4 As shown in Figure 1, it is a structural diagram of the feature matching network.

[0090] The feature matching network consists of a MLP, a 9-layer graph neural attention network alternating between multi-head self-attention and cross-attention, and an optimization module.

[0091] In one embodiment, and{ } to traverse and match, for , the target feature points and target feature descriptors are used as the input of the graph neural network (i.e., feature matching network). First, the target feature points and target feature vectors of the input network are encoded, and the MLP network is used for feature encoding. is the encoding output of the i-th target feature point, is the target descriptor, is the position of the target feature point. Formula (1):

[0092] (1)

[0093] The position of the target feature point and the target descriptor are encoded into the same feature (i.e., the target encoding map), so that the network can consider the similarity of feature description and position at the same time when matching.

[0094] for , perform the same operation as above to obtain the remote sensing coding map.

[0095] Then the target encoding graph And the remote sensing coding map is input into Attention-GNN to obtain the extended confidence matrix for matching , and finally the matching result is obtained through the Sinkhorn algorithm.

[0096] In a specific embodiment, in the graph neural attention network stage, the descriptors obtained from the feature point extraction network are used as node features of the graph neural network (GNN) to preliminarily represent the local information of the key points, including the coordinate positions and the similarity between the descriptors.

[0097] For the feature vector with dimension M, position encoding is performed using the formula , obtaining a more accurate M-dimensional feature vector that effectively expresses the potential correlation between nodes. The descriptors of the two input images are processed using a graph neural network architecture consisting of nine layers of alternating multi-head self-attention (MHSA) and cross-attention. Through a message-passing mechanism, graph neural networks effectively fuse the spatial information of keypoints with descriptor features, capturing the global relationship between keypoints in an image pair.

[0098] During the optimal transmission optimization phase, the node feature vectors updated by the graph neural network become context-enhanced M-dimensional descriptors, incorporating global contextual information between key points. Subsequently, an extended confidence matrix is ​​constructed, and the final matching matrix is ​​solved through 20 iterations of the Sinkhorn algorithm. Ultimately, accurate matching results are output.

[0099] In step S106, the center point of the optimal remote sensing image is transformed into a coordinate system and mapped into an actual geographic coordinate system to obtain a visual positioning result of the current position of the UAV.

[0100] Specifically, based on the optimal remote sensing image, its homography matrix HM is calculated;

[0101] Based on the homography matrix HM, the optimal remote sensing image is transformed into a perspective to obtain the absolute position;

[0102] Map the absolute position to the actual geographic coordinate system to obtain the visual positioning result of the drone's current position.

[0103] More specifically, the absolute position of the center point of the drone image is obtained by perspective transformation, as shown in formula (2):

[0104] (2)

[0105] in, and are the center points H / 2 and W / 2 of the drone image respectively, and The pixel position of the drone on the satellite map is then mapped to the actual geographic coordinate system. The position information of the center point of the drone image is the current positioning output of the drone, and the visual positioning latitude and longitude information of the drone's current position is obtained.

[0106] In a specific embodiment, the performance of the present application can be illustrated by the test results of the following data.

[0107] 1. Experimental Conditions

[0108] This application is implemented on a 12th Gen Intel(R) Core(TM) i5-12400F2.50 GHz CPU, 64G memory, NVIDIA RTX 3090 graphics card, Ubuntu20.04 operating system, based on Pytorch1.7.1 and Python3.9 language environment. In the feature extraction network module, N and M are set to 512.

[0109] In terms of data sets, we conducted multi-scene and multi-season shooting and collection, and simulated actual application scenarios. The drone was fixed at a flight altitude of 200 meters, the video was recorded at 60fps, and the interval was set to 20 frames. The size is Aerial images taken by drones, satellite images obtained through Google Maps, map level 19, cut into Satellite map slices are generated. The real position is determined by the drone’s own GPS.

[0110] 2. Experimental Results

[0111] First, the proposed method's positioning capabilities were verified in a real-world environment. In an urban setting, following the implementation plan, the initial flight position of the drone was obtained and fed into the algorithm. Visual positioning of the drone was then performed, and the error between the algorithm's predicted geographic location and the GPS position was calculated to evaluate positioning accuracy.

[0112] Here, the positioning error is used as the average indicator to evaluate the positioning results of the algorithm and the positioning error of GPS. As shown in formula (3):

[0113] (3)

[0114] N is the number of positioning times, are the longitude and latitude coordinates of the UAV visual positioning, are the GPS latitude and longitude coordinates, where The calculation formula is shown in (4):

[0115] (4)

[0116] in, = It is half the difference in latitude, in radians; It is half the longitude difference, in radians, and the radius value is set to 6378.137 kilometers.

[0117] Subsequently, extensive tests were conducted in multiple different scenarios (such as industrial parks, parks, suburbs, campuses, fields, and villages), and the experimental results further verified the generalization ability of the algorithm in various environments.

[0118] Table 1 Multi-scenario test results

[0119]

[0120] As shown in Table 1, the flight scenarios cover cities, industrial areas, parks, villages, schools, suburbs, etc., with an average error of 8.12 meters, which shows that the positioning system has a high accuracy as a whole.

[0121] like Figure 5 The following figure shows the visual positioning results of the drone. The solid green circle represents the real coordinates of the drone's position during flight, and the hollow white circle represents the position obtained through visual positioning. It can be seen that the positioning positions are mostly in the real coordinates, verifying the matching accuracy and positioning performance of the model. Furthermore, the position estimation strategy can effectively improve positioning accuracy and ensure real-time performance.

[0122] Through the above-mentioned UAV visual positioning method based on cross-domain collaboration between air and space, combined with the feature point extraction network and feature matching network, robust feature point extraction and matching can still be achieved in complex scenarios; through dynamic prediction of the position range, the computational complexity of matching is reduced and efficiency is improved; the fusion positioning of remote sensing images and UAV images can adapt to scenarios in non-GNSS environments; from data collection to positioning result output, the method has a high degree of automation.

[0123] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly indicate the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0124] In the embodiments of the present disclosure, unless otherwise expressly specified or limited, the terms "installed," "connected," "connected," "fixed," and the like should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integration; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present disclosure based on specific circumstances.

[0125] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0126] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the appended claims.

Claims

1. A UAV visual positioning method based on air-space cross-domain collaboration, characterized by: The method includes: Get the drone image of the current view; The feature point extraction network is used to extract features from UAV images to obtain fine-grained feature maps and coarse-grained feature maps, and the fine-grained feature maps and coarse-grained feature maps are reconstructed to obtain fusion features; The feature point extraction network is used to extract feature points and generate descriptors on the fused feature map to obtain target feature points and their corresponding target descriptors; According to the historical location information of the UAV, several remote sensing images are obtained, and feature extraction is performed on each remote sensing image using a feature point extraction network to obtain several groups of remote sensing feature points and their corresponding remote sensing descriptors; Based on all remote sensing feature points and their corresponding remote sensing descriptors, a feature matching network is used to traverse and match the target feature points and the target descriptors, and the remote sensing image with the most matching feature points is selected as the optimal remote sensing image; specifically, the method includes: using an MLP network to positionally encode the target feature points, and then performing feature encoding with the target descriptors to obtain a target encoding map; for a pair of remote sensing feature points and their corresponding descriptors, using an MLP network to positionally encode the remote sensing feature points, and then performing feature encoding with the remote sensing descriptors to obtain a remote sensing encoding map; the target encoding map and all remote sensing encoding maps are respectively input into a graph neural attention network to update and obtain an enhanced target encoding map and an enhanced remote sensing encoding map; an extended confidence matrix is ​​constructed based on the enhanced target encoding map and the enhanced remote sensing encoding map, and the final matching matrix is ​​iteratively solved by the Sinkhorn algorithm; based on the final matching matrix, the remote sensing image with the most matching feature points is selected as the optimal remote sensing image; The coordinate transformation of the center point of the optimal remote sensing image is performed and mapped to the actual geographic coordinate system to obtain the visual positioning result of the current position of the UAV.

2. The UAV visual positioning method based on air-space cross-domain collaboration according to claim 1 is characterized in that: The method further includes: Before the first takeoff, obtain the corresponding satellite map based on the drone's location information; The satellite map is divided into remote sensing images of N×M pixels at fixed pixel intervals through a sliding window.

3. The UAV visual positioning method based on air-space cross-domain collaboration according to claim 2 is characterized in that: The feature point extraction network includes a feature extraction module and a feature point-descriptor generation module; wherein, The feature extraction module includes a fine-grained feature extractor, a coarse-grained feature extractor, and a feature reconstruction module. The fine-grained feature extractor includes a convolutional network, the convolutional network includes a convolution block-1, and the convolution block-1 includes 3 layers of maximum pooling and 6 layers of first convolution. The feature reconstruction module includes a convolution block-2, and the convolution block-2 includes 3 layers of second convolution. The feature point-descriptor generation module includes convolution block-3 and the third convolution with M channels.

4. The UAV visual positioning method based on air-space cross-domain collaboration according to claim 3 is characterized in that: The steps of extracting features from drone images using a feature point extraction network to obtain fine-grained feature maps and coarse-grained feature maps, and reconstructing the fine-grained feature maps and coarse-grained feature maps to obtain fused features include: In the coarse-grained feature extractor, the drone image is divided into several non-overlapping image blocks; Flatten each image block and map it to a multidimensional space through a linear layer to generate the first feature matrix; Add position encoding to the first feature matrix and process it through an 8-layer Transformer encoder to generate the second feature matrix; Perform a Reshape operation on the second feature matrix to generate a coarse-grained feature map; In the fine-grained feature extractor, a convolutional network is used to extract local features to obtain a fine-grained feature map; After upsampling the coarse-grained feature map, it is concatenated with the fine-grained feature map along the channel dimension to obtain a concatenated feature map. The spliced ​​feature map is passed through convolution block-2 for feature reconstruction to generate a fused feature map.

5. The UAV visual positioning method based on air-space cross-domain collaboration according to claim 4 is characterized in that: The step of extracting feature points and generating descriptors from the fused feature map using a feature point extraction network to obtain target feature points and their corresponding target descriptors includes: The fused feature map is input into convolution block-3, and a feature point heat map is generated through a convolution layer with 65 channels; Input the fused feature map into the third convolution with a channel number of M to generate a descriptor feature map; Perform Softmax operation on the feature point heat map and restore the original image resolution through Reshape to obtain the target feature points; The descriptor feature map is input into the descriptor incremental mapping module for optimization to obtain the target descriptor.

6. The UAV visual positioning method based on air-space cross-domain collaboration according to claim 5 is characterized in that: The feature matching network includes MLP, a 9-layer graph neural attention network that alternates multi-head self-attention and cross-attention, and an optimization module.

7. The UAV visual positioning method based on air-space cross-domain collaboration according to claim 6 is characterized in that: The steps of performing coordinate transformation on the center point of the optimal remote sensing image and mapping it to the actual geographic coordinate system to obtain the visual positioning result of the current position of the UAV include: Based on the optimal remote sensing image, calculate its homography matrix HM; Based on the homography matrix HM, the optimal remote sensing image is transformed into a perspective to obtain the absolute position; Map the absolute position to the actual geographic coordinate system to obtain the visual positioning result of the drone's current position.

Images