Scene matching method and device for unmanned aerial vehicle navigation

Abstract

The embodiment of the application provides a scene matching method and device for unmanned aerial vehicle navigation, which comprises the following steps: obtaining real-time images of an unmanned aerial vehicle and preset reference images, performing gray processing by using a preset brightness method, extracting first and second feature point sets of the gray real-time images and the reference images by using a set feature extraction network, adding a virtual node to each feature point set, constructing third and fourth feature point sets, wherein the virtual node is used to absorb feature points that cannot be matched in the attention matching process, inputting the third and fourth feature point sets into a set attention mechanism matching network, iteratively updating state vectors of each feature point by using a multi-layer stacked attention module, outputting a confidence score based on a preset multi-layer perception machine according to the state vectors, and then calculating a score matrix, decoding a matching point pair set according to the score matrix, realizing unmanned aerial vehicle scene matching navigation, and the application can improve the scene matching efficiency and accuracy of unmanned aerial vehicle navigation.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a scene matching method and apparatus for unmanned aerial vehicle (UAV) navigation. Background Technology

[0002] When unmanned aerial vehicles (UAVs) perform autonomous navigation missions in environments where GNSS signals are denied or interfered with (such as valleys, urban canyons, and tunnels), scene matching navigation technology is an important supplementary solution. This technology acquires real-time images through onboard visual sensors and matches them with pre-stored georeferenced maps to calculate the UAV's absolute position and heading.

[0003] Existing scene matching methods are mainly divided into two categories. The first category consists of traditional manual feature matching algorithms, such as SIFT, SURF, and ORB. These methods rely on manually designed local feature descriptors. Although they are relatively computationally efficient, they are sensitive to changes in lighting, viewpoint rotation, seasonal changes, and differences between heterogeneous images. They have poor robustness in complex environments and are prone to localization failures. The second category consists of deep learning-based feature matching methods, such as SuperPoint and SuperGlue. These methods employ dual-branch convolutional neural networks and attention mechanisms, significantly improving matching accuracy. However, their deep network structure, large number of parameters, and high computational latency make them difficult to meet the stringent real-time requirements of UAV embedded platforms. Furthermore, existing deep learning feature extraction networks generally use max pooling or stride convolution for downsampling, leading to the loss of high-frequency detail information and affecting the stability of heterogeneous image matching.

[0004] Therefore, there is an urgent need for a scene matching method for UAV navigation that can improve scene matching efficiency and accuracy, and enable autonomous navigation of lightweight UAVs. Summary of the Invention

[0005] To address the problems in the prior art, this application provides a scene matching method and apparatus for UAV navigation, which can improve the scene matching efficiency and accuracy of UAV navigation.

[0006] To solve at least one of the above problems, this application provides the following technical solution: Firstly, this application provides a scene matching method for UAV navigation, comprising: Acquire real-time images and preset reference images captured by a drone, and perform grayscale conversion on the real-time images and the reference images based on a preset brightness method to determine the corresponding grayscale real-time images and grayscale reference images respectively. Based on the set feature extraction network, features are extracted from the grayscale real-time image and the grayscale reference image, respectively determining the corresponding first feature point set and second feature point set. Virtual nodes are added to the first feature point set and the second feature point set respectively, and the corresponding third feature point set and fourth feature point set are determined. The virtual nodes are used to participate in information aggregation during the attention matching process and absorb unmatched feature points. The third and fourth feature point sets are input into a pre-defined attention mechanism matching network. The attention mechanism matching network iteratively updates the state vector of each feature point through a multi-layer stacked attention module. Based on the state vector obtained by the multi-layer perceptron based on the attention module, a confidence score is determined for each feature point. When the confidence score meets a pre-defined convergence condition, the final state vector of each feature point is determined. Based on the final state vector, a score matrix is ​​determined between the third and fourth feature point sets. Based on the score matrix, a corresponding set of matching point pairs is determined, thereby realizing scene matching for UAV navigation.

[0007] Furthermore, the defined feature extraction network includes: Construct a feature extraction network, including encoder and decoder structures; The encoder structure employs several cascaded processing units, each of which sequentially includes a lightweight convolution module and a Haar wavelet downsampling module. The lightweight convolution module includes a deep convolutional layer, a convolutional block attention layer, and two cascaded 1×1 pointwise convolutional layers. The layers of the lightweight convolution module are connected by residuals. The Haar wavelet downsampling module is used to achieve a halving of spatial resolution while preserving frequency domain information. The decoder structure employs transposed convolution to upsample the output that has been downsampled by the encoder structure.

[0008] Further, the step of extracting features from the grayscale real-time image and the grayscale reference image according to the set feature extraction network, and determining the corresponding first feature point set and second feature point set respectively, includes: The lightweight convolution module extracts features from the grayscale real-time image and the grayscale reference image, determines the corresponding feature maps, and inputs the obtained feature maps into the Haar wavelet downsampling module. The Haar wavelet downsampling module performs a two-dimensional discrete Haar wavelet transform on the feature map output by the lightweight convolution module to determine four sub-maps: low-frequency approximate component, horizontal high-frequency component, vertical high-frequency component, and diagonal high-frequency component. The four sub-maps are then concatenated in the channel dimension and fused by a 1×1 convolution to determine the corresponding concatenated feature map. The spliced ​​feature map is upsampled according to the decoder structure, and the upsampled output is decoded by two parallel decoder heads to determine the corresponding feature point heatmap and descriptor map respectively. The non-maximum suppression algorithm is used to extract the coordinates of the feature points in the feature point heatmap, and the descriptor vector of each feature point is extracted at the corresponding position in the descriptor map. A first feature point set is constructed based on the descriptor vectors of the feature points corresponding to the grayscale real-time image, and a second feature point set is constructed based on the descriptor vectors of the feature points corresponding to the grayscale reference image.

[0009] Furthermore, the attention module includes: A self-attention unit is used to perform self-attention calculations among all feature points within the same image, and the formula is:

[0010] in, For self-attention score, Let i be the query vector for point i. Let i be the key vector. Rotation encoding for the relative positions between points i and j; The cross-attention unit is used to perform bidirectional attention calculation between feature points in the real-time image and the reference image, and the formula is as follows:

[0011] in, For the score of cross attention, The values ​​belong to the corresponding index positions in the two graphs.

[0012] Furthermore, the attention mechanism matching network iteratively updates the state vector of each feature point through multiple stacked attention modules, including: For the initial state vector of each feature point, self-attention calculation is performed according to the self-attention unit. The self-attention calculation result is residually connected with the initial state vector to determine the corresponding intermediate state vector. The cross-attention unit receives the intermediate state vector, performs cross-attention calculation, and performs residual connection between the cross-attention calculation result and the intermediate state vector to determine the corresponding updated state vector.

[0013] Further, the step of determining the confidence score corresponding to each feature point based on the state vector obtained by the attention module of the preset multilayer perceptron, and determining the final state vector of each feature point when the confidence score satisfies the preset convergence condition, includes: The updated state vector corresponding to each feature point is input into a preset multilayer perceptron to determine the confidence score of the feature point output by the multilayer perceptron. When the confidence score meets the preset convergence condition, the attention mechanism matching network stops iterating and outputs the state vector of the current layer feature point as the final state vector. The convergence condition includes the following among all feature points in the current layer: the proportion of the number of feature points with a confidence score greater than a preset confidence threshold to the total number of feature points exceeds the global proportion threshold.

[0014] Further, the step of determining the score matrix between the third feature point set and the fourth feature point set based on the final state vector, and determining the corresponding set of matching point pairs based on the score matrix, includes: Based on the final state vector, construct the cosine similarity matrix between the third feature point set and the fourth feature point set; The cosine similarity matrix is ​​matched for feature points using the nearest neighbor algorithm to determine the corresponding set of matching point pairs.

[0015] Secondly, this application provides a scene matching device for unmanned aerial vehicle (UAV) navigation, comprising: The image acquisition module is used to acquire real-time images and preset reference images captured by the drone, and to convert the real-time images and the reference images to grayscale based on a preset brightness method, so as to determine the corresponding grayscale real-time images and grayscale reference images respectively. The feature extraction module is used to extract features from the grayscale real-time image and the grayscale reference image according to the set feature extraction network, determine the corresponding first feature point set and second feature point set respectively, add virtual nodes to the first feature point set and the second feature point set respectively, and determine the corresponding third feature point set and fourth feature point set. The virtual nodes are used to participate in information aggregation during the attention matching process and absorb unmatched feature points. The feature point scene matching module is used to input the third feature point set and the fourth feature point set into a preset attention mechanism matching network. The attention mechanism matching network iteratively updates the state vector of each feature point through a multi-layer stacked attention module. Based on the state vector obtained by the multi-layer perceptron based on the attention module, it determines the confidence score corresponding to each feature point. When the confidence score meets the preset convergence condition, it determines the final state vector of each feature point. Based on the final state vector, it determines the score matrix between the third feature point set and the fourth feature point set. Based on the score matrix, it determines the corresponding set of matching point pairs, thereby realizing scene matching for UAV navigation.

[0016] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the scene matching method for UAV navigation.

[0017] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the scene matching method for UAV navigation described above.

[0018] Fifthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the scene matching method for UAV navigation described above.

[0019] As can be seen from the above technical solution, this application provides a scene matching method and apparatus for UAV navigation. By acquiring real-time images of the UAV and a preset reference image, grayscale processing is performed using a preset brightness method. A set feature extraction network is used to extract the first and second feature point sets of the grayscale real-time image and the reference image respectively. Virtual nodes are added to each feature point set to construct the third and fourth feature point sets. The virtual nodes are used to absorb unmatchable feature points during the attention matching process. The third and fourth feature point sets are input into a set attention mechanism matching network. The state vectors of each feature point are iteratively updated through multi-layer stacked attention modules. Based on a preset multi-layer perceptron, a confidence score is output according to the state vector, and a score matrix is ​​calculated. A set of matching point pairs is generated by decoding the score matrix to realize UAV scene matching navigation, thereby improving the scene matching efficiency and accuracy of UAV navigation. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the scene matching method for UAV navigation in the embodiments of this application; Figure 2 This is a structural diagram of the scene matching device for UAV navigation in the embodiments of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The acquisition, storage, use, and processing of data in this application comply with relevant laws and regulations.

[0024] Considering the problems of poor robustness in complex environments, high computational latency, and low matching stability in UAV autonomous navigation scene matching methods, this application provides a scene matching method and apparatus for UAV navigation. It acquires real-time UAV images and a preset reference image, performs grayscale processing using a preset brightness method, and extracts first and second feature point sets from the grayscale real-time image and the reference image using a set feature extraction network. Virtual nodes are added to each feature point set to construct third and fourth feature point sets. The virtual nodes are used to absorb unmatched feature points during the attention matching process. The third and fourth feature point sets are input into a set attention mechanism matching network. The state vectors of each feature point are iteratively updated through multi-layer stacked attention modules. A confidence score is output based on the state vectors using a preset multi-layer perceptron, and a score matrix is ​​calculated. A set of matching point pairs is generated based on the score matrix, realizing UAV scene matching navigation. This improves the scene matching efficiency and accuracy of UAV navigation.

[0025] To improve the efficiency and accuracy of scene matching in UAV navigation, this application provides an embodiment of a scene matching method for UAV navigation, see [link to embodiment]. Figure 1 The scene matching method for UAV navigation specifically includes the following: Step S101: Acquire real-time images and preset reference images captured by the drone, and perform grayscale conversion on the real-time images and the reference images based on the preset brightness method, and determine the corresponding grayscale real-time images and grayscale reference images respectively; Optionally, in this embodiment, the research background of this solution is as follows: Currently, visual navigation technology for unmanned aerial vehicles (UAVs) plays a central role in autonomous localization and path planning in complex environments. Among these, image-based scene matching methods are widely used in areas where GNSS is unavailable or signal-limited, such as low-altitude areas, mountainous regions, and urban canyons, due to their advantages of not requiring external signals and strong anti-interference capabilities. However, existing scene matching navigation technologies still have significant limitations. Traditional feature matching algorithms such as SIFT, SURF, and ORB, while achieving high matching stability in the early stages, rely on manually designed features and are sensitive to changes in lighting, viewpoint rotation, and temporal phase differences, making them difficult to adapt to the frequently changing external environment during UAV flight.

[0026] Currently, most mainstream deep learning scene matching methods employ a two-branch feature extraction structure, such as SuperPoint, SuperGlue, and their subsequent improved models. These methods utilize convolutional neural networks to extract local features and then achieve global matching through attention mechanisms or Transformer structures. While they significantly outperform traditional algorithms in matching accuracy, their deep computational paths and large number of parameters make them difficult to deploy on embedded platforms. Especially in scenarios such as high-speed drone flight and real-time obstacle avoidance, this high computational latency directly leads to information mismatch between image frames, affecting the continuity and stability of pose calculation. Furthermore, existing models generally exhibit redundant computation in the feature extraction stage: the network needs to perform convolution or fully connected operations on the entire image, even processing regions lacking effective features. This indiscriminate feature encoding wastes inference resources and limits further improvements in real-time performance.

[0027] On the other hand, existing convolutional neural networks are prone to losing key local geometric features during image compression. For UAV aerial images, ground textures are often complex, with sparse distribution and significant scale differences in ground features. This causes convolutional features to blur edges and weaken textures during spatial downsampling, affecting the accuracy of keypoint detection. Furthermore, traditional downsampling methods (such as max pooling or stride convolution) discard frequency domain information during dimensionality reduction, making the model unstable when matching images from different sources or across different time periods. These problems are particularly pronounced in scene matching across seasons and lighting conditions, often leading to a sharp decrease in the number of matching points and affecting the reliability of attitude estimation results. Therefore, how to improve the geometric fidelity and global stability of feature extraction while ensuring model lightweightness has become a key technical challenge in the current field of UAV visual matching.

[0028] In summary, to address the problems faced by UAV visual navigation scene matching methods, a scene matching solution that balances versatility, high accuracy, and lightweight design is needed.

[0029] This solution addresses the aforementioned issues by constructing a lightweight, integrated solution encompassing feature encoding and matching optimization. This improves scene matching efficiency, thereby enhancing the autonomous navigation performance of UAVs in complex environments.

[0030] Specifically, during autonomous navigation, the drone needs to match the captured images with pre-stored reference images (such as satellite imagery) in real time for real-time positioning.

[0031] However, the images acquired in real time by the airborne vision sensor and the pre-stored reference image may originate from different sensors or be captured at different times. This can lead to significant differences in pixels, color, and brightness, interfering with the stability of the feature matching algorithm. To effectively reduce the impact of these heterogeneous image differences, this embodiment first performs grayscale conversion on all input images before formally performing feature encoding.

[0032] Specifically, this solution does not use a simple average or maximum value method, but instead employs a luminance method for grayscale conversion. This method uses a weighted average based on the human eye's sensitivity to different color channels, with the mapping function being:

[0033] Where (u,v) are the pixel coordinates, and R, G, B are the pixel values ​​of the red, green, and blue channels, respectively. These represent the weight coefficients for the corresponding channels. Compared to other linear projection methods, the luminance method can better preserve key feature information such as texture, structure, and contour of the original image during the conversion process, with minimal distortion. Simultaneously, uniformly converting color images into single-channel grayscale images effectively reduces the storage requirements of subsequent baseline images and the computational burden on the feature extraction network.

[0034] After preprocessing using the luminance method, grayscale real-time images and grayscale reference images are obtained, and feature encoding is then performed on these two image sets.

[0035] Step S102: Based on the set feature extraction network, perform feature extraction on the grayscale real-time image and the grayscale reference image, determine the corresponding first feature point set and second feature point set respectively, add virtual nodes to the first feature point set and the second feature point set respectively, and determine the corresponding third feature point set and fourth feature point set. The virtual nodes are used to participate in information aggregation during the attention matching process and absorb unmatched feature points. Optionally, in this embodiment, the real-time image and the reference image obtained from the previous grayscale processing are input into a lightweight feature coding network (the structure of the feature coding network is described in detail in subsequent step S201).

[0036] The feature extraction network embeds and transforms features layer by layer into the image through an encoder-decoder structure, and finally outputs two feature maps with the same spatial resolution: one is a feature point probability heatmap, and the other is a dense descriptor map.

[0037] A non-maximum suppression algorithm is performed on the feature point heatmap. The highest response values ​​of several pixels are selected as feature point coordinates through local neighborhood search. Then, based on these coordinates, a fixed-dimensional feature vector is extracted from the corresponding spatial location in the descriptor map to form an initial first feature point set (from the real-time image) and a second feature point set (from the reference image). Each feature point set contains several feature points with coordinate information and descriptor information.

[0038] Based on this, structural expansion operations (a pre-step for attention mechanism matching, defining the input dimension) are performed on the two feature point sets mentioned above respectively: A virtual node is added to each feature point set. This virtual node is not an actual detected feature point in the image, but a specially constructed node with learnable parameters or a fixed initial state. Formally, the first feature point set and the added virtual node together constitute the third feature point set; the second feature point set and the added virtual node together constitute the fourth feature point set.

[0039] The core role of virtual nodes is reflected in the subsequent attention matching process: when the matching network aggregates information across images, virtual nodes participate in self-attention and cross-attention calculations just like ordinary feature points, receiving information from other real feature points while also passing their own state information to other nodes. For those real feature points that cannot find a corresponding match in another image, the matching network can point their matching object to the virtual node, thus forming an "absorption" mechanism.

[0040] Through this step, firstly, virtual nodes effectively solve the problem of some feature points being unable to match due to occlusion, changes in viewpoint, or image boundary cropping in real-world scenarios, avoiding mismatches caused by forced matching; secondly, by uniformly converging unmatched feature points to virtual nodes, the scoring matrix output by the matching network becomes more regular in dimension, facilitating subsequent matrix operations and loss calculations; thirdly, the introduction of virtual nodes transforms the matching task from "finding one-to-one correspondences" into a joint problem of "classification + matching," improving the model's ability to process asymmetric feature point sets.

[0041] Step S103: Input the third feature point set and the fourth feature point set into a set attention mechanism matching network. The attention mechanism matching network iteratively updates the state vector of each feature point through multi-layer stacked attention modules. Based on the state vector obtained by the multi-layer perceptron based on the attention module, the confidence score corresponding to each feature point is determined. When the confidence score meets the preset convergence condition, the final state vector of each feature point is determined. Based on the final state vector, the score matrix between the third feature point set and the fourth feature point set is determined. Based on the score matrix, the corresponding matching point pair set is determined, thereby realizing scene matching for UAV navigation.

[0042] Optionally, in this embodiment, after feature point extraction and virtual node addition are completed, an attention-based matching network is used to perform iterative information fusion and matching relationship calculation on the two sets of feature points, and finally output a reliable set of matching point pairs.

[0043] The third feature point set (from the real-time image) and the fourth feature point set (from the reference image) after adding virtual nodes are jointly input into the attention mechanism matching network (the structure of which is described in detail in subsequent step S401).

[0044] The attention mechanism matching network employs multi-layered stacked attention modules as its basic processing unit, with each layer containing two cascaded sub-operations: self-attention and cross-attention. In the self-attention stage, each feature point pays attention to all other feature points within the same image, thus integrating the geometric and contextual information within that image. In the cross-attention stage, each feature point in the real-time image pays attention to all feature points in the reference image, and vice versa, thereby achieving cross-image information exchange. Both attention operations use residual connections to superimpose updated information onto the original state vector, allowing each feature point's state vector to gradually fuse its own intra-image relationships with cross-image matching cues in each layer iteration.

[0045] After each attention module has finished processing, a pre-defined multilayer perceptron (MLP) is invoked. This MLP takes the state vector of each feature point output from the current layer as input and independently calculates a confidence score for each feature point. This score, ranging from 0 to 1, reflects the stability of the current state vector of that feature point and the degree of matching convergence. A higher score indicates that the feature point has formed a reliable feature representation, and its matching decision tends to be stable.

[0046] Subsequently, the system determines whether the current layer meets the preset convergence conditions.

[0047] The condition is set as follows: among all feature points (including virtual nodes), the proportion of feature points with a confidence score exceeding a certain threshold (e.g., 0.9) reaches a global threshold (e.g., 95%). If this condition is met, the system immediately terminates the iterative computation of subsequent layers and outputs the state vector of each feature point in the current layer as the final state vector. This dynamic early stopping mechanism avoids performing unnecessary subsequent computations on simple or rapidly converging image pairs, thereby significantly reducing the average matching latency.

[0048] If the convergence condition is not met and the current layer has not reached the network's preset maximum number of layers, the system inputs the updated state vector of the current layer into the next layer's attention module, repeating the self-attention, cross-attention, and confidence evaluation process described above. As the number of layers increases, the geometric verification and information interaction between feature points become more sufficient, and the state vector gradually converges.

[0049] After all iterative layers have been executed (whether the maximum number of layers has been reached or the process has stopped early), the final state vector after matching the third and fourth feature point sets is obtained.

[0050] The system calculates a score matrix based on the two sets of state vectors in the final output. The rows of this matrix correspond to each feature point in the third feature point set, and the columns correspond to each feature point in the fourth feature point set. The value of each element in the matrix represents the similarity (e.g., cosine similarity) between the corresponding two feature points. Finally, the system performs a decoding operation on this score matrix, employing a "nearest neighbor" strategy: a pair of feature points is considered a valid match only if the column index corresponding to the maximum similarity of a real-time image feature point in the row direction of the score matrix matches the row index corresponding to the maximum similarity of the reference image feature point in the column direction.

[0051] All results containing virtual nodes are discarded. Finally, the system outputs a reliable set of matching point pairs that does not contain virtual nodes. The UAV then uses these matching point pairs to calculate its pose and obtain the correct UAV pose, thus enabling UAV navigation.

[0052] This example demonstrates how this embodiment combines lightweight multi-level feature perception with frequency domain information preservation to construct an autonomous navigation scheme for unmanned aerial vehicles (UAVs) that optimizes feature encoding and feature point matching.

[0053] As described above, the scene matching method for UAV navigation provided in this application can acquire real-time UAV images and preset reference images, perform grayscale processing using a preset brightness method, extract first and second feature point sets from the grayscale real-time images and reference images respectively using a set feature extraction network, and add virtual nodes to each feature point set to construct third and fourth feature point sets. The virtual nodes are used to absorb unmatched feature points during the attention matching process. The third and fourth feature point sets are input into a set attention mechanism matching network, and the state vectors of each feature point are iteratively updated through multi-layer stacked attention modules. Based on a preset multi-layer perceptron, a confidence score is output according to the state vector, and a score matrix is ​​calculated. A set of matching point pairs is generated by decoding the score matrix to realize UAV scene matching navigation, thereby improving the scene matching efficiency and accuracy of UAV navigation.

[0054] In one embodiment of the scene matching method for UAV navigation in this application, it may further include the following: Step S201: Construct a feature extraction network, including an encoder structure and a decoder structure; Step S202: The encoder structure adopts several cascaded processing units. Each processing unit includes a lightweight convolution module and a Haar wavelet downsampling module in sequence. The lightweight convolution module includes a depthwise convolutional layer, a convolutional block attention layer, and two cascaded 1×1 pointwise convolutional layers. The layers of the lightweight convolution module are connected by residuals. The Haar wavelet downsampling module is used to achieve a halving of spatial resolution while retaining frequency domain information. Step S203: The decoder structure uses transposed convolution to upsample the output that has been downsampled by the encoder structure.

[0055] Optionally, in this embodiment, this step is the construction of a feature extraction network.

[0056] Specifically, this embodiment designs a lightweight encoder-decoder network to solve the problems of large computational load and poor real-time performance of traditional networks.

[0057] The network first reduces the resolution layer by layer through a downsampling module, performing core feature processing in the low-resolution space, and finally restores the spatial resolution through an upsampling operation. The rationale behind this design is that it greatly expands the network's receptive field, enabling the network to determine feature uniqueness based on global context, while concentrating the vast majority of computation on low-resolution feature maps, thus meeting the real-time requirements of UAVs.

[0058] Furthermore, this network combines an attention mechanism with Haar wavelet downsampling. The network first maps the image into the embedding space through an image patch embedding layer; then, the feature map is sequentially processed through n repeated processing units, each consisting of a lightweight convolutional module (responsible for feature refinement) and a Haar wavelet downsampling module (responsible for reducing resolution); finally, the output stage generates a high-resolution feature point map and a corresponding descriptor map through a parallel decoding head (containing transposed convolution and 1x1 convolution).

[0059] The detailed structure of each part of the lightweight encoder-decoder network is as follows: 1. Coding Network - Image Embedding Layer The image embedding layer is the starting module of the entire feature extraction network. Its function is to efficiently embed the input image, which has an H×W spatial resolution, into a single image. Original image of each channel Converted into the initial feature map .

[0060] This operation is achieved through a single convolution, which is functionally equivalent to segmenting the image into a series of non-overlapping image patches and independently mapping the pixel information of each image patch to a... In the dimensional eigenvectors.

[0061] Specifically, this layer uses There are *n* convolutional kernels, each with a spatial size of P×P and a stride of P, which can be expressed by the following formula:

[0062] in, The input image has dimensions of B represents the batch size; This represents a two-dimensional convolution operation; Weight is the set of convolution kernel weights, which contains... There are 3 independent convolutional kernels, each with a dimension of 1. ; The bias term; Stride is the convolution stride; Kernel Size is the kernel size; and the output of the operation... Dimensions This is the initial feature map for subsequent processing.

[0063] Understandably, the initial feature map includes each image that has undergone preliminary embedding processing from both the grayscale real-time image set and the grayscale reference image set.

[0064] 2. Encoding Network - Lightweight Convolutional Module The lightweight convolutional module is the core feature processing unit of this network. It accepts a feature map as input. Let the feature map input be... Furthermore, it achieves lightweight and efficient feature extraction through a parallel structure containing identity connections and main logic.

[0065] In the main logic path, the input feature map First, it passes through a depthwise convolutional module. The calculation process is as follows: for an input feature map with C channels... (The module has C independent two-dimensional convolution kernels), the c-th independent two-dimensional spatial convolution kernel and The cth channel Perform a convolution operation to obtain the result for channel C in the output image. This operation is repeated independently for all C channels, and the module output is denoted as... Its c-th channel is The formula is expressed as follows:

[0066] Finally, a feature map with C channels is obtained. This operation performs spatial convolution independently on each input channel without cross-mixing between channels, aiming to extract spatial features (such as edges and textures) at low computational cost.

[0067] The output feature map of the subsequent deep convolution The data is fed into a Convolutional Block Attention (CBAM) module, which adaptively refines the features to focus on more important information. The CBAM consists of two concatenated sub-modules. First, The channel attention module (CAM) is then used to determine "what to focus on." This is achieved through global average pooling (...). ) and global max pooling ( The process aggregates channel information. Global average pooling involves independently calculating the average value of H×W pixels for each channel of the (C, H, W) feature map, ultimately obtaining a C-dimensional vector representing its channel-wise average response information. Global max pooling, on the other hand, involves independently finding the maximum pixel value among H×W pixels for each channel, ultimately obtaining a C-dimensional vector representing its channel-wise most salient feature response information. The results of both methods are fed into two multilayer perceptrons (MLPs), and finally, a channel attention map is generated using a sigmoid activation function. The calculation process can be summarized as follows:

[0068] Output of the channel attention module (Right now Multiply by each channel It is then fed into the Spatial Attention Module (SAM), which is used to determine "where to focus".

[0069] Unlike CAM, which compresses spatial dimensions (reducing H×W to 1×1), SAM aims to aggregate information along the channel axis to generate a two-dimensional spatial feature descriptor. Specifically, SAM processes the input feature map... Perform channel-based average pooling and max pooling This means that for each spatial pixel coordinate (i, j) on the feature map, the algorithm calculates the average and maximum values ​​of that point across all input channels.

[0070] This operation produces two independent two-dimensional feature maps, both of which are [size missing]. Subsequently, these two feature maps are concatenated along the channel dimension to form a feature map with 2 channels and a size of ( The concatenated feature map is then fused using a 1×1 convolution to compress the number of channels from 2 back to 1. Finally, a sigmoid activation function is used to generate the final two-dimensional spatial attention map. (size is) The calculation process can be summarized as follows:

[0071] Finally, output the CAM With this spatial attention map Element-wise multiplication is performed to obtain the output feature map after complete refinement by CBAM. .

[0072] The output is then fed into a composite structure consisting of two 1x1 pointwise convolutions, which is used to efficiently mix and transform channel information. First, the feature map undergoes a first 1x1 convolution, which proportionally expands the channel dimensions. Next, this expanded feature map passes through a batch normalization (BN) layer and a ReLU activation function. Then, it is fed into a second 1x1 convolution, which projects its channel dimensions back to the original dimensions. This "expansion-activation-compression" operation can be represented as:

[0073] Finally, residual connections are performed on this module. The output feature map calculated in the main processing path... , compared with the original input feature map retained at the beginning of the process (Identity path) performs element-wise addition. This operation helps alleviate the vanishing gradient problem in deep networks, making the network easier to train. The final output of this module... It can be represented as:

[0074] 3. Encoding Network - Haar Wavelet Downsampling To address the issue of traditional downsampling methods (such as max pooling or stride convolution) losing excessive texture and structural information when reducing resolution, this embodiment employs Haar wavelet transform to downsample the final output of the convolution module. Due to its orthogonality and compactness, the Haar wavelet can effectively reduce spatial resolution while preserving the original signal information to the maximum extent. A two-dimensional discrete wavelet transform is used to decompose the input feature map into four sub-maps. The width and height of these four sub-maps are reduced to half the original image size, while the number of channels remains unchanged. They are: Low-frequency approximation components represent the main structural and contour information of the image.

[0075] Horizontal high-frequency detail component, capturing edges and textures in the horizontal direction.

[0076] Vertical high-frequency detail component, capturing edges and textures in the vertical direction.

[0077] Diagonal high-frequency detail component, capturing edges and textures in the diagonal direction.

[0078] The four subgraphs mentioned above are based on two fundamental one-dimensional functions: the Haar scaling function. and Haar wavelet function The results are obtained through combined processing. These two one-dimensional functions define the most basic pairwise transformation operation. For any one-dimensional sequence, this operation treats it as consecutive and non-overlapping pairs of pixels and computes an output value for each pair. Since only one output value is produced for every two input pixels, the length of the transformed data is reduced to half the original length. The scaling function... As a low-pass filter, its function is to calculate the average value of the pixel pair to preserve smooth background information. Wavelet function As a high-pass filter, its function is to calculate the difference between the pixel pairs to extract the edge details of abrupt changes.

[0079] The two-dimensional transformation processing of feature maps utilizes the separability of images, that is, it is achieved by sequentially applying the above one-dimensional transformation in the horizontal direction (x-axis) and the vertical direction (y-axis):

[0080]

[0081]

[0082]

[0083] Here Only indicate a combination of operations, such as This means that after calculating the average value in pairs along the x-axis, the interpolation is calculated in pairs along the y-axis. This process is performed on each channel of the feature map. This combined operation splits the original image into 2x2 blocks, and each 2x2 pixel block is decomposed into four values, each belonging to a corresponding sub-image, so that the four sub-images contain complete information of the original image in different frequency bands and directions.

[0084] In subsequent network processing, these four sub-images are first concatenated along the channel dimension. This produces a new image with a channel depth four times that of the original feature map. Then, a 1x1 convolutional layer (followed by batch normalization and ReLU) is applied to this concatenated feature map. The purpose of this 1x1 convolution is to fuse the information from the four sub-images and simultaneously adjust the number of channels to match the input requirements of the next stage. In this way, this embodiment preserves richer feature information.

[0085] 4. Decoding Network - Output Stage After the last repeating unit of the feature extraction network is processed, the subsequent output stage is responsible for mapping the deep features into the information needed for specific feature point matching.

[0086] In practice, the terminal feature map is directly split into two parallel, independent decoding head branches. The first branch is the feature point decoding head, which uses a dedicated 1×1 convolutional layer to project high-dimensional features onto a probability space to generate a feature point heatmap. The second branch is the descriptor decoding head, which maps the same terminal feature map to a high-dimensional vector space in parallel through another independent 1×1 convolutional layer to generate a descriptor map.

[0087] Subsequently, the outputs of both convolutional layers are upsampled to the size of the original image to ensure the accuracy of coordinate localization. This embodiment uses transposed convolution as the upsampling method. Through a set of convolutional kernels continuously optimized during training, each pixel on the low-resolution feature map is projected back to a high-resolution grid, reconstructing lost spatial details in the process. By stacking several transposed convolutional layers, the system can intelligently and layer-by-layer recover richly detailed high-resolution features. Finally, the system performs 1x1 convolutions on the upsampled high-resolution feature map to adjust the number of channels, obtaining a feature point heatmap (only one channel) and a descriptor map (the number of channels is the same as the preset descriptor dimension). The precise pixel coordinates (x, y) of the feature points are then determined based on a non-maximum suppression algorithm. This algorithm first selects all pixels with scores higher than a certain confidence threshold T on the heatmap, forming a candidate set S. Then, the algorithm iteratively selects the highest-scoring point from S. And retain it, while removing it from S. All other candidate points in the neighborhood (i.e., within the radius R of the point). This process is repeated until S is empty, thus ensuring that the final output feature points are the most salient points in their local regions.

[0088] Meanwhile, the system can directly extract the corresponding descriptor vector at the same (x, y) position on the corresponding descriptor graph for subsequent feature matching tasks.

[0089] Through step S203, this embodiment constructs a lightweight feature extraction network that can adapt to the complex situations and computational efficiency requirements in the autonomous navigation process of UAVs.

[0090] In one embodiment of the scene matching method for UAV navigation in this application, it may further include the following: Step S301: Extract features from the grayscale real-time image and the grayscale reference image using the lightweight convolution module, determine the corresponding feature maps, and input the obtained feature maps into the Haar wavelet downsampling module; Step S302: The Haar wavelet downsampling module performs a two-dimensional discrete Haar wavelet transform on the feature map output by the lightweight convolution module to determine the corresponding low-frequency approximate component, horizontal high-frequency component, vertical high-frequency component and diagonal high-frequency component. The four sub-maps are then concatenated in the channel dimension and fused by a 1×1 convolution to determine the corresponding concatenated feature map. Step S303: Upsample the spliced ​​feature map according to the decoder structure, and decode the upsampled output through two parallel decoder heads to determine the corresponding feature point heatmap and descriptor map respectively; Step S304: The non-maximum suppression algorithm is used to extract the coordinates of the feature points in the feature point heatmap, and the descriptor vector of each feature point is extracted at the corresponding position in the descriptor map. A first feature point set is constructed based on the descriptor vector of the feature points corresponding to the grayscale real-time image, and a second feature point set is constructed based on the descriptor vector of the feature points corresponding to the grayscale reference image.

[0091] Optionally, in this embodiment, this step is the application process of the lightweight feature extraction network after the feature extraction network is constructed in step S201 above.

[0092] Specifically, the real-time image obtained from the preceding grayscale processing and the baseline image are respectively input into a lightweight convolutional module for feature extraction. This lightweight convolutional module sequentially performs depthwise convolution, a block convolutional attention module (CBAM), and two cascaded 1×1 pointwise convolutions, and adds the module input and output through residual connections. Depthwise convolution performs spatial convolution independently on each input channel with extremely low computational cost, extracting local texture information such as edges and corners; the CBAM module first filters out information-rich channels through channel attention, and then locates discriminative pixel regions in the image through spatial attention; the two 1×1 pointwise convolutions complete the information mixing and dimensionality transformation between channels; the residual connections superimpose the original input features onto the processing result, ensuring smooth gradient backpropagation during deep network training. The feature map output by this module retains key structural information while maintaining the same or slightly reduced spatial resolution compared to the input image. Subsequently, this feature map is fed into a Haar wavelet downsampling module for further processing.

[0093] The Haar wavelet downsampling module performs a two-dimensional discrete Haar wavelet transform on the input feature map. Through a combination of low-pass and high-pass filters, the feature map is divided into several non-overlapping 2×2 pixel blocks based on spatial location. For each pixel block, four output values ​​are calculated: the average of the four values ​​within the pixel block (low-frequency approximation component), the horizontal difference (horizontal high-frequency component), the vertical difference (vertical high-frequency component), and the diagonal difference (diagonal high-frequency component). These four components correspond to the contour structure information, horizontal edge texture, vertical edge texture, and diagonal detail information of the original feature map, respectively. After the transform, the spatial resolution of each component is reduced to half that of the original feature map, but the four components combined completely preserve all the information of the original feature map without the irreversible information loss seen in traditional pooling operations. Subsequently, the system concatenates these four components along the channel dimension to obtain a feature map with four times the number of channels. A 1×1 convolutional layer then performs channel fusion and compression on this concatenated result, generating a concatenated feature map with a channel number adapted to the subsequent network input.

[0094] Subsequently, the decoder structure splits the feature map into two parallel decoder heads: the feature point decoder head projects the feature map into a single-channel feature point heatmap through a 1×1 convolution, where each pixel value in the heatmap represents the probability that a feature point exists at that location; the descriptor decoder head projects the feature map into a multi-channel descriptor map through another independent 1×1 convolution, where each spatial location corresponds to a fixed-dimensional descriptor vector.

[0095] During the decoding process of each decoding head in the decoder, transposed convolution is used for layer-by-layer upsampling. By inserting zero values ​​between pixels in the feature map and performing standard convolution operations, the low-resolution feature map is gradually restored to high resolution. Unlike fixed upsampling methods such as bilinear interpolation, the kernel parameters of transposed convolution can be learned during training, enabling the network to intelligently reconstruct compressed details in the image according to task requirements. Finally, the decoding head outputs its respective results, feature point heatmaps, and descriptor maps.

[0096] A non-maximum suppression algorithm is used to process the feature point heatmap. First, all pixels in the heatmap with a confidence level above a preset threshold are selected as candidate points. Then, they are sorted from highest to lowest confidence level, and the highest-scoring point is retained while all other candidate points within its neighborhood are deleted. This process is repeated until the candidate point set is empty. The remaining points are the precise coordinates of the feature points. The system extracts descriptor vectors from the corresponding positions in the descriptor map based on these coordinates. All feature points and their descriptors extracted from the real-time image are aggregated into a first feature point set, and all feature points and their descriptors extracted from the baseline image are aggregated into a second feature point set.

[0097] Through step S304, this embodiment successfully ensures that the output feature points are locally unique in space, and each feature point carries a discriminative vector representation that can be used for subsequent matching.

[0098] In one embodiment of the scene matching method for UAV navigation in this application, it may further include the following: Step S401: Self-attention unit, used to perform self-attention calculation among all feature points within the same image, the formula is:

[0099] in, For self-attention score, Let i be the query vector for point i. Let i be the key vector. Rotation encoding for the relative positions between points i and j; Step S402: Cross-attention unit, used to perform bidirectional attention calculation between feature points in the real-time image and the reference image, the formula is:

[0100] in, For the score of cross attention, The values ​​belong to the corresponding index positions in the two graphs.

[0101] Optionally, in this embodiment, this step is the construction process of the attention mechanism matching network.

[0102] Specifically, after obtaining feature points and their descriptors from two images (real-time and baseline images) through a lightweight feature extraction network, an efficient and accurate matching algorithm is needed to establish the correspondence between them. Traditional feature matching algorithms (such as ORB) rely heavily on the similarity of local descriptors. When there are large scale changes, rotations, viewpoint differences, or heterogeneous differences between images, the descriptors themselves will change, leading to a high mismatch rate or even failure to match effectively.

[0103] To address this issue, this embodiment employs a high-precision feature matching algorithm based on a deep learning attention mechanism. This algorithm can simulate a graph neural network, utilize the global contextual relationships of feature points to infer matching pairs, and incorporates lightweight improvements to meet the real-time requirements of UAV platforms.

[0104] Specifically, we first define the input to the attention mechanism matching algorithm (i.e., step S102). We take the feature point sets of two images (e.g., image A (first feature point set) has m features, and image B (second feature point set) has n features) as input, where each feature is a d-dimensional descriptor. To handle situations where the number of feature points selected for the two images may be unequal in reality, and where feature points cannot be matched due to occlusion or viewpoint, an additional "unmatched" node is added to each of the two feature point sets. Therefore, the actual input processed is (m+1) features from image A (third feature point set) and (n+1) features from image B (fourth feature point set).

[0105] The matching process is achieved by stacking multiple processing layers with identical structures. To meet the stringent real-time requirements of UAV navigation, this embodiment limits the total number of network layers to a small number, in order to strike a balance between matching accuracy and computational speed.

[0106] In the inference of each layer, the algorithm executes two key steps sequentially: self-attention and cross-attention units. The information computed by each attention unit is superimposed on the feature vector output by the previous unit through residual connections. The input to this step is the third and fourth feature point sets.

[0107] The first step is the self-attention unit. This structure enhances the context awareness of features. In this step, each feature point in image A pays attention to all other feature points in image A (the same applies to image B; this process occurs symmetrically and independently on images A and B). In this way, the state vector of each feature point is updated (overlaid) with feature information from its neighborhood and even the entire image. This process introduces Rotation Position Encoding (RoPE), making the attention score depend not only on feature similarity but also on the relative influence between points. Self-attention score (The attention given by point i to point j) can be expressed as:

[0108] in, Let i be the query vector for point i. Let i be the key vector. Rotation encoding of the relative positions between points i and j.

[0109] Following the self-attention enhancement features, the second step in this layer is the cross-attention layer. This layer is the core of the matching process. Here, feature points in image A that already contain internal contextual information are allowed to attend to all points in image B (and vice versa). The network evaluates the similarity and matchability between all feature point pairs in the two images through this bidirectional cross-attention computation. The computed cross-attention information is symmetrically superimposed again onto the feature vectors of images A and B via residual connections (i.e., image A state superimposed with image B information, and image B state superimposed with image A information). The cross-attention score... It can be represented as:

[0110] in, The values ​​belong to the corresponding index positions in the two graphs.

[0111] Through step S402, this embodiment successfully constructed an attention mechanism matching network, laying a solid foundation for subsequent feature point matching.

[0112] In one embodiment of the scene matching method for UAV navigation in this application, it may further include the following: Step S501: For the initial state vector of each feature point, perform self-attention calculation according to the self-attention unit, and perform residual connection between the self-attention calculation result and the initial state vector to determine the corresponding intermediate state vector; Step S502: The cross-attention unit receives the intermediate state vector, performs cross-attention calculation, and performs residual connection between the cross-attention calculation result and the intermediate state vector to determine the corresponding updated state vector.

[0113] Optionally, in this embodiment, this step is the application process of the attention mechanism matching network.

[0114] Self-attention computation is performed on the initial state vector of each feature point. The self-attention mechanism applies to all feature points within the same image; that is, feature points within the real-time image and feature points within the reference image are mutually attenuating, and the computation processes for the two images are symmetrical and independent. Specifically, for each feature point in the current image, the system calculates its attention weight with all other feature points in the image. The weight depends on the similarity of the current state vectors of the two points and their relative geometric relationship. Subsequently, based on these attention weights, the system performs weighted fusion of the state vectors of all other feature points, generating an attention result that aggregates the contextual information of the entire image. This result reflects the relative position of the current feature point within its own image, its local neighborhood structure, and its global semantic environment.

[0115] After calculating the self-attention result, a residual concatenation operation is performed between it and the initial state vector of the current feature point, i.e., the two are added element-wise to obtain the intermediate state vector of that feature point. This preserves the original individual information of the feature point while superimposing contextual information extracted from other feature points in the entire image, so that the updated vector neither loses the original features nor loses the richer global perception capability.

[0116] Subsequently, the intermediate state vectors obtained above are fed into the cross-attention unit. Unlike self-attention, cross-attention operates between two different sets of feature points: feature points in the real-time image attend to all feature points in the reference image, and feature points in the reference image also attend to all feature points in the real-time image. The system calculates the attention weight between the intermediate state vector of each feature point in the real-time image and the intermediate state vectors of all feature points in the reference image, and then performs a weighted fusion of the state vectors of the feature points in the reference image to obtain the cross-image attention result. This result reflects the similarity assessment between the current feature point and all potential matching points in another image.

[0117] Finally, the system performs a residual concatenation between the cross-attention calculation result and the intermediate state vector of the feature point, adding them element-wise to obtain the updated state vector of the feature point. This allows the feature point to retain its internal image context while further incorporating cross-image correlation information, providing a richer and more accurate feature representation for subsequent iterative matching levels.

[0118] Through step S502, the attention mechanism matching network of this embodiment performs "intra-image information aggregation" and "inter-image information exchange" sequentially in each attention module by executing self-attention and cross-attention in sequence, realizing progressive feature enhancement from local to global and from single image to dual image, laying the foundation for subsequent feature point matching.

[0119] In one embodiment of the scene matching method for UAV navigation in this application, it may further include the following: Step S601: Input the updated state vector corresponding to each feature point into a preset multilayer perceptron, and determine the confidence score of the feature point output by the multilayer perceptron. Step S602: When the confidence score meets the preset convergence condition, the attention mechanism matching network stops iterating and outputs the state vector of the current layer feature point as the final state vector. The convergence condition includes the following among all feature points in the current layer: the proportion of the number of feature points with a confidence score greater than the preset confidence threshold to the total number of feature points exceeds the global proportion threshold.

[0120] Optionally, in this embodiment, the algorithm performs a dynamic early stopping process after the cross-attention calculation of the Lth layer is completed.

[0121] Specifically, to avoid wasting unnecessary computation on "simple" image pairs, we insert a compact multilayer perceptron (MLP) classifier that infers the confidence score of the predicted assignment for each point in images A and B, respectively. As shown in the formula below:

[0122] in, This is the state vector of feature point i in the current layer. When the confidence level of a point... Above a certain preset threshold At that point, the system considers the point "trustworthy." Subsequently, the system calculates whether the percentage of all "trustworthy" points exceeds a global threshold. The final judgment conditions are as follows:

[0123] If the condition is met after the inference at layer L (i.e., the proportion of "credible" points is high enough), the inference process will stop early and enter the decoding stage with the final feature vector from layer L. If the criterion is not met, and the current layer number L is less than the set maximum layer limit, the algorithm will enter layer L+1 with the updated feature vector and repeat the self-attention and cross-attention processes described above. The self-attention at layer L+1 will use the matching guesses and mutual information from the cross-attention at layer L to perform internal context verification, thereby correcting errors and resolving fuzzy matches in the cross-attention at layer L+1.

[0124] Through step S602, this embodiment successfully improves matching efficiency by introducing a multilayer perceptron to control the early stopping of the attention matching process.

[0125] In one embodiment of the scene matching method for UAV navigation in this application, it may further include the following: Step S701: Based on the final state vector, construct the cosine similarity matrix of the third feature point set and the fourth feature point set; Step S702: Perform feature point matching on the cosine similarity matrix according to the nearest neighbor algorithm to determine the corresponding set of matching point pairs.

[0126] Optionally, in this embodiment, after the above-mentioned weighting and stopping control, whether the algorithm stops early at layer L or stops after reaching the maximum number of layers, it will eventually enter the decoding stage.

[0127] The decoding stage is used to decode the final feature vector set output when the algorithm stops. (Real-time graph) and The baseline image is converted into specific matching pairs. During decoding, the final score matrix between the feature set (m+1 features) of image A and the feature set (n+1 features) of image B is first calculated using matrix multiplication. The matrix The value is the cosine similarity between the i-th feature vector of A and the j-th feature vector of B. Finally, the algorithm... Score matrix Perform "nearest neighbor" decoding on the above: points in graph A And a point in Figure B Only when exist The The maximum value in the row is located at Column, and exist The The maximum value in the column is located at Only when a row is matched is it considered a valid match. All other cases are considered unmatched. Furthermore, all points whose best match is in the last row or last column of the matrix are considered unmatched.

[0128] A high-precision matching feature point set between the real-time image and the baseline image is obtained through a feature matching algorithm.

[0129] in Let (u, v) be the pixel coordinates in the real-time image. The geographic coordinates (x, y) in the baseline map. Finally, through a two-step solution module, the absolute position coordinates and heading of the UAV at the current moment are estimated from the matching point set, thus completing the UAV navigation.

[0130] Through step S702, this embodiment successfully constructs a similarity matrix for feature matching and achieves efficient and lightweight UAV navigation.

[0131] To improve the efficiency and accuracy of scene matching in UAV navigation, this application provides an embodiment of a scene matching device for UAV navigation that implements all or part of the aforementioned scene matching method for UAV navigation. See [link to embodiment]. Figure 2 The scene matching device for UAV navigation specifically includes the following components: The image acquisition module 10 is used to acquire real-time images and preset reference images captured by the drone, and to convert the real-time images and the reference images to grayscale based on the preset brightness method, so as to determine the corresponding grayscale real-time images and grayscale reference images respectively. Feature extraction module 20 is used to extract features from the grayscale real-time image and the grayscale reference image according to the set feature extraction network, determine the corresponding first feature point set and second feature point set respectively, add virtual nodes to the first feature point set and the second feature point set respectively, and determine the corresponding third feature point set and fourth feature point set. The virtual nodes are used to participate in information aggregation during the attention matching process and absorb unmatched feature points. The feature point scene matching module 30 is used to input the third feature point set and the fourth feature point set into a preset attention mechanism matching network. The attention mechanism matching network iteratively updates the state vector of each feature point through a multi-layer stacked attention module. Based on the state vector obtained by the preset multilayer perceptron based on the attention module, it determines the confidence score corresponding to each feature point. When the confidence score meets the preset convergence condition, it determines the final state vector of each feature point. Based on the final state vector, it determines the score matrix between the third feature point set and the fourth feature point set. Based on the score matrix, it determines the corresponding set of matching point pairs, thereby realizing scene matching for UAV navigation.

[0132] As described above, the scene matching device for UAV navigation provided in this application can acquire real-time images of the UAV and a preset reference image, perform grayscale processing using a preset brightness method, extract the first and second feature point sets of the grayscale real-time image and the reference image respectively using a set feature extraction network, and add virtual nodes to each feature point set to construct the third and fourth feature point sets. The virtual nodes are used to absorb unmatched feature points during the attention matching process. The third and fourth feature point sets are input into a set attention mechanism matching network, and the state vector of each feature point is iteratively updated through a multi-layer stacked attention module. Based on a preset multi-layer perceptron, a confidence score is output according to the state vector, and a score matrix is ​​calculated. The set of matching point pairs is generated by decoding the score matrix to realize UAV scene matching navigation, thereby improving the scene matching efficiency and accuracy of UAV navigation.

[0133] To further illustrate this solution, this application also provides a specific application example of using the above-mentioned scene matching device for UAV navigation to implement the scene matching method for UAV navigation, which specifically includes the following: Optionally, in this embodiment, the training of each system model includes: The system model in this embodiment includes a lightweight feature extraction network and a high-precision matching network, and its final performance is guaranteed by a two-stage supervised training strategy.

[0134] The training data uses a heterogeneous image set containing known real matching point pairs. In the first training phase, the system's backbone network, the feature extractor, is trained end-to-end with the complete matching network, which includes all iterative layers. During this phase, dynamic early stopping is disabled, allowing the network to focus on learning the matching task itself. The training objective of this phase is to optimize the final score matrix output after the maximum number of iterations. Supervision is performed using negative log-likelihood loss, which ensures that for each feature point in graph A (assuming it has m feature points), the loss function guarantees the accuracy of the prediction. The true matching point of the target in graph B (assuming it has n feature points). The corresponding predicted probability The matching loss is maximized. It can be represented as:

[0135] After the backbone network training in the first phase converges, the system enters the second training phase. In this phase, all weights of the backbone network (feature extractor and matching network) are frozen, and the system focuses on training a dynamic early stopping MLP network. This MLP extracts feature vectors at the output of each layer of the backbone network and outputs the corresponding confidence score. This score reflects the convergence and stability of the current feature vector state.

[0136] Supervision labels for this training phase It is dynamically generated as follows: For an input sample, firstly, the backbone network trained in the first stage (without early stopping) is run to its maximum layer number, and its output at the final layer is recorded. Matching is performed as described above, and the matching result is stored in a decision vector (length m+n, each position value is the index of its matching point, and unmatched points are marked as -1). Then, for the same input, the model's output at layer L is observed, and its decision vector calculated based on the current layer's output is compared with the decision vector of the aforementioned final layer. If the matching result of the two decision matrices for the i-th vector is consistent, then... , indicating "converged"; if inconsistent, then This indicates "not converged". Therefore, a binary cross-entropy loss function is used for training.

[0137] The two-stage training strategy described above ensures that the backbone network focuses on maximizing matching accuracy, while the early stopping MLP focuses on efficiently determining whether the matching decision for each point has converged, thus achieving real-time matching process with minimal sacrifice of accuracy.

[0138] Optionally, in this embodiment, the system workflow is as follows: The UAV scene matching navigation system in this embodiment integrates a lightweight feature extraction module and a high-precision real-time matching module. It is a closed-loop navigation scheme from image input to positioning output. The workflow of this system is divided into two stages: offline reference map construction and online real-time navigation.

[0139] During the offline construction phase, the system first loads a high-resolution benchmark map and preprocesses it using the Luminance grayscale conversion algorithm. Subsequently, a lightweight feature extraction network is used to process the benchmark map, and the extracted feature points, descriptors, and their corresponding geographic coordinates are stored in an offline feature benchmark map database for use during online navigation.

[0140] During the online real-time navigation phase, the system enters a real-time loop. The real-time scene image captured by the UAV first undergoes the same grayscale conversion and lightweight feature extraction as in the offline phase. Then, the system feeds the real-time extracted feature set together with the candidate benchmark feature set retrieved from the database into a high-precision matching algorithm. Through iterative self-attention and cross-attention, supplemented by a dynamic early stopping mechanism, high-confidence matching point pairs are output. Finally, the heading of the obtained high-confidence matching point set is calculated using coordinate estimation based on K-means clustering and heading calculation based on multi-point vector angle differences, outputting the UAV's current precise coordinates and heading. This positional information can be fed back to update the search area for the next frame, thereby improving the overall frame rate of the system.

[0141] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor performs the above-described scene matching method for UAV navigation.

[0142] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described scene matching method for UAV navigation.

[0143] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described scene matching method for UAV navigation.

[0144] In this embodiment of the invention, by acquiring real-time images of a UAV and a preset reference image, grayscale processing is performed using a preset brightness method. A set feature extraction network is used to extract the first and second feature point sets of the grayscale real-time image and the reference image, respectively. Virtual nodes are added to each feature point set to construct the third and fourth feature point sets. The virtual nodes are used to absorb unmatched feature points during the attention matching process. The third and fourth feature point sets are input into a set attention mechanism matching network. The state vectors of each feature point are iteratively updated through multi-layer stacked attention modules. Based on a preset multi-layer perceptron, a confidence score is output according to the state vector, and a score matrix is ​​calculated. A set of matching point pairs is generated by decoding the score matrix to realize UAV scene matching navigation, thereby improving the scene matching efficiency and accuracy of UAV navigation.

[0145] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0147] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0148] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0149] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A scene matching method for UAV navigation, characterized in that, The method includes: Acquire real-time images and preset reference images captured by a drone, and perform grayscale conversion on the real-time images and the reference images based on a preset brightness method to determine the corresponding grayscale real-time images and grayscale reference images respectively. Based on the set feature extraction network, features are extracted from the grayscale real-time image and the grayscale reference image, respectively determining the corresponding first feature point set and second feature point set. Virtual nodes are added to the first feature point set and the second feature point set respectively, and the corresponding third feature point set and fourth feature point set are determined. The virtual nodes are used to participate in information aggregation during the attention matching process and absorb unmatched feature points. The third and fourth feature point sets are input into a pre-defined attention mechanism matching network. The attention mechanism matching network iteratively updates the state vector of each feature point through a multi-layer stacked attention module. Based on the state vector obtained by the multi-layer perceptron based on the attention module, a confidence score is determined for each feature point. When the confidence score meets a pre-defined convergence condition, the final state vector of each feature point is determined. Based on the final state vector, a score matrix is ​​determined between the third and fourth feature point sets. Based on the score matrix, a corresponding set of matching point pairs is determined, thereby realizing scene matching for UAV navigation.

2. The scene matching method for UAV navigation according to claim 1, characterized in that, The defined feature extraction network includes: Construct a feature extraction network, including encoder and decoder structures; The encoder structure employs several cascaded processing units, each of which sequentially includes a lightweight convolution module and a Haar wavelet downsampling module. The lightweight convolution module includes a deep convolutional layer, a convolutional block attention layer, and two cascaded 1×1 pointwise convolutional layers. The layers of the lightweight convolution module are connected by residuals. The Haar wavelet downsampling module is used to achieve a halving of spatial resolution while preserving frequency domain information. The decoder structure employs transposed convolution to upsample the output that has been downsampled by the encoder structure.

3. The scene matching method for UAV navigation according to claim 2, characterized in that, The step of extracting features from the grayscale real-time image and the grayscale reference image according to the set feature extraction network, and determining the corresponding first feature point set and second feature point set respectively, includes: The lightweight convolution module extracts features from the grayscale real-time image and the grayscale reference image, determines the corresponding feature maps, and inputs the obtained feature maps into the Haar wavelet downsampling module. The Haar wavelet downsampling module performs a two-dimensional discrete Haar wavelet transform on the feature map output by the lightweight convolution module to determine four sub-maps: low-frequency approximate component, horizontal high-frequency component, vertical high-frequency component, and diagonal high-frequency component. The four sub-maps are then concatenated in the channel dimension and fused by a 1×1 convolution to determine the corresponding concatenated feature map. The spliced ​​feature map is upsampled according to the decoder structure, and the upsampled output is decoded by two parallel decoder heads to determine the corresponding feature point heatmap and descriptor map respectively. The non-maximum suppression algorithm is used to extract the coordinates of the feature points in the feature point heatmap, and the descriptor vector of each feature point is extracted at the corresponding position in the descriptor map. A first feature point set is constructed based on the descriptor vectors of the feature points corresponding to the grayscale real-time image, and a second feature point set is constructed based on the descriptor vectors of the feature points corresponding to the grayscale reference image.

4. The scene matching method for UAV navigation according to claim 1, characterized in that, The attention module includes: A self-attention unit is used to perform self-attention calculations among all feature points within the same image, and the formula is: in, For self-attention score, Let i be the query vector for point i. Let i be the key vector. Rotation encoding for the relative positions between points i and j; The cross-attention unit is used to perform bidirectional attention calculation between feature points in the real-time image and the reference image, and the formula is as follows: in, For the score of cross attention, The values ​​belong to the corresponding index positions in the two graphs.

5. The scene matching method for UAV navigation according to claim 4, characterized in that, The attention mechanism matching network iteratively updates the state vector of each feature point through multiple stacked attention modules, including: For the initial state vector of each feature point, self-attention calculation is performed according to the self-attention unit. The self-attention calculation result is residually connected with the initial state vector to determine the corresponding intermediate state vector. The cross-attention unit receives the intermediate state vector, performs cross-attention calculation, and performs residual connection between the cross-attention calculation result and the intermediate state vector to determine the corresponding updated state vector.

6. The scene matching method for UAV navigation according to claim 5, characterized in that, The step of determining the confidence score corresponding to each feature point based on the state vector obtained by the attention module of the preset multilayer perceptron, and determining the final state vector of each feature point when the confidence score satisfies the preset convergence condition, includes: The updated state vector corresponding to each feature point is input into a preset multilayer perceptron to determine the confidence score of the feature point output by the multilayer perceptron. When the confidence score meets the preset convergence condition, the attention mechanism matching network stops iterating and outputs the state vector of the current layer feature point as the final state vector. The convergence condition includes the following among all feature points in the current layer: the proportion of the number of feature points with a confidence score greater than a preset confidence threshold to the total number of feature points exceeds the global proportion threshold.

7. The scene matching method for UAV navigation according to claim 1, characterized in that, The step of determining a score matrix between the third feature point set and the fourth feature point set based on the final state vector, and determining the corresponding set of matching point pairs based on the score matrix, includes: Based on the final state vector, construct the cosine similarity matrix between the third feature point set and the fourth feature point set; The cosine similarity matrix is ​​matched for feature points using the nearest neighbor algorithm to determine the corresponding set of matching point pairs.

8. A scene matching device for UAV navigation, characterized in that, The device includes: The image acquisition module is used to acquire real-time images and preset reference images captured by the drone, and to convert the real-time images and the reference images to grayscale based on a preset brightness method, so as to determine the corresponding grayscale real-time images and grayscale reference images respectively. The feature extraction module is used to extract features from the grayscale real-time image and the grayscale reference image according to the set feature extraction network, determine the corresponding first feature point set and second feature point set respectively, add virtual nodes to the first feature point set and the second feature point set respectively, and determine the corresponding third feature point set and fourth feature point set. The virtual nodes are used to participate in information aggregation during the attention matching process and absorb unmatched feature points. The feature point scene matching module is used to input the third feature point set and the fourth feature point set into a preset attention mechanism matching network. The attention mechanism matching network iteratively updates the state vector of each feature point through a multi-layer stacked attention module. Based on the state vector obtained by the multi-layer perceptron based on the attention module, it determines the confidence score corresponding to each feature point. When the confidence score meets the preset convergence condition, it determines the final state vector of each feature point. Based on the final state vector, it determines the score matrix between the third feature point set and the fourth feature point set. Based on the score matrix, it determines the corresponding set of matching point pairs, thereby realizing scene matching for UAV navigation.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the scene matching method for UAV navigation as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the scene matching method for UAV navigation as described in any one of claims 1 to 7.

Images