A UAV assisted positioning method and device based on image matching

Abstract

The present invention discloses a method and device for assisting drone positioning based on image matching. The method comprises: obtaining an overhead image captured by a drone at the current moment to obtain an image of the drone to be positioned; extracting drone feature information from the image of the drone to be positioned using a target image matching model, and matching the drone feature information with satellite feature information from multiple satellite images to obtain a target satellite image; pairing feature points of the image of the drone to be positioned and the target satellite image to obtain first paired feature points; calculating a first homography matrix between the image of the drone to be positioned and the target satellite image; and using the first homography matrix to map the position information in the target satellite image to the image of the drone to be positioned to obtain first predicted position information of the drone. This solution can capture highly robust multi-scale object features, improve the accuracy of image matching, and enhance the precision of drone positioning.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and device for assisting unmanned aerial vehicle (UAV) positioning based on image matching. Background Art

[0002] A UAV (Unmanned Aerial Vehicle) is an unmanned flying device. With its advantages of good maneuverability, flexibility, small size, high intelligence and low detection, it is widely used in material transportation, military reconnaissance, industrial inspection, remote sensing mapping, rescue operations, weather monitoring, geographic mapping and crop production.

[0003] Satellite positioning systems, inertial positioning systems, and visual positioning systems are mainstream and mature methods for drone positioning. Satellite positioning systems, a common method for drone positioning, are often subject to interference and rejection during use. In military applications, they are even susceptible to malicious radio interference or shielding. Inertial positioning requires heavy sensors and requires precise knowledge of the carrier's position at the start of flight. Inertial measurements are used to estimate position changes after the start of flight, resulting in large cumulative errors. Visual positioning systems utilize various sensors onboard the drone to acquire information about its current environment at specific angles. This information is then paired with satellite imagery or other pre-processed information to determine the drone's own position.

[0004] The positioning targets of drone systems must not only meet strict time constraints, but also strict positioning errors. While drone visual positioning systems offer strong anti-interference capabilities, no cumulative positioning errors, and low operational dependency, they also face challenges that cannot be ignored: 1) Satellite images and drone images are heterogeneous. Due to the different acquisition methods used for the two image sources, drone images and remote sensing images may differ significantly in imaging style, time scale, and coverage scale. 2) Drones all have a certain flight altitude, and the actual coverage area of ​​the images is extremely rich in content, with sparse textures and details. Time changes further increase the difficulty of finding identical target pairs, reducing the success rate of image matching. 3) Drones are embedded platforms with limited computing power and are not suitable for deploying large-scale neural network models. Designing an efficient image matching algorithm without sacrificing accuracy is highly challenging. Summary of the Invention

[0005] The present invention aims to at least solve the technical problems existing in the prior art. To this end, in a first aspect, the present invention proposes a drone-assisted positioning method based on image matching, the method comprising:

[0006] During the flight of the drone, an overhead image captured by the drone at the current moment is obtained to obtain an image of the drone to be located at the current moment;

[0007] Inputting the drone image to be located into a target image matching model, extracting drone feature information of the drone image to be located using the target image matching model, and matching the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located;

[0008] Extracting feature points of the image of the unmanned aerial vehicle to be positioned and the target satellite image, and performing feature point pairing on the image of the unmanned aerial vehicle to be positioned and the target satellite image to obtain first paired feature points;

[0009] Calculating a first homography matrix between the image of the unmanned aerial vehicle to be located and the target satellite image using the position coordinate information of the first paired feature points;

[0010] The first homography matrix is ​​used to map the position information in the target satellite image to the image of the unmanned aerial vehicle to be located, so as to obtain the first predicted position information of the unmanned aerial vehicle at the current moment.

[0011] Optionally, the target image matching model includes an image matching layer and a downsampling layer, and extracting the drone feature information of the drone image to be located by using the target image matching model includes:

[0012] Downsampling the image of the drone to be located using the downsampling layer to obtain a downsampled image;

[0013] The image matching layer is used to extract and combine multi-scale features of the downsampled image to obtain drone feature information.

[0014] Optionally, the image matching layer includes a spatial attention mechanism unit, a depthwise separable convolution unit, and a global response normalization unit. The image matching layer is used to extract and combine multi-scale features of the downsampled image to obtain drone feature information, including:

[0015] Dynamically adjusting weights of different feature vectors in the downsampled image using the spatial attention mechanism unit to obtain first feature information;

[0016] Inputting the first feature information into the depthwise separable convolution unit, and performing a separate convolution on the feature information of each channel using the convolution kernel of each channel to obtain feature information corresponding to each channel;

[0017] Use a 1x1 convolution kernel to linearly combine the feature information corresponding to each of the channels to obtain second feature information;

[0018] The second feature information is input into the global response normalization unit, and global feature aggregation, feature normalization and feature calibration are performed on the second feature information to obtain drone feature information.

[0019] Optionally, the utilizing the spatial attention mechanism unit to dynamically adjust weights of different feature vectors in the downsampled image to obtain first feature information includes:

[0020] Performing a global maximum pooling operation and an average pooling operation on the downsampled image to obtain two one-dimensional vectors having the same number of channels as the feature vector of the downsampled image;

[0021] Input the two one-dimensional vectors into a convolution layer with a convolution kernel size of 7 to perform a convolution operation to obtain a convolution feature vector;

[0022] Input the convolution feature vector into the activation function for activation to obtain the weight coefficient corresponding to each feature vector;

[0023] The weight coefficient is multiplied by a feature vector corresponding to the downsampled image to obtain first feature information.

[0024] Optionally, inputting the second feature information into the global response normalization unit, performing global feature aggregation, feature normalization, and feature calibration on the second feature information to obtain the UAV feature information includes:

[0025] Performing feature aggregation on the feature information of each channel in the second feature information using a global function to obtain a first vector corresponding to each channel; the first vector is a scalar that aggregates the statistical features of the feature information of the current channel;

[0026] Normalizing each of the first vectors using a standard division normalization function to obtain a feature normalization score; the feature normalization score represents the importance of the current channel relative to other channels;

[0027] The second feature information is calibrated in response using the feature normalization score and preset learnable parameters to obtain drone feature information.

[0028] Optionally, before obtaining the overhead image captured by the drone at the current moment, the method further includes:

[0029] Acquire a historical data set, wherein the historical data set includes n matching pairs, each of which is a historical drone image and a historical satellite image closest to a geographical location represented by a pixel center of the historical drone image;

[0030] Performing feature descriptions on the historical UAV images and historical satellite images in the n matching pairs respectively to obtain n historical UAV image feature information and n historical satellite image feature information;

[0031] Calculate the distance information between each pair of historical UAV image feature information and historical satellite image feature information respectively;

[0032] Determining, based on the distance information, a non-matching historical satellite image that is closest to the characteristic information of each of the historical drone images, and determining a non-matching historical drone image that is closest to the characteristic information of each of the historical satellite images;

[0033] Determining first distance information of the non-matching historical satellite image corresponding to the historical drone image in each matching pair, and second distance information of the non-matching historical drone image corresponding to the historical satellite image in the matching pair;

[0034] Selecting an image corresponding to a smaller value of the first distance information and the second distance information as the most similar negative sample of the matching pair, and forming a paired feature triplet with the matching pair and the corresponding most similar negative sample;

[0035] The paired feature triplets are used as training data to train the initial image matching model to obtain a target image matching model.

[0036] Optionally, matching the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located includes:

[0037] Matching the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a plurality of matching degrees;

[0038] The matching degrees are sorted in descending order, and the satellite image ranked first is obtained as the target satellite image that matches the drone image to be located.

[0039] Optionally, after obtaining a target satellite image that matches the image of the unmanned aerial vehicle to be located, the method further includes:

[0040] Obtain the top K satellite images as candidate satellite images;

[0041] Determine the homography matrix between each of the to-be-located UAV images and the candidate satellite images to obtain K second homography matrices;

[0042] The second homography matrix is ​​used to map the position information in the candidate satellite image to the image of the unmanned aerial vehicle to be located, thereby obtaining a plurality of candidate predicted position information of the unmanned aerial vehicle.

[0043] Optionally, after obtaining the first predicted position information of the UAV at the current moment, the method further includes:

[0044] Acquire flight data of the UAV before the current moment, and fit the flight trajectory of the UAV using the flight data and the IMM model to obtain second predicted position information of the UAV at the current moment;

[0045] Determining an area within a preset distance range of the second predicted position information as the predicted position range of the drone at the current moment;

[0046] determining whether the first predicted location information is within the predicted location range;

[0047] If the target position is within the predicted position range, the first predicted position information is determined as the target position information of the UAV at the current moment;

[0048] If it is not within the predicted position range, selecting at least one target predicted position information within the predicted position range from the multiple candidate predicted position information;

[0049] Obtaining at least one candidate satellite image corresponding to the at least one target predicted position information, and obtaining a satellite image that best matches the image of the drone to be located among the at least one candidate satellite image;

[0050] The candidate predicted position information corresponding to the most matching satellite image is determined as the target position information of the UAV at the current moment.

[0051] A second aspect of the present invention provides a drone assisted positioning device based on image matching, the device comprising:

[0052] The UAV image acquisition module is used to acquire the overhead image collected by the UAV at the current moment during the UAV flight process, and obtain the image of the UAV to be located at the current moment;

[0053] a matching module, configured to input the drone image to be located into a target image matching model, extract drone feature information of the drone image to be located using the target image matching model, and match the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located;

[0054] A feature point pairing module is used to extract feature points of the image of the unmanned aerial vehicle to be positioned and the target satellite image, and perform feature point pairing on the image of the unmanned aerial vehicle to be positioned and the target satellite image to obtain a first paired feature point;

[0055] a homography matrix calculation module, configured to calculate a first homography matrix between the image of the unmanned aerial vehicle to be located and the target satellite image using the position coordinate information of the first paired feature points;

[0056] A mapping module is used to map the position information in the target satellite image to the image of the unmanned aerial vehicle to be located using the first homography matrix to obtain the first predicted position information of the unmanned aerial vehicle at the current moment.

[0057] A third aspect of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the image matching-based drone assisted positioning method as described in the first aspect.

[0058] In a sixth aspect, the present invention proposes a computer-readable storage medium, which stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the image matching-based drone assisted positioning method as described in the first aspect.

[0059] The embodiments of the present invention have the following beneficial effects:

[0060] In an embodiment of the present invention, during the flight of a drone, an overhead image captured by the drone at a current moment is obtained to obtain the drone image to be located at the current moment; the drone image to be located is input into a target image matching model, the drone feature information of the drone image to be located is extracted using the target image matching model, and the drone feature information is matched with the satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located; feature points of the drone image to be located and the target satellite image are extracted, and the feature points of the drone image to be located and the target satellite image are paired to obtain a first paired feature point; the position coordinate information of the first paired feature point is used to calculate a first homography matrix between the drone image to be located and the target satellite image; the position information in the target satellite image is mapped to the drone image to be located using the first homography matrix to obtain the first predicted position information of the drone at the current moment. This solution utilizes a learning-based target image matching network for feature extraction and matching, and uses satellite images that have obvious visual differences from the drone images to be located for image matching. It can capture highly robust multi-scale object features that are not easily disturbed by changes in the external environment, improve the accuracy of image matching, and enhance the accuracy of drone positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flowchart of a method for assisting drone positioning based on image matching provided by an embodiment of the present invention;

[0062] Figure 2 This is a diagram of the overall framework for image processing using an image matching network according to an embodiment of the present invention;

[0063] Figure 3 An internal structure diagram of an image matching layer provided by an embodiment of the present invention;

[0064] Figure 4 A diagram showing the internal structure of a multi-scale convolutional network provided by an embodiment of the present invention;

[0065] Figure 5 A schematic diagram of generating paired feature triples provided by an embodiment of the present invention;

[0066] Figure 6 This is a structural block diagram of a drone-assisted positioning device based on image matching provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0068] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "multiple" means two or more. In addition, the use of "based on" or "according to" means openness and inclusiveness, because the process, steps, calculations or other actions "based on" or "according to" one or more of the conditions or values ​​may be based on additional conditions or values ​​beyond the stated in practice.

[0069] Figure 1 This is a flowchart of a method for assisting drone positioning based on image matching provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0070] Step 101: During the flight of a drone, an overhead image captured by the drone at a current moment is acquired to obtain an image of the drone to be located at the current moment.

[0071] Drones can be controlled by remote controllers or autopilot systems to achieve a variety of shooting angles and effects. For example, drones can overlook the earth from a high vantage point to obtain aerial images.

[0072] During the flight, the drone continuously captures overhead images at preset intervals. The currently captured overhead image is used as the drone image to be located, and the location of the drone image will be determined later.

[0073] Step 102: Input the drone image to be located into a target image matching model, use the target image matching model to extract drone feature information of the drone image to be located, and match the drone feature information with satellite feature information of multiple preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located.

[0074] The drone stores multiple pre-collected satellite images of the flight area. The satellite images carry location information. The satellite images carrying location information can be used as a matching set to be matched with the drone image at the current moment.

[0075] Specifically, the feature information of the drone image is matched with the feature information of the satellite image to obtain a target satellite image that matches the drone image to be located.

[0076] Step 103: extract feature points of the image of the unmanned aerial vehicle to be positioned and the target satellite image, and perform feature point pairing on the image of the unmanned aerial vehicle to be positioned and the target satellite image to obtain first paired feature points.

[0077] LoFTR is a feature point detection method based on deep learning. The model learns the entire feature extraction and matching process directly from input to output through end-to-end training, and can achieve efficient feature point detection on complex images.

[0078] According to the characteristics of the UAV embedded platform, the present invention constructs a pixel relationship analysis model based on the LoFTR feature point detection algorithm to pair feature points of image pairs, calculates the position of each feature point on the image of the UAV to be located corresponding to the target satellite image, obtains the feature points on the target satellite image corresponding to each feature point on the image of the UAV to be located, and obtains multiple first paired feature points.

[0079] Step 104: Calculate a first homography matrix between the image of the drone to be located and the target satellite image using the position coordinate information of the first paired feature points.

[0080] The transformation relationship between point pairs on two images with different viewpoints can be expressed in homogeneous coordinates through projective transformation. Projective transformation is also called homography, and the matrix corresponding to the projective transformation is called the homography matrix. Using the first homography matrix, the image of the drone to be located can be mapped to the coordinate system of the target satellite image. This process can be expressed as:

[0081] X2=HX1 X1,X2∈R 3 (1)

[0082] Where X2 represents the drone image to be located, X1 represents the target satellite image, H represents the first homography matrix, and R represents a real number.

[0083] Formula (1) can be expanded to obtain:

[0084]

[0085] Among them, (x1, y1, z1) represents the feature points in X2, (x2, y2, z2) represents the feature points in X1, and the matrix represents the first homography matrix H.

[0086] By calculating the first homography matrix between the paired target satellite image and the image of the drone to be located, we can map the location coordinates in the target satellite image to the image of the drone to be located. This allows us to locate the drone using the coordinate information contained in the satellite image. The method for solving the first homography matrix is ​​well-known and will not be further described here.

[0087] Step 105: Use the first homography matrix to map the location information in the target satellite image to the image of the drone to be located, and obtain first predicted location information of the drone at the current moment.

[0088] The first homography matrix is ​​used to map the location information in the target satellite image to the image of the drone to be located, thereby obtaining the first predicted location information. In this way, the real-time location coordinates of the drone can be obtained based on the drone image captured by the drone.

[0089] To meet the real-time and precision requirements for UAV system positioning in satellite-denied environments, this solution improves existing UAV visual positioning methods. Considering the significant differences in imaging style, temporal scale, and coverage between heterogeneous images, a learning-based approach is used to extract multi-scale image features of key targets. The aerial imagery captured by the UAV in real time is efficiently matched with pre-stored satellite imagery containing high-precision location information. The image matching results are used to determine the UAV's real-time positioning coordinates. Furthermore, the real-time nature of the task is ensured by optimizing the image matching retrieval space and using lightweight operators.

[0090] In summary, in an embodiment of the present invention, during the flight of a drone, an overhead image collected by the drone at the current moment is obtained to obtain the drone image to be located at the current moment; the drone image to be located is input into a target image matching model, and the drone feature information of the drone image to be located is extracted using the target image matching model, and the drone feature information is matched with the satellite feature information of multiple satellite images with preset location information to obtain a target satellite image that matches the drone image to be located; feature points of the drone image to be located and the target satellite image are extracted, and the feature points of the drone image to be located and the target satellite image are paired to obtain a first paired feature point; the position coordinate information of the first paired feature point is used to calculate the first homography matrix between the drone image to be located and the target satellite image; the first homography matrix is ​​used to map the position information in the target satellite image to the drone image to be located to obtain the first predicted position information of the drone at the current moment. This solution utilizes a learning-based target image matching network for feature extraction and matching, and uses satellite images that have obvious visual differences from the drone images to be located for image matching. It can capture highly robust multi-scale object features that are not easily disturbed by changes in the external environment, improve the accuracy of image matching, and enhance the accuracy of drone positioning.

[0091] In one possible implementation, the target image matching model includes an image matching layer and a downsampling layer. Step 102 uses the target image matching model to extract drone feature information of the drone image to be located, including:

[0092] Step 1021: Use the downsampling layer to downsample the image of the drone to be located to obtain drone feature information.

[0093] Step 1022: Use the image matching layer to extract and combine multi-scale features of the downsampled image to obtain drone feature information.

[0094] In steps 1021 and 1022, after the image is input into the image matching network (MSD-Net), multi-scale features are first extracted and fused through the stacked downsampling layer (Downsample Layer) and image matching layer (MSD Block). Before generating the final feature representation, MSD-Net performs global average pooling (GAP), layer normalization (Layer Normalization), and linear mapping operations on the processed features to further improve the robustness and generalization ability of the feature representation.

[0095] Figure 2 This is a diagram of the overall framework for image processing using an image matching network provided by an embodiment of the present invention.

[0096] like Figure 2 As shown in the figure, the image matching network mainly consists of 4 stacked downsample layers (downsampling layers) and MSD blocks (image matching layers).

[0097] After the drone image is processed by the Downsample Layer and MSD Block, global average pooling (GAP), layer normalization (Layer Normalization) and linear mapping operations (linear) are performed to finally obtain the feature description of the drone image.

[0098] Satellite images are also processed by the image matching network to obtain feature descriptions of the satellite images. When processing drone images and satellite images in the image matching network, a weight sharing strategy is adopted.

[0099] Specifically, in order to reduce the amount of data, reduce the computing resources when processing feature maps, perceive information in a wider spatial range, improve the model's understanding and abstraction of the input image, and avoid model overfitting, the downsample layer is used to gradually reduce the size of the feature map and increase the dimension of the feature map. The first downsample layer module consists of a convolution kernel of size 4×4 and stride 4 and a layer normalization layer. The following three downsample layer modules are composed of a layer normalization layer and a size 2×2 , and a convolution kernel with a stride of 2. Because the stride and kernel size used are identical, convolution without overlap is achieved. Compared to larger kernels, a 2×2 convolution kernel has fewer parameters, reducing computational complexity. Especially in deep neural networks, reducing the number of parameters helps mitigate the risk of overfitting and improve model training efficiency.

[0100] In one possible implementation, the image matching layer includes a spatial attention mechanism unit, a depthwise separable convolution unit, and a global response normalization unit, and step 1022 includes:

[0101] Step 10221: Utilize the spatial attention mechanism unit to dynamically adjust the weights of different feature vectors in the downsampled image to obtain first feature information;

[0102] Step 10222: Input the first feature information into the depthwise separable convolution unit, and perform a separate convolution on the feature information of each channel using the convolution kernel of each channel to obtain feature information corresponding to each channel;

[0103] Step 10223: Use a 1x1 convolution kernel to linearly combine the feature information corresponding to each channel to obtain second feature information;

[0104] Step 10224: Input the second feature information into the global response normalization unit, perform global feature aggregation, feature normalization, and feature calibration on the second feature information to obtain drone feature information.

[0105] In steps 10221 to 10224, in the specific implementation, the image matching layer is mainly composed of three operators: spatial attention mechanism unit (Spitial Attention, SA), depthwise separable convolution unit (Depthwise Separable Convolution) and global response normalization unit (Global Response Normalization, GRN).

[0106] Spatial attention (SA) is a common technique in deep learning that allows a model to pay varying degrees of attention to different spatial locations when processing input data. In SA, the model learns to dynamically adjust the degree of attention it pays to different locations in the input data, specifically by adjusting the weights of different feature vectors in the downsampled image. This allows the model to more flexibly handle spatial variations.

[0107] Depthwise separable convolution is a convolution operation in convolutional neural networks that aims to reduce the number of parameters and computational complexity while maintaining the expressiveness of the model. A normal convolution operation involves convolving all depths (channels) of the input tensor and then combining these convolution results using a fully connected layer. The depthwise separable convolution in this solution splits this operation into the following two independent steps:

[0108] Depthwise convolution: Performs a separate convolution on each channel of the input to produce feature maps of multiple channels. Each input channel has a corresponding convolution kernel, but these convolution kernels are independent and not mixed together as in traditional convolution.

[0109] Point-wise convolution: Use 1x1 convolution kernel (point-wise convolution) to linearly combine the output channels of depthwise convolution into the final output. This step is similar to the fully connected layer, but because the depthwise convolution is performed first, the number of parameters is greatly reduced.

[0110] The Global Response Normalization (GRN) unit consists of three parts: global feature aggregation, feature normalization, and feature calibration. First, a set of aggregated values ​​is obtained through global feature aggregation. These aggregated values ​​are then enhanced using a response normalization function and normalized using standard division normalization.

[0111] Figure 3 This is a diagram of the internal structure of an image matching layer provided by an embodiment of the present invention.

[0112] Figure 4 A diagram of the internal structure of a multi-scale convolutional network provided by an embodiment of the present invention.

[0113] like Figure 3 As shown in , the downsampled image first passes through the Multi-scale Conv (multi-scale convolutional network) for convolution operation. Multi-scale Conv uses the spatial attention mechanism to focus on specific robust areas. Figure 4 As shown, Multi-scale Conv uses point convolution (1*1Conv), regular convolution (3*3Conv), and dilated convolution (3*3DilatedConv) to extract information at different scales from feature maps. The information extracted by the three convolution kernels is aggregated and sent to the inverse bottleneck layer, extracting more information from the high-dimensional feature metric space and preventing information loss. The GRN layer alleviates the problem of feature space collapse without increasing FLOPS, increasing the contrast and selection diversity of individual neurons in response to stimuli.

[0114] Multi-scale perception enables neural networks to simultaneously consider information at multiple scales or levels when processing input data. By considering information at different levels of the network, the network can better handle both small and large objects and is more robust to large scale variations. Furthermore, multi-scale perception helps the model better understand the semantic information of different regions in an image, allowing for more flexible processing of both fine-grained and coarse-grained semantic information in an image, thereby improving the model's ability to learn complex and diverse patterns.

[0115] In one possible implementation, step 10221 includes:

[0116] Step 102211: Perform a global maximum pooling operation and an average pooling operation on the downsampled image to obtain two one-dimensional vectors having the same number of channels as the feature vector of the downsampled image;

[0117] Step 102212: Input the two one-dimensional vectors into a convolution layer with a convolution kernel size of 7 to perform a convolution operation to obtain a convolution feature vector;

[0118] Step 102213: Input the convolution feature vector into the activation function for activation to obtain the weight coefficient corresponding to each feature vector;

[0119] Step 102214: multiply the weight coefficient by the feature vector corresponding to the downsampled image to obtain first feature information.

[0120] In steps 102211-102214, the global max pooling operation extracts the maximum value from each channel of the feature map, resulting in a one-dimensional vector with the same number of channels. Each element of this vector represents the maximum activation value on the corresponding channel and can be regarded as a global feature representation of that channel.

[0121] The global average pooling operation calculates the average value of each channel in the feature map, resulting in a one-dimensional vector with the same number of channels. Each element of this vector represents the average activation value of the corresponding channel, which can be regarded as a global statistical feature of the channel.

[0122] If the number of channels of the downsampled image is C, then after the global maximum pooling and average pooling operations, two C×1×1 one-dimensional vectors are obtained.

[0123] The two one-dimensional vectors are concatenated and fed into a convolution layer with a kernel size of 7. The resulting convolution feature vector is then activated using a Sigmoid activation function to obtain the weight coefficient Mc. Mc × the input feature is calculated to obtain the scaled new feature, the first feature information.

[0124] In one possible implementation, step 10224 includes:

[0125] Step 102241: Perform feature aggregation on the feature information of each channel in the second feature information using a global function to obtain a first vector corresponding to each channel; the first vector is a scalar that aggregates the statistical features of the feature information of the current channel;

[0126] Step 102242: Normalize each of the first vectors using a standard division normalization function to obtain a feature normalization score; the feature normalization score represents the importance of the current channel relative to other channels;

[0127] Step 102243: Use the feature normalization score and the preset learnable parameters to perform response calibration on the second feature information to obtain drone feature information.

[0128] In steps 102241 to 102243, GRN (global response normalization) consists of three parts: global feature aggregation, feature normalization, and feature calibration.

[0129] Specifically, for an input vector X∈R H×W×C , first transform a spatial feature map X i Through global functions Aggregate into vector gx:

[0130]

[0131] in, Represents a global function, X∈R H×W×C Represents the input vector, and gx represents the first vector aggregated.

[0132] This global function It can be regarded as a simple pooling layer. The present invention uses L2-norm (L2 norm) for feature aggregation, which can achieve better results. A set of aggregated values ​​is obtained through global feature aggregation in is a scalar that aggregates the statistical features of the i-th channel.

[0133] Next, the response normalization function The aggregated values ​​are enhanced and normalized using the following standard division normalization:

[0134]

[0135] Among them, ||X i || is the result after regularization of the i-th channel, is the result of L2-norm normalization, i.e., the feature normalization score. For the i-th channel, Equation (4) calculates the relative importance of that channel relative to all other channels. Similar to other forms of normalization, this step creates a feature competition between channels through mutual suppression.

[0136] Finally, the calculated feature normalization score is used to calibrate the response of the original input. The specific formula is as follows:

[0137]

[0138] in, Represents the feature normalization score of Xi. Multiplying the feature normalization score with Xi will obtain the response calibration result of the feature information, that is, the UAV feature information.

[0139] To simplify the neural network optimization process, the present invention improves formula (5). Two additional learnable parameters, γ and β, are introduced into the final formula of the GRN and initialized to zero. A residual connection is added between the input and output layers of the GRN. The final GRN module calculation formula is as follows:

[0140]

[0141] Among them, γ and β are learnable parameters, represents the normalized score of Xi’s features.

[0142] This setting allows the model to perform an identity function in the initial stage and gradually adjust the parameters to adapt to the training as the model training process progresses.

[0143] To prevent the degradation of understanding ability that occurs as the depth of the neural network increases, this solution adds residual connections between the input and output layers of the GRN. Residual connections connect modules in the neural network by adding skip connections.

[0144] In the residual connection of the MSD Block, assuming that the original input is x, the output after a layer of nonlinear transformation is:

[0145] H(x)=F(x)+β×x (7)

[0146] Where β is the weight adjustment coefficient. F(x) represents the residual part learned by the neural network, and x represents the input.

[0147] In a possible implementation, before step 101, a process of training the target image matching model is further included, specifically including the following steps:

[0148] Step 201: Acquire a historical data set, wherein the historical data set includes n matching pairs, each of which is a historical drone image and a historical satellite image closest to a geographical location represented by a pixel center of the historical drone image.

[0149] Step 202: Perform feature description on each of the n matching pairs of historical UAV images and historical satellite images to obtain n historical UAV image feature information and n historical satellite image feature information;

[0150] Step 203: Calculate the distance information between each pair of historical UAV image feature information and historical satellite image feature information;

[0151] Step 204: Determine, based on the distance information, a non-matching historical satellite image that is closest to the characteristic information of each of the historical drone images, and determine a non-matching historical drone image that is closest to the characteristic information of each of the historical satellite images;

[0152] Step 205: Determine first distance information of the non-matching historical satellite image corresponding to the historical drone image in each matching pair, and second distance information of the non-matching historical drone image corresponding to the historical satellite image in the matching pair;

[0153] Step 206: Select the image corresponding to the smaller value of the first distance information and the second distance information as the most similar negative sample of the matching pair, and form a paired feature triplet with the matching pair and the corresponding most similar negative sample;

[0154] Step 207: Use the paired feature triples as training data to train the initial image matching model to obtain a target image matching model.

[0155] In steps 201-207, the specific process for collecting drone datasets involves initializing the drone's position, flight attitude, and other information after takeoff. Then, video is recorded and location information is recorded from the collection start point to the end point, at the set flight speed and altitude. When creating a drone image dataset, image frames are captured from the flight video stream at specific time intervals, and the corresponding location information is recorded. Datasets collected in this manner are prone to the model being unable to identify the most similar image among a large number of highly similar images. Specifically, the image coverage area between adjacent frames overlaps significantly, the content contained in adjacent images is less distinguishable, and adjacent images appear to visually "move" slightly apart. This prevents the model from effectively distinguishing between two highly similar adjacent frames, leading to positioning failure.

[0156] To address the above problems, this solution designs a most similar negative sample sampling strategy to construct a model training dataset, fully mining and utilizing the information between highly similar neighboring images, maximizing the feature distance between positive samples and the nearest negative samples in a batch, significantly enhancing the network's feature representation capabilities and similar sample discrimination capabilities, and improving the accuracy of visual matching and positioning.

[0157] First, a historical dataset is obtained. The historical dataset includes n matching pairs, and each matching pair includes a paired historical drone image and a historical satellite image.

[0158] The matching pairs can be represented as Where A is a historical drone image, and P is a positive satellite image of the historical drone image A. A positive satellite image is one whose pixel center represents the geographic location closest to the drone image.

[0159] The feature descriptions of the historical UAV images and historical satellite images in the n matching pairs are performed respectively, and then the distance information between each historical UAV image and historical satellite image is calculated respectively to obtain a feature distance representation matrix D = cdist (a, p) with a matrix size of n × n.

[0160] in, a i and p j Corresponding to A i and P jThe feature description vector of .

[0161] Figure 5 A schematic diagram of generating paired feature triples provided by an embodiment of the present invention.

[0162] like Figure 5 As shown in the figure, a1, a2, a3, and a4 represent historical drone images, and p1, p2, p3, and p4 represent historical satellite images. Feature descriptions are performed on historical drone and satellite images, and the distance between each pair of historical drone and satellite image feature information is calculated to obtain a feature distance representation matrix D. The diagonal elements of distance matrix D represent the feature distances between matching pairs of images. The off-diagonal elements represent the feature distances between non-matching historical drone and satellite images.

[0163] Calculate and match pairs (A i , P i ) The calculation formula for the closest similar negative sample is as follows:

[0164] j min =argmin j＝1..n，j≠i d(a i , p j )

[0165] k min =argmin k＝1.n，k≠i d(a k , p i ) (8)

[0166] Among them, j min For a i The nearest non-matching image The superscript, k min For p i The nearest non-matching image Corner mark.

[0167] like Figure 5 As shown by Construct a feature descriptor quadruple and select and The smaller value in is taken as the most similar negative sample n min The final paired feature triplet (a i ,p i ,n min ).

[0168] The paired feature triplets are used as training data to train the initial image matching model and obtain the target image matching model.

[0169] This solution designs the most similar negative sample sampling strategy to construct the model training data set, fully mines and utilizes the information between high-similarity neighboring images, maximizes the feature distance between the positive samples and the nearest negative samples in a batch, significantly enhances the network feature representation ability and similar sample discrimination ability, and improves the accuracy of visual matching and positioning.

[0170] In one possible implementation, step 102 includes:

[0171] Step 1021: Match the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a plurality of matching degrees;

[0172] Step 1022: Sort the matching degrees in descending order, and obtain the satellite image ranked first as the target satellite image that matches the drone image to be located.

[0173] In step 1021 and step 1022, a method for acquiring a target satellite image is given, that is, the satellite image ranked first in matching degree is selected as the target satellite image that matches the drone image to be located.

[0174] In a possible implementation, after step 1022, the method further includes:

[0175] Step 1023: Obtain the top K ranked K satellite images as candidate satellite images;

[0176] Step 1024: Determine the homography matrix between each of the to-be-located UAV images and the candidate satellite images to obtain K second homography matrices;

[0177] Step 1025: Use the second homography matrix to map the location information in the candidate satellite image to the image of the drone to be located, to obtain multiple candidate predicted location information of the drone.

[0178] In steps 1023-1025, the top K satellite images ranked by matching scores are selected as candidate satellite images. Second homography matrices are calculated between the image of the drone to be located and each candidate satellite image. Using each second homography, the location information in the candidate satellite images is mapped to the image of the drone to be located, resulting in multiple candidate predicted locations for the drone. These multiple candidate predicted locations are then combined with the first predicted location information to produce a single set of predicted location information.

[0179] If it is subsequently determined that the first predicted position information does not meet the requirements, the predicted position information in the result set can be used to calibrate the positioning.

[0180] Specifically, the method for calibrating positioning may be performed after step 105, and specifically includes the following steps:

[0181] Step 301: Acquire flight data of the UAV before the current moment, and use the flight data and the IMM model to fit the flight trajectory of the UAV to obtain second predicted position information of the UAV at the current moment;

[0182] Step 302: Determine an area within a preset distance range of the second predicted location information as the predicted location range of the UAV at the current moment;

[0183] Step 303: Determine whether the first predicted location information is within the predicted location range;

[0184] Step 304: If the target position is within the predicted position range, the first predicted position information is determined as the target position information of the UAV at the current moment;

[0185] Step 305: If the target position is not within the predicted position range, select at least one target predicted position information within the predicted position range from the plurality of candidate predicted position information;

[0186] Step 306: Obtain at least one candidate satellite image corresponding to the at least one target predicted position information, and obtain a satellite image that best matches the image of the drone to be located among the at least one candidate satellite image;

[0187] Step 307: Determine the candidate predicted position information corresponding to the most matching satellite image as the target position information of the UAV at the current moment.

[0188] In steps 301 to 307, the present invention rationally analyzes the positioning points of the previous flight through an image search space optimization algorithm based on trajectory prediction, fits the subsequent flight trajectory of the UAV, and thus estimates the confidence of the next UAV visual positioning result, evaluates the credibility of the positioning, and optimizes the subsequent satellite image library search space.

[0189] Specifically, the UAV trajectory is estimated using the Interacting Multiple Model (IMM) model. To better describe the IMM model, the following conventions are made:

[0190] 1. The model contains a total of n motion models Where i represents the i-th model and the algorithm iterates k discrete times.

[0191] 2. The system measurement value at the kth moment is Y k .

[0192] 3. There is correlation between models, according to the k-1 moment model i and the measured value Y k-1 Ability to determine the transition probability of model j at time k

[0193] 4. Use M k and the measured value Y at time k k The probability of being able to accurately describe the motion of an object is

[0194] The input of the IMM algorithm in this scheme is the flight data of the UAV before the current moment, including acceleration, speed, turning rate, linear velocity, angular acceleration and other flight data.

[0195] In the implementation of the IMM algorithm, the input is first mixed and the optimal estimate of each model is calculated at time k-1. and the covariance matrix The estimated states and covariance matrices related to each model are weightedly fused to obtain the optimal state x and covariance matrix P of the target under the k-1 moment fusion state:

[0196]

[0197] in Represents the correlation coefficient of fusion. Based on the weight of the correlation, the models are weighted fused to obtain the fusion estimation state and covariance matrix of the i-th model. Considering that each model has a correctness probability, each element of P needs to be multiplied by the corresponding model confidence Then normalize it to get the final weight:

[0198]

[0199] Then the state equation and covariance matrix corresponding to the model are predicted and updated. The prediction steps are as follows:

[0200]

[0201] The update steps are as follows:

[0202]

[0203] Calculate the true value of the filter of each model at time k and the covariance matrix

[0204] In order to evaluate the quality of the model's estimation of the motion state, we use Assess the quality of model j's predictions:

[0205]

[0206] in The greater the deviation between the predicted value of model j and the measured value, the less effective the model is. Update the confidence of model j

[0207]

[0208] in represents the confidence after fusion, To achieve the normalized denominator coefficient. The updated confidence λ can be used to fuse the models according to the weights to obtain the output target state x of the IMM filtering algorithm. k and the covariance matrix P k :

[0209]

[0210] By fitting the UAV's flight trajectory through the above steps and using the second predicted position information generated by trajectory analysis, we can effectively estimate the satellite image library search range at the next moment and perform a confidence assessment on the upcoming visual positioning results.

[0211] The confidence assessment method is as follows: the area within the preset distance range of the second predicted location information is determined as the drone's predicted location range at the current moment; then a determination is made as to whether the first predicted location information is within the predicted location range. If so, the prediction result of the first predicted location information is determined to be accurate and the first predicted location information is used as the target location information. If not, there is a significant deviation between the first predicted location information and the predicted location range, indicating a possible location error or environmental change.

[0212] To ensure the accuracy of the results, the deviated results are removed to eliminate potential mismatches or anomalies. Turning to the remaining result set, multiple candidate predicted position information is obtained, which is the candidate predicted position information calculated according to the second homography matrix in step 1025.

[0213] At least one target predicted location information within the predicted location range is selected from the plurality of candidate predicted location information. At least one candidate satellite image corresponding to the at least one target predicted location information is obtained, and a satellite image that best matches the image of the drone to be located is obtained from the at least one candidate satellite image.

[0214] The candidate predicted location information corresponding to the most matching satellite image is taken as the most reliable positioning result and is determined as the target location information.

[0215] After obtaining the best-matching image pair, this solution calculates the position prediction result through the position acquisition algorithm of image registration and evaluates the confidence through the IMM model. If there is a significant deviation from the estimated satellite library search space, the positions within the search space will be sorted in the remaining result set to determine the most reliable positioning result, thereby further improving the accuracy of drone positioning.

[0216] Figure 6 This is a structural block diagram of a drone assisted positioning device based on image matching provided by an embodiment of the present invention. Figure 6 As shown, the apparatus 400 includes:

[0217] The UAV image acquisition module 401 is used to acquire the overhead image captured by the UAV at the current moment during the UAV flight process, and obtain the image of the UAV to be located at the current moment;

[0218] Matching module 402 is configured to input the drone image to be located into a target image matching model, extract drone feature information of the drone image to be located using the target image matching model, and match the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located;

[0219] The feature point pairing module 403 is configured to extract feature points of the image of the unmanned aerial vehicle to be positioned and the target satellite image, and perform feature point pairing on the image of the unmanned aerial vehicle to be positioned and the target satellite image to obtain first paired feature points.

[0220] A homography matrix calculation module 404 is configured to calculate a first homography matrix between the image of the drone to be located and the target satellite image using the position coordinate information of the first paired feature points;

[0221] The mapping module 405 is configured to map the location information in the target satellite image to the image of the drone to be located using the first homography matrix to obtain first predicted location information of the drone at the current moment.

[0222] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0223] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, which can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0224] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0225] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A UAV assisted positioning method based on image matching, characterized in that: The method comprises: During the flight of the drone, an overhead image captured by the drone at the current moment is obtained to obtain an image of the drone to be located at the current moment; Inputting the drone image to be located into a target image matching model, extracting drone feature information of the drone image to be located using the target image matching model, and matching the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located; Extracting feature points of the image of the unmanned aerial vehicle to be positioned and the target satellite image, and performing feature point pairing on the image of the unmanned aerial vehicle to be positioned and the target satellite image to obtain first paired feature points; Calculating a first homography matrix between the image of the unmanned aerial vehicle to be located and the target satellite image using the position coordinate information of the first paired feature points; Mapping the location information in the target satellite image to the image of the unmanned aerial vehicle to be located using the first homography matrix to obtain first predicted location information of the unmanned aerial vehicle at the current moment; The target image matching model includes an image matching layer and a downsampling layer. The extracting the drone feature information of the drone image to be located by using the target image matching model includes: Downsampling the image of the drone to be located using the downsampling layer to obtain a downsampled image; Extracting and combining multi-scale features of the downsampled image using the image matching layer to obtain drone feature information; The image matching layer includes a spatial attention mechanism unit, a depth-wise separable convolution unit, and a global response normalization unit. The image matching layer is used to extract and combine multi-scale features of the downsampled image to obtain drone feature information, including: Dynamically adjusting weights of different feature vectors in the downsampled image using the spatial attention mechanism unit to obtain first feature information; Inputting the first feature information into the depthwise separable convolution unit, and performing a separate convolution on the feature information of each channel using the convolution kernel of each channel to obtain feature information corresponding to each channel; Use a 1x1 convolution kernel to linearly combine the feature information corresponding to each of the channels to obtain second feature information; The second feature information is input into the global response normalization unit, and global feature aggregation, feature normalization and feature calibration are performed on the second feature information to obtain drone feature information.

2. The method according to claim 1, characterized in that The method of dynamically adjusting the weights of different feature vectors in the downsampled image by using the spatial attention mechanism unit to obtain first feature information includes: Performing a global maximum pooling operation and an average pooling operation on the downsampled image to obtain two one-dimensional vectors having the same number of channels as the feature vector of the downsampled image; Input the two one-dimensional vectors into a convolution layer with a convolution kernel size of 7 to perform a convolution operation to obtain a convolution feature vector; Input the convolution feature vector into the activation function for activation to obtain the weight coefficient corresponding to each feature vector; The weight coefficient is multiplied by a feature vector corresponding to the downsampled image to obtain first feature information.

3. The method according to claim 1, characterized in that Inputting the second feature information into the global response normalization unit, performing global feature aggregation, feature normalization, and feature calibration on the second feature information to obtain UAV feature information, includes: Performing feature aggregation on the feature information of each channel in the second feature information using a global function to obtain a first vector corresponding to each channel; the first vector is a scalar that aggregates the statistical features of the feature information of the current channel; Normalizing each of the first vectors using a standard division normalization function to obtain a feature normalization score; the feature normalization score represents the importance of the current channel relative to other channels; The second feature information is calibrated in response using the feature normalization score and preset learnable parameters to obtain drone feature information.

4. The method according to claim 1, wherein Before obtaining the overhead image collected by the drone at the current moment, the method further includes: Acquire a historical data set, wherein the historical data set includes n matching pairs, each of which is a historical drone image and a historical satellite image closest to a geographical location represented by a pixel center of the historical drone image; Performing feature descriptions on the historical UAV images and historical satellite images in the n matching pairs respectively to obtain n historical UAV image feature information and n historical satellite image feature information; Calculate the distance information between each pair of historical UAV image feature information and historical satellite image feature information respectively; Determining, based on the distance information, a non-matching historical satellite image that is closest to the characteristic information of each of the historical drone images, and determining a non-matching historical drone image that is closest to the characteristic information of each of the historical satellite images; Determining first distance information of the non-matching historical satellite image corresponding to the historical drone image in each matching pair, and second distance information of the non-matching historical drone image corresponding to the historical satellite image in the matching pair; Selecting an image corresponding to a smaller value of the first distance information and the second distance information as the most similar negative sample of the matching pair, and forming a paired feature triplet with the matching pair and the corresponding most similar negative sample; The paired feature triplets are used as training data to train the initial image matching model to obtain a target image matching model.

5. The method according to claim 1, wherein The matching of the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located includes: Matching the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a plurality of matching degrees; The matching degrees are sorted in descending order, and the satellite image ranked first is obtained as the target satellite image that matches the drone image to be located.

6. The method according to claim 5, characterized in that After obtaining the target satellite image that matches the image of the unmanned aerial vehicle to be located, the method further includes: Obtain the top K satellite images as candidate satellite images; Determine the homography matrix between each of the to-be-located UAV images and the candidate satellite images to obtain K second homography matrices; The second homography matrix is ​​used to map the position information in the candidate satellite image to the image of the unmanned aerial vehicle to be located, thereby obtaining a plurality of candidate predicted position information of the unmanned aerial vehicle.

7. The method according to claim 6, characterized in that After obtaining the first predicted position information of the UAV at the current moment, the method further includes: Acquire flight data of the UAV before the current moment, and fit the flight trajectory of the UAV using the flight data and the IMM model to obtain second predicted position information of the UAV at the current moment; Determining an area within a preset distance range of the second predicted position information as the predicted position range of the drone at the current moment; determining whether the first predicted location information is within the predicted location range; If the target position is within the predicted position range, the first predicted position information is determined as the target position information of the UAV at the current moment; If it is not within the predicted position range, selecting at least one target predicted position information within the predicted position range from the multiple candidate predicted position information; Obtaining at least one candidate satellite image corresponding to the at least one target predicted position information, and obtaining a satellite image that best matches the image of the drone to be located among the at least one candidate satellite image; The candidate predicted position information corresponding to the most matching satellite image is determined as the target position information of the UAV at the current moment.

8. An image matching-based UAV auxiliary positioning device, characterized in that: The device comprises: The UAV image acquisition module is used to acquire the overhead image collected by the UAV at the current moment during the UAV flight process, and obtain the image of the UAV to be located at the current moment; a matching module, configured to input the drone image to be located into a target image matching model, extract drone feature information of the drone image to be located using the target image matching model, and match the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located; A feature point pairing module is used to extract feature points of the image of the unmanned aerial vehicle to be positioned and the target satellite image, and perform feature point pairing on the image of the unmanned aerial vehicle to be positioned and the target satellite image to obtain a first paired feature point; a homography matrix calculation module, configured to calculate a first homography matrix between the image of the unmanned aerial vehicle to be located and the target satellite image using the position coordinate information of the first paired feature points; a mapping module, configured to map the position information in the target satellite image to the image of the unmanned aerial vehicle to be located using the first homography matrix, to obtain first predicted position information of the unmanned aerial vehicle at the current moment; The target image matching model includes an image matching layer and a downsampling layer, and the matching module is specifically used to: Downsampling the image of the drone to be located using the downsampling layer to obtain a downsampled image; Extracting and combining multi-scale features of the downsampled image using the image matching layer to obtain drone feature information; The image matching layer includes a spatial attention mechanism unit, a depth-wise separable convolution unit, and a global response normalization unit. The image matching layer is used to extract and combine multi-scale features of the downsampled image to obtain drone feature information, including: Dynamically adjusting weights of different feature vectors in the downsampled image using the spatial attention mechanism unit to obtain first feature information; Inputting the first feature information into the depthwise separable convolution unit, and performing a separate convolution on the feature information of each channel using the convolution kernel of each channel to obtain feature information corresponding to each channel; Use a 1x1 convolution kernel to linearly combine the feature information corresponding to each of the channels to obtain second feature information; The second feature information is input into the global response normalization unit, and global feature aggregation, feature normalization and feature calibration are performed on the second feature information to obtain drone feature information.

Images