Unmanned aerial vehicle auxiliary positioning method and device based on image matching

Abstract

The invention discloses an unmanned aerial vehicle auxiliary positioning method and device based on image matching, and the method comprises the steps: obtaining a high-angle image collected by an unmanned aerial vehicle at a current moment, and obtaining a to-be-positioned unmanned aerial vehicle image; extracting unmanned aerial vehicle feature information of the to-be-positioned unmanned aerial vehicle image by using a target image matching model, and matching the unmanned aerial vehicle feature information with the satellite feature information of the plurality of satellite images to obtain a target satellite image; performing feature point pairing on the to-be-positioned unmanned aerial vehicle image and the target satellite image to obtain a first pairing feature point; calculating a first homography matrix between the to-be-positioned unmanned aerial vehicle image and the target satellite image; and mapping the position information in the target satellite image to the to-be-positioned unmanned aerial vehicle image by using the first homography matrix to obtain first predicted position information of the unmanned aerial vehicle. According to the scheme, high-robustness multi-scale object features can be captured, the image matching accuracy is improved, and the unmanned aerial vehicle positioning precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image matching-based unmanned aerial vehicle auxiliary positioning method and device. Background Art

[0002] Unmanned Aerial Vehicle (UAV) is an unmanned flying device. With its advantages of good maneuverability, flexibility, small size, high intelligence and low detectability, it is widely used in material transportation, military reconnaissance, industrial inspection, remote sensing mapping, rescue operations, meteorological monitoring, geographic mapping and crop production.

[0003] Satellite positioning systems, inertial positioning systems and visual positioning systems are mainstream and mature methods for drone positioning. As a common method for drone positioning, satellite positioning systems are often interfered with and rejected during use. Even in the military field, radio is prone to malicious interference or shielding. The sensors required for inertial positioning are heavy, and the position of the carrier must be accurately known at the beginning of the flight. The inertial measurement value is used to estimate the position change that occurs after the start, resulting in a large cumulative error defect in the system. The visual positioning system uses various sensors carried by the drone itself to obtain information about the current environment at a certain angle, and pairs the obtained information with satellite images or other pre-processed information to obtain its own location information.

[0004] The target of the UAV system positioning needs to meet not only strict time constraints, but also strict positioning errors. As a positioning method with strong anti-interference ability, no positioning cumulative error and low working dependence, the UAV visual positioning system also has problems that cannot be ignored: 1) Satellite images and UAV images are heterogeneous images. Due to the different acquisition methods of the two image sources, there may be significant differences in imaging style, time scale and coverage scale between UAV images and remote sensing images. 2) UAVs have a certain flight altitude, and the content in the actual coverage range of the image is extremely rich, with sparse textures and details. Time changes further increase the difficulty of finding the same target pair and reduce the success rate of image matching. 3) UAVs are embedded platforms with low computing power and do not have the conditions to deploy large-scale neural network models. It is very challenging to design an efficient image matching algorithm without losing accuracy. Summary of the invention

[0005] The present invention aims to at least solve the technical problems existing in the prior art. To this end, the first aspect of the present invention proposes a drone assisted positioning method based on image matching, the method comprising:

[0006] During the flight of the drone, an overhead image collected by the drone at the current moment is acquired to obtain an image of the drone to be located at the current moment;

[0007] Inputting the unmanned aerial vehicle image to be located into a target image matching model, extracting unmanned aerial vehicle feature information of the unmanned aerial vehicle image to be located using the target image matching model, and matching the unmanned aerial vehicle feature information with satellite feature information of a plurality of preset satellite images carrying location information, to obtain a target satellite image that matches the unmanned aerial vehicle image to be located;

[0008] Extracting feature points of the unmanned aerial vehicle image to be located and the target satellite image, and performing feature point pairing on the unmanned aerial vehicle image to be located and the target satellite image to obtain first paired feature points;

[0009] Calculating a first homography matrix between the image of the unmanned aerial vehicle to be located and the target satellite image using the position coordinate information of the first paired feature points;

[0010] The first homography matrix is ​​used to map the location information in the target satellite image to the image of the unmanned aerial vehicle to be located, so as to obtain the first predicted location information of the unmanned aerial vehicle at the current moment.

[0011] Optionally, the target image matching model includes an image matching layer and a downsampling layer, and the step of extracting the drone feature information of the drone image to be located by using the target image matching model includes:

[0012] Using the downsampling layer to downsample the image of the unmanned aerial vehicle to be located, to obtain a downsampled image;

[0013] The image matching layer is used to extract and combine multi-scale features of the downsampled image to obtain drone feature information.

[0014] Optionally, the image matching layer includes a spatial attention mechanism unit, a depth-separable convolution unit, and a global response normalization unit, and the image matching layer is used to extract and combine multi-scale features of the downsampled image to obtain drone feature information, including:

[0015] Using the spatial attention mechanism unit, dynamically adjusting the weights of different feature vectors in the downsampled image to obtain first feature information;

[0016] Inputting the first feature information into the depthwise separable convolution unit, and performing a separate convolution on the feature information input into each channel using the convolution kernel of each channel to obtain feature information corresponding to each channel;

[0017] Using a 1x1 convolution kernel to linearly combine the feature information corresponding to each of the channels to obtain second feature information;

[0018] The second feature information is input into the global response normalization unit, and global feature aggregation, feature normalization and feature calibration are performed on the second feature information to obtain UAV feature information.

[0019] Optionally, the using the spatial attention mechanism unit to dynamically adjust the weights of different feature vectors in the downsampled image to obtain the first feature information includes:

[0020] Performing a global maximum pooling operation and an average pooling operation on the downsampled image to obtain two one-dimensional vectors having the same number of channels as the feature vector of the downsampled image;

[0021] Input the two one-dimensional vectors into a convolution layer with a convolution kernel size of 7 to perform a convolution operation to obtain a convolution feature vector;

[0022] Input the convolution feature vector into the activation function for activation, and obtain the weight coefficient corresponding to each feature vector;

[0023] The weight coefficient is multiplied by a feature vector corresponding to the down-sampled image to obtain first feature information.

[0024] Optionally, the step of inputting the second feature information into the global response normalization unit, performing global feature aggregation, feature normalization and feature calibration on the second feature information, and obtaining the drone feature information includes:

[0025] The feature information of each channel in the second feature information is respectively subjected to feature aggregation by a global function to obtain a first vector corresponding to each channel; the first vector is a scalar that aggregates the statistical features of the feature information of the current channel;

[0026] Using a standard division normalization function to normalize each of the first vectors, respectively, to obtain a feature normalization score; the feature normalization score represents the importance of the current channel relative to other channels;

[0027] The second feature information is calibrated in response to the feature normalization score and the preset learnable parameters to obtain the drone feature information.

[0028] Optionally, before acquiring the overhead image collected by the drone at the current moment, the method further includes:

[0029] Acquire a historical data set, wherein the historical data set includes n matching pairs, wherein the matching pairs are a historical drone image and a historical satellite image that is closest to a geographical location represented by a pixel center of the historical drone image;

[0030] Performing feature descriptions on the historical UAV images and historical satellite images in the n matching pairs respectively to obtain n historical UAV image feature information and n historical satellite image feature information;

[0031] Calculate the distance information between each pair of historical UAV image feature information and historical satellite image feature information respectively;

[0032] Determine, based on the distance information, a non-matching historical satellite image that is closest to the characteristic information of each of the historical drone images, and determine a non-matching historical drone image that is closest to the characteristic information of each of the historical satellite images;

[0033] Determine first distance information of the non-matching historical satellite image corresponding to the historical drone image in each of the matching pairs, and second distance information of the non-matching historical drone image corresponding to the historical satellite image in the matching pair;

[0034] Selecting an image corresponding to a smaller value of the first distance information and the second distance information as the most similar negative sample of the matching pair, and forming a paired feature triplet with the matching pair and the corresponding most similar negative sample;

[0035] The paired feature triplets are used as training data to train the initial image matching model to obtain a target image matching model.

[0036] Optionally, matching the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located includes:

[0037] Matching the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a plurality of matching degrees;

[0038] The matching degrees are sorted in descending order, and the satellite image ranked first is obtained as the target satellite image that matches the drone image to be located.

[0039] Optionally, after obtaining the target satellite image that matches the image of the unmanned aerial vehicle to be located, the method further includes:

[0040] Obtain the top K satellite images as candidate satellite images;

[0041] Determine the homography matrix between each of the to-be-located UAV images and the candidate satellite images to obtain K second homography matrices;

[0042] The position information in the candidate satellite image is mapped to the image of the unmanned aerial vehicle to be located using the second homography matrix to obtain multiple candidate predicted position information of the unmanned aerial vehicle.

[0043] Optionally, after obtaining the first predicted position information of the drone at the current moment, the method further includes:

[0044] Acquire flight data of the UAV before the current moment, and fit the flight trajectory of the UAV using the flight data and the IMM model to obtain second predicted position information of the UAV at the current moment;

[0045] Determine an area within a preset distance range of the second predicted position information as a predicted position range of the drone at the current moment;

[0046] Determining whether the first predicted location information is within the predicted location range;

[0047] If it is within the predicted position range, determining the first predicted position information as the target position information of the UAV at the current moment;

[0048] If it is not within the predicted position range, selecting at least one target predicted position information within the predicted position range from the multiple candidate predicted position information;

[0049] Acquire at least one candidate satellite image corresponding to the at least one target predicted position information, and acquire a satellite image that best matches the image of the unmanned aerial vehicle to be located among the at least one candidate satellite image;

[0050] The candidate predicted position information corresponding to the most matching satellite image is determined as the target position information of the UAV at the current moment.

[0051] A second aspect of the present invention provides a drone auxiliary positioning device based on image matching, the device comprising:

[0052] The UAV image acquisition module is used to acquire the overhead image collected by the UAV at the current moment during the flight of the UAV, and obtain the image of the UAV to be located at the current moment;

[0053] A matching module is used to input the drone image to be located into a target image matching model, extract drone feature information of the drone image to be located using the target image matching model, and match the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located;

[0054] A feature point pairing module is used to extract feature points of the unmanned aerial vehicle image to be located and the target satellite image, and perform feature point pairing on the unmanned aerial vehicle image to be located and the target satellite image to obtain a first pairing feature point;

[0055] A homography matrix calculation module, used to calculate a first homography matrix between the image of the unmanned aerial vehicle to be located and the target satellite image using the position coordinate information of the first paired feature points;

[0056] A mapping module is used to map the location information in the target satellite image to the image of the unmanned aerial vehicle to be located using the first homography matrix to obtain the first predicted location information of the unmanned aerial vehicle at the current moment.

[0057] The third aspect of the present invention proposes an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the drone assisted positioning method based on image matching as described in the first aspect.

[0058] The sixth aspect of the present invention proposes a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the image matching-based drone assisted positioning method as described in the first aspect.

[0059] The embodiments of the present invention have the following beneficial effects:

[0060] In an embodiment of the present invention, during the flight of a drone, an overhead image collected by the drone at a current moment is obtained to obtain the drone image to be located at the current moment; the drone image to be located is input into a target image matching model, the drone feature information of the drone image to be located is extracted using the target image matching model, and the drone feature information is matched with the satellite feature information of a plurality of satellite images that are preset to carry location information to obtain a target satellite image that matches the drone image to be located; feature points of the drone image to be located and the target satellite image are extracted, and feature points of the drone image to be located and the target satellite image are paired to obtain a first paired feature point; the position coordinate information of the first paired feature point is used to calculate a first homography matrix between the drone image to be located and the target satellite image; the position information in the target satellite image is mapped to the drone image to be located using the first homography matrix to obtain the first predicted position information of the drone at the current moment. This scheme uses a learning-based target image matching network for feature extraction and matching, and uses satellite images that have obvious visual differences from the drone images to be located for image matching. It can capture highly robust multi-scale object features that are not easily disturbed by changes in the external environment, improve the accuracy of image matching, and improve the accuracy of drone positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flowchart of a method for assisting unmanned aerial vehicle positioning based on image matching provided by an embodiment of the present invention;

[0062] Figure 2 An overall framework diagram of image processing using an image matching network provided in an embodiment of the present invention;

[0063] Figure 3 An internal structure diagram of an image matching layer provided by an embodiment of the present invention;

[0064] Figure 4 An internal structure diagram of a multi-scale convolutional network provided by an embodiment of the present invention;

[0065] Figure 5 A schematic diagram of generating a pairing feature triple provided by an embodiment of the present invention;

[0066] Figure 6 A structural block diagram of a drone auxiliary positioning device based on image matching provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "multiple" means two or more. In addition, the use of "based on" or "according to" means openness and inclusiveness, because the process, steps, calculations or other actions "based on" or "according to" one or more of the conditions or values ​​may be based on additional conditions or values ​​beyond the described values ​​in practice.

[0069] Figure 1 FIG. 1 is a flowchart of a method for assisting drone positioning based on image matching provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0070] Step 101: During the flight of the drone, an overhead image collected by the drone at a current moment is acquired to obtain an image of the drone to be located at the current moment.

[0071] Drones can be controlled by remote controllers or autopilot systems to achieve a variety of shooting angles and effects. For example, drones can look down on the earth from a high altitude to obtain overhead images.

[0072] During the flight of the drone, the overhead images are continuously collected at preset time intervals. The overhead images collected at the current moment are used as the drone images to be located, and the position of the drone images will be located later.

[0073] Step 102: input the drone image to be located into a target image matching model, extract drone feature information of the drone image to be located using the target image matching model, and match the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located.

[0074] The drone stores multiple pre-collected satellite images of the flight area. The satellite images carry location information. The satellite images carrying location information can be used as a to-be-matched set to be matched with the drone images at the current moment.

[0075] Specifically, the feature information of the drone image is matched with the feature information of the satellite image to obtain a target satellite image that matches the drone image to be located.

[0076] Step 103: extract feature points of the unmanned aerial vehicle image to be located and the target satellite image, and perform feature point pairing on the unmanned aerial vehicle image to be located and the target satellite image to obtain first paired feature points.

[0077] LoFTR is a feature point detection method based on deep learning. The model learns the entire feature extraction and matching process directly from input to output through end-to-end training, and can realize efficient feature point detection on complex images.

[0078] According to the characteristics of the UAV embedded platform, the present invention builds a pixel relationship analysis model based on the LoFTR feature point detection algorithm to pair feature points of image pairs, calculates the positions of each feature point on the image of the UAV to be located corresponding to the target satellite image, obtains the feature points on the target satellite image corresponding to each feature point on the image of the UAV to be located, and obtains multiple first paired feature points.

[0079] Step 104: Calculate a first homography matrix between the image of the unmanned aerial vehicle to be located and the target satellite image using the position coordinate information of the first paired feature points.

[0080] The point pairs on two images with different viewpoints can be expressed in homogeneous coordinates through projective transformation, which is also called homography. The matrix corresponding to the projective transformation is called the homography matrix. Using the first homography matrix, the image of the drone to be located can be mapped to the coordinate system of the target satellite image. This process can be expressed as:

[0081] X2=HX1 X1,X2∈R 3 (1)

[0082] Among them, X2 represents the drone image to be located, X1 represents the target satellite image, H represents the first homography matrix, and R represents a real number.

[0083] By expanding formula (1), we can get:

[0084]

[0085] Among them, (x1, y1, z1) represents the feature points in X2, (x2, y2, z2) represents the feature points in X1, and the matrix represents the first homography matrix H.

[0086] By finding the first homography matrix of the paired target satellite image and the image of the drone to be located, the position coordinates in the target satellite image can be mapped to the image of the drone to be located, that is, the coordinate information carried by the satellite image is used to locate the drone position. The method for solving the first homography matrix is ​​a prior art and will not be repeated here.

[0087] Step 105: Use the first homography matrix to map the location information in the target satellite image to the image of the unmanned aerial vehicle to be located, so as to obtain the first predicted location information of the unmanned aerial vehicle at the current moment.

[0088] The first homography matrix is ​​used to map the location information in the target satellite image to the image of the drone to be located, thereby obtaining the first predicted location information. In this way, the real-time location coordinates of the drone can be obtained based on the drone image taken by the drone.

[0089] In order to meet the real-time and accuracy requirements of UAV system positioning in satellite denial environments, this solution improves the existing UAV visual positioning method. Considering the significant differences in imaging style, time scale, and coverage scale of heterogeneous images, a learning-based method is used to extract multi-scale image features of key targets, and the overhead images collected by the UAV in real time are efficiently matched with the satellite images with high-precision location information pre-stored in the aircraft. The real-time positioning coordinates of the UAV are determined by the image matching results, and the real-time nature of the task is met by optimizing the image matching retrieval space and using lightweight operators.

[0090] In summary, in an embodiment of the present invention, during the flight of a drone, an overhead image collected by the drone at the current moment is obtained to obtain the drone image to be located at the current moment; the drone image to be located is input into a target image matching model, and the drone feature information of the drone image to be located is extracted using the target image matching model, and the drone feature information is matched with the satellite feature information of multiple satellite images that are preset to carry location information to obtain a target satellite image that matches the drone image to be located; feature points of the drone image to be located and the target satellite image are extracted, and feature points of the drone image to be located and the target satellite image are paired to obtain a first paired feature point; the position coordinate information of the first paired feature point is used to calculate a first homography matrix between the drone image to be located and the target satellite image; the position information in the target satellite image is mapped to the drone image to be located using the first homography matrix to obtain the first predicted position information of the drone at the current moment. This scheme uses a learning-based target image matching network for feature extraction and matching, and uses satellite images that have obvious visual differences from the drone images to be located for image matching. It can capture highly robust multi-scale object features that are not easily disturbed by changes in the external environment, improve the accuracy of image matching, and improve the accuracy of drone positioning.

[0091] In a possible implementation, the target image matching model includes an image matching layer and a downsampling layer. In step 102, extracting the drone feature information of the drone image to be located using the target image matching model includes:

[0092] Step 1021: Use the downsampling layer to downsample the image of the drone to be located to obtain drone feature information.

[0093] Step 1022: Use the image matching layer to extract and combine multi-scale features of the down-sampled image to obtain drone feature information.

[0094] In step 1021-step 1022, after the image is input into the image matching network (MSD-Net), multi-scale features are first extracted and fused through the stacked downsampling layer (Downsample Layer) and image matching layer (MSD Block). Before generating the final feature representation, MSD-Net performs global average pooling (Global Average Pooling, GAP), layer normalization (Layer Normalization) and linear mapping operations on the processed features to further improve the robustness and generalization ability of feature expression.

[0095] Figure 2 An overall framework diagram of image processing using an image matching network provided in an embodiment of the present invention.

[0096] like Figure 2 As shown, the image matching network mainly includes 4 stacked downsample layers (downsampling layers) and MSD Block (image matching layer).

[0097] After the drone image is processed by Dowmsample Layer and MSD Block, global average pooling (GAP), layer normalization (Layer Normalization) and linear mapping operations (linear) are performed to finally obtain the feature description of the drone image.

[0098] Satellite images are also processed by the image matching network to obtain the feature description of the satellite images. When processing drone images and satellite images in the image matching network, a weight sharing strategy is adopted.

[0099] Specifically, in order to reduce the amount of data, reduce the computing resources when processing feature maps, perceive information in a wider spatial range, improve the model's understanding and abstraction of the input image, and avoid model overfitting, the downsample layer is used to gradually reduce the size of the feature map and increase the dimension of the feature map. The first Downsample layer module consists of a convolution kernel of size 4×4 and stride 4 and a Layer Normalization layer. The following three Downsample layer modules consist of a Layer Normalization layer and a size 2×2 , stride 2 convolution kernel. Since the stride and size of the convolution kernel are exactly the same, non-overlapping convolution can be achieved. Compared with larger convolution kernels, 2×2 convolution kernels have fewer parameters, thus reducing computational complexity. Especially in deep neural networks, reducing the number of parameters helps reduce the risk of overfitting and improve the training efficiency of the model.

[0100] In a possible implementation, the image matching layer includes a spatial attention mechanism unit, a depth-separable convolution unit, and a global response normalization unit, and step 1022 includes:

[0101] Step 10221: using the spatial attention mechanism unit, dynamically adjusting the weights of different feature vectors in the downsampled image to obtain first feature information;

[0102] Step 10222: input the first feature information into the depthwise separable convolution unit, and use the convolution kernel of each channel to perform a separate convolution on the feature information input into each channel to obtain feature information corresponding to each channel;

[0103] Step 10223: Use a 1x1 convolution kernel to linearly combine the feature information corresponding to each of the channels to obtain second feature information;

[0104] Step 10224: input the second feature information into the global response normalization unit, perform global feature aggregation, feature normalization and feature calibration on the second feature information to obtain UAV feature information.

[0105] In steps 10221 to 10224, in the specific implementation, the image matching layer is mainly composed of three operators: spatial attention mechanism unit (Spitial Attention, SA), depthwise separable convolution unit (Depthwise Separable Convolution) and global response normalization unit (Global Response Normalization, GRN).

[0106] Spatial attention mechanism (SA) is a common technique in deep learning, which is used for the model to pay different degrees of attention to different spatial locations when processing input data. In SA, the model learns to dynamically adjust the degree of attention to different locations in the input data, that is, adjust the weights of different feature vectors in the downsampled image, so that the model can handle spatial changes more flexibly.

[0107] Depthwise separable convolution is a convolution operation in convolutional neural networks that aims to reduce the number of parameters and computational complexity while maintaining the expressiveness of the model. A normal convolution operation involves convolving all depths (channels) of the input tensor and then combining these convolution results using a fully connected layer. The depthwise separable convolution in this solution splits this operation into the following two independent steps:

[0108] Depth convolution: Perform a separate convolution on each channel of the input to produce feature maps of multiple channels. Each input channel has a corresponding convolution kernel, but these convolution kernels are independent and not mixed together as in traditional convolution.

[0109] Point-wise convolution: Use 1x1 convolution kernel (point-wise convolution) to linearly combine the output channels of the depth convolution into the final output. This step is similar to the fully connected layer, but because the depth convolution is performed first, the number of parameters is greatly reduced.

[0110] The global response normalization unit (GRN) consists of three parts: global feature aggregation, feature normalization, and feature calibration. First, a set of aggregated values ​​is obtained by global feature aggregation, then the aggregated values ​​are enhanced by the response normalization function, and then the aggregated values ​​are normalized using standard division normalization.

[0111] Figure 3 An internal structure diagram of an image matching layer provided in an embodiment of the present invention.

[0112] Figure 4 An internal structure diagram of a multi-scale convolutional network provided in an embodiment of the present invention.

[0113] like Figure 3 As shown in , the downsampled image first passes through the Multi-scale Conv (multi-scale convolutional network) for convolution operation. Multi-scale Conv uses the spatial attention mechanism to focus on specific robust areas. Figure 4 As shown in the figure, Multi-scale Conv uses point convolution (1*1Conv), regular convolution (3*3Conv) and dilated convolution (3*3dilatedConv) to extract information of different scales from feature maps respectively, and then aggregates the information extracted by the three convolution kernels and sends it to the inverse bottleneck layer to extract more information in the high-dimensional feature metric space and avoid information loss. The GRN layer alleviates the problem of feature space collapse without increasing FLOPS, and increases the contrast and selection diversification of single neurons to stimuli.

[0114] Multi-scale perception allows the neural network to consider information at multiple scales or levels when processing input data. By considering information at different levels of the network, the network can better handle small and large objects, and is more robust to situations with large scale changes. In addition, multi-scale perception can help the model better understand the semantic information of different regions in the image, and can more flexibly handle the situation of processing fine-grained and coarse-grained semantic information in the image at the same time, improving the model's ability to learn complex and diverse patterns.

[0115] In a possible implementation, step 10221 includes:

[0116] Step 102211: performing a global maximum pooling operation and an average pooling operation on the downsampled image to obtain two one-dimensional vectors having the same number of channels as the feature vector of the downsampled image;

[0117] Step 102212: Input the two one-dimensional vectors into a convolution layer with a convolution kernel size of 7 to perform a convolution operation to obtain a convolution feature vector;

[0118] Step 102213: Input the convolution feature vector into an activation function for activation to obtain a weight coefficient corresponding to each feature vector;

[0119] Step 102214: multiply the weight coefficient by the feature vector corresponding to the downsampled image to obtain first feature information.

[0120] In step 102211-step 102214, the global maximum pooling operation extracts the maximum value of each channel of the feature map, thereby obtaining a one-dimensional vector with the same number of channels. Each element of this vector represents the maximum activation value on the corresponding channel, so it can be regarded as a global feature representation of the channel.

[0121] The global average pooling operation calculates the average value of each channel of the feature map to obtain a one-dimensional vector with the same number of channels. Each element of this vector represents the average activation value on the corresponding channel, which can be regarded as a global statistical feature of the channel.

[0122] If the number of channels of the downsampled image is C, two C×1×1 one-dimensional vectors are obtained after the global maximum pooling and average pooling operations.

[0123] After concatenating the two one-dimensional vectors, they are sent to a convolution layer with a convolution kernel size of 7 for operation. The convolution feature vector obtained is activated by the Sigmoid function to obtain the weight coefficient Mc. Mc×input feature is calculated to obtain the scaled new feature, i.e., the first feature information.

[0124] In one possible implementation, step 10224 includes:

[0125] Step 102241, feature-aggregating the feature information of each channel in the second feature information through a global function to obtain a first vector corresponding to each channel; the first vector is a scalar that aggregates the statistical features of the feature information of the current channel;

[0126] Step 102242: normalize each of the first vectors using a standard division normalization function to obtain a feature normalization score; the feature normalization score represents the importance of the current channel relative to other channels;

[0127] Step 102243: calibrate the response of the second feature information using the feature normalization score and the preset learnable parameters to obtain the drone feature information.

[0128] In step 102241-step 102243, GRN (global response normalization) consists of three parts: global feature aggregation, feature normalization and feature calibration.

[0129] Specifically, for an input vector X∈R H×W×C , first transform a spatial feature map X i Through the global function Aggregate into vector gx:

[0130]

[0131] in, represents a global function, X∈R H×W×C represents the input vector, and gx represents the first vector aggregated.

[0132] This global function It can be regarded as a simple pooling layer. The present invention uses L2-norm (L2 norm) for feature aggregation, which can achieve better results. A set of aggregated values ​​is obtained through global feature aggregation in is a scalar that aggregates the statistical features of the i-th channel.

[0133] Next, the response normalization function The aggregated values ​​are enhanced and normalized using the following standard division normalization:

[0134]

[0135] Among them, ||X i || is the result after regularization of the i-th channel, is the result of L2-norm normalization, i.e., the feature normalization score. For the i-th channel, the above formula (4) calculates the relative importance of the channel relative to all other channels. Similar to other forms of normalization, this step creates a feature competition between channels through mutual suppression.

[0136] Finally, the calculated feature normalization score is used to calibrate the response of the original input. The specific formula is as follows:

[0137]

[0138] in, Represents the feature normalization score of Xi. Multiplying the feature normalization score by Xi can obtain the response calibration result of the feature information, that is, the UAV feature information.

[0139] In order to simplify the neural network optimization process, the present invention improves formula (5). Two learnable additional parameters γ and β are introduced into the final formula of GRN and initialized to zero, and a residual connection is added between the input layer and the output layer of GRN. The final GRN module calculation formula is as follows:

[0140]

[0141] Among them, γ and β are learnable parameters, Represents the feature normalized score of Xi.

[0142] This setting allows the model to perform an identity function in the initial stage and gradually adjust parameters to adapt to training as the model training process progresses.

[0143] In order to prevent the phenomenon of comprehension degradation as the depth of the neural network increases, this scheme adds residual connections between the input layer and the output layer of the GRN. The residual connection connects the modules in the neural network by adding skip connections.

[0144] In the residual connection of the MSD Block, assuming that the original input is x, the output after a layer of nonlinear transformation is:

[0145] H(x)=F(x)+β×x (7)

[0146] Among them, β is the weight adjustment coefficient. F(x) represents the residual part learned by the neural network, and x represents the input.

[0147] In a possible implementation manner, before step 101, a process of training the target image matching model is further included, which specifically includes the following steps:

[0148] Step 201: Acquire a historical data set, wherein the historical data set includes n matching pairs, wherein the matching pairs are a historical drone image and a historical satellite image that is closest to a geographical location represented by a pixel center of the historical drone image;

[0149] Step 202: performing feature description on the historical UAV images and historical satellite images in the n matching pairs respectively to obtain feature information of n historical UAV images and feature information of n historical satellite images;

[0150] Step 203, respectively calculating the distance information between every two pieces of historical drone image feature information and historical satellite image feature information;

[0151] Step 204: Determine, based on the distance information, a non-matching historical satellite image that is closest to the characteristic information of each of the historical drone images, and determine a non-matching historical drone image that is closest to the characteristic information of each of the historical satellite images;

[0152] Step 205: Determine first distance information of the non-matching historical satellite image corresponding to the historical drone image in each matching pair, and second distance information of the non-matching historical drone image corresponding to the historical satellite image in the matching pair;

[0153] Step 206: Select an image corresponding to a smaller value of the first distance information and the second distance information as the most similar negative sample of the matching pair, and form a paired feature triplet with the matching pair and the corresponding most similar negative sample;

[0154] Step 207: Use the paired feature triples as training data to train the initial image matching model to obtain a target image matching model.

[0155] In step 201-step 207, the specific process of collecting the drone data set is to complete the initialization of the position, flight attitude and other information after the drone takes off, and then record the video and position information from the collection starting point to the end point according to the set flight speed and flight altitude. When making a drone image data set, image frames are intercepted from the flight video stream at specific time intervals, and the corresponding position information of the image is recorded. The data set collected in this way is prone to the situation where the model cannot identify the most similar image among a large number of high-similarity images. Specifically, the image coverage area between adjacent frames has a high degree of overlap, the content contained in adjacent images has a low degree of distinction, and the adjacent images visually show a small "move" phenomenon, causing the model to be unable to effectively distinguish between two images of adjacent frames with high similarity, which leads to positioning failure.

[0156] To address the above problems, this solution designs the most similar negative sample sampling strategy to construct a model training data set, fully mines and utilizes the information between highly similar neighboring images, maximizes the feature distance between the positive samples in a batch and the nearest negative samples, significantly enhances the network feature representation ability and similar sample discrimination ability, and improves the accuracy of visual matching and positioning.

[0157] First, a historical data set is obtained, which includes n matching pairs, each of which includes a paired historical drone image and a historical satellite image.

[0158] The matching pairs can be represented as Where A refers to the historical drone image, and P is the positive sample satellite image of the historical drone image A. The positive sample refers to the satellite image whose geographical location represented by the center of the pixel is closest to the location of the drone image.

[0159] The historical UAV images and historical satellite images in the n matching pairs are characterized respectively, and then the distance information between each historical UAV image and historical satellite image is calculated respectively, and a feature distance representation matrix D=cdist(a, p) with a matrix size of n×n is obtained.

[0160] in, a i and p j Corresponding to A i and P jThe feature description vector of .

[0161] Figure 5 A schematic diagram of generating a pairing feature triplet provided in an embodiment of the present invention.

[0162] like Figure 5 As shown in the figure, a1, a2, a3, and a4 represent historical drone images, and p1, p2, p3, and p4 represent historical satellite images. The historical drone images and historical satellite images are characterized, and the distance information between each pair of historical drone image feature information and historical satellite image feature information is calculated to obtain the feature distance representation matrix D. The diagonal of the distance matrix D represents the feature distance between the matching pairs of images that match each other. The elements in the non-diagonal position are the feature distances between the non-matching historical drone images and historical satellite images.

[0163] Calculate and match pairs (A i , P i )The calculation formula of the most similar negative sample with the closest distance is as follows:

[0164] j min =argmin j＝1..n，j≠i d(a i , p j )

[0165] k min =argmin k＝1.n，k≠i d(a k , p i ) (8)

[0166] Among them, j min For a i The nearest non-matching image The superscript of k min For i The nearest non-matching image Corner mark.

[0167] like Figure 5 As shown by Construct a feature descriptor quadruple and select and The smaller value in is taken as the most similar negative sample n min The final paired feature triplet (a i ,p i ,n min ).

[0168] The paired feature triplets are used as training data to train the initial image matching model to obtain the target image matching model.

[0169] This scheme designs the most similar negative sample sampling strategy to construct the model training data set, fully mines and utilizes the information between high-similarity neighboring images, maximizes the feature distance between the positive samples and the nearest negative samples in a batch, significantly enhances the network feature representation ability and similar sample discrimination ability, and improves the accuracy of visual matching and positioning.

[0170] In a possible implementation, step 102 includes:

[0171] Step 1021, matching the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a plurality of matching degrees;

[0172] Step 1022: sort the matching degrees in descending order, and obtain the satellite image ranked first as the target satellite image that matches the drone image to be located.

[0173] In step 1021-step 1022, a method for acquiring a target satellite image is given, that is, the satellite image ranked first in matching degree is selected as the target satellite image that matches the drone image to be located.

[0174] In a possible implementation manner, after step 1022, the method further includes:

[0175] Step 1023: Acquire the top K satellite images as candidate satellite images;

[0176] Step 1024: determine the homography matrix between each of the to-be-located UAV images and the candidate satellite images to obtain K second homography matrices;

[0177] Step 1025: Use the second homography matrix to map the location information in the candidate satellite image to the image of the unmanned aerial vehicle to be located, so as to obtain multiple candidate predicted location information of the unmanned aerial vehicle.

[0178] In step 1023-step 1025, the K satellite images ranked in the top K in terms of matching degree are used as candidate satellite images, and the second homography matrix between the image of the unmanned aerial vehicle to be located and each candidate satellite image is calculated. Using each second homography matrix, the location information in the candidate satellite image is mapped to the image of the unmanned aerial vehicle to be located, and multiple candidate predicted location information of the unmanned aerial vehicle is obtained. In this way, multiple candidate predicted location information is combined with the first predicted location information to obtain a result set of predicted location information.

[0179] If it is subsequently determined that the first predicted position information does not meet the requirements, the predicted position information in the result set can be used to calibrate the positioning.

[0180] Specifically, the calibration positioning method may be performed after step 105, and specifically includes the following steps:

[0181] Step 301: Acquire flight data of the UAV before the current moment, and fit the flight trajectory of the UAV using the flight data and the IMM model to obtain second predicted position information of the UAV at the current moment;

[0182] Step 302: Determine an area within a preset distance range of the second predicted position information as a predicted position range of the UAV at the current moment;

[0183] Step 303: Determine whether the first predicted location information is within the predicted location range;

[0184] Step 304: If the target position is within the predicted position range, the first predicted position information is determined as the target position information of the UAV at the current moment;

[0185] Step 305: If the target position is not within the prediction position range, selecting at least one target prediction position information within the prediction position range from the plurality of candidate prediction position information;

[0186] Step 306: obtaining at least one candidate satellite image corresponding to the at least one target predicted position information, and obtaining a satellite image that best matches the image of the unmanned aerial vehicle to be located among the at least one candidate satellite image;

[0187] Step 307: Determine the candidate predicted position information corresponding to the most matching satellite image as the target position information of the UAV at the current moment.

[0188] In steps 301 to 307, the present invention reasonably analyzes the positioning points of the previous flight through an image search space optimization algorithm based on trajectory prediction, fits the subsequent flight trajectory of the UAV, thereby estimating the confidence of the next UAV visual positioning result, evaluating the credibility of the positioning, and optimizing the subsequent satellite image library search space.

[0189] Specifically, the UAV trajectory is estimated through the IMM (Interacting Multiple Model) model. To better describe the IMM model, the following conventions are made:

[0190] 1. The model contains a total of n motion models Where i represents the i-th model and the algorithm iterates k discrete times.

[0191] 2. The system measurement value at the kth moment is Y k .

[0192] 3. There is correlation between models. According to the k-1 moment model i and the measured value Y k-1 Able to determine the transition probability of model j at time k

[0193] 4. Use M k and the measured value Y at time k k The probability of being able to accurately describe the motion of an object is

[0194] The input of the IMM algorithm in this scheme is the flight data of the UAV before the current moment, including acceleration, speed, turning rate, linear velocity, angular acceleration and other flight data.

[0195] In the implementation of the IMM algorithm, the input is first mixed, and the optimal estimate of each model is calculated at time k-1 and the covariance matrix The estimated states and covariance matrices related to each model are weighted and fused to obtain the optimal state x and covariance matrix P of the target under the fusion state at time k-1:

[0196]

[0197] in Represents the correlation coefficient of fusion. Based on the weight of the correlation, the models are weighted fused to obtain the fusion estimation state and covariance matrix of the i-th model. Considering that each model has a correctness probability, each element of P needs to be multiplied by the corresponding model confidence Then normalization is performed to obtain the final weights:

[0198]

[0199] Then the state equation and covariance matrix corresponding to the model are predicted and updated. The prediction steps are as follows:

[0200]

[0201] The update steps are as follows:

[0202]

[0203] Calculate the true value of the filter of each model at time k and the covariance matrix

[0204] In order to evaluate the quality of the model's estimation of the motion state, we use Evaluate the quality of predictions of model j:

[0205]

[0206] in The greater the deviation between the predicted value of model j and the measured value, the less effective the model is. Then, the model quality evaluation parameter Update the confidence of model j

[0207]

[0208] in represents the confidence after fusion, The denominator coefficient is used to achieve normalization. The updated confidence λ can be used to fuse the models according to the weights to obtain the output target state x of the IMM filtering algorithm. k And the covariance matrix P k :

[0209]

[0210] Through the above steps, the flight trajectory of the UAV is fitted. With the help of the second predicted position information generated by trajectory analysis, we can effectively estimate the satellite image library search range at the next moment and conduct a confidence assessment on the visual positioning results to be obtained.

[0211] The confidence evaluation method is as follows: determine the area within the preset distance range of the second predicted position information as the predicted position range of the drone at the current moment; determine whether the first predicted position information is within the predicted position range. If it is within the predicted position range, the prediction result of the first predicted position information is determined to be accurate, and the first predicted position information is used as the target position information. If it is not within the predicted position range, it means that there is a significant deviation between the first predicted position information and the predicted position range, indicating that position error or environmental changes may have occurred.

[0212] In order to ensure the accuracy of the results, the deviated results are removed to eliminate potential mismatches or abnormalities. Turning to the remaining result set, a plurality of candidate prediction position information is obtained, which is calculated according to the second homography matrix in step 1025.

[0213] At least one target predicted position information located within the predicted position range is selected from the plurality of candidate predicted position information. At least one candidate satellite image corresponding to the at least one target predicted position information is obtained, and a satellite image that best matches the image of the unmanned aerial vehicle to be located is obtained from the at least one candidate satellite image.

[0214] The candidate predicted position information corresponding to the most matching satellite image is taken as the most reliable positioning result and is determined as the target position information.

[0215] After obtaining the best matching image pair, this scheme calculates the position prediction result through the position acquisition algorithm of image registration, and evaluates the confidence through the IMM model. If there is a significant deviation from the estimated satellite library search space, the positions in the search space will be sorted in the remaining result set to determine the most reliable positioning result, thereby further improving the accuracy of UAV positioning.

[0216] Figure 6 is a structural block diagram of a drone auxiliary positioning device based on image matching provided by an embodiment of the present invention. Figure 6 As shown, the device 400 includes:

[0217] The drone image acquisition module 401 is used to acquire the overhead image collected by the drone at the current moment during the flight of the drone, and obtain the drone image to be located at the current moment;

[0218] The matching module 402 is used to input the drone image to be located into the target image matching model, extract the drone feature information of the drone image to be located using the target image matching model, and match the drone feature information with the satellite feature information of multiple satellite images carrying location information to obtain a target satellite image that matches the drone image to be located;

[0219] The feature point pairing module 403 is used to extract feature points of the unmanned aerial vehicle image to be located and the target satellite image, and perform feature point pairing on the unmanned aerial vehicle image to be located and the target satellite image to obtain a first pairing feature point;

[0220] A homography matrix calculation module 404 is used to calculate a first homography matrix between the image of the unmanned aerial vehicle to be located and the target satellite image using the position coordinate information of the first paired feature points;

[0221] The mapping module 405 is used to map the position information in the target satellite image to the image of the unmanned aerial vehicle to be located by using the first homography matrix to obtain the first predicted position information of the unmanned aerial vehicle at the current moment.

[0222] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0223] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0224] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that comply with the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0225] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A UAV assisted positioning method based on image matching, characterized in that: The method comprises: During the flight of the drone, an overhead image collected by the drone at the current moment is acquired to obtain an image of the drone to be located at the current moment; Inputting the unmanned aerial vehicle image to be located into a target image matching model, extracting unmanned aerial vehicle feature information of the unmanned aerial vehicle image to be located using the target image matching model, and matching the unmanned aerial vehicle feature information with satellite feature information of a plurality of preset satellite images carrying location information, to obtain a target satellite image that matches the unmanned aerial vehicle image to be located; Extracting feature points of the unmanned aerial vehicle image to be located and the target satellite image, and performing feature point pairing on the unmanned aerial vehicle image to be located and the target satellite image to obtain first paired feature points; Calculating a first homography matrix between the image of the unmanned aerial vehicle to be located and the target satellite image using the position coordinate information of the first paired feature points; The first homography matrix is ​​used to map the location information in the target satellite image to the image of the unmanned aerial vehicle to be located, so as to obtain the first predicted location information of the unmanned aerial vehicle at the current moment.

2. The method according to claim 1, characterized in that The target image matching model includes an image matching layer and a downsampling layer. The step of extracting the drone feature information of the drone image to be located by using the target image matching model includes: The downsampling layer is used to downsample the image of the unmanned aerial vehicle to be located to obtain a downsampled image; The image matching layer is used to extract and combine multi-scale features of the down-sampled image to obtain drone feature information.

3. The method according to claim 2, characterized in that The image matching layer includes a spatial attention mechanism unit, a depth-separable convolution unit, and a global response normalization unit. The image matching layer is used to extract and combine multi-scale features of the downsampled image to obtain drone feature information, including: Using the spatial attention mechanism unit, dynamically adjusting weights of different feature vectors in the downsampled image to obtain first feature information; Inputting the first feature information into the depthwise separable convolution unit, and performing a separate convolution on the feature information input into each channel using the convolution kernel of each channel to obtain feature information corresponding to each channel; Using a 1x1 convolution kernel to linearly combine the feature information corresponding to each of the channels to obtain second feature information; The second feature information is input into the global response normalization unit, and global feature aggregation, feature normalization and feature calibration are performed on the second feature information to obtain UAV feature information.

4. The method according to claim 3, characterized in that The method of using the spatial attention mechanism unit to dynamically adjust the weights of different feature vectors in the downsampled image to obtain first feature information includes: Performing a global maximum pooling operation and an average pooling operation on the downsampled image to obtain two one-dimensional vectors having the same number of channels as the feature vector of the downsampled image; Input the two one-dimensional vectors into a convolution layer with a convolution kernel size of 7 to perform a convolution operation to obtain a convolution feature vector; Input the convolution feature vector into the activation function for activation, and obtain the weight coefficient corresponding to each feature vector; The weight coefficient is multiplied by a feature vector corresponding to the down-sampled image to obtain first feature information.

5. The method according to claim 3, characterized in that: The step of inputting the second feature information into the global response normalization unit, performing global feature aggregation, feature normalization, and feature calibration on the second feature information, and obtaining the UAV feature information includes: The feature information of each channel in the second feature information is respectively subjected to feature aggregation by a global function to obtain a first vector corresponding to each channel; the first vector is a scalar that aggregates the statistical features of the feature information of the current channel; Using a standard division normalization function to normalize each of the first vectors, respectively, to obtain a feature normalization score; the feature normalization score represents the importance of the current channel relative to other channels; The second feature information is calibrated in response to the feature normalization score and the preset learnable parameters to obtain the drone feature information.

6. The method according to claim 3, characterized in that Before obtaining the overhead image collected by the drone at the current moment, the method further includes: Acquire a historical data set, wherein the historical data set includes n matching pairs, wherein the matching pairs are a historical drone image and a historical satellite image that is closest to a geographical location represented by a pixel center of the historical drone image; Performing feature descriptions on the historical UAV images and historical satellite images in the n matching pairs respectively to obtain n historical UAV image feature information and n historical satellite image feature information; Calculate the distance information between each pair of historical UAV image feature information and historical satellite image feature information respectively; Determine, based on the distance information, a non-matching historical satellite image that is closest to the characteristic information of each of the historical drone images, and determine a non-matching historical drone image that is closest to the characteristic information of each of the historical satellite images; Determine first distance information of the non-matching historical satellite image corresponding to the historical drone image in each of the matching pairs, and second distance information of the non-matching historical drone image corresponding to the historical satellite image in the matching pair; Selecting an image corresponding to a smaller value of the first distance information and the second distance information as the most similar negative sample of the matching pair, and forming a paired feature triplet with the matching pair and the corresponding most similar negative sample; The paired feature triplets are used as training data to train the initial image matching model to obtain a target image matching model.

7. The method according to claim 1, characterized in that The step of matching the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located includes: Matching the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a plurality of matching degrees; The matching degrees are sorted in descending order, and the satellite image ranked first is obtained as the target satellite image that matches the drone image to be located.

8. The method according to claim 7, characterized in that After obtaining the target satellite image that matches the image of the unmanned aerial vehicle to be located, the method further includes: Obtain the top K satellite images as candidate satellite images; Determine the homography matrix between each of the to-be-located UAV images and the candidate satellite images to obtain K second homography matrices; The position information in the candidate satellite image is mapped to the image of the unmanned aerial vehicle to be located using the second homography matrix to obtain multiple candidate predicted position information of the unmanned aerial vehicle.

9. The method according to claim 8, characterized in that After obtaining the first predicted position information of the drone at the current moment, the method further includes: Acquire flight data of the UAV before the current moment, and fit the flight trajectory of the UAV using the flight data and the IMM model to obtain second predicted position information of the UAV at the current moment; Determine an area within a preset distance range of the second predicted position information as a predicted position range of the drone at the current moment; Determining whether the first predicted location information is within the predicted location range; If it is within the predicted position range, determining the first predicted position information as the target position information of the UAV at the current moment; If it is not within the predicted position range, selecting at least one target predicted position information within the predicted position range from the multiple candidate predicted position information; Acquire at least one candidate satellite image corresponding to the at least one target predicted position information, and acquire a satellite image that best matches the image of the unmanned aerial vehicle to be located among the at least one candidate satellite image; The candidate predicted position information corresponding to the most matching satellite image is determined as the target position information of the UAV at the current moment.

10. An image matching-based drone auxiliary positioning device, characterized in that: The device comprises: The UAV image acquisition module is used to acquire the overhead image collected by the UAV at the current moment during the flight of the UAV, and obtain the image of the UAV to be located at the current moment; A matching module is used to input the drone image to be located into a target image matching model, extract drone feature information of the drone image to be located using the target image matching model, and match the drone feature information with satellite feature information of a plurality of preset satellite images carrying location information to obtain a target satellite image that matches the drone image to be located; A feature point pairing module is used to extract feature points of the unmanned aerial vehicle image to be located and the target satellite image, and perform feature point pairing on the unmanned aerial vehicle image to be located and the target satellite image to obtain a first pairing feature point; A homography matrix calculation module, used to calculate a first homography matrix between the image of the unmanned aerial vehicle to be located and the target satellite image using the position coordinate information of the first paired feature points; A mapping module is used to map the location information in the target satellite image to the image of the unmanned aerial vehicle to be located using the first homography matrix to obtain the first predicted location information of the unmanned aerial vehicle at the current moment.

Images