An unmanned aerial vehicle visual positioning method based on adaptive selection of satellite image and airborne image matching results

Abstract

The application provides a UAV visual positioning method based on adaptive selection of satellite image and airborne image matching results, which comprises the following steps: slicing a satellite image to construct an image database of the satellite image slices; taking an initial frame as a query image to retrieve satellite image slices from the image database, and performing image matching on the initial frame and the satellite image slices to calculate a positioning result of the initial frame; performing image matching on a subsequent frame and its previous frame, constructing a matching relationship between the subsequent frame and the satellite image by using the matching relationship between the previous frame and the satellite image, and calculating a positioning result of the subsequent frame; taking a corresponding pixel point of a center point of an input frame at a current time on the satellite image as a center to cut out a satellite image slice, performing image matching on the input frame and the satellite image slice, and calculating a positioning result of the input frame; and evaluating the positioning results and adaptively selecting a final UAV positioning result according to an evaluation index.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more specifically to a UAV visual positioning method based on adaptive selection of satellite image and airborne image matching results. Background Technology

[0002] Unmanned aerial vehicle (UAV) visual positioning technology aims to solve the problem of UAV positioning when Global Navigation Satellite Systems (GNSS) are unavailable. This technology estimates the absolute pose of the UAV on Earth using satellite imagery and images taken by the UAV's onboard camera, enabling UAV positioning without the use of GNSS.

[0003] UAV visual positioning technology is mainly divided into two categories: template matching-based methods and feature matching-based methods. Template matching-based methods use images captured by the UAV as templates, performing a sliding window matching process on a reference satellite image. Similarity metrics such as normalized distance, mutual information, and phase correlation are used to measure the similarity between the UAV-captured image and satellite image slices, thereby determining the UAV's absolute position. Feature matching-based methods include feature extraction and feature matching. Feature extraction extracts salient key points and their feature descriptors from the UAV-captured image and the satellite image. Feature matching matches the feature descriptors of the key points in the two images, constructing matching point pairs between the UAV-captured image and the satellite image. The absolute position of the UAV is then calculated using the positions of these matching points on the satellite image.

[0004] In UAV visual positioning technology, template-based methods are subject to numerous limitations. They often require UAV images and satellite images to maintain consistency in scale and rotation angle, and the UAV images must be captured vertically downwards to ensure a consistent viewpoint with the satellite images, thus severely restricting their application. Feature-based methods face the challenge of difficult matching between UAV and satellite images. Due to differences in image acquisition time and methods, factors such as lighting and scene conditions also vary, making feature matching difficult. This invention divides the input frame into an initial frame and subsequent frames. For the initial frame, it compares the positioning results obtained based on global positioning with those obtained based on local satellite image matching. For subsequent frames, it compares the positioning results obtained based on visual odometry with those obtained based on local satellite image matching. By comparing these results, the better positioning result is adaptively selected and used as the absolute position of the UAV, enabling accurate positioning of the UAV even in areas where matching aerial and satellite images is difficult. Summary of the Invention

[0005] The purpose of this invention is to provide a UAV visual positioning method based on adaptive selection of satellite image and airborne image matching results. This method divides the input frame into an initial frame and subsequent frames. For the initial frame, it compares the positioning result obtained based on global positioning with the positioning result obtained based on local satellite image matching. For subsequent frames, it compares the positioning result obtained based on visual odometry with the positioning result obtained based on local satellite image matching. By comparing these results and adaptively selecting the better positioning result, and using this as the UAV's absolute position, accurate positioning of the UAV is achieved even in areas where matching aerial and satellite images is difficult.

[0006] This invention provides a UAV visual positioning method based on adaptive selection of satellite image and airborne image matching results, comprising:

[0007] (1.1) Data preparation: Sliding window slicing and slice rotation enhancement are performed on satellite images, and an image database of satellite image slices is constructed based on these slice images and the enhanced slice images;

[0008] (1.2) Global positioning of the initial frame: The initial frame is used as the query image. Satellite image slices are retrieved from the image database. The initial frame and satellite image slices are matched to calculate the positioning result of the UAV at the time corresponding to the initial frame.

[0009] (1.3) Visual odometry of subsequent frames: The subsequent frames are matched with the previous frame. The matching relationship between the previous frame and the satellite image is used to construct the matching relationship between the subsequent frames and the satellite image, and then the positioning result of the UAV at the corresponding time of the subsequent frame is calculated.

[0010] (1.4) Local satellite image matching and positioning: Taking the corresponding pixel point of the center point of the input frame at the current time on the satellite image as the center, a satellite image slice is cropped, the input frame is matched with the satellite image slice, and the positioning result of the UAV at the corresponding time of the input frame is calculated.

[0011] (1.5) Adaptive positioning selection: The positioning results are evaluated, and the final UAV positioning result is adaptively selected based on the evaluation indicators.

[0012] The UAV visual positioning method based on adaptive selection of satellite image and airborne image matching results, as described above, is characterized in that the data preparation includes:

[0013] (2.1) Using a sliding window method, the wide-area satellite image is cropped according to a cropping size of 1000×1000 and a window overlap rate of 0.8 to obtain multiple satellite image slices;

[0014] (2.2) In order to further fit different rotation angles of the aerial image, each slice image was rotated clockwise three times at 90°, 180° and 270° respectively, resulting in four slice images including the original slice image.

[0015] (2.3) Using the NetVLAD method, extract image feature descriptors for all slice images and save all slice images and their image feature descriptors into an image database.

[0016] The UAV visual positioning method based on adaptive selection of satellite image and airborne image matching results, as described above, is characterized in that the global positioning of the initial frame includes:

[0017] (3.1) For the initial frame, the NetVLAD method is used to extract image feature descriptors, and the cosine similarity is calculated with the image feature descriptors in the image database. The top 5 satellite image slices with the highest cosine similarity are then extracted.

[0018] (3.2) For these 5 satellite image slices, each image slice is matched with the initial frame. The image matching process is to first use the SuperPoint algorithm to extract key points and their feature descriptors, and then use the LightGlue algorithm to match the key point feature descriptors, hereinafter referred to as SPLG, to obtain matching point pairs.

[0019] (3.3) Take the satellite image slice with the most matching point pairs, calculate the homography matrix based on the matching point pairs between the initial frame and the satellite image slice, and then perform the UAV latitude and longitude position solution to obtain the UAV latitude and longitude position and average reprojection error at the time corresponding to the initial frame. If the number of matching point pairs is less than 6 or the average reprojection error is greater than 100, the positioning fails, and the positioning failure is returned as the positioning result. The next frame is used as the initial frame to start the calculation again.

[0020] The UAV visual positioning method based on adaptive selection of satellite image and airborne image matching results, as described above, is characterized in that the visual odometry of subsequent frames includes:

[0021] (4.1) Perform image matching based on the AKAZE algorithm on the subsequent frame and its previous frame, and calculate the homography matrix from the subsequent frame to its previous frame based on the matching point pairs.

[0022] (4.2) Perform matrix multiplication on the homography matrix from the previous frame to the satellite image and the homography matrix from the subsequent frame to its previous frame to obtain the homography matrix from the subsequent frame to the satellite image. Then, perform the UAV latitude and longitude position calculation to obtain the UAV latitude and longitude position and average reprojection error at the corresponding time of the subsequent frame. If the number of matching point pairs is less than 6 or the average reprojection error is greater than 100, the positioning fails and the positioning failure is returned as the positioning result. The next frame is then used as the initial frame.

[0023] The UAV visual positioning method based on adaptive selection of satellite image and airborne image matching results, as described above, is characterized in that the local satellite image matching positioning includes:

[0024] (5.1) If the input frame is the initial frame, the homography matrix estimated by the "global positioning of the initial frame" is considered as the homography matrix from the current input frame to the satellite image. If the input frame is a subsequent frame, the homography matrix estimated by the "visual odometry of the subsequent frame" is considered as the homography matrix from the current input frame to the satellite image.

[0025] (5.2) Using the homography matrix from the current input frame to the satellite image, calculate the corresponding pixel point of the center point of the input frame on the satellite image, and crop out a local satellite image on the wide-area satellite image with the pixel point as the center and the size of 1000×1000.

[0026] (5.3) Using the homography matrix from the current input frame to the satellite image, perform homography transformation on the input frame to obtain the homography-transformed input frame;

[0027] (5.4) Perform image matching between the homography-transformed input frame and the local satellite image. The matching algorithm is SPLG to obtain matching point pairs. Here, the matching point pairs are "matching point pairs between the homography-transformed input frame and the local satellite image".

[0028] (5.5) Perform an inverse transformation on the matching points of the input frame after homography transformation to restore them to the input frame, thus obtaining the matching point pairs between the input frame and the satellite image;

[0029] (5.6) Using the matching point pairs between the input frame and the satellite image, calculate the homography matrix from the input frame to the satellite image, and then perform the UAV latitude and longitude position solution to obtain the UAV's latitude and longitude position and average reprojection error at the time corresponding to the input frame.

[0030] The UAV visual positioning method described above, which adaptively selects based on the matching results of satellite and airborne images, is characterized in that the adaptive positioning selection includes:

[0031] (6.1) If the input frame is the initial frame, the average reprojection error calculated by "global positioning of the initial frame" and "local satellite image matching positioning" is compared to obtain the minimum average reprojection error at the time of the initial frame; if the input frame is a subsequent frame, the average reprojection error calculated by "visual odometry of the subsequent frame" and "local satellite image matching positioning" is compared to obtain the minimum average reprojection error at the time of the subsequent frame.

[0032] (6.2) The positioning result corresponding to the minimum average reprojection error, including the latitude and longitude position of the UAV and the homography matrix of the corresponding input frame to the satellite image, is used as the final UAV positioning result corresponding to the input frame time.

[0033] The UAV visual positioning method based on adaptive selection of satellite image and airborne image matching results, as described above, is characterized by the following steps in solving the UAV's latitude and longitude position:

[0034] (7.1) Using the homography matrix to obtain matching point pairs between aerial images and satellite images: Set one center point, four corner points and four midpoints of the side of the aerial image as anchor points, and use the homography matrix to calculate the corresponding points of the nine anchor points on the satellite image, thereby obtaining nine pairs of matching point pairs between the aerial image and the satellite image.

[0035] (7.2) Transformation of satellite image matching point pixel coordinates into projected coordinate system coordinates: Using the latitude and longitude information of the pixels carried by the satellite image, and the conversion formula from latitude and longitude to the projected coordinate system, the two-dimensional pixel coordinates of the satellite image matching points are transformed into two-dimensional coordinates in the projected coordinate system.

[0036] (7.3) Satellite image matching points are converted into three-dimensional points: When the UAV reaches a certain flight altitude, the ground can be considered flat. Therefore, a z-axis is added to the projected coordinate system and the z-coordinate is set to 0, so as to convert the satellite image matching points into three-dimensional points.

[0037] (7.4) Calculate the UAV pose and average reprojection error in the projected coordinate system: Construct a 3D-2D matching relationship using the two-dimensional pixel coordinates of the matching points in the aerial image and the three-dimensional coordinates of the matching points in the satellite image. Use the PnP algorithm to solve the UAV pose in the projected coordinate system. At the same time, use this pose to project the three-dimensional points onto the two-dimensional pixel plane. Calculate the pixel error between each projected point and the corresponding matching point. Take the average value to obtain the average reprojection error. The purpose of calculating the average reprojection error is to provide an evaluation index for the positioning results.

[0038] (7.5) Obtain the latitude and longitude position of the UAV using the UAV pose in the projected coordinate system: Since we only care about the latitude and longitude position of the UAV, we use the UAV pose in the projected coordinate system to obtain the two-dimensional position of the UAV in the projected coordinate system. Then, we use the conversion formula from the projected coordinate system to latitude and longitude to convert the two-dimensional position of the UAV in the projected coordinate system into the latitude and longitude position of the UAV.

[0039] Compared to existing technologies, this invention distinguishes the input frame into an initial frame and subsequent frames. For the initial frame, it compares the positioning result obtained based on global positioning with the positioning result obtained based on local satellite image matching. For subsequent frames, it compares the positioning result obtained based on visual odometry with the positioning result obtained based on local satellite image matching. By comparing the above results, the better positioning result is adaptively selected and used as the absolute position of the UAV. This enables the UAV to achieve accurate positioning even in areas where matching aerial and satellite images is difficult. Attached Figure Description

[0040] 1. Flowchart of UAV visual positioning method based on adaptive selection of satellite image and airborne image matching results

[0041] 2. Flowchart for solving the latitude and longitude position of a UAV Detailed Implementation

[0042] To make the purpose, technical solution and advantages of this method clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments.

[0043] like Figure 1 As shown, the UAV visual positioning method based on adaptive selection of satellite image and airborne image matching results proposed in this invention includes the following steps:

[0044] (1) Data preparation: Sliding window slicing and slice rotation enhancement are performed on satellite images, and an image database of satellite image slices is constructed based on these slice images and the enhanced slice images;

[0045] (2) Global positioning of the initial frame: The initial frame is used as the query image. Satellite image slices are retrieved from the image database, and the initial frame and satellite image slices are matched to calculate the positioning result of the UAV at the time corresponding to the initial frame.

[0046] (3) Visual odometry of subsequent frames: The subsequent frames are matched with the previous frame. The matching relationship between the previous frame and the satellite image is used to construct the matching relationship between the subsequent frames and the satellite image, and then the positioning result of the UAV at the corresponding time of the subsequent frame is calculated.

[0047] (4) Local satellite image matching and positioning: Taking the corresponding pixel point of the center point of the input frame at the current time on the satellite image as the center, the satellite image slice is cropped, the input frame is matched with the satellite image slice, and the positioning result of the UAV at the corresponding time of the input frame is calculated.

[0048] (5) Adaptive positioning selection: The positioning results are evaluated, and the final UAV positioning result is adaptively selected based on the evaluation indicators.

[0049] like Figure 1 The data preparation shown mainly includes: cropping the wide-field satellite image using a sliding window method with a cropping size of 1000×1000 and a window overlap rate of 0.8 to obtain multiple satellite image slices; in order to further fit different rotation angles of the aerial image, each slice image is rotated clockwise three times at 90°, 180° and 270° respectively to obtain four slice images including the original slice image; using the NetVLAD method, image feature descriptors are extracted from all slice images, and all slice images and their image feature descriptors are saved as an image database.

[0050] like Figure 1 The global localization of the initial frame mainly includes: for the initial frame, extracting image feature descriptors using the NetVLAD method, calculating cosine similarity with image feature descriptors in the image database, and extracting the top 5 satellite image slices with the highest cosine similarity; for each of these 5 satellite image slices, performing image matching with the initial frame. The image matching process involves first extracting key points and their feature descriptors using the SuperPoint algorithm for both the image slices and the initial frame, then matching the key point feature descriptors using the LightGlue algorithm (hereinafter referred to as SPLG) to obtain matching point pairs; taking the satellite image slice with the most matching point pairs, calculating the homography matrix based on the matching point pairs between the initial frame and this satellite image slice, and then performing the following... Figure 2 The method for solving the latitude and longitude position of the UAV shown in the figure obtains the latitude and longitude position of the UAV and the average reprojection error at the time corresponding to the initial frame. If the number of matching point pairs is less than 6 or the average reprojection error is greater than 100, the positioning fails, and the positioning failure is returned as the positioning result. The next frame is then used as the initial frame to start the calculation again.

[0051] like Figure 1The visual odometry for subsequent frames mainly includes: image matching between the subsequent frame and its previous frame. Due to the short time interval and small image changes, the matching difficulty is low. Therefore, the traditional AKAZE algorithm with rotation invariance is used for image matching. The homography matrix from the subsequent frame to its previous frame is calculated based on the matching point pairs. Matrix multiplication is performed between the homography matrix from the previous frame to the satellite image and the homography matrix from the subsequent frame to its previous frame to obtain the homography matrix from the subsequent frame to the satellite image. Then, the following steps are performed: Figure 2 The latitude and longitude position of the UAV shown is obtained by solving the latitude and longitude position of the UAV at the corresponding time in the subsequent frame and the average reprojection error. If the number of matching point pairs is less than 6 or the average reprojection error is greater than 100, the positioning fails and the positioning failure is returned as the positioning result, and the next frame is used as the initial frame.

[0052] like Figure 1 The local satellite image matching and localization shown mainly includes: if the input frame is the initial frame, the homography matrix estimated by "global localization of the initial frame" is considered as the homography matrix from the current input frame to the satellite image; if the input frame is a subsequent frame, the homography matrix estimated by "visual odometry of the subsequent frame" is considered as the homography matrix from the current input frame to the satellite image; using the homography matrix from the current input frame to the satellite image, the corresponding pixel point of the center point of the input frame on the satellite image is calculated, and a local satellite image is cropped from the wide-format satellite image with a size of 1000×1000, centered on this pixel point; using the homography matrix from the current input frame to the satellite image... The homography matrix is ​​used to perform a homography transformation on the input frame, resulting in a homography-transformed input frame that maintains a consistent viewpoint with the local satellite image, thus improving matching quality. The homography-transformed input frame is then matched with the local satellite image using the SPLG matching algorithm to obtain matching point pairs. These matching point pairs are defined as "matching point pairs between the homography-transformed input frame and the local satellite image." The matching points of the homography-transformed input frame are then inversely transformed to restore them onto the input frame, yielding matching point pairs between the input frame and the satellite image. Using these matching point pairs, the homography matrix from the input frame to the satellite image is calculated, and then the following steps are performed: Figure 2 The latitude and longitude position of the UAV shown is obtained by solving the latitude and longitude position of the UAV at the corresponding time of the input frame and the average reprojection error.

[0053] like Figure 1 The adaptive positioning selection shown mainly includes: if the input frame is the initial frame, comparing the average reprojection errors calculated by "global positioning of the initial frame" and "local satellite image matching positioning" respectively; if the input frame is a subsequent frame, comparing the average reprojection errors calculated by "visual odometry of the subsequent frame" and "local satellite image matching positioning" respectively; selecting the UAV latitude and longitude position corresponding to the positioning result with the smaller average reprojection error and the corresponding homography matrix from the input frame to the satellite image to achieve the final UAV positioning.

[0054] like Figure 2 As shown, the process of determining the latitude and longitude of a UAV includes the following steps:

[0055] (1) Using the homography matrix to obtain matching point pairs between aerial images and satellite images: Set one center point, four corner points and four midpoints of the aerial images as anchor points, and use the homography matrix to calculate the corresponding points of the nine anchor points on the satellite images, thereby obtaining nine pairs of matching point pairs between aerial images and satellite images.

[0056] (2) Transform the pixel coordinates of the matching point in the satellite image into the coordinates of the projected coordinate system: Using the latitude and longitude information of the pixel points carried by the satellite image, and the conversion formula from latitude and longitude to the projected coordinate system, the two-dimensional pixel coordinates of the matching point in the satellite image are transformed into two-dimensional coordinates in the projected coordinate system.

[0057] (3) Satellite image matching points are converted into three-dimensional points: When the UAV reaches a certain flight altitude, the ground can be considered flat. Therefore, a z-axis is added to the projected coordinate system and the z-coordinate is set to 0, so that the satellite image matching points are converted into three-dimensional points.

[0058] (4) Calculate the UAV pose and average reprojection error in the projection coordinate system: Construct a 3D-2D matching relationship using the 3D coordinates of the matching points in the satellite image and the 2D pixel coordinates of the matching points in the aerial image. Use the PnP algorithm to solve the UAV pose in the projection coordinate system. At the same time, use this pose to project the 3D points onto the 2D pixel plane, calculate the pixel error between each projection point and the corresponding matching point, and take the average value to obtain the average reprojection error. The purpose of calculating the average reprojection error is to provide an evaluation index for the positioning results.

[0059] (5) Obtain the latitude and longitude position of the UAV using the UAV pose in the projected coordinate system: Since we only care about the latitude and longitude position of the UAV, we use the UAV pose in the projected coordinate system to obtain the two-dimensional position of the UAV in the projected coordinate system. Then, we use the conversion formula from the projected coordinate system to latitude and longitude to convert the two-dimensional position of the UAV in the projected coordinate system into the latitude and longitude position of the UAV.

[0060] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0061] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1.A method for adaptive selection of unmanned aerial vehicle (UAV) visual positioning based on matching results of satellite images and aerial images, characterized in that, include: (1.1) Data preparation: Sliding window slicing and slice rotation enhancement are performed on satellite images, and an image database of satellite image slices is constructed based on these slice images and the enhanced slice images; (1.2) Global positioning of the initial frame: The initial frame is used as the query image. Satellite image slices are retrieved from the image database, and the initial frame and satellite image slices are matched to calculate the positioning result of the UAV at the time corresponding to the initial frame. (1.3) Visual odometry of subsequent frames: The subsequent frame is matched with the previous frame using the AKAZE algorithm. The homography matrix from the subsequent frame to the previous frame is calculated based on the matching point pairs. The homography matrix from the previous frame to the satellite image and the homography matrix from the subsequent frame to the previous frame are multiplied by matrix to obtain the homography matrix from the subsequent frame to the satellite image. Then, the latitude and longitude position of the UAV is solved to obtain the latitude and longitude position and average reprojection error of the UAV at the corresponding time of the subsequent frame. If the number of matching point pairs is less than 6 or the average reprojection error is greater than 100, the positioning fails and the positioning failure is returned as the positioning result. The next frame is used as the initial frame. (1.4) Local satellite image matching and positioning: Taking the corresponding pixel point of the center point of the input frame at the current time on the satellite image as the center, a satellite image slice is cropped, the input frame is matched with the satellite image slice, and the positioning result of the UAV at the corresponding time of the input frame is calculated. (1.5) Adaptive positioning selection: The positioning results are evaluated. If the input frame is the initial frame, the average reprojection error calculated by "global positioning of the initial frame" and "local satellite image matching positioning" is compared to obtain the minimum average reprojection error at the time of the initial frame. If the input frame is a subsequent frame, the average reprojection error calculated by "visual odometry of the subsequent frame" and "local satellite image matching positioning" is compared to obtain the minimum average reprojection error at the time of the subsequent frame. The positioning result corresponding to the minimum average reprojection error, including the latitude and longitude position of the UAV and the homography matrix from the input frame to the satellite image, is taken as the final UAV positioning result at the time of the input frame. 2.The method of claim 1, wherein, The data preparation includes: (2.1) Using a sliding window method, the wide-area satellite image is cropped according to a cropping size of 1000×1000 and a window overlap rate of 0.8 to obtain multiple satellite image slices; (2.2) In order to further fit different rotation angles of the aerial image, each slice image was rotated clockwise three times at 90°, 180° and 270° respectively, resulting in four slice images including the original slice image. (2.3) Using the NetVLAD method, extract image feature descriptors for all slice images and save all slice images and their image feature descriptors into an image database. 3.The method of claim 1, wherein, The global positioning of the initial frame includes: (3.1) For the initial frame, the NetVLAD method is used to extract image feature descriptors, and the cosine similarity is calculated with the image feature descriptors in the image database. The top 5 satellite image slices with the highest cosine similarity are then extracted. (3.2) For these 5 satellite image slices, each image slice is matched with the initial frame. The image matching process is to first use the SuperPoint algorithm to extract key points and their feature descriptors, and then use the LightGlue algorithm to match the key point feature descriptors, hereinafter referred to as SPLG, to obtain matching point pairs; (3.3) Take the satellite image slice with the most matching point pairs, calculate the homography matrix based on the matching point pairs between the initial frame and the satellite image slice, and then perform the UAV latitude and longitude position solution to obtain the UAV latitude and longitude position and average reprojection error at the time corresponding to the initial frame. If the number of matching point pairs is less than 6 or the average reprojection error is greater than 100, the positioning fails, and the positioning failure is returned as the positioning result. The next frame is used as the initial frame to start the calculation again. 4.The method of claim 1, wherein, The local satellite image matching and positioning includes: (4.1) If the input frame is the initial frame, the homography matrix estimated by "global positioning of the initial frame" is considered as the homography matrix from the current input frame to the satellite image. If the input frame is a subsequent frame, the homography matrix estimated by "visual odometry of the subsequent frame" is considered as the homography matrix from the current input frame to the satellite image. (4.2) Using the homography matrix from the current input frame to the satellite image, calculate the corresponding pixel on the satellite image of the center point of the input frame, and crop out a local satellite image on the wide-area satellite image with the pixel as the center and the size of 1000×1000. (4.3) Using the homography matrix from the current input frame to the satellite image, perform homography transformation on the input frame to obtain the homography-transformed input frame; (4.4) Perform image matching between the homography-transformed input frame and the local satellite image. The matching algorithm is SPLG to obtain matching point pairs. Here, the matching point pairs are "matching point pairs between the homography-transformed input frame and the local satellite image". (4.5) Perform an inverse transformation on the matching points of the input frame after homography transformation to restore them to the input frame, thus obtaining the matching point pairs between the input frame and the satellite image; (4.6) Using the matching point pairs between the input frame and the satellite image, calculate the homography matrix from the input frame to the satellite image, and then perform the UAV latitude and longitude position solution to obtain the UAV's latitude and longitude position and average reprojection error at the time corresponding to the input frame. 5.The method of claim 1, wherein, The determination of the UAV's latitude and longitude position includes the following steps: (5.1) Using the homography matrix to obtain matching point pairs between aerial images and satellite images: Set one center point, four corner points and four midpoints of the side of the aerial image as anchor points, and use the homography matrix to calculate the corresponding points of the nine anchor points on the satellite image, thereby obtaining nine pairs of matching point pairs between the aerial images and satellite images. (5.2) Transform the pixel coordinates of the matching point in the satellite image into the coordinates of the projected coordinate system: Using the latitude and longitude information of the pixel points carried by the satellite image, and the conversion formula from latitude and longitude to the projected coordinate system, the two-dimensional pixel coordinates of the matching point in the satellite image are transformed into two-dimensional coordinates in the projected coordinate system. (5.3) Satellite image matching points are converted into three-dimensional points: When the UAV reaches a certain flight altitude, the ground can be considered flat. Therefore, a z-axis is added to the projected coordinate system and the z-coordinate is set to 0, so as to convert the satellite image matching points into three-dimensional points. (5.4) Calculate the UAV pose and average reprojection error in the projected coordinate system: Construct a 3D-2D matching relationship using the two-dimensional pixel coordinates of the matching points in the aerial image and the three-dimensional coordinates of the matching points in the satellite image. Use the PnP algorithm to solve the UAV pose in the projected coordinate system. At the same time, use this pose to project the three-dimensional points onto the two-dimensional pixel plane, calculate the pixel error between each projection point and the corresponding matching point, and take the average value to obtain the average reprojection error. The purpose of calculating the average reprojection error is to provide an evaluation index for the positioning results. (5.5) Obtain the latitude and longitude position of the UAV using the UAV pose in the projected coordinate system: Since we only care about the latitude and longitude position of the UAV, we use the UAV pose in the projected coordinate system to obtain the two-dimensional position of the UAV in the projected coordinate system. Then, we use the conversion formula from the projected coordinate system to latitude and longitude to convert the two-dimensional position of the UAV in the projected coordinate system into the latitude and longitude position of the UAV.

Images