Unmanned aerial vehicle accumulative-error-free grading positioning method based on satellite map

Abstract

The invention discloses an unmanned aerial vehicle accumulative-error-free hierarchical positioning method based on a satellite map. The method comprises the steps of satellite picture off-line feature sparse processing, map data cutting and feature compression. Carrying out matching positioning on the aerial image and a satellite image reference image, and constructing a space-time consistent transmission framework of'deep learning satellite image matching + optical flow homography matrix transmission '; carrying out homography matrix transmission on the aerial image through optical flow tracking, taking a homography matrix obtained through satellite image matching as a correction value, carrying out back calculation on geographic latitude and longitude coordinates corresponding to feature points of each frame of aerial image in real time, and providing a data source for a visual inertial odometer; in combination with a visual inertial odometer, map coordinate points and SLAM rear-end measurement updating are tightly coupled, latitude and longitude coordinates of the map points are used for endowing visual positioning with absolute position information, and a positioning result of the visual inertial odometer is optimized and corrected. According to the invention, denial flight is realized, the trajectory is stable in the flight process, no accumulated error exists, and the positioning precision is high.

Description

Technical Field

[0001] This invention belongs to the field of scene matching visual positioning technology, specifically relating to a UAV hierarchical positioning method without cumulative error based on satellite maps. Background Technology

[0002] With the rapid development of fields such as drones and autonomous driving, high-precision real-time positioning technology has become a core industry requirement. Inertial navigation systems (INS) offer high autonomy, relying on no external information and radiating no information outwards, while maintaining high short-term accuracy. However, positioning errors accumulate rapidly over time, and high-precision INS are not only bulky and large but also expensive. Global Positioning Navigation Systems (GPS) offer high positioning accuracy, with errors not accumulating over time, but suffer from poor autonomy, susceptibility to interference, and low update frequency, making it difficult to meet the requirements of high-dynamic real-time control. Since a single navigation system often cannot meet practical requirements, integrated navigation systems, primarily based on INS and supplemented by other navigation systems, have become the mainstream. Integrated navigation systems can significantly improve the accuracy and reliability of navigation systems.

[0003] To address the aforementioned problems, this invention proposes a scene matching and positioning scheme that combines visual odometry. Using regional features as characteristics, an image sensor is used to acquire images of the flight area and match them with a reference image stored on the aircraft to obtain the aircraft's position data. This corrects the navigation cumulative error caused by visual odometry during long-term flight, thereby achieving accurate navigation. Summary of the Invention

[0004] The purpose of this invention is to propose a satellite map-based UAV hierarchical positioning method without cumulative error, which achieves tight coupling positioning with visual inertial odometry, enabling UAVs to achieve robust real-time positioning in denied environments and realize stable and efficient real-time UAV position estimation.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A satellite map-based UAV hierarchical positioning method without cumulative error includes:

[0007] Offline feature sparsification processing for satellite images, map data cropping, and feature compression;

[0008] Aerial images are matched and located with satellite imagery reference images to construct a spatiotemporal consistency transfer framework of "deep learning satellite image matching + optical flow homography matrix transfer";

[0009] Aerial images are used to transfer homography matrix via optical flow tracking. The homography matrix obtained by matching satellite images is used as the correction value. The geographic latitude and longitude coordinates corresponding to the feature points of each frame of aerial images are calculated in real time to provide a data source for visual inertial odometry.

[0010] By combining visual inertial odometry, map coordinates are tightly coupled with SLAM backend measurement updates. The latitude and longitude coordinates of map points are used to provide absolute position information for visual positioning, thereby optimizing and correcting the visual inertial odometry positioning results.

[0011] Furthermore, the offline feature sparsity processing of the satellite imagery includes:

[0012] Large-scale offline clear satellite maps, cropped step by step;

[0013] A 40% overlap rate corresponds to high-resolution cropping;

[0014] Low texture entropy values ​​are culled.

[0015] Feature points and descriptor pre-extracted data.

[0016] Furthermore, the large-scale offline clear satellite map is cropped in stages, including: using publicly available satellite map data as the reference map for scene matching, and cropping the large scene map into a small area map of 500m*500m.

[0017] Furthermore, the 40% overlap rate corresponds to fine-tuning of resolution, including: using the GDAL library to crop the satellite map into a map image with a resolution of 1920*1080, with a 40% overlap rate as the benchmark during cropping, and the upper left pixel containing latitude and longitude coordinates and latitude and longitude scale.

[0018] Furthermore, the low texture entropy value removal includes: determining an information entropy threshold as a filtering threshold by using the data histogram and the maximum inter-class variance, and removing images with entropy values ​​less than the threshold.

[0019] Furthermore, the pre-extraction of feature points and descriptors includes: offline extraction of image feature points and descriptors using the Superpoint deep learning network and conversion into NumPy data format; naming corresponding folders with information such as map scene name, map ID, and scale; and pre-training global feature information of aerial and satellite images using the MixVPR network, pre-collecting a large number of aerial images and corresponding satellite images for global model training to learn the global semantic information of captured images.

[0020] Furthermore, the spatiotemporal consistent transfer framework for "deep learning satellite image matching + optical flow homography matrix transfer" includes:

[0021] Global satellite image search;

[0022] Image similarity filtering selects the optimal matching image based on the cosine similarity of feature point vectors.

[0023] Feature point extraction and feature point pair matching: extract superpoint feature points from aerial images and the best matching image, and use superglue to perform image matching to obtain feature point pairs;

[0024] Feature point pairs are used to calculate the homography matrix between satellite images and aerial photographs;

[0025] Aerial image optical flow tracking homography matrix transfer;

[0026] Satellite image matching homography matrix correction;

[0027] Real-time calculation of the geographic latitude and longitude coordinates corresponding to feature points in each frame of aerial photography.

[0028] Furthermore, the global satellite image search includes: matching pre-trained global features of satellite images with global features of real-time aerial images; acquiring aerial images and removing weak textures; performing MixVPR image filtering and matching to obtain a candidate image set, and then using a cosine similarity search method to select the optimal matching image.

[0029] Furthermore, the similarity-based image selection, feature point extraction, feature point pair matching, and feature point pair calculation of the homography matrix between satellite images and aerial images include: for the optimal matching image, using the deep learning network Superpoint to extract feature points and descriptors from the aerial image, using the image matching network Superglue to match the aerial image with the satellite image, obtaining a set of feature point pairs between the aerial image and the matched satellite image, and calculating the homography transformation matrix between the two.

[0030] Furthermore, the aerial image optical flow tracking homography matrix transfer, satellite image matching homography matrix correction, and real-time back-calculation of the geographic latitude and longitude coordinates corresponding to the feature points of each frame of the aerial image include: using the aerial image frame matched with the satellite image as the key frame, performing optical flow feature tracking in the aerial image to calculate the matching relationship of feature points between the previous and next frames in real time, and obtaining feature point pairs; obtaining the homography transfer matrix between each frame and the key frame by calculating the inter-frame homography transformation matrix, and then calculating the geographic 3D feature coordinates corresponding to the pixel feature points of the aerial image in that frame.

[0031] Furthermore, the combination of visual inertial odometry tightly couples map coordinates with SLAM backend measurement updates, using map point latitude and longitude coordinates to provide absolute position information for visual positioning, and optimizes and corrects the visual inertial odometry positioning results, including:

[0032] By combining visual inertial odometry, map coordinates are tightly coupled with SLAM backend measurement updates. The latitude and longitude coordinates of map points are used to provide absolute position information for visual positioning, thereby optimizing and correcting the visual inertial odometry positioning results.

[0033] If the image acquisition device is a gimbal, the real-time position of the UAV can be deduced by inversely solving the image's optical center pixel coordinates and homography matrix. The real-time position can then be integrated into the visual inertial odometry positioning, adding a positioning correction step and improving positioning accuracy and robustness.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] This invention provides a satellite map-based UAV hierarchical positioning method without cumulative error. It mainly uses visual inertial odometry and combines scene matching characteristics. It updates the homography matrix in real time by using optical flow tracking of aerial images, and then converts image feature points into geographic latitude and longitude coordinates and back-solves the UAV position back projection. The latitude and longitude information associated with the image feature points and the UAV back projection position are used for measurement and updating, realizing tight coupling positioning with visual inertial odometry. This enables the UAV to be robustly positioned in real time in denied environments. The UAV position is obtained using geographic latitude and longitude coordinates to achieve denied flight. Attached Figure Description

[0036] Figure 1 This is an overall flowchart of the satellite map-based UAV hierarchical positioning method without cumulative error according to an embodiment of the present invention;

[0037] Figure 2 System module functional block diagram;

[0038] Figure 3 Experimental positioning results diagram. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0041] The overall flowchart of a satellite map-based UAV hierarchical positioning method without cumulative error in this embodiment is as follows: Figure 1 As shown, the functional block diagram of each system module is as follows: Figure 2 As shown.

[0042] The specific steps are as follows:

[0043] Step 1: Offline feature sparsity processing of satellite images. This step reduces storage space and improves data processing efficiency by cropping, compressing and extracting features from map data.

[0044] Step 1.1: This invention uses publicly available satellite map data, such as Baidu satellite map, Bing satellite map, Google satellite map, or orthophotos captured and stitched by drones as the reference map for scene matching. For large-scale offline clear satellite map images, it is necessary to crop them in steps. Map cropping software is used to crop large scene maps into small-area maps of 500m*500m to reduce the amount of subsequent cropping data calculation.

[0045] Step 1.2: Use the GDAL library to crop the satellite map to a 1920*1080 resolution image. A 40% overlap rate is used during cropping to ensure a stable transition of the search baseline image during subsequent scene matching. Geographic coordinate information can be obtained from the top-left corner pixel information and the image's latitude and longitude scale.

[0046] Step 1.3: Entropy Value Removal of Low-Texture Regions. This step calculates the image information entropy to filter and remove low-texture regions such as deserts and lakes, ensuring the validity of satellite image matching and positioning results. The specific calculation method is as follows:

[0047]

[0048] Where L is the upper limit of grayscale value (255), p(i) represents the probability of grayscale value i in the image, and H is the information entropy value. The value of image information entropy directly reflects its information distribution characteristics: low entropy values ​​correspond to simple image structures with highly concentrated information, while high entropy values ​​characterize complex texture features with uniformly distributed information. The change in entropy value is positively correlated with the richness of image details. The information entropy threshold is determined by the data histogram and the maximum inter-class variance as the screening threshold, and images with entropy values ​​less than the threshold are removed.

[0049] Step 1.4 addresses the issue of large data volume in high-resolution satellite imagery by proposing a data compression method that uses global and local feature points and descriptors to represent satellite images, transforming the storage of images into the storage of feature points and descriptors. The specific method is as follows:

[0050] Step 1.4.1: Use the Superpoint deep learning network to extract image feature points and descriptors offline.

[0051] Step 1.4.2: Convert the feature points and descriptor data extracted by the deep learning network into NumPy data format and store them in the benchmark map folder with a fixed naming format.

[0052] Step 1.4.3: Name the corresponding folder with information such as map scene name, map ID, and scale. During subsequent matching and positioning, the corresponding map image will be searched using the relevant information as the matching reference map for matching and positioning.

[0053] Step 1.5: Use the MixVPR network to pre-train the global feature information of aerial images and satellite images. A large number of aerial images and corresponding satellite images are collected in advance for global model training to learn the global semantic information of captured images. Global feature vectors are extracted from the satellite images obtained in Step 1.4 and stored on the airborne board to ensure the real-time screening process during subsequent flights.

[0054] Step 2: Matching and locating the aerial image with the satellite baseline image. This is the core image matching part. It involves matching real-time aerial images with satellite images to construct a homography transformation matrix between the aerial and satellite images. The steps include: aerial image acquisition, global satellite image search, aerial image feature extraction, deep learning matching of aerial and satellite images, and homography transformation calculation. The specific method is as follows: Step 2.1: This step first acquires the aerial image and performs weak texture removal on the image, as follows.

[0055] Step 2.1.1: Acquire real-time aerial images. Use a drone equipped with a self-stabilized gimbal camera to fly at high altitude and take real-time aerial images of the ground scene from above. Publish the images to the onboard board via ROS messages.

[0056] Step 2.1.2, Low-texture culling. Read the real-time aerial image from the drone and calculate the information entropy of the aerial image using the information entropy method in Step 1.3. If the entropy value is less than the threshold, the information in that area is low, and the image is culled. Visual odometry is then used to transition this segment of the positioning process.

[0057] Step 2.2, Global Satellite Image Search. The airborne board receives image messages and learns global features of the aerial image through the MixVPR network. It then performs image filtering and matching with the pre-trained satellite images from Step 1.5 to obtain the satellite image corresponding to the current frame's aerial image. The specific steps are as follows:

[0058] Step 2.2.1, Aerial Image Feature Extraction. The global descriptor of the image is obtained using the MixVPR network.

[0059] Step 2.2.2, Image Filtering by Similarity. The global feature descriptor of the aerial image is matched against a pre-trained global feature database using an approximate nearest neighbor search algorithm. The goal is to find satellite images in the database that are most similar in features to the real-time aerial image. The matching search employs a cosine similarity search method, with the specific formula as follows:

[0060] Two vectors X = (x1, x2, ..., xn) d) and Y = (y1, y2, ..., y d The cosine distance is expressed as:

[0061]

[0062] Among them, X i Let X represent the value of the i-th dimension, and Y represent the value of the i-th dimension. i Let X represent the value of the i-th dimension of vector Y, COS(X,Y) be the cosine similarity between vectors X and Y, and dist(X,Y) be the cosine distance between vectors X and Y. The smaller the cosine distance, the more similar the two vectors are. This method can search for the k nearest neighbor images of the target image in the database as candidate images.

[0063] Step 2.2.3: Based on the final position and timestamp of the previous frame, the searched candidate images are eliminated by position. If the time interval is less than 5 seconds, the satellite image candidate images that are more than 500m away from the previous frame position are eliminated.

[0064] Step 2.3, Feature point extraction: Select the image with the highest similarity after screening in the candidate images as the matching image, and use the feature extraction deep learning network Superpoint to extract the feature points of the aerial image and match them with the descriptors.

[0065] Step 2.4, Feature point pair matching: Use the deep learning image matching network Superglue to perform feature matching between the two images to obtain a set of feature point pairs between the aerial image and the matched satellite image.

[0066] Step 2.5: Solve the homography matrix between the satellite image and the aerial image using feature point pairs. Use the matched set of feature point pairs to solve for the homography transformation matrix H between the aerial image and the satellite image. x The specific calculation method is as follows:

[0067]

[0068] Where (x) (1,i) ,y (1,i) Let (x) be the pixel coordinates of the i-th feature point in the aerial image. (2,i) ,y (2,i) Let be the pixel coordinates of the i-th feature point in the satellite image that matches it. By using multiple sets of feature point pairs (more than 4 sets), the homography transformation matrix between the two images can be calculated using the least squares method.

[0069] Using the homography transformation matrix H x The feature points of the aerial image are projected onto the satellite image, and the average pixel error between the projected feature points and the matched feature points is calculated to verify the matching result. The specific formula is as follows:

[0070]

[0071] Among them, (x' (1,i) ,y' (1,i) ) is the feature point (x) of the aerial image (1,i) ,y (1,i) The projected coordinates of the pixel are given. The average pixel error is calculated as follows:

[0072]

[0073] Where MPE is the pixel average error, N is the number of matched feature points, (x (2,i) ,y (2,i) ) represents the coordinates of the satellite image and the projected coordinates (x'). (1,i) ,y' (1,i) The pixel coordinates of the i-th feature point matched.

[0074] If the reprojection error is greater than the threshold, the matching is considered incorrect, and the image matching calculation is re-selected. If there is no successfully matched image in the candidate images, the image matching for that frame is skipped, and local positioning is maintained by visual inertial odometry.

[0075] Preferably, the satellite image retrieval scheme may use other deep learning image retrieval methods to replace the satellite image retrieval of this scheme, and other similarity elimination methods to replace the related schemes for satellite image retrieval mismatch elimination, such as Euclidean distance similarity, depending on the image matching effect.

[0076] Preferably, for the image matching part, a deep learning network for image matching such as Lightglue can be used instead of Superglue for satellite image matching.

[0077] Step 3: Construct a spatiotemporal consistent transfer framework of "deep learning satellite image matching + optical flow homography matrix transfer". The aerial image is transferred for homography matrix by optical flow tracking. The homography matrix obtained by satellite image matching is used as the correction value. The geographic latitude and longitude coordinates corresponding to the feature points of each frame of aerial image are calculated in real time to provide a data source for visual inertial odometry.

[0078] Step 3.1: Real-time calculation of the matching relationship between feature points in previous and next frames is performed by optical flow feature tracking to obtain feature point pairs.

[0079] Step 3.2: Track the matching relationship through inter-frame optical flow features and calculate the inter-frame homography transformation matrix in real time. The calculation method is the same as in step 2.5.

[0080] Step 3.3, calculate the homography transfer matrix. First, define the homography matrix of the most recent satellite image matching as H. x1 The current frame aerial image and H x1The inter-frame homography matrices between the corresponding aerial images are H1, H2…H n Then the current frame aerial image and H x1 The homography matrix relationship of the corresponding satellite image map can be calculated by the following formula:

[0081] H = H x1 ·H1·H2·…·H n

[0082] Where H is the calculated aerial image of that frame and H x1 The homography transformation matrix of the corresponding satellite image map.

[0083] Step 3.4: Using the obtained H homography matrix, the geographic 3D feature coordinates corresponding to the pixel feature points of the aerial image frame can be calculated, providing a data foundation for subsequent 3D feature point visual odometry measurement updates. The specific calculation method is as follows:

[0084] Step 3.4.1: Calculate the pixel feature points corresponding to the satellite image map using the optical flow feature points of the aerial image frame and the homography matrix H. The calculation method is the same as in step 2.5.

[0085] Step 3.4.3: Using satellite image map information, calculate the geographic 3D coordinates corresponding to the feature pixels of the satellite image. The formula for calculating latitude and longitude is as follows:

[0086] x = x (0,0) +a i ·d x

[0087] y = y (0,0) +b i ·d y

[0088] Among them, (x (0,0) ,y (0,0) (d) represents the latitude and longitude coordinates of the upper left corner of the satellite image map. x ,d y (a) represents the pixel width of the satellite image map, i.e., the actual ground distance corresponding to a single pixel, in latitude and longitude units. This information can be obtained by searching for the map information using the satellite image map ID. Additionally, (a) i ,b i ) represents the pixel coordinates of the i-th feature point.

[0089] Altitude information can be obtained by searching the DEM elevation information database using latitude and longitude, which will give you the geographic coordinates of the 3D point corresponding to the aerial image features, including longitude, latitude, and altitude information.

[0090] Step 3.5: When a new satellite image match appears, clear the inter-frame homography matrix and continue to perform optical flow homography matrix transfer with that frame.

[0091] Step 4: Combining visual inertial odometry, the map coordinates are tightly coupled with the SLAM backend measurement update. The map points' latitude and longitude coordinates are used to provide absolute position information for visual positioning, optimizing and correcting the visual inertial odometry positioning results. If the image acquisition device is a gimbal, the real-time position of the UAV can be deduced from the image's optical center pixel coordinates and homography matrix. This real-time position is then integrated into the visual inertial odometry positioning, adding a positioning correction step and improving positioning accuracy and robustness.

[0092] Step 4.1: Acquire aerial images and synchronized inertial measurement instrument data in real time to construct a visual inertial odometry system.

[0093] The state variables include IMU state variables, sliding window keyframe state variables, and SLAM feature point state variables, as detailed below:

[0094] The IMU state variables are:

[0095]

[0096] in, For rotation from the global coordinate system to the IMU coordinate system, The position of the IMU in the global coordinate system. Let b be the velocity of the IMU in the global coordinate system. a ,b g These are the acceleration bias and the gyroscope bias, respectively.

[0097] The feature point state variables are:

[0098]

[0099] in Let be the global coordinates of the i-th feature point.

[0100] The inverse depth state variable of MSCKF feature points is:

[0101] [ρ1,ρ2,…,ρ n ]

[0102] Where ρ i is the inverse depth of the i-th MSCKF feature point.

[0103] Step 4.2, the overall visual localization process is divided into: front-end feature extraction, tightly coupled model construction, EKF update and joint optimization, and sliding window keyframe maintenance. The specific steps are as follows:

[0104] Step 4.2.1: Perform optical flow feature tracking through the front-end vision processing module to extract stable features.

[0105] Step 4.2.2: The backend uses an MSCKF filter based on a sliding window to jointly construct a tightly coupled optimization model with the IMU state and the feature point position.

[0106] Step 4.2.3: Perform EKF update equation to jointly optimize visual reprojection residuals and IMU pre-integration residuals, where SLAM feature points use global XYZ parameterization and MSCKF feature points use inverse depth parameterization.

[0107] Step 4.2.4: Maintain a sliding window of 10-15 frames by marginalizing old keyframes.

[0108] Step 4.3: If there are feature points in the current frame that match satellite images, add the 3D geographic coordinates calculated in step 3.4 to the SLAM feature points, set the weight of this type of point set to high, and update the SLAM feature points to add global positioning information measurement correction to the positioning algorithm.

[0109] Step 4.4: If the image data is acquired vertically downwards by the gimbal device, the latitude and longitude coordinates corresponding to the optical center pixel in the image can represent the real-time position of the UAV and be used to integrate the visual inertial odometry (VIO) correction positioning results. The UAV position back-projection 3D feature point corresponding to the optical center pixel obtained in Step 3.4 can be used as the UAV position measurement value for positioning correction, improving the UAV's positioning stability and robustness.

[0110] Preferably, the visual positioning scheme can use other positioning algorithms such as VINS, ORBSLAM3, etc. to replace the visual inertial odometry positioning scheme of this scheme.

[0111] Preferably, the optical flow homography transformation transfer method can use forward feature point transfer instead of homography transformation transfer, and transfer the matched geographic points to subsequent keyframes in the form of optical flow, in order to replace the homography matrix to solve for latitude and longitude points.

Claims

1. A satellite map-based UAV hierarchical positioning method without cumulative error, characterized in that, include: Offline feature sparsification processing for satellite images, map data cropping, and feature compression; Aerial images are matched and located with satellite imagery reference images to construct a spatiotemporal consistency transfer framework of "deep learning satellite image matching + optical flow homography matrix transfer"; Aerial images are used to transfer homography matrix via optical flow tracking. The homography matrix obtained by matching satellite images is used as the correction value. The geographic latitude and longitude coordinates corresponding to the feature points of each frame of aerial images are calculated in real time to provide a data source for visual inertial odometry. By combining visual inertial odometry, map coordinates are tightly coupled with SLAM backend measurement updates. The latitude and longitude coordinates of map points are used to provide absolute position information for visual positioning, thereby optimizing and correcting the visual inertial odometry positioning results.

2. The satellite map-based UAV hierarchical positioning method without cumulative error as described in claim 1, characterized in that, The offline feature sparsity processing of the satellite imagery includes: Large-scale offline clear satellite maps, cropped step by step; A 40% overlap rate corresponds to high-resolution cropping; Low texture entropy values ​​are culled. Feature points and descriptor pre-extracted data.

3. The satellite map-based UAV hierarchical positioning method without cumulative error as described in claim 2, characterized in that, The large-scale offline clear satellite map is cropped in stages, including: using publicly available satellite map data as the reference map for scene matching, and cropping the large scene map into a small area map of 500m*500m.

4. The satellite map-based UAV hierarchical positioning method without cumulative error as described in claim 2, characterized in that, The 40% overlap rate corresponds to fine-tuning of resolution, including: using the GDAL library to crop the satellite map into a map image with a resolution of 1920*1080, with a 40% overlap rate as the benchmark during cropping, and the upper left pixel containing latitude and longitude coordinates and latitude and longitude scale.

5. The satellite map-based UAV hierarchical positioning method without cumulative error as described in claim 2, characterized in that, The low texture entropy removal includes: determining an information entropy threshold using the data histogram and the maximum inter-class variance as a filtering threshold, and removing images with entropy values ​​less than the threshold.

6. The satellite map-based UAV hierarchical positioning method without cumulative error as described in claim 2, characterized in that, The feature point and descriptor pre-extraction data includes: offline extraction of image feature points and descriptors using the Superpoint deep learning network and conversion into NumPy data format; naming corresponding folders with information such as map scene name, map ID, and scale; and pre-training global feature information of aerial and satellite images using the MixVPR network, pre-collecting a large number of aerial images and corresponding satellite images for global model training to learn the global semantic information of captured images.

7. The satellite map-based UAV hierarchical positioning method without cumulative error as described in claim 1, characterized in that, The spatiotemporal consistent transfer framework for "deep learning satellite image matching + optical flow homography matrix transfer" includes: Global satellite image search; Image similarity filtering selects the optimal matching image based on the cosine similarity of feature point vectors. Feature point extraction and feature point pair matching: extract superpoint feature points from aerial images and the best matching image, and use superglue to perform image matching to obtain feature point pairs; Feature point pairs are used to calculate the homography matrix between satellite images and aerial photographs; Aerial image optical flow tracking homography matrix transfer; Satellite image matching homography matrix correction; Real-time calculation of the geographic latitude and longitude coordinates corresponding to feature points in each frame of aerial photography.

8. The satellite map-based UAV hierarchical positioning method without cumulative error as described in claim 7, characterized in that, The global satellite image search includes: matching global features of pre-trained satellite images with global features of real-time aerial images; acquiring aerial images and removing weak textures; performing MixVPR image filtering and matching to obtain a candidate image set, and then using the cosine similarity search method to select the optimal matching image.

9. The satellite map-based UAV hierarchical positioning method without cumulative error as described in claim 7, characterized in that, The similarity-based image selection, feature point extraction, feature point pair matching, and feature point pair calculation of the homography matrix between satellite images and aerial images include: for the optimal matching image, using the deep learning network Superpoint to extract feature points and descriptors from the aerial image, using the image matching network Superglue to match the aerial image with the satellite image, obtaining a set of feature point pairs between the aerial image and the matched satellite image, and calculating the homography transformation matrix between the two.

10. The satellite map-based UAV hierarchical positioning method without cumulative error as described in claim 7, characterized in that, The process of transferring the homography matrix for aerial image optical flow tracking, correcting the homography matrix for satellite image matching, and calculating the geographic latitude and longitude coordinates corresponding to the feature points of each frame of aerial imagery in real time includes: using the aerial image frame that matches the satellite image as the key frame, performing optical flow feature tracking in the aerial imagery to calculate the matching relationship between the feature points of the preceding and following frames in real time, and obtaining feature point pairs; obtaining the homography transfer matrix between each frame and the key frame by calculating the inter-frame homography transformation matrix, and then calculating the geographic 3D feature coordinates corresponding to the pixel feature points of that frame of aerial imagery.

11. The satellite map-based UAV hierarchical positioning method without cumulative error as described in claim 1, characterized in that, The method of combining visual inertial odometry tightly couples map coordinates with SLAM backend measurement updates, using map point latitude and longitude coordinates to provide absolute position information for visual positioning, and optimizing and correcting visual inertial odometry positioning results, including: By combining visual inertial odometry, map coordinates are tightly coupled with SLAM backend measurement updates. The latitude and longitude coordinates of map points are used to provide absolute position information for visual positioning, thereby optimizing and correcting the visual inertial odometry positioning results. If the image acquisition device is a gimbal, the real-time position of the UAV can be deduced by inversely solving the image's optical center pixel coordinates and homography matrix. The real-time position can then be integrated into the visual inertial odometry positioning, adding a positioning correction step and improving positioning accuracy and robustness.

Images