Unmanned aerial vehicle visual geographic positioning system and method in GNSS denial environment

Abstract

The invention relates to an unmanned aerial vehicle visual geographic positioning system and method in a GNSS denial environment, which significantly improves the positioning performance in GNSS signal limited scenes (such as urban canyons, mountainous areas, forests and the like), provides reliable guarantee for unmanned aerial vehicle autonomous navigation, and realizes high-precision, high-efficiency and high-robustness visual geographic positioning in a complex environment. The system comprises a self-adaptive global matching module, a geographical perception mixed training module, an elevation constraint separation optimization module and a gridding random sampling arbitration module. All the modules are interconnected with a control bus through a data bus; the output end of the self-adaptive global matching module is connected with the input end of the geographical perception mixed training module; and the elevation constraint separation optimization module and the gridding random sampling arbitration module form a parallel processing pipeline.

Description

Technical Field

[0001] This invention relates to the field of UAV navigation and multi-mode composite trajectory testing technology, and in particular to a UAV visual geolocation system in a GNSS denied environment, and a UAV visual geolocation method in a GNSS denied environment. Background Technology

[0002] In environments denied by Global Navigation Satellite System (GNSS), UAV positioning primarily employs traditional technologies such as Inertial Navigation (INS), Simultaneous Localization and Mapping (SLAM), and Terrain Reference Navigation (TRN). INS achieves fully autonomous positioning based on inertial measurement units, but errors accumulate over time. SLAM performs well in structured scenes through multi-sensor fusion, but its positioning accuracy significantly decreases in high-altitude, large-scale outdoor environments due to difficulties in matching long-distance viewpoint features. TRN utilizes prior terrain data matching to provide absolute position, but it is prone to failure or drift errors in flat or dynamic terrain.

[0003] Visual geolocation methods improve positioning capabilities in large-scale high-altitude scenes by fusing prior information from aerial imagery and satellite maps. However, existing methods still have significant drawbacks:

[0004] 1. The global matching phase uses an exhaustive strategy, which is inefficient and has a high probability of mismatch when dealing with large-scale satellite image databases.

[0005] 2. The model training relies on data from non-task regions, and its generalization ability across regional scenarios is insufficient, resulting in a decrease in matching accuracy.

[0006] 3. The local matching mid-beam adjustment (BA) optimization has constraint conflicts, and the fixed elevation prior causes an imbalance between horizontal and vertical accuracy.

[0007] 4. System errors, spatiotemporal parallax, and mismatched points in satellite imagery can easily lead to divergence in the optimization process, resulting in poor positioning stability. Summary of the Invention

[0008] To overcome the shortcomings of existing technologies, the technical problem to be solved by this invention is to provide a visual geolocation system for UAVs in GNSS denied environments, which significantly improves positioning performance in GNSS signal-limited scenarios (such as urban canyons, mountainous areas, forests, etc.), provides reliable assurance for UAV autonomous navigation, and achieves high-precision, high-efficiency, and high-robust visual geolocation in complex environments.

[0009] The technical solution of this invention is: a visual geolocation system for unmanned aerial vehicles (UAVs) in GNSS-denied environments, comprising:

[0010] The adaptive global matching module dynamically optimizes the search strategy based on the UAV kinematic model. By adaptively adjusting the search box parameters, it prioritizes matching satellite images corresponding to the current flight area of ​​the UAV.

[0011] The geographic perception hybrid training module integrates a general dataset of aerial photography and satellite imagery with a task area dataset. The task area dataset generates pseudo-labels through gridded segmentation of satellite images and trains the visual location recognition model in a proportional manner, enhancing the model's adaptability to cross-regional geographic features.

[0012] The elevation constraint separation optimization module adopts a parallel BA optimization architecture. The planar constraint branch fixes the elevation of the three-dimensional point to the reference plane to improve the horizontal position accuracy, while the DEM constraint branch introduces digital elevation model data to improve the vertical direction estimation. Finally, the optimal position output is formed through selective fusion.

[0013] The gridded random sampling arbitration module uniformly divides satellite image feature points into an M×N grid. Within each grid, a subset of feature points is randomly sampled for multiple basis analysis (BA) optimizations. The results are verified using an Euclidean distance threshold. All modules are interconnected via a data bus and a control bus: the output of the adaptive global matching module is connected to the input of the geographic perception hybrid training module; the elevation constraint separation optimization module and the gridded random sampling arbitration module form a parallel processing pipeline. Consistency is ensured.

[0014] This invention significantly improves positioning performance in GNSS signal-constrained scenarios (such as urban canyons, mountainous areas, and forests) through multi-module collaborative innovation, providing reliable assurance for UAV autonomous navigation and achieving high-precision, high-efficiency, and highly robust visual geolocation in complex environments.

[0015] A method for operating a UAV visual geolocation system in a GNSS denied environment is also provided, which includes the following steps:

[0016] (1) System initialization stage: The host computer requests test data packets. The data module generates data packets according to the motion constraint formula and sends them to the host computer. If the test is correct, proceed to step (2). If the data cannot be matched, check the system connection status. After troubleshooting, repeat step (1).

[0017] (2) Parameter configuration stage: Calculate the optimal search box parameters based on the UAV flight trajectory and satellite image database range, ensure that the area to be matched is completely within the field of view, and start the grayscale acquisition mode;

[0018] (3) Data acquisition stage: Input the aerial image sequence into the system and continuously acquire grayscale information. When a sufficient number of feature points are detected at the same time, it indicates that the matching conditions have been met. The image enhancement unit reads the enhancement parameters and starts the distance acquisition mode.

[0019] (4) Data processing stage: In the cache unit, the image information is transmitted to the image enhancement unit for further processing, and at the same time, it is transmitted to the cache module for backup. The image enhancement unit reads the current frame information from the cache module according to the configuration, reads the historical frame information from the cache module and processes it. The processed result is transmitted to the cache module and the position calculation unit at the same time.

[0020] (5) Position calculation stage: The position calculation unit calculates the pose information and transmits it to the cache module through the data cache unit. The cache module then transmits the position information to the host computer.

[0021] (5) Result optimization stage: The host computer performs calculations based on the obtained position information to obtain three-dimensional results.

[0022] The positioning results are then processed by noise reduction and filtering. Attached Figure Description

[0023] Figure 1 This is a flowchart of the UAV visual geolocation method under GNSS denied environment according to the present invention.

[0024] Figure 2 This is the global matching algorithm of the present invention.

[0025] Figure 3 This is a process for separating and optimizing elevation constraints based on BA optimization. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0027] To make the description of this disclosure more detailed and complete, illustrative descriptions of embodiments and specific examples of the present invention are provided below; however, these are not the only forms of implementing or utilizing the specific examples of the present invention. The embodiments cover features of multiple specific examples and methods and steps for constructing and operating these specific examples, and their order. However, other specific examples may also be used to achieve the same or equivalent functions and order of steps.

[0028] like Figure 1 As shown, this UAV visual geolocation system in a GNSS-denied environment includes:

[0029] The adaptive global matching module dynamically optimizes the search strategy based on the UAV kinematic model. By adaptively adjusting the search box parameters, it prioritizes matching satellite images corresponding to the current flight area of ​​the UAV.

[0030] The geographic perception hybrid training module integrates a general dataset of aerial photography and satellite imagery with a task area dataset. The task area dataset generates pseudo-labels through gridded segmentation of satellite images and trains the visual location recognition model in a proportional manner, enhancing the model's adaptability to cross-regional geographic features.

[0031] The elevation constraint separation optimization module adopts a parallel BA optimization architecture. The planar constraint branch fixes the elevation of the three-dimensional point to the reference plane to improve the horizontal position accuracy, while the DEM constraint branch introduces digital elevation model data to improve the vertical direction estimation. Finally, the optimal position output is formed through selective fusion.

[0032] The gridded random sampling arbitration module divides the satellite image feature points into an M×N grid, randomly selects a subset of feature points in each grid for multiple BA optimizations, and verifies the consistency of the results through the Euclidean distance threshold.

[0033] Each module is interconnected with the control bus via a data bus: the output of the adaptive global matching module is connected to the input of the geographic perception hybrid training module, and the elevation constraint separation optimization module and the gridded random sampling arbitration module form a parallel processing pipeline.

[0034] This invention significantly improves positioning performance in GNSS signal-constrained scenarios (such as urban canyons, mountainous areas, and forests) through multi-module collaborative innovation, providing reliable assurance for UAV autonomous navigation and achieving high-precision, high-efficiency, and highly robust visual geolocation in complex environments.

[0035] Preferably, the search box parameters of the adaptive global matching module are dynamically adjusted according to the UAV's motion state, wherein the search box width W is set to 3 to 10, and the increment step dW is set to 2 to 5, in order to adapt to the needs of different flight scenarios.

[0036] Preferably, the dataset construction of the geographic perception hybrid training module adopts a multi-source data fusion strategy, and the mixing ratio of the general dataset and the task area dataset is determined by the grid search method, with the optimal ratio being 4:1.

[0037] Preferably, in the parallel processing architecture of the elevation constraint separation optimization module, the plane constraint optimization unit uses the least squares method to solve the problem, and the DEM constraint optimization unit calculates the phase value based on the phase measurement principle by combining a lookup table and an iterator.

[0038] Preferably, the grid division of the gridded random sampling arbitration module adopts a uniform partitioning strategy, and the number of grids M×N is adaptively determined according to the image resolution, wherein M=8 and N=6 is the optimal configuration.

[0039] Preferably, the adaptive global matching module includes: a motion constraint analysis unit, a search box dynamic adjustment unit, and a matching execution unit; the geographic perception hybrid training module includes: a dataset construction unit, a model training unit, and a feature extraction and fusion unit; the elevation constraint separation optimization module includes: a plane constraint optimization unit, a DEM constraint optimization unit, and a multi-source data fusion unit; the gridded random sampling arbitration module includes: a gridded partitioning unit, a random sampling unit, and a multi-round arbitration decision unit.

[0040] like Figure 2 As shown, based on the time series constraints of the UAV flight trajectory ||P k -P k-1 ||≤(T k -T k-1 )·V m The system dynamically adjusts the search box parameters; it starts a grayscale acquisition mode to match aerial images with a satellite image database, switching to distance acquisition mode when feature points are detected; during feature point matching, image information is transmitted to the image enhancement unit for processing and simultaneously to the cache module for backup; the image enhancement unit reads the current frame information from the cache module according to its configuration, reads historical frame information from the cache module and processes it, transmitting the processed results to both the cache module and the position calculation unit; the position calculation unit calculates pose information and transmits it to the cache module through the data cache unit, which then transmits the position information to the host computer; the host computer calculates the 3D positioning result based on the obtained position information, performs denoising and filtering on the positioning result, and calculates the spatial characteristics of horizontal position accuracy, elevation accuracy, and 3D positioning error, as well as system performance indicators such as matching accuracy, positioning stability, and algorithm robustness.

[0041] Preferably, the position calculation unit employs a multi-sensor fusion algorithm, combining IMU data and visual information, and uses Kalman filtering to achieve pose optimization.

[0042] A method for operating a UAV visual geolocation system in a GNSS denied environment is also provided, which includes the following steps:

[0043] (1) System initialization phase: The host computer requests test data packets. The data module generates data packets according to the motion constraint formula and sends them to the host computer. If the test is correct, proceed to step (2). If the data cannot be matched, check the system connection status. After troubleshooting, repeat step (1).

[0044] (2) Parameter configuration stage: Calculate the optimal search box parameters based on the UAV flight trajectory and satellite image database range, ensure that the area to be matched is completely within the field of view, and start the grayscale acquisition mode;

[0045] (3) Data acquisition stage: Input the aerial image sequence into the system and continuously acquire grayscale information. When a sufficient number of feature points are detected at the same time, it indicates that the matching conditions have been met. The image enhancement unit reads the enhancement parameters and starts the distance acquisition mode.

[0046] (4) Data processing stage: In the cache unit, the image information is transmitted to the image enhancement unit for further processing, and at the same time, it is transmitted to the cache module for backup. The image enhancement unit reads the current frame information from the cache module according to the configuration, reads the historical frame information from the cache module and processes it. The processed result is transmitted to the cache module and the position calculation unit at the same time.

[0047] (5) Position calculation stage: The position calculation unit calculates the pose information and transmits it to the cache module through the data cache unit. The cache module then transmits the position information to the host computer.

[0048] (6) Result optimization stage: The host computer calculates based on the obtained position information to obtain the three-dimensional positioning result, and performs noise reduction and filtering on the positioning result;

[0049] Preferably, the image enhancement process in step (4) includes the following sub-steps: first, histogram equalization is performed, then Gaussian filtering is used for noise reduction, and finally edge enhancement is performed to improve the accuracy of feature point detection.

[0050] Preferably, the pose calculation in step (5) employs the PnP algorithm based on feature point matching, combined with the RANSAC method to remove mismatched points, and uses a BA-based elevation constraint separation optimization process (such as...). Figure 3 As shown); the denoising process in step (6) adopts the wavelet threshold denoising algorithm, and the filtering process adopts the extended Kalman filter.

[0051] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A UAV visual geolocation system in a GNSS-denied environment, characterized in that: It comprises: An adaptive global matching module, which dynamically optimizes the search strategy based on the UAV kinematic model, and preferentially matches the satellite image corresponding to the current flight area of the UAV through the adaptive adjustment of the search frame parameters; A geographic perception hybrid training module, which fuses the general data set and the task area data set of aerial and satellite images, generates pseudo-labels through satellite image grid segmentation, and proportionally mixes and trains the visual position recognition model to enhance the adaptability of the model to cross-regional geographic features; An elevation constraint separation optimization module, which adopts a parallel BA optimization architecture, fixes the elevation of three-dimensional points to the datum plane in the plane constraint branch to improve the horizontal position accuracy, and introduces digital elevation model data in the DEM constraint branch to improve the vertical direction estimation, and finally forms the optimal position output through selective fusion; A grid random sampling arbitration module, which uniformly divides the satellite image feature points into MxN grids, randomly extracts a subset of feature points in each grid for multiple BA optimizations, and checks the consistency through the Euclidean distance threshold; Each module is interconnected through a data bus and a control bus: the output end of the adaptive global matching module is connected with the input end of the geographic perception hybrid training module, and the elevation constraint separation optimization module and the grid random sampling arbitration module form a parallel processing pipeline.

2. The UAV visual geolocating system in a GNSS denial environment according to claim 1, characterized in that: The search frame parameters of the adaptive global matching module are dynamically adjusted according to the motion state of the UAV, wherein the search frame width W is set to 3-10, and the increase step dW is set to 2-5 to adapt to the needs of different flight scenarios.

3. The UAV visual geolocating system in a GNSS denial environment according to claim 2, characterized in that: The data set construction of the geographic perception hybrid training module adopts a multi-source data fusion strategy, and the mixed proportion of the general data set and the task area data set is determined by the grid search method, and the optimal proportion is 4:

1.

4. The UAV visual geolocating system in a GNSS denial environment according to claim 3, characterized in that: In the parallel processing architecture of the elevation constraint separation optimization module, the plane constraint optimization unit solves by the least squares method, and the DEM constraint optimization unit calculates the phase value by combining the look-up table and the iterator based on the phase measurement principle.

5. The UAV visual geolocating system in a GNSS denial environment according to claim 4, characterized in that: The grid division of the grid random sampling arbitration module adopts a uniform partitioning strategy, and the grid number MxN is adaptively determined according to the image resolution, wherein M=8 and N=6 are the optimal configuration.

6. The UAV visual geolocating system in a GNSS denial environment according to claim 5, characterized in that: The adaptive global matching module comprises: a motion constraint analysis unit, a search frame dynamic adjustment unit, and a matching execution unit; the geographic perception hybrid training module comprises: a data set construction unit, a model training unit, and a feature extraction and fusion unit; the elevation constraint separation optimization module comprises: a plane constraint optimization unit, a DEM constraint optimization unit, and a multi-source data fusion unit; and the grid random sampling arbitration module comprises: a grid partitioning unit, a random sampling unit, and a multi-round arbitration decision unit. According to the time sequence constraint of the flight trajectory of the unmanned aerial vehicle k -P k-1 ‖≤(T k -T k-1 )·V m , dynamically adjusting the search box parameters; the system starts the gray scale acquisition mode, matches the aerial photograph with the satellite map database, and converts to the distance acquisition mode when the feature points are detected; in the feature point matching process, the image information is transmitted to the image enhancement unit for processing, and at the same time is transmitted to the cache module for backup; the image enhancement unit reads the current frame information from the cache module according to the configuration, reads the historical frame information from the cache module and processes it, and the processed result is transmitted to the cache module and the position solving unit at the same time; the position solving unit solves the position information, which is transmitted to the cache module through the data cache unit, and the cache module transmits the position information to the host computer; the host computer calculates according to the obtained position information, obtains the three-dimensional positioning result, and performs denoising and filtering processing on the positioning result, calculates the horizontal position accuracy, the height accuracy, the spatial characteristics of the three-dimensional positioning error, and the system performance indexes of the matching accuracy, the positioning stability and the algorithm robustness.

7. The UAV visual geolocating system in a GNSS denial environment according to claim 6, characterized in that: The position solving unit adopts a multi-sensor fusion algorithm, combines IMU data and visual information, and realizes pose optimization through Kalman filtering.

8. The method of claim 7, wherein the GNSS-denied environment is a GNSS- denied urban environment. It comprises the following steps: (1) System initialization stage: the host computer requests a test data packet, the data module generates a data packet according to the motion constraint formula and sends it to the host computer, if the detection is correct, step (2) is performed, if the data cannot be matched, the system connection state is checked, and after the fault is eliminated, step (1) is repeated; (2) Parameter configuration stage: according to the flight trajectory of the unmanned aerial vehicle and the range of the satellite map database, the optimal search frame parameters are calculated to ensure that the to-be-matched region is completely in the field of view, and the gray level acquisition mode is started; (3) Data acquisition stage: input the aerial image sequence into the system, and continuously acquire gray level information; when a sufficient number of feature points are detected at the same time, it indicates that the matching condition has been met, the image enhancement unit reads the enhancement parameters, and the distance acquisition mode is started; (4) Data processing stage: in the cache unit, the image information is transmitted to the image enhancement unit for further processing, and is also transmitted to the cache module for backup; the image enhancement unit reads the current frame information from the cache module according to the configuration, reads the historical frame information from the cache module and processes it, and the processed result is transmitted to the cache module and the position solving unit at the same time; (5) Position solving stage: the position solving unit solves the pose information, which is transmitted to the cache module through the data cache unit, and the cache module transmits the position information to the host computer; (6) Result optimization stage: the host computer calculates according to the obtained position information to obtain the three-dimensional positioning result, and performs denoising and filtering processing on the positioning result.

9. The method of claim 8, wherein: The image enhancement processing in step (4) includes the following sub-steps: first, histogram equalization is performed, then Gaussian filtering is used for denoising, and finally edge enhancement processing is performed to improve the accuracy of feature point detection.

10. The method of claim 9, wherein the GNSS-denied environment is a GNSS- denied urban environment. The pose solving in step (5) uses a PnP algorithm based on feature point matching, combines with a RANSAC method to remove false matching points, and uses a height constraint separation optimization process based on BA optimization; the denoising processing in step (6) uses a wavelet threshold denoising algorithm, and the filtering processing uses an extended Kalman filter.

Images