A high-precision geographic information dynamic surveying and mapping system fusing laser radar and unmanned aerial vehicle

Abstract

The application discloses a kind of high-precision geographic information dynamic surveying and mapping systems of fusing laser radar and unmanned aerial vehicle, including data acquisition module based on unmanned aerial vehicle platform and the multiple sensor load carried synchronous acquisition multidimensional geographic information data, acquisition pre-processing module is used to carry out space-time synchronization and preliminary purification to multi-source sensor data, real-time data processing module is used to carry out real-time solution and analysis to the data after purification, dynamic modeling analysis module is used to generate high-precision real scene model and carry out time series change analysis by fusing multi-source data, visualization application module is used for the three-dimensional display of achievement, interaction and application integration;The application solves the problems of low efficiency, environmental constraints and inability to update dynamically by multi-sensor collaborative sensing and edge real-time processing, realizes all-weather dynamic geographic information acquisition and processing with centimeter-level precision and minute-level response in complex terrain, and is especially suitable for geological disaster monitoring, smart city management and other scenarios.

Description

Technical Field

[0001] This invention relates to the field of geographic surveying and mapping technology, and in particular to a high-precision dynamic geographic information surveying and mapping system that integrates lidar and unmanned aerial vehicles (UAVs). Background Technology

[0002] Traditional geographic information mapping mainly relies on manual ground surveying or satellite remote sensing technology. Manual surveying is inefficient, costly, and difficult to cover complex terrains (such as mountains and forests). Although satellite remote sensing has a wide coverage, its resolution is limited (usually at the meter level) and is greatly affected by weather.

[0003] In recent years, mapping solutions using drones equipped with optical cameras or single-line lidar have become increasingly popular. However, drone optical cameras are easily affected by lighting conditions and cloud cover, making it impossible to penetrate vegetation to obtain three-dimensional information about the ground surface. Single-line lidar has low scanning efficiency, making it difficult to meet the needs of large-area, high-precision mapping. While multi-line lidar offers high point cloud density, it is expensive and poorly compatible with drones. Furthermore, existing technologies are mostly static mapping, unable to achieve real-time updates of geographic information, and use offline processing for massive amounts of data, which cannot meet emergency mapping needs. Therefore, this invention proposes a high-precision dynamic geographic information mapping system that integrates lidar and drones to solve the problems existing in the prior art. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to propose a high-precision dynamic geographic information mapping system that integrates lidar and UAVs. This system, through multi-sensor collaborative perception and real-time edge processing, solves the problems of low efficiency, environmental limitations, and inability to dynamically update traditional mapping methods. It achieves all-weather dynamic geographic information acquisition and processing with centimeter-level accuracy and minute-level response in complex terrain, and is particularly suitable for scenarios such as geological disaster monitoring and smart city management.

[0005] To achieve the objectives of this invention, the following technical solution is provided: a high-precision dynamic geographic information mapping system integrating lidar and UAV, comprising a data acquisition module, a data acquisition and preprocessing module, a real-time data processing module, a dynamic modeling and analysis module, and a visualization application module. The data acquisition module synchronously acquires multi-dimensional geographic information data based on a UAV platform and its onboard multi-sensor payload. The data acquisition and preprocessing module is used for spatiotemporal synchronization and preliminary purification of multi-source sensor data. The real-time data processing module is used for real-time calculation and analysis of the purified data. The dynamic modeling and analysis module is used for fusing multi-source data to generate a high-precision real-scene model and performing temporal change analysis. The visualization application module is used for 3D display, interaction, and application integration of the results.

[0006] Further improvements are made in the following aspects: The UAV platform is used to provide flight vehicle and high-precision positioning and attitude determination capabilities. The multi-sensor payload includes a multi-line lidar, a high-resolution optical camera, and auxiliary sensors. The multi-line lidar obtains ground point clouds by penetrating vegetation through multi-echo detection. The high-resolution optical camera uses a full-frame CMOS sensor to obtain images after ground feature classification and texture enhancement. The auxiliary sensors include a temperature and humidity sensor and a barometer for environmental parameter compensation.

[0007] Further improvements are made in that: the acquisition and preprocessing module includes a spatiotemporal synchronization unit and a preliminary filtering unit. The spatiotemporal synchronization unit achieves nanosecond-level time synchronization between the multi-sensor payload and the UAV platform through the PTP precise time protocol, and uses the UAV pose data to perform spatial registration on the multi-sensor payload to generate raw point clouds and images with geographic coordinates. The preliminary filtering unit uses an improved moving least squares method to filter out outlier noise points and dynamic interference points in the point cloud data.

[0008] Further improvements are made in the following aspects: The real-time data processing module includes an edge computing unit, an incremental SLAM unit, and a dynamic target tracking unit. The edge computing unit performs high-load parallel computing tasks based on a real-time point cloud segmentation model. The incremental SLAM unit constructs a local 3D map in real time based on inter-frame matching of LiDAR point clouds and visual feature point matching, combined with RTK-GPS constraining the global coordinate system. The dynamic target tracking unit analyzes the motion trajectory through point cloud clustering and optical flow methods, and outputs the speed and direction parameters of the dynamic target.

[0009] Further improvements are made in that the real-time point cloud segmentation model is a lightweight deep learning model based on the improved PointNet++ network architecture. Through multi-layer feature perception and hierarchical aggregation, it performs real-time semantic segmentation of ground, vegetation, buildings and dynamic targets in large-scale point cloud data.

[0010] Further improvements are made in that: the dynamic target tracking unit includes a point cloud clustering subunit and a trajectory tracking subunit. The point cloud clustering subunit uses the density-based DBSCAN spatial clustering algorithm to cluster the dynamic target point cloud segmented in real time to identify independent moving individuals. The trajectory tracking subunit is used to combine the point cloud clustering results between adjacent frames with the optical flow information of the optical image sequence, and uses an extended Kalman filter to estimate and track the position, velocity and direction of motion of the moving individuals.

[0011] Further improvements are made in that: the dynamic modeling and analysis module includes a data fusion unit, a change detection unit, and a dynamic scene modeling unit. The data fusion unit performs data fusion based on a multi-source data fusion engine to generate a three-dimensional real-world model with complete texture and high accuracy. The change detection unit is used to identify terrain changes. The dynamic scene modeling unit generates a dynamic geographic information layer based on a long short-term memory network model.

[0012] Further improvements are made in that the multi-source data fusion engine specifically uses Kalman filtering to fuse the geometric information of the lidar point cloud, the texture information of the camera image, and the pose and motion state of the UAV platform, thereby optimizing the point cloud accuracy and generating a textured 3D real-world model.

[0013] The change detection unit first uses the iterative nearest point algorithm to finely register the currently acquired 3D point cloud with the historical benchmark 3D model. Then, by calculating the point cloud density difference or distance field change within the voxel grid and combining morphological opening and closing operations to filter out noise, it extracts the difference region exceeding the preset threshold, thereby realizing the automatic identification and quantification of the change region.

[0014] The long short-term memory network model is used to learn the target historical trajectory sequence output by the real-time data processing module, predict its future short-term movement trajectory, and overlay the predicted trajectory with the real-scene 3D model to generate a four-dimensional spatiotemporal geographic information layer containing static scene and dynamic target information.

[0015] Further improvements are made in that: the visualization application module includes a 3D visualization unit and an emergency response control unit. The 3D visualization unit realizes the streaming loading and real-time rendering of massive 3D models based on the WebGL technology of the Web-based 3D visualization platform, and provides interactive interfaces for marking change areas, dynamic trajectory playback and data export. When the emergency response control unit receives a change alarm from the dynamic modeling and analysis module that exceeds the safety threshold, it automatically generates an encrypted monitoring flight mission and sends it to the UAV platform, while pushing the alarm information to the designated terminal.

[0016] The beneficial effects of this invention are as follows: By deeply integrating multi-line lidar, high-resolution multispectral camera and high-precision combined navigation system, this invention constructs an integrated air-space-ground collaborative perception network, which significantly improves the completeness and accuracy of data acquisition in complex terrain environments.

[0017] Relying on airborne edge computing and optimized real-time processing algorithms, it enables online processing and intelligent analysis of massive point cloud and image data. Based on multi-source data fusion and dynamic modeling technology, it generates centimeter-level precision real-scene 3D models and dynamic geographic information layers, realizing dynamic monitoring of geographic information. It also has all-weather operation and rapid response capabilities, providing efficient, reliable and real-time updated spatial information solutions for scenarios such as geological disaster early warning, smart city management, and emergency mapping. Attached Figure Description

[0018] Figure 1 This is a flowchart of the system architecture of the present invention. Detailed Implementation

[0019] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0020] Traditional geographic information mapping mainly relies on manual ground surveying or satellite remote sensing technology. Manual surveying is inefficient, costly, and difficult to cover complex terrains (such as mountains and forests). Although satellite remote sensing has a wide coverage, its resolution is limited (usually at the meter level) and is greatly affected by weather.

[0021] In recent years, mapping solutions using drones equipped with optical cameras or single-line lidar have become increasingly popular, but significant drawbacks still exist:

[0022] Limitations of a single sensor: Optical cameras are easily affected by light and fog, and cannot penetrate vegetation to obtain three-dimensional information of the ground surface; Single-line lidar has low scanning efficiency (thousands of points per second), making it difficult to meet the needs of large-area high-precision mapping; Although multi-line lidar has high point cloud density, it is expensive and has poor compatibility with UAVs (high power consumption and high payload requirements).

[0023] Insufficient dynamic mapping capabilities: Existing technologies are mostly static mapping (single flight data collection), which cannot achieve real-time updates of geographic information; for dynamic targets (such as moving vehicles, landslide deformation), there is a lack of real-time perception and modeling capabilities.

[0024] Low data processing efficiency: The stitching, denoising, and classification of massive point cloud data rely on offline processing, which takes several hours or even days, and cannot meet the needs of emergency mapping.

[0025] according to Figure 1 As shown in the figure, this embodiment provides a high-precision dynamic geographic information mapping system that integrates lidar and UAV, including a data acquisition module, a data acquisition and preprocessing module, a real-time data processing module, a dynamic modeling and analysis module, and a visualization application module.

[0026] The data acquisition module synchronously collects multi-dimensional geographic information data based on the UAV platform and its onboard multi-sensor payload;

[0027] The UAV platform provides flight vehicles and high-precision positioning and attitude determination capabilities. It adopts a lightweight hexacopter UAV and is equipped with a high-precision RTK-GPS / IMU integrated navigation system with a positioning accuracy of ±1cm+1ppm and an attitude angle accuracy of ±0.1°. It supports autonomous flight path planning, obstacle avoidance based on vision and millimeter-wave radar, and long-endurance operation with a flight time of ≥60 minutes. The autonomous flight path planning supports setting the flight altitude of 50-100m, the heading overlap rate of 80%, and the lateral overlap rate of 70% according to the survey area, and presets control points through RTK-GPS base stations.

[0028] The multi-sensor payload is integrated into the UAV platform, including a multi-line lidar, a high-resolution optical camera and auxiliary sensors. The multi-line lidar obtains the ground point cloud by penetrating vegetation through multi-echo detection. The multi-line lidar is a 16-line / 32-line hybrid solid-state lidar with a ranging range of 0.5-200m, a point frequency of ≥300,000 points / second, and a field of view of 360° horizontally and 30° vertically.

[0029] The high-resolution optical camera uses a full-frame CMOS sensor with no less than 24 million effective pixels, and is equipped with a global shutter. It supports RGB+NIR multispectral imaging to acquire images after ground feature classification and texture enhancement.

[0030] Auxiliary sensors include temperature and humidity sensors and barometers, used for environmental parameter compensation.

[0031] The acquisition and preprocessing module is used to perform spatiotemporal synchronization and preliminary purification of multi-source sensor data, including a spatiotemporal synchronization unit and a preliminary filtering unit;

[0032] The spatiotemporal synchronization unit achieves nanosecond-level time synchronization between multi-sensor payloads and UAV platforms through the PTP precise time protocol, and uses UAV pose data to perform spatial registration of multi-sensor payloads to generate raw point clouds and images with geographic coordinates.

[0033] The preliminary filtering unit uses an improved moving least squares (MLS) method to filter out outlier noise points and dynamic interference points in the point cloud data. Specifically, the improved moving least squares method introduces a local neighborhood weighting function to optimize the surface fitting. The weighting coefficients decrease exponentially with the increase of the distance from the point to the fitted surface, so as to suppress instantaneous noise such as birds and dust reflection points.

[0034] The real-time data processing module is connected to the acquisition and preprocessing module and is used to perform real-time calculation and analysis on the purified data, including an edge computing unit, an incremental SLAM unit, and a dynamic target tracking unit.

[0035] The edge computing unit performs high-load parallel computing tasks based on the real-time point cloud segmentation model. The real-time point cloud segmentation model is a lightweight deep learning model based on the improved PointNet++ network architecture. Through multi-layer feature perception and hierarchical aggregation, it performs real-time semantic segmentation of ground, vegetation, buildings and dynamic targets in large-scale point cloud data. It is equipped with a GPU acceleration card with computing power ≥10TOPS.

[0036] The incremental SLAM unit is based on inter-frame matching of LiDAR point cloud and visual feature point matching, combined with RTK-GPS constraining the global coordinate system to build a local 3D map in real time. It adopts a tightly coupled SLAM framework that fuses LiDAR point cloud features and visual features. The front end extracts curvature feature points in the LiDAR point cloud and performs joint matching with ORB feature points in the optical image. The back end optimizes the fusion of LiDAR odometry, visual odometry and RTK-GPS observations through factor graph optimization to achieve real-time positioning with centimeter-level accuracy and dense point cloud map construction.

[0037] The dynamic target tracking unit analyzes the motion trajectory through point cloud clustering and optical flow, and outputs the velocity and orientation parameters of the dynamic target, including a point cloud clustering subunit and a trajectory tracking subunit.

[0038] The point cloud clustering subunit uses the density-based DBSCAN spatial clustering algorithm to cluster the dynamic target point cloud segmented in real time to identify independent moving individuals. Specifically, the neighborhood radius ε=0.5m and the minimum number of points MinPts=5 are set to identify moving target point cloud clusters.

[0039] The trajectory tracking subunit combines the point cloud clustering results between adjacent frames with the optical flow information of the optical image sequence, and uses an extended Kalman filter to estimate and track the position, velocity and direction of motion of the moving individual.

[0040] The dynamic modeling and analysis module is connected to the real-time data processing module. It is used to fuse multi-source data to generate a high-precision real-scene model and perform time-series change analysis. It includes a data fusion unit, a change detection unit and a dynamic scene modeling unit.

[0041] The data fusion unit performs data fusion based on a multi-source data fusion engine to generate a 3D real-world model with complete texture and high accuracy. Specifically, the multi-source data fusion engine uses Kalman filtering to fuse the geometric information of the LiDAR point cloud, the texture information of the camera image, and the pose and motion state of the UAV platform to optimize the point cloud accuracy and generate a textured 3D real-world model with an accuracy of ±3cm. The state vector of the Kalman filter includes the 3D coordinates of the point cloud, the image texture coordinates, and the attitude angle of the UAV platform.

[0042] The change detection unit is used to identify terrain changes. Specifically, it first uses the iterative nearest point algorithm to finely register the currently acquired 3D point cloud with the historical benchmark 3D model. Then, it calculates the point cloud density difference or distance field change within the voxel grid, and combines morphological opening and closing operations to filter out noise, extracting the difference region that exceeds the preset threshold, thereby realizing the automatic identification and quantification of the change region.

[0043] The dynamic scene modeling unit generates a dynamic geographic information layer based on a long short-term memory network model. The long short-term memory network model is used to learn the target historical trajectory sequence output by the real-time data processing module, predict its future short-term movement trajectory, and overlay the predicted trajectory with the real-scene 3D model to generate a four-dimensional spatiotemporal geographic information layer containing static scene and dynamic target information. The network input is a sequence of historical trajectory points of the moving target (time interval 1s), with 64 neurons in the hidden layer, and outputs the trajectory prediction coordinates for the next 5s.

[0044] The visualization application module and dynamic modeling and analysis module are used for the three-dimensional display, interaction and application integration of results, including a three-dimensional visualization unit and an emergency response control unit;

[0045] The 3D visualization unit is based on WebGL technology of the Web-based 3D visualization platform to realize the streaming loading and real-time rendering of massive 3D models, and provides interactive interfaces for variable area annotation, dynamic trajectory playback and data export. The variable area annotation of the Web-based 3D visualization platform supports manual correction of automatic detection results, and the dynamic target trajectory playback can adjust the time axis rate (0.5x-2x). The interactive interface is compatible with OGC WMS / WFS standards.

[0046] When the emergency response control unit receives an alarm from the dynamic modeling and analysis module indicating a change exceeding a safety threshold, it automatically generates an encrypted monitoring flight mission and sends it to the UAV platform. Simultaneously, it pushes the alarm information to designated terminals. The safety threshold changes include the area threshold for the difference region (e.g., a single-frame change area > 5m²). 2 For unmanned aerial vehicle (UAV) platforms, the re-flight mission planning adopts a spiral-style encrypted route and the flight altitude is reduced to 30-50m when the target speed threshold (e.g., speed > 2m / s) or dynamic target speed threshold (e.g., speed > 2m / s) is set.

[0047] The system's modules communicate with each other via onboard gigabit Ethernet and wireless data transmission links. Data storage adopts a distributed storage architecture that combines onboard SSDs with cloud backup, supporting breakpoint resume and data redundancy verification.

[0048] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A high-precision dynamic geographic information mapping system integrating lidar and UAV, characterized in that: The system includes a data acquisition module, a data acquisition and preprocessing module, a real-time data processing module, a dynamic modeling and analysis module, and a visualization application module. The data acquisition module synchronously acquires multi-dimensional geographic information data based on an unmanned aerial vehicle (UAV) platform and its onboard multi-sensor payload. The data acquisition and preprocessing module performs spatiotemporal synchronization and preliminary purification of multi-source sensor data. The real-time data processing module performs real-time calculation and analysis on the purified data. The dynamic modeling and analysis module fuses multi-source data to generate a high-precision real-scene model and performs temporal change analysis. The visualization application module is used for 3D display, interaction, and application integration of the results.

2. The high-precision dynamic geographic information mapping system integrating lidar and UAV as described in claim 1, characterized in that: The unmanned aerial vehicle platform is used to provide flight vehicles and high-precision positioning and attitude determination capabilities. The multi-sensor payload includes a multi-line lidar, a high-resolution optical camera, and auxiliary sensors. The multi-line lidar obtains ground point clouds by penetrating vegetation through multi-echo detection. The high-resolution optical camera uses a full-frame CMOS sensor to obtain images after ground feature classification and texture enhancement. The auxiliary sensors include a temperature and humidity sensor and a barometer for environmental parameter compensation.

3. The high-precision dynamic geographic information mapping system integrating lidar and UAV as described in claim 1, characterized in that: The acquisition and preprocessing module includes a spatiotemporal synchronization unit and a preliminary filtering unit. The spatiotemporal synchronization unit achieves nanosecond-level time synchronization between the multi-sensor payload and the UAV platform through the PTP precise time protocol, and uses the UAV pose data to perform spatial registration on the multi-sensor payload to generate raw point clouds and images with geographic coordinates. The preliminary filtering unit uses an improved moving least squares method to filter out outlier noise points and dynamic interference points in the point cloud data.

4. The high-precision dynamic geographic information mapping system integrating lidar and UAV as described in claim 1, characterized in that: The real-time data processing module includes an edge computing unit, an incremental SLAM unit, and a dynamic target tracking unit. The edge computing unit performs high-load parallel computing tasks based on a real-time point cloud segmentation model. The incremental SLAM unit constructs a local 3D map in real time based on inter-frame matching of LiDAR point clouds and visual feature point matching, combined with RTK-GPS constraining the global coordinate system. The dynamic target tracking unit analyzes the motion trajectory through point cloud clustering and optical flow methods, and outputs the velocity and direction parameters of the dynamic target.

5. A high-precision dynamic geographic information mapping system integrating lidar and UAV as described in claim 4, characterized in that: The real-time point cloud segmentation model is a lightweight deep learning model based on the improved PointNet++ network architecture. Through multi-layer feature perception and hierarchical aggregation, it performs real-time semantic segmentation of ground, vegetation, buildings and dynamic targets in large-scale point cloud data.

6. A high-precision dynamic geographic information mapping system integrating lidar and UAV as described in claim 4, characterized in that: The dynamic target tracking unit includes a point cloud clustering subunit and a trajectory tracking subunit. The point cloud clustering subunit uses the density-based DBSCAN spatial clustering algorithm to cluster the dynamic target point cloud segmented in real time to identify independent moving individuals. The trajectory tracking subunit combines the point cloud clustering results between adjacent frames with the optical flow information of the optical image sequence, and uses an extended Kalman filter to estimate and track the position, velocity and direction of motion of the moving individuals.

7. A high-precision dynamic geographic information mapping system integrating lidar and UAV as described in claim 1, characterized in that: The dynamic modeling and analysis module includes a data fusion unit, a change detection unit, and a dynamic scene modeling unit. The data fusion unit performs data fusion based on a multi-source data fusion engine to generate a three-dimensional real-world model with complete texture and high accuracy. The change detection unit is used to identify terrain changes. The dynamic scene modeling unit generates a dynamic geographic information layer based on a long short-term memory network model.

8. A high-precision dynamic geographic information mapping system integrating lidar and UAV as described in claim 7, characterized in that: The multi-source data fusion engine specifically uses Kalman filtering to fuse the geometric information of LiDAR point clouds, the texture information of camera images, and the pose and motion state of the UAV platform, thereby optimizing the point cloud accuracy and generating a textured 3D real-world model. The change detection unit first uses the iterative nearest point algorithm to finely register the currently acquired 3D point cloud with the historical benchmark 3D model. Then, by calculating the point cloud density difference or distance field change within the voxel grid and combining morphological opening and closing operations to filter out noise, it extracts the difference region exceeding the preset threshold, thereby realizing the automatic identification and quantification of the change region. The long short-term memory network model is used to learn the target historical trajectory sequence output by the real-time data processing module, predict its future short-term movement trajectory, and overlay the predicted trajectory with the real-scene 3D model to generate a four-dimensional spatiotemporal geographic information layer containing static scene and dynamic target information.

9. A high-precision dynamic geographic information mapping system integrating lidar and UAV as described in claim 1, characterized in that: The visualization application module includes a 3D visualization unit and an emergency response control unit. The 3D visualization unit uses WebGL technology on a web-based 3D visualization platform to achieve streaming loading and real-time rendering of massive 3D models, and provides interactive interfaces for marking change areas, replaying dynamic trajectories, and exporting data. When the emergency response control unit receives an alarm from the dynamic modeling and analysis module indicating a change exceeding a safety threshold, it automatically generates an encrypted monitoring flight mission and sends it to the UAV platform, while simultaneously pushing the alarm information to a designated terminal.

Images