SLAM mobile measurement system data acquisition and processing method and system

Abstract

The embodiment of the present application relates to the technical field of unmanned aerial vehicle surveying and mapping, and specifically discloses a SLAM mobile measurement system data acquisition and processing method and system. The embodiment of the present application sets flight task parameters of an unmanned aerial vehicle and time synchronization configuration of multiple sensors, carries out multi-source data acquisition and preprocessing, acquires multi-source synchronous data, constructs a sparse point cloud map, reconstructs a dense point cloud map and carries out modeling processing, carries out loop detection and map error optimization, generates an optimized three-dimensional map, and carries out application display. The flight task parameters of the unmanned aerial vehicle and the time synchronization configuration of the multiple sensors can be set, the dense point cloud map can be reconstructed and modeling processing can be carried out, the three-dimensional space map can be generated, loop detection and map error optimization can be carried out, the optimized three-dimensional map can be generated and applied and displayed, the multi-source data can be effectively aligned, the time error can be reduced, the precision and stability of the SLAM system can be improved, the three-dimensional modeling precision can be improved, and the high-quality surveying and mapping can be achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of unmanned aerial vehicle surveying and mapping, and particularly relates to a SLAM mobile measurement system data acquisition and processing method and system. BACKGROUND

[0002] Unmanned aerial vehicle surveying and mapping refers to a technical means of acquiring geographic space data in low-altitude autonomous or remote-controlled flight by using an unmanned aerial vehicle platform carrying high-precision sensors (such as cameras, laser radars, GNSS, etc.), and generating surveying and mapping results such as digital orthophoto maps (DOM), digital elevation models (DEM), and three-dimensional real scene models (3D models) through data processing and modeling.

[0003] Compared with traditional surveying and mapping methods, unmanned aerial vehicle surveying and mapping has the advantages of high operation efficiency, low cost, fast and flexible data acquisition, and adaptation to complex terrain, and is widely used in land resource investigation, agricultural remote sensing, urban planning, disaster monitoring, engineering surveying, and other fields.

[0004] In the prior art, the data acquisition and processing of the SLAM mobile measurement system have the problems of time stamp errors or sampling frequency differences of multiple sensors, which leads to the fact that subsequent data cannot be effectively aligned, the accumulation of time errors affects the accuracy and stability of the SLAM system, and the dense reconstruction has the problem of redundant point clouds, which leads to insufficient three-dimensional modeling accuracy and difficulty in being used for high-quality surveying and mapping. SUMMARY

[0005] The purpose of the embodiments of the application is to provide a SLAM mobile measurement system data acquisition and processing method and system, aiming to solve the problems proposed in the background art.

[0006] To achieve the above-mentioned purpose, the embodiments of the application provide the following technical solutions:

[0007] The SLAM mobile measurement system data acquisition and processing method specifically includes the following steps:

[0008] Receiving an unmanned aerial vehicle surveying and mapping task, setting flight task parameters of the unmanned aerial vehicle and time synchronization configurations of multiple sensors, and performing initialization processing on the SLAM mobile measurement system;

[0009] According to the flight task parameters and the time synchronization configurations, performing multi-source data acquisition and preprocessing to acquire multi-source synchronous data;

[0010] Performing SLAM positioning on the multi-source synchronous data to construct a sparse point cloud map;

[0011] Based on the sparse point cloud map, reconstructing a dense point cloud map, and performing modeling processing on the dense point cloud map to generate a three-dimensional space map;

[0012] Loop detection and map error optimization are performed on the three-dimensional space map to generate an optimized three-dimensional map, and the three-dimensional space map is applied and displayed.

[0013] The SLAM mobile measurement system data acquisition and processing system is applied to the SLAM mobile measurement system data acquisition and processing method, and comprises a surveying task processing unit, a multi-source data acquisition unit, a sparse point cloud construction unit, a dense point cloud reconstruction unit and a map application display unit.

[0014] The surveying task processing unit is configured to receive a UAV surveying task, set flight task parameters of the UAV and time synchronization configurations of multi-sensors, and initialize the SLAM mobile measurement system.

[0015] The multi-source data acquisition unit is configured to acquire multi-source synchronous data by performing multi-source data acquisition and preprocessing according to the flight task parameters and the time synchronization configurations.

[0016] The sparse point cloud construction unit is configured to construct a sparse point cloud map by performing SLAM positioning on the multi-source synchronous data.

[0017] The dense point cloud reconstruction unit is configured to reconstruct a dense point cloud map based on the sparse point cloud map, perform modeling processing on the dense point cloud map, and generate a three-dimensional space map.

[0018] The map application display unit is configured to perform loop detection and map error optimization on the three-dimensional space map to generate an optimized three-dimensional map, and apply and display the three-dimensional space map.

[0019] Compared with the prior art, the present application has the following advantages:

[0020] The present application sets flight task parameters of the UAV and time synchronization configurations of multi-sensors, acquires multi-source synchronous data by performing multi-source data acquisition and preprocessing, constructs a sparse point cloud map, reconstructs a dense point cloud map and performs modeling processing, performs loop detection and map error optimization to generate an optimized three-dimensional map, and applies and displays the three-dimensional space map. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application.

[0022] Figure 1 A flow chart of the method provided by the embodiment of the present application is shown.

[0023] Figure 2 An application architecture diagram of the system provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present application more clear, the following will further describe the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0025] It can be understood that in the prior art, the data acquisition and processing of the SLAM mobile measurement system has the problem of time stamp error or sampling frequency difference of multiple sensors, which leads to that the subsequent data cannot be effectively aligned, the accumulation of time error affects the accuracy and stability of the SLAM system, and the dense reconstruction has the problem of point cloud redundancy, which leads to insufficient three-dimensional modeling accuracy and difficulty in high-quality mapping.

[0026] To solve the above problems, the embodiment of the present application sets the flight task parameters of the unmanned aerial vehicle and the time synchronization configuration of the multiple sensors by receiving the unmanned aerial vehicle mapping task, initializes the SLAM mobile measurement system, collects and preprocesses the multi-source data according to the flight task parameters and the time synchronization configuration, acquires multi-source synchronous data, performs SLAM positioning on the multi-source synchronous data, constructs a sparse point cloud map, reconstructs a dense point cloud map based on the sparse point cloud map, performs modeling processing on the dense point cloud map, generates a three-dimensional space map, performs loop detection and map error optimization on the three-dimensional space map, generates an optimized three-dimensional map, and applies and displays the three-dimensional space map. The flight task parameters of the unmanned aerial vehicle and the time synchronization configuration of the multiple sensors can be set, the dense point cloud map can be reconstructed and modeling processed, the three-dimensional space map can be generated, loop detection and map error optimization can be performed, the optimized three-dimensional map can be generated and applied and displayed, the multi-source data can be effectively aligned, the time error can be reduced, the accuracy and stability of the SLAM system can be improved, the three-dimensional modeling accuracy can be improved, and the high-quality mapping can be realized.

[0027] Figure 1 A flow chart of the method provided by the embodiment of the present application is shown.

[0028] Specifically, the SLAM mobile measurement system data acquisition and processing method specifically comprises the following steps:

[0029] Step S101, receiving a UAV mapping task, setting flight task parameters of the UAV and time synchronization configuration of the multi-sensor, and initializing a SLAM mobile measurement system.

[0030] In the embodiment of the application, by receiving a UAV mapping task, the target mapping area and the mapping task type are determined, and then the flight task parameters of the UAV (including flight path, height, speed, and route overlap) are set according to the target mapping area, and the time synchronization configuration of the multi-sensor is set according to the mapping task type (specifically, the synchronization protocol of the camera, LiDAR, IMU, GNSS, and other sensors is configured to provide a unified space-time reference), and the SLAM mobile measurement system is initialized to initialize the UAV position, attitude, and map state, and obtain key SLAM parameters.

[0031] Specifically, in the preferred embodiment provided by the application, the receiving of the UAV mapping task, the setting of the flight task parameters of the UAV and the time synchronization configuration of the multi-sensor, and the initialization of the SLAM mobile measurement system specifically include the following steps:

[0032] Receiving a UAV mapping task, determining a target mapping area and a mapping task type;

[0033] Setting flight task parameters of the UAV according to the target mapping area;

[0034] Setting time synchronization configuration of the multi-sensor according to the mapping task type;

[0035] Initializing a SLAM mobile measurement system, initializing the UAV position, attitude, and map state, and obtaining key SLAM parameters.

[0036] In the preferred embodiment provided by the application, setting the time synchronization configuration of the multi-sensor according to the mapping task type specifically includes the following steps:

[0037] Determining the mapping task type, and selecting one of the sensors as the master sensor according to the mapping task type, the master sensor clock as the master clock, and the other sensors as the slave sensors, the slave sensor clock as the slave clock;

[0038] In the preferred embodiment provided by the application, the mapping task type includes high-precision mapping, fast scanning, and balanced precision and speed requirements; if the mapping task type is high-precision mapping, the IMU is selected as the master clock; if the mapping task type is fast scanning, the global clock of the laser radar is selected as the master clock; if the mapping task type is balanced precision and speed requirements, the priority score of each sensor clock is calculated, and the sensor clock with the highest priority score is selected as the master clock.

[0039] According to the mapping task type, different timestamp alignment modes are selected for alignment, and a synchronization mode parameter and a corrected sensor timestamp table are obtained;

[0040] Different timestamp alignment modes include a strict alignment mode and a dynamic compensation mode, the strict alignment mode is to perform phase locking between other slave clock signals and a master clock through a phase-locked loop circuit of an FPGA chip, and to forcibly align the clock edges, and the dynamic compensation mode is to calculate a compensation amount according to a sensor inherent delay parameter and a transmission delay, to uniformly quantize the compensation amount according to the master clock, and to obtain a slave sensor timestamp dynamic compensation parameter table.

[0041] In the embodiment of the application, high-precision mapping tasks have very high requirements for time synchronization accuracy, inertial measurement unit (IMU) data is usually collected at a high frequency and is stable in timing, and therefore the IMU clock is selected as the master clock to ensure that the inertial data is strictly synchronized with other sensor data. The fast scanning task emphasizes data collection speed, and the laser radar (LiDAR) is the main data source, and therefore the global clock of the laser radar is selected as the master clock to ensure that the laser point cloud data timestamp is accurate and supports fast real-time processing. The balanced mode combines sensor performance indicators to calculate clock priority scores, and selects the most suitable clock as the master clock, taking into account accuracy and efficiency.

[0042] And different timestamp alignment strategies can be adopted according to different task requirements: strict alignment mode: all slave clock signal edges are locked with the master clock through hardware means (such as a phase-locked loop (PLL) in an FPGA chip), phase synchronization at the hardware level is achieved, and the timestamps are forcibly aligned on the hardware, which is suitable for mapping tasks with very high requirements for time accuracy. Dynamic compensation mode: based on sensor inherent delay (such as signal acquisition and transmission delay) parameters, a compensation amount is calculated, the slave sensor timestamp is dynamically adjusted based on the master clock, and this mode is suitable for scenarios with high real-time requirements and low strict phase-locked requirements for hardware.

[0043] Therefore, the multi-sensor time synchronization configuration based on the mapping task type to select the master clock and the synchronization mode, in combination with the strict alignment of the hardware phase-locked loop and the software dynamic compensation, can guarantee the time consistency of the multi-sensor data, improve the mapping accuracy, flexibly adapt to different mapping task requirements, and improve the system application range.

[0044] Further, the SLAM mobile measurement system data acquisition and processing method further includes the following steps:

[0045] Step S102, according to the flight task parameters and the time synchronization configuration, multi-source data acquisition and preprocessing are performed to obtain multi-source synchronous data.

[0046] In the embodiment of the present application, multi-source data acquisition is performed according to the flight task parameters and the time synchronization configuration, multi-source acquisition data (including image, IMU, point cloud and other multi-source data) is obtained, the time stamp corresponding to the multi-source acquisition data is determined, the time stamp data is recorded, noise filtering processing is performed on the multi-source acquisition data, multi-source filtered data is obtained, and then time stamp alignment synchronization processing is performed on the multi-source filtered data according to the time stamp data, and multi-source synchronization data is obtained.

[0047] Specifically, in the preferred embodiment provided by the present application, the multi-source data acquisition and preprocessing according to the flight task parameters and the time synchronization configuration to obtain multi-source synchronization data specifically includes the following steps:

[0048] Multi-source data acquisition is performed according to the flight task parameters and the time synchronization configuration, and multi-source acquisition data is obtained;

[0049] The time stamp corresponding to the multi-source acquisition data is determined, and the time stamp data is recorded;

[0050] Noise filtering processing is performed on the multi-source acquisition data, and multi-source filtered data is obtained;

[0051] The multi-source filtered data is subjected to time stamp alignment synchronization processing according to the time stamp data, and multi-source synchronization data is obtained.

[0052] In the preferred embodiment provided by the present application, the multi-source filtered data is subjected to time stamp alignment synchronization processing according to the time stamp data to obtain multi-source synchronization data specifically includes the following steps:

[0053] The hardware trigger moment of the master sensor is extracted as the reference point of the global time axis, and a time sequence is generated at a fixed interval to obtain a global time axis sequence of the master sensor;

[0054] The compensation amount and the clock drift rate of each slave sensor are read from the slave sensor time stamp dynamic compensation parameter table;

[0055] The first arrival time of the slave sensor data recorded at system startup is taken as the initial time, and the original time stamp of the slave sensor is superimposed with the compensation amount of the slave sensor as the current time stamp;

[0056] According to the initial time and the current time stamp, the running time length of the slave sensor since the initial time is calculated, and the clock drift rate linearly increasing with time is superimposed to obtain the corrected time stamp data of the slave sensor;

[0057] When the main sensor hardware trigger time arrives, according to the highest sampling frequency of the current slave sensor, the dynamic window width is calculated, and on the basis of the dynamic window width, the window width is expanded forward and backward to the global time axis sequence of the main sensor with the main sensor hardware trigger time as the center, to obtain the dynamic window range corresponding to the main sensor trigger time;

[0058] All filtered data of the slave sensor are traversed, and data points of the slave sensor corrected timestamp data whose timestamps fall within the dynamic window range are extracted, to obtain a data point set of the slave sensor within the dynamic window range;

[0059] The absolute time difference between the corrected timestamp of each data point in the data point set of the slave sensor within the dynamic window range and the main clock is calculated.

[0060] Based on the size of the absolute time difference, the fusion weight is obtained by using exponential decay calculation, and the multiple data points of the same sensor are weighted and averaged according to the fusion weight, to obtain multi-source synchronous data.

[0061] In the embodiment of the application, the hardware trigger time of the main sensor is taken as the global time axis reference, the timestamp of the slave sensor is corrected in real time through fixed delay compensation and clock drift rate, and the long-time error caused by crystal oscillator temperature drift is eliminated. The window width is dynamically adjusted according to the sampling frequency of the slave sensor. For example, the window width of a 100Hz GNSS sensor is 5ms, while the window width of a 1000Hz laser radar is 0.5ms, avoiding data truncation of high-frequency sensors or noise introduced by wide window of low-frequency sensors. And the weight is calculated by using the exponential decay function, and the weight of the data with smaller time difference is higher, to ensure the alignment accuracy. Through the three-layer cooperation of "hardware reference + software optimization + closed-loop correction", the three core problems of frequency difference, clock drift and data loss in multi-sensor synchronization are solved, and high-precision and high-robustness time synchronization guarantee is provided for the SLAM mobile measurement system.

[0062] Further, the SLAM mobile measurement system data acquisition and processing method further comprises the following steps:

[0063] Step S103, SLAM positioning is performed on the multi-source synchronous data to construct a sparse point cloud map.

[0064] In the embodiment of the application, image feature extraction (key points are extracted in the image, inter-frame matching or optical flow tracking is performed) is performed on the multi-source synchronous data, geometric feature data is recorded, unmanned aerial vehicle pose analysis (unmanned aerial vehicle pose is estimated by using IMU and visual / laser data, and a nonlinear optimization algorithm is used to improve the estimation accuracy) is performed on the multi-source synchronous data, unmanned aerial vehicle pose data is recorded, and then a sparse point cloud map is constructed according to the geometric feature data and the unmanned aerial vehicle pose data (a preliminary sparse point cloud map is constructed according to the estimated pose, and is used as the basis for global mapping).

[0065] Specifically, in the preferred embodiments provided by the present application, the SLAM positioning of the multi-source synchronous data and the construction of the sparse point cloud map specifically include the following steps:

[0066] Image feature extraction is performed on the multi-source synchronous data, and geometric feature data is recorded;

[0067] Unmanned aerial vehicle pose analysis is performed on the multi-source synchronous data, and unmanned aerial vehicle pose data is recorded;

[0068] According to the geometric feature data and the unmanned aerial vehicle pose data, a sparse point cloud map is constructed.

[0069] In the preferred embodiments provided by the present application, the construction of the sparse point cloud map according to the geometric feature data and the unmanned aerial vehicle pose data specifically includes the following steps:

[0070] According to the motion speed and angular speed in the unmanned aerial vehicle pose data, the pose change rate is calculated, the current scene type is judged according to the pose change rate, and different sensor weights are given to the sensors according to the scene type; the scene type includes dynamic scene and static scene, and in the static scene: the laser radar weight 0.7 and the camera weight 0.3 are read from the parameter table; in the dynamic scene: the IMU weight 0.6 and the laser radar weight 0.4 are read; if the feature points are observed by multiple sensors (such as laser radar and camera detecting the same corner point), the average value of the weights is taken.

[0071] The motion blur coefficient is calculated according to the instantaneous angular speed of the unmanned aerial vehicle and the exposure time of the camera;

[0072] The feature matching score is obtained according to the stability of each feature point in the geometric feature data in time or space;

[0073] The feature matching score corresponding to each feature point in the geometric feature data is fused with the sensor weight, and the motion blur coefficient is subtracted, to obtain a feature point list with reliability score;

[0074] The RGB image in the geometric feature data is input into a pre-trained lightweight semantic segmentation model to obtain a pixel-level semantic label;

[0075] For the region with the pixel-level semantic label being "vehicle" or "pedestrian", the feature point motion vector is calculated in combination with the unmanned aerial vehicle motion direction;

[0076] According to the comparison between the feature point motion vector and the unmanned aerial vehicle pose change, if the direction of the feature point motion vector is inconsistent with the unmanned aerial vehicle pose change, it is marked as a dynamic interference point;

[0077] Filtering the dynamic interference points in the feature point list with reliability scores to obtain a filtered static feature point list;

[0078] Taking the filtered static feature point list as a basic data source, dividing a current UAV visual field into a preset size of uniform area grid, and each uniform area grid covering the same physical range;

[0079] According to the semantic label distribution in the uniform area grid, judging the complexity, reserving different numbers of filtered static feature points in the corresponding uniform area grid according to the complexity, and merging the filtered static feature points in the same grid which are derived from different sensors and have a spatial distance less than a preset spatial threshold, to obtain an optimized sparse point cloud map.

[0080] In the embodiment of the application, different sensors are configured with signal weights according to the task type (static / dynamic). For example, IMU data is more reliable in a dynamic scene, and the laser radar has higher precision in a static scene. In addition, a blur coefficient is calculated using the angular velocity of the UAV and the camera exposure time to reduce the error influence of visual features in high-speed motion.

[0081] In addition, a low-computational-overhead semantic segmentation network is used to quickly identify dynamic objects such as vehicles and pedestrians in the image, compare the feature point motion direction with the UAV pose change direction, and exclude abnormal moving points to achieve semantic-assisted dynamic object filtering.

[0082] By dividing the visual field into a grid, dynamically controlling the number of features retained in each grid according to the semantic label (building, ground, etc.), and avoiding redundancy, the coordinates and descriptors of multi-source features (such as laser radar corner points + visual corner points) at the same position are fused, and the reconstruction of non-structural areas is skipped to improve the uniqueness and recognition of features.

[0083] Further, the SLAM mobile measurement system data acquisition and processing method further comprises the following steps:

[0084] In step S104, based on the sparse point cloud map, a dense point cloud map is reconstructed, the dense point cloud map is modeled and processed, and a three-dimensional space map is generated.

[0085] In the embodiment of the application, based on the sparse point cloud map, a structure light, depth map or stereo matching method is used to reconstruct a dense point cloud map (a structure light, depth map or stereo matching method is used to reconstruct a dense point cloud, and adjacent frame images and depth information fusion are introduced to improve the reconstruction accuracy), then voxel filtering and outlier rejection are performed on the dense point cloud map, and point cloud registration and merging are performed on the dense point cloud map (ICP or NDT algorithm is used for point cloud registration and merging), and then the processed point cloud is converted into a grid model, and texture is superimposed to generate a three-dimensional space map.

[0086] Specifically, in the preferred embodiments provided by the present application, the step of reconstructing a dense point cloud map based on the sparse point cloud map and modeling the dense point cloud map to generate a three-dimensional space map specifically comprises the following steps:

[0087] Reconstructing a dense point cloud map based on the sparse point cloud map by using structured light, depth map or stereo matching method;

[0088] Performing voxel filtering and outlier rejection on the dense point cloud map;

[0089] Performing point cloud registration and merging on the dense point cloud map;

[0090] Converting a grid model and superimposing texture to generate a three-dimensional space map.

[0091] In the preferred embodiments provided by the present application, the step of reconstructing a dense point cloud map based on the sparse point cloud map by using structured light, depth map or stereo matching method specifically comprises the following steps:

[0092] Extracting a semantic label distribution map from the optimized sparse point cloud map;

[0093] Defining a reconstruction area priority according to the semantic label distribution map to obtain a region priority strategy table;

[0094] Performing near-field high-resolution reconstruction, middle-field adaptive reconstruction and far-field low-power consumption reconstruction based on the region priority strategy table to obtain near-field point cloud, middle-field point cloud and far-field point cloud;

[0095] The near-field high-resolution reconstruction specifically comprises the following steps: projecting an encoded raster pattern to a target area, capturing a deformed raster image by using a binocular camera, calculating depth data of the deformed raster image according to the parallax of left and right cameras to obtain an initial visual depth map, and generating a 0.5mm resolution point cloud by using the initial visual depth map to obtain near-field high-precision point cloud;

[0096] The middle-field adaptive reconstruction specifically comprises the following steps: adjusting the distance between the binocular cameras in real time according to the target distance, combining the initial visual depth map of multiple frames of images in the flight of the unmanned aerial vehicle, eliminating noise by weighted average to obtain a final visual depth map, and generating middle-field adaptive point cloud by using the final visual depth map;

[0097] The far-field low-power consumption reconstruction specifically comprises the following steps: generating a continuous surface by using Bezier surface fitting based on the laser radar points in the optimized sparse point cloud map, and limiting the density of interpolation points to 10mm interval in the process of generating the continuous surface to avoid over-fining, thereby obtaining far-field low-density point cloud;

[0098] Based on the semantic label distribution map, the near-field point cloud, the middle-field point cloud and the far-field point cloud are aligned with the laser radar points in the optimized sparse point cloud map, and then are weightedly fused with the optimized sparse point cloud map to obtain a cross-modal optimized dense point cloud;

[0099] According to the semantic label mutation area detected by the semantic label distribution map, a semantic boundary mask map is obtained.

[0100] An original high-resolution RGB image is obtained from multi-source synchronous data, and a semantic boundary mask map is applied to obtain an RGB image of a boundary area.

[0101] The RGB image of the boundary area is input into a lightweight GAN network for upsampling to obtain a super-resolution boundary image.

[0102] Sub-pixel level disparity calculation is performed on the super-resolution image to generate a high-precision edge point cloud, and the boundary points in the cross-modal optimized dense point cloud are replaced by the high-precision edge point cloud to obtain a final optimized dense point cloud map.

[0103] In the embodiment of the application, the semantic label (such as building, vegetation) directly affects the weight distribution: high-priority semantic areas (such as buildings) give higher weight (0.8) to visual data during fusion because they need to retain details; low-priority areas (such as vegetation) reduce the visual weight (0.4) to avoid over-optimizing non-critical areas. And the semantic label helps step 3 to select high-confidence alignment reference points (such as building corners, road markings), instead of randomly selecting points, to improve the ICP registration accuracy.

[0104] And through hierarchical ICP alignment, the scale and position deviations caused by different reconstruction strategies in step are eliminated to ensure seamless splicing of multi-resolution point clouds. The near field relies on visual details, the far field relies on the stability of the laser radar, and the middle field serves as a transition layer to avoid model discontinuity caused by resolution mutation through weight fusion, and to take advantage of the respective advantages of multi-modal data,

[0105] The obtained cross-modal optimized point cloud has high global consistency, providing an accurate spatial reference for subsequent semantic edge super-resolution enhancement. The cross-modal optimized point cloud provides a coarse calibration point cloud, and on this basis, local enhancement is performed for the semantic boundary to greatly improve the accuracy of key areas such as building-vegetation boundaries, thereby improving the overall modeling accuracy

[0106] Further, the SLAM mobile measurement system data acquisition and processing method further comprises the following steps:

[0107] Step S105, loop detection and map error optimization are performed on the three-dimensional space map to generate an optimized three-dimensional map, and the three-dimensional space map is applied and displayed.

[0108] In the embodiment of the present application, the loop detection is performed on the three-dimensional space map (the map optimization factor graph is constructed by using the loop detection algorithm), then the accumulated error is eliminated by using the back-end optimization algorithm, the pose error is evaluated, the map and the trajectory are corrected (the pose error is evaluated by comparing the GNSS real trajectory with the SLAM estimated trajectory, and the map and the trajectory are corrected according to the error), the optimized three-dimensional map is generated, the map application platform is determined, and then the three-dimensional space map is applied and displayed on the map application platform.

[0109] Specifically, in the preferred embodiment provided by the present application, the loop detection and map error optimization on the three-dimensional space map, the generation of the optimized three-dimensional map, and the application and display of the three-dimensional space map specifically include the following steps:

[0110] The loop detection is performed on the three-dimensional space map, and the accumulated error is eliminated.

[0111] The pose error is evaluated, and the map and the trajectory are corrected.

[0112] The optimized three-dimensional map is generated.

[0113] The map application platform is determined.

[0114] The three-dimensional space map is applied and displayed on the map application platform.

[0115] Further, Figure 2 The application architecture diagram of the system provided by the embodiment of the present application is shown.

[0116] In another preferred embodiment provided by the present application, the SLAM mobile measurement system data acquisition and processing system is applied to the SLAM mobile measurement system data acquisition and processing method, and the system includes a surveying task processing unit 101, a multi-source data acquisition unit 102, a sparse point cloud construction unit 103, a dense point cloud reconstruction unit 104, and a map application display unit 105, wherein:

[0117] The surveying task processing unit 101 is used to receive the unmanned aerial vehicle surveying task, set the flight task parameters of the unmanned aerial vehicle and the time synchronization configuration of the multi-sensor, and perform the initialization processing on the SLAM mobile measurement system.

[0118] In the embodiment of the present application, the surveying task processing unit 101 receives the unmanned aerial vehicle surveying task, determines the target surveying area and the surveying task type, sets the flight task parameters (including the flight path, height, speed, and route overlap degree) of the unmanned aerial vehicle according to the target surveying area, sets the time synchronization configuration of the multi-sensor (specifically, the synchronization protocol of the camera, LiDAR, IMU, GNSS, and other sensors is configured to provide a unified space-time reference) according to the surveying task type, and performs initialization processing on the SLAM mobile measurement system to initialize the position, attitude, and map state of the unmanned aerial vehicle and obtain key SLAM parameters.

[0119] The multi-source data acquisition unit 102 is configured to acquire and pre-process multi-source data and obtain multi-source synchronous data according to the flight task parameters and the time synchronization configuration.

[0120] In the embodiment of the present application, the multi-source data acquisition unit 102 acquires multi-source data (including image, IMU, point cloud, and other multi-source data) according to the flight task parameters and the time synchronization configuration, determines the time stamp corresponding to the multi-source data, records the time stamp data, performs noise filtering processing on the multi-source data to obtain multi-source filtered data, and then performs time stamp alignment and synchronization processing on the multi-source filtered data according to the time stamp data to obtain multi-source synchronous data.

[0121] The sparse point cloud construction unit 103 is configured to perform SLAM positioning on the multi-source synchronous data and construct a sparse point cloud map.

[0122] In the embodiment of the present application, the sparse point cloud construction unit 103 extracts image features from the multi-source synchronous data (extracts key points in the image and performs inter-frame matching or optical flow tracking), records the geometric feature data, analyzes the unmanned aerial vehicle pose from the multi-source synchronous data (estimates the unmanned aerial vehicle pose by combining the IMU and visual / laser data, and improves the estimation accuracy by using a nonlinear optimization algorithm), records the unmanned aerial vehicle pose data, and then constructs a sparse point cloud map according to the geometric feature data and the unmanned aerial vehicle pose data (projects the feature points according to the estimated pose to construct a preliminary sparse point cloud map as the basis for global mapping).

[0123] The dense point cloud reconstruction unit 104 is configured to reconstruct a dense point cloud map based on the sparse point cloud map, perform modeling processing on the dense point cloud map, and generate a three-dimensional space map.

[0124] In the embodiment of the present application, the dense point cloud reconstruction unit 104 reconstructs the dense point cloud map based on the sparse point cloud map by using the structured light, depth map or stereo matching method (reconstructs the dense point cloud by using the structured light, depth map or stereo matching method, and introduces the fusion of adjacent frame images and depth information to improve the reconstruction accuracy), then performs voxel filtering and outlier rejection on the dense point cloud map, and performs point cloud registration and merging (performs point cloud registration and merging by using the ICP or NDT algorithm) on the dense point cloud map, and then converts the processed point cloud into a grid model, superimposes texture, and generates a three-dimensional space map.

[0125] The map application display unit 105 is configured to perform loop detection and map error optimization on the three-dimensional space map, generate an optimized three-dimensional map, and display the three-dimensional space map.

[0126] In the embodiment of the present application, the map application display unit 105 performs loop detection on the three-dimensional space map (constructs a graph optimization factor graph by using a loop detection algorithm), then eliminates accumulated errors by using a back-end optimization algorithm, evaluates pose errors, corrects the map and the trajectory (compares the GNSS real trajectory with the SLAM estimated trajectory to evaluate the pose errors, corrects the map and the trajectory according to the errors), generates an optimized three-dimensional map, determines a map application platform, and displays the three-dimensional space map on the map application platform.

[0127] It should be understood that, although each step in the flowchart of each embodiment of the present application is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in each embodiment can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer readable storage medium, and when the program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0129] Any combination of the technical features of the above-mentioned embodiments can be combined. In order to make the description simple, all possible combinations of the technical features in the above-mentioned embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0130] The above-mentioned embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be understood as limiting the scope of the present application. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

[0131] The above-mentioned is only the preferred embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for data acquisition and processing of a SLAM mobile measurement system, characterized in that, The method specifically comprises the following steps: Receiving a UAV surveying task, setting flight task parameters of the UAV and time synchronization configuration of multiple sensors, and initializing a SLAM mobile surveying system; According to the flight task parameters and the time synchronization configuration, multi-source data acquisition and preprocessing are performed to obtain multi-source synchronous data; SLAM positioning is performed on the multi-source synchronous data to construct a sparse point cloud map; Based on the sparse point cloud map, a dense point cloud map is reconstructed, modeling processing is performed on the dense point cloud map, and a three-dimensional space map is generated; Loop detection and map error optimization are performed on the three-dimensional space map to generate an optimized three-dimensional map, and application display is performed on the three-dimensional space map; The time synchronization configuration of the multiple sensors comprises: Determining a surveying task type, and selecting one of the sensors as a master sensor according to the surveying task type, taking the master sensor clock as a master clock, and taking the other sensors as slave sensors, taking the slave sensor clock as a slave clock; The surveying task type comprises high-precision surveying, rapid scanning, and balanced precision and speed requirements; if the surveying task type is high-precision surveying, the IMU is selected as the master clock; if the surveying task type is rapid scanning, the global clock of the laser radar is selected as the master clock; if the surveying task type is balanced precision and speed requirements, the priority scores of the sensor clocks are calculated, and the sensor clock with the highest priority score is selected as the master clock; According to the surveying task type, different timestamp alignment modes are selected for alignment to obtain synchronization mode parameters and a corrected sensor timestamp table; The different timestamp alignment modes comprise a strict alignment mode and a dynamic compensation mode; the strict alignment mode is to perform phase locking on the other slave clock signals and the master clock through a phase-locked loop circuit of an FPGA chip, and to forcibly align the clock edges; the dynamic compensation mode is to calculate a compensation amount according to sensor inherent delay parameters and transmission delay, to uniformly quantize the compensation amount according to the master clock, and to obtain a slave sensor timestamp dynamic compensation parameter table; The reconstruction of the dense point cloud map comprises: Based on the sparse point cloud map, a semantic label distribution map is generated by extracting the semantic labels of each feature point; a region priority strategy table is obtained by defining the reconstruction region priority according to the semantic label distribution map; near-field high-resolution reconstruction, middle-field adaptive reconstruction, and far-field low-power consumption reconstruction are performed by using a structured light, a depth map, or a stereo matching method respectively according to the region priority strategy table, to obtain near-field point clouds, middle-field point clouds, and far-field point clouds.

2. The SLAM mobile survey system data acquisition and processing method of claim 1, wherein, The receiving of the UAV surveying task, the setting of the flight task parameters of the UAV and the time synchronization configuration of the multiple sensors, and the initialization of the SLAM mobile surveying system specifically comprise the following steps: Receiving a UAV surveying task, determining a target surveying area and a surveying task type; According to the target surveying area, setting flight task parameters of the UAV; According to the surveying task type, setting time synchronization configuration of the multiple sensors; Initializing the SLAM mobile surveying system, initializing the position, attitude, and map state of the UAV, and obtaining key SLAM parameters.

3. The SLAM mobile survey system data acquisition and processing method of claim 2, wherein, The multi-source data acquisition and preprocessing according to the flight task parameters and the time synchronization configuration comprises the following steps: According to the flight task parameters and the time synchronization configuration, multi-source data acquisition is performed to obtain multi-source acquisition data; Determine the timestamp corresponding to the multi-source acquisition data, and record the timestamp data; Noise filtering processing is performed on the multi-source acquisition data to obtain multi-source filtered data; According to the timestamp data, the multi-source filtered data is subjected to timestamp alignment synchronization processing to obtain multi-source synchronization data.

4. The SLAM mobile survey system data acquisition and processing method of claim 3, wherein, The multi-source data acquisition and preprocessing according to the flight task parameters and the time synchronization configuration comprises the following steps: Extract the hardware trigger time of the main sensor as the reference point of the global time axis, generate a time sequence at a fixed interval, and obtain the global time axis sequence of the main sensor; Read the compensation amount and clock drift rate of each slave sensor from the slave sensor timestamp dynamic compensation parameter table; Take the first arrival time of the slave sensor data recorded at system startup as the initial time, and superimpose the compensation amount of the slave sensor on the original timestamp of the slave sensor to obtain the current timestamp; According to the initial time and the current timestamp, calculate the running time of the slave sensor since the initial time, and superimpose the clock drift rate that increases linearly with time to obtain the corrected timestamp data of the slave sensor; When the main sensor hardware trigger time arrives, calculate the dynamic window width according to the current highest sampling frequency of the slave sensor, and expand the window width before and after the global time axis sequence of the main sensor based on the dynamic window width, with the main sensor hardware trigger time as the center, to obtain the dynamic window range corresponding to the main sensor trigger time; Traverse all the filtered data of the slave sensors, extract the data points of the corrected timestamp data of the slave sensors that fall within the dynamic window range, and obtain the data point set of the slave sensors within the dynamic window range; Calculate the absolute time difference between the corrected timestamp of each data point in the data point set of the slave sensors within the dynamic window range and the main clock; Based on the size of the absolute time difference, the fusion weight is obtained by exponential decay calculation, and the multiple data points of the same sensor are weighted and averaged according to the fusion weight to obtain the multi-source synchronization data.

5. The SLAM mobile survey system data acquisition and processing method of claim 4, wherein, The SLAM positioning of the multi-source synchronization data and the construction of the sparse point cloud map comprise the following steps: Image feature extraction is performed on the multi-source synchronization data, and geometric feature data is recorded; The multi-source synchronization data is subjected to unmanned aerial vehicle pose analysis, and unmanned aerial vehicle pose data is recorded; According to the geometric feature data and the unmanned aerial vehicle pose data, a sparse point cloud map is constructed.

6. The SLAM mobile survey system data acquisition and processing method of claim 5, wherein, The construction of the sparse point cloud map according to the geometric feature data and the unmanned aerial vehicle pose data comprises the following steps: According to the motion speed and angular speed in the unmanned aerial vehicle pose data, the pose change rate is calculated, the current scene type is judged according to the pose change rate, and different sensor weights are assigned to the sensors according to the scene type; The motion blur coefficient is calculated according to the instantaneous angular speed of the unmanned aerial vehicle and the exposure time of the camera; According to the stability of each feature point in the geometric feature data in time or space to obtain a feature matching score; Fusing the feature matching score corresponding to each feature point in the geometric feature data with the sensor weight, and subtracting the motion blur coefficient to obtain a feature point list with reliability score; Inputting the RGB image in the geometric feature data into a pre-trained lightweight semantic segmentation model to obtain a pixel-level semantic label; For the region with the pixel-level semantic label being "vehicle" or "pedestrian", calculating a feature point motion vector in combination with the unmanned aerial vehicle motion direction; According to the comparison between the feature point motion vector and the unmanned aerial vehicle pose change, if the direction of the feature point motion vector is inconsistent with the unmanned aerial vehicle pose change, it is marked as a dynamic interference point; Filtering the dynamic interference point in the feature point list with reliability score to obtain a filtered static feature point list; Taking the filtered static feature point list as a basic data source, dividing the current unmanned aerial vehicle field of view into a preset size of uniform area grid, and each uniform area grid covers the same physical range; According to the semantic label distribution in the uniform area grid, judging the complexity, according to the complexity, retaining different numbers of filtered static feature points in the corresponding uniform area grid, and merging the filtered static feature points with a spatial distance less than a preset spatial threshold in the same grid to obtain an optimized sparse point cloud map.

7. The SLAM mobile survey system data acquisition and processing method of claim 6, wherein, The reconstruction of the dense point cloud map based on the sparse point cloud map and the modeling processing of the dense point cloud map to generate a three-dimensional space map specifically includes the following steps: Reconstructing the dense point cloud map based on the sparse point cloud map by using a structured light, a depth map or a stereo matching method; Performing voxel filtering and outlier rejection on the dense point cloud map; Performing point cloud registration and merging on the dense point cloud map; Converting the grid model and superimposing the texture to generate a three-dimensional space map.

8. The SLAM mobile survey system data acquisition and processing method of claim 7, wherein, The reconstruction of the dense point cloud map based on the sparse point cloud map by using a structured light, a depth map or a stereo matching method specifically includes the following steps: Extracting the semantic label of each feature point from the optimized sparse point cloud map to generate a semantic label distribution map; Defining a reconstruction area priority according to the semantic label distribution map to obtain an area priority strategy table; Based on the area priority strategy table, performing near-field high-resolution reconstruction, middle-field adaptive reconstruction and far-field low-power consumption reconstruction to obtain near-field point cloud, middle-field point cloud and far-field point cloud; The near-field high-resolution reconstruction specifically includes: projecting an encoded raster pattern to a target area, capturing a deformed raster image through a binocular camera, calculating the depth data of the deformed raster image according to the parallax of the left and right cameras to obtain an initial visual depth map, and generating a 0.5mm resolution point cloud by using the initial visual depth map to obtain near-field high-precision point cloud; The middle-field adaptive reconstruction specifically includes: adjusting the distance between the binocular cameras in real time according to the target distance, combining the initial visual depth map of multiple frames of images in the unmanned aerial vehicle flight, eliminating noise through weighted average to obtain a final visual depth map, and generating a middle-field adaptive point cloud by using the final visual depth map. The far-field low-power reconstruction specifically refers to: using the laser radar points in the optimized sparse point cloud map, generating a continuous surface through Bezier surface fitting, and limiting the interpolation point density to 10 mm interval in the continuous surface generation process to avoid over-refinement, and obtaining a far-field low-density point cloud; Based on the semantic label distribution map, the near-field point cloud, the middle-field point cloud and the far-field point cloud are aligned with the laser radar points in the optimized sparse point cloud map, and then are weightedly fused with the optimized sparse point cloud map to obtain a cross-modal optimized dense point cloud; According to the semantic label distribution map, a semantic boundary mask map is detected, an original high-resolution RGB image is obtained from the multi-source synchronous data, and the semantic boundary mask map is applied to obtain an RGB image of the boundary region; The RGB image of the boundary region is input into a lightweight GAN network for upsampling to obtain a super-resolution boundary image; Sub-pixel level disparity calculation is performed on the super-resolution image to generate a high-precision edge point cloud, and the boundary points in the cross-modal optimized dense point cloud are replaced by the high-precision edge point cloud to obtain a final optimized dense point cloud map.

9. The SLAM mobile survey system data acquisition and processing method of claim 8, wherein, The loop detection and map error optimization of the three-dimensional space map, the generation of an optimized three-dimensional map, and the application and display of the three-dimensional space map specifically include the following steps: Loop detection is performed on the three-dimensional space map to eliminate cumulative errors; Pose error is evaluated, and the map and the trajectory are corrected; An optimized three-dimensional map is generated; A map application platform is determined; The three-dimensional space map is applied and displayed on the map application platform.

10. A SLAM mobile measurement system data acquisition and processing system, the system being applied to the SLAM mobile measurement system data acquisition and processing method of any one of claims 1 to 9, characterized in that, The system includes a surveying task processing unit, a multi-source data acquisition unit, a sparse point cloud construction unit, a dense point cloud reconstruction unit and a map application display unit, wherein: The surveying task processing unit is configured to receive a UAV surveying task, set flight task parameters of a UAV and time synchronization configurations of multiple sensors, and initialize a SLAM mobile measurement system; The multi-source data acquisition unit is configured to acquire and pre-process multi-source synchronous data according to the flight task parameters and the time synchronization configurations; The sparse point cloud construction unit is configured to perform SLAM positioning on the multi-source synchronous data to construct a sparse point cloud map; The dense point cloud reconstruction unit is configured to reconstruct a dense point cloud map based on the sparse point cloud map, and perform modeling processing on the dense point cloud map to generate a three-dimensional space map; The map application display unit is configured to perform loop detection and map error optimization on the three-dimensional space map to generate an optimized three-dimensional map, and apply and display the three-dimensional space map.

Images