SLAM mobile measurement system data acquisition and processing method and system

Abstract

The embodiment of the invention relates to the technical field of unmanned aerial vehicle surveying and mapping, and particularly discloses an SLAM mobile measurement system data acquisition and processing method and system. According to the embodiment of the invention, flight mission parameters of the unmanned aerial vehicle and time synchronization configuration of multiple sensors are set; performing multi-source data acquisition and preprocessing to obtain multi-source synchronous data; constructing a sparse point cloud map; reconstructing a dense point cloud map, and carrying out modeling processing; loopback detection and graph error optimization are carried out, an optimized three-dimensional map is generated, and application display is carried out. Flight mission parameters of an unmanned aerial vehicle and time synchronization configuration of multiple sensors can be set, a dense point cloud map is reconstructed, modeling processing is carried out, a three-dimensional space map is generated, loopback detection and graph error optimization are carried out, and an optimized three-dimensional map is generated, applied and displayed, so that effective alignment of multi-source data can be realized, the time error is reduced, and the accuracy of the three-dimensional space map is improved. The precision and stability of the SLAM system are improved, the three-dimensional modeling precision can be improved, and the method can be used for high-quality surveying and mapping.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle (UAV) surveying and mapping, and in particular relates to a data acquisition and processing method and system for a SLAM mobile measurement system. Background Art

[0002] Drone surveying and mapping refers to the use of drone platforms equipped with high-precision sensors (such as cameras, lidar, GNSS, etc.) to obtain geospatial data during low-altitude autonomous or remote-controlled flight, and through data processing and modeling, generate digital orthophoto maps (DOM), digital elevation models (DEM), three-dimensional real-scene models (3D models) and other surveying and mapping results.

[0003] Compared with traditional surveying and mapping methods, drone surveying and mapping has the advantages of high operating efficiency, low cost, fast and flexible data acquisition, and adaptability to complex terrain. It is widely used in land and resources surveys, agricultural remote sensing, urban planning, disaster monitoring, engineering surveying and other fields.

[0004] In the existing technology, the data acquisition and processing of SLAM mobile measurement systems have the problem of timestamp errors or sampling frequency differences among multiple sensors, which makes it impossible to effectively align subsequent data. The accumulation of time errors affects the accuracy and stability of the SLAM system. In addition, dense reconstruction has the problem of excessive point cloud redundancy, resulting in insufficient three-dimensional modeling accuracy and difficulty in use for high-quality surveying and mapping. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a data acquisition and processing method and system for a SLAM mobile measurement system, aiming to solve the problems raised in the background technology.

[0006] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions: The SLAM mobile measurement system data acquisition and processing method specifically comprises the following steps: Receive UAV mapping tasks, set the UAV's flight mission parameters and multi-sensor time synchronization configuration, and initialize the SLAM mobile measurement system; Perform multi-source data acquisition and preprocessing according to the flight mission parameters and the time synchronization configuration to obtain multi-source synchronized data; Performing SLAM positioning on the multi-source synchronous data to construct a sparse point cloud map; Based on the sparse point cloud map, a dense point cloud map is reconstructed, and the dense point cloud map is modeled to generate a three-dimensional space map; Perform loop closure detection and graph error optimization on the three-dimensional space map to generate an optimized three-dimensional map, and apply and display the three-dimensional space map.

[0007] A SLAM mobile measurement system data acquisition and processing system, which is applied to the above-mentioned SLAM mobile measurement system data acquisition and processing method, and includes a surveying and mapping task processing unit, a multi-source data acquisition unit, a sparse point cloud construction unit, a dense point cloud reconstruction unit, and a map application display unit, wherein: The mapping task processing unit is used to receive UAV mapping tasks, set the UAV flight mission parameters and multi-sensor time synchronization configuration, and initialize the SLAM mobile measurement system; A multi-source data acquisition unit, configured to perform multi-source data acquisition and preprocessing according to the flight mission parameters and the time synchronization configuration, and obtain multi-source synchronized data; A sparse point cloud construction unit, configured to perform SLAM positioning on the multi-source synchronous data and construct a sparse point cloud map; A dense point cloud reconstruction unit is used to reconstruct a dense point cloud map based on the sparse point cloud map, and perform modeling processing on the dense point cloud map to generate a three-dimensional space map; The map application display unit is used to perform loop detection and map error optimization on the three-dimensional space map, generate an optimized three-dimensional map, and display the three-dimensional space map in an application.

[0008] Compared with the prior art, the present invention has the following beneficial effects: The embodiment of the present invention sets the flight mission parameters of the drone and the time synchronization configuration of multiple sensors; performs multi-source data acquisition and preprocessing to obtain multi-source synchronized data; constructs a sparse point cloud map; reconstructs a dense point cloud map and performs modeling processing; performs loop detection and graph error optimization to generate an optimized three-dimensional map, and then displays the application. The embodiment of the present invention can set the flight mission parameters of the drone and the time synchronization configuration of multiple sensors, reconstruct a dense point cloud map and perform modeling processing to generate a three-dimensional space map, then perform loop detection and graph error optimization to generate and display the optimized three-dimensional map. This can achieve effective alignment of multi-source data, reduce time error, improve the accuracy and stability of the SLAM system, and improve the accuracy of three-dimensional modeling, and can be used for high-quality surveying and mapping. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention.

[0010] Figure 1 A flow chart of a method provided by an embodiment of the present invention is shown.

[0011] Figure 2 The application architecture diagram of the system provided by the embodiment of the present invention is shown. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0013] It is understandable that in the existing technology, the data acquisition and processing of the SLAM mobile measurement system has the problem of timestamp error or sampling frequency difference among multiple sensors, which makes it impossible to effectively align subsequent data. The accumulation of time errors affects the accuracy and stability of the SLAM system, and the dense reconstruction has the problem of excessive point cloud redundancy, resulting in insufficient three-dimensional modeling accuracy and making it difficult to use for high-quality surveying and mapping.

[0014] To solve the above problems, the embodiment of the present invention receives a UAV surveying and mapping task, sets the UAV's flight mission parameters and the time synchronization configuration of multiple sensors, and initializes the SLAM mobile measurement system; performs multi-source data acquisition and preprocessing according to the flight mission parameters and the time synchronization configuration to obtain multi-source synchronous data; performs SLAM positioning on the multi-source synchronous data to construct a sparse point cloud map; reconstructs a dense point cloud map based on the sparse point cloud map, and performs modeling processing on the dense point cloud map to generate a three-dimensional space map; performs loop detection and graph error optimization on the three-dimensional space map to generate an optimized three-dimensional map, and then displays the three-dimensional space map in an application. The embodiment of the present invention can set the UAV's flight mission parameters and the time synchronization configuration of multiple sensors, reconstruct the dense point cloud map and perform modeling processing to generate a three-dimensional space map, then perform loop detection and graph error optimization, generate and display the optimized three-dimensional map in an application, and can achieve effective alignment of multi-source data, reduce time error, improve the accuracy and stability of the SLAM system, and improve the accuracy of three-dimensional modeling, which can be used for high-quality surveying and mapping.

[0015] Figure 1 A flow chart of a method provided by an embodiment of the present invention is shown.

[0016] Specifically, the SLAM mobile measurement system data acquisition and processing method includes the following steps: Step S101: receiving a UAV surveying and mapping mission, setting the UAV's flight mission parameters and multi-sensor time synchronization configuration, and initializing the SLAM mobile measurement system.

[0017] In an embodiment of the present invention, by receiving a UAV surveying and mapping task, the target surveying and mapping area and the type of surveying and mapping task are determined, and then the UAV's flight mission parameters (including flight path, altitude, speed, route overlap, etc.) are set according to the target surveying and mapping area. In addition, according to the type of surveying and mapping task, the time synchronization configuration of multiple sensors is set (specifically: configuring the synchronization protocol of sensors such as cameras, LiDAR, IMU, GNSS, and providing a unified time and space reference), and the SLAM mobile measurement system is initialized to initialize the UAV's position, posture, and map status, and obtain key SLAM parameters.

[0018] Specifically, in the preferred embodiment provided by the present invention, the receiving of the UAV surveying and mapping task, setting the UAV flight mission parameters and the time synchronization configuration of the multi-sensor, and initializing the SLAM mobile measurement system specifically includes the following steps: Receive UAV mapping tasks, determine the target mapping area and mapping task type; Setting the flight mission parameters of the UAV according to the target mapping area; According to the type of surveying and mapping task, set the time synchronization configuration of multiple sensors; Initialize the SLAM mobile measurement system, initialize the drone's position, attitude, map status, and obtain key SLAM parameters.

[0019] In a preferred embodiment of the present invention, setting the time synchronization configuration of multiple sensors according to the surveying and mapping task type specifically includes the following steps: Determine the type of surveying and mapping task, and select one of the sensors as the master sensor according to the surveying and mapping task type, the master sensor clock as the master clock, and the other sensors as slave sensors, the slave sensor clocks as slave clocks; Among them, the surveying and mapping task types include high-precision surveying and mapping, fast scanning, and the need to balance accuracy and speed. If the surveying and mapping task type is high-precision surveying and mapping, the IMU is selected as the master clock; if the surveying and mapping task type is fast scanning, the global clock of the lidar is selected as the master clock; if the surveying and mapping task type requires a balance between accuracy and speed, the priority score of each sensor clock is calculated, and the sensor clock with the highest priority score is selected as the master clock; According to the type of surveying and mapping task, different timestamp alignment modes are selected for alignment to obtain synchronization mode parameters and the corrected sensor timestamp table; Among them, different timestamp alignment modes include strict alignment mode and dynamic compensation mode. The strict alignment mode uses the phase-locked loop circuit of the FPGA chip to phase-lock other slave clock signals with the master clock and force the alignment of clock edges. The dynamic compensation mode calculates the compensation amount based on the sensor's inherent delay parameters and transmission delay, quantizes the supplementary amount according to the master clock, and obtains the dynamic compensation parameter table of the slave sensor timestamp.

[0020] In this embodiment of the present invention, high-precision mapping tasks require extremely high time synchronization accuracy. Inertial measurement unit (IMU) data is typically collected at a high frequency and has stable timing. Therefore, the IMU clock is selected as the master clock to ensure strict synchronization of inertial data with other sensor data. Rapid scanning tasks emphasize data acquisition speed. LiDAR is the primary data source. The LiDAR's global clock is selected as the master clock to ensure accurate timestamps for laser point cloud data and support fast real-time processing. Balanced mode combines sensor performance indicators, calculates clock priority scores, and selects the most appropriate clock as the master clock, balancing accuracy and efficiency.

[0021] Different timestamp alignment strategies can be adopted based on different task requirements: Strict Alignment Mode: This locks the signal edges of all slave clocks to the master clock through hardware means (such as the phase-locked loop (PLL) within the FPGA chip), achieving hardware-level phase synchronization. Timestamps are forced to be aligned in hardware, making it suitable for surveying and mapping tasks that require extremely high time accuracy. Dynamic Compensation Mode: This calculates compensation based on the sensor's inherent delay parameters (such as signal acquisition and transmission delay), dynamically adjusting the slave sensor timestamps based on the master clock. This is suitable for scenarios with high real-time requirements and low requirements for strict hardware phase locking.

[0022] Therefore, the multi-sensor time synchronization configuration of selecting the master clock and synchronization mode based on the surveying and mapping task type, combined with strict alignment of the hardware phase-locked loop and dynamic compensation of the software can ensure the time consistency of multi-sensor data, improve the surveying and mapping accuracy, flexibly adapt to the requirements of different surveying and mapping tasks, and expand the scope of application of the system.

[0023] Furthermore, the SLAM mobile measurement system data acquisition and processing method further includes the following steps: Step S102 : performing multi-source data collection and preprocessing according to the flight mission parameters and the time synchronization configuration to obtain multi-source synchronized data.

[0024] In an embodiment of the present invention, multi-source data acquisition is performed according to flight mission parameters and time synchronization configuration to obtain multi-source acquired data (including multi-source data such as images, IMU, point clouds, etc.), and the timestamp corresponding to the multi-source acquired data is determined, the timestamp data is recorded, and noise filtering processing is performed on the multi-source acquired data to obtain multi-source filtered data. Then, according to the timestamp data, the multi-source filtered data is synchronized and timestamp aligned to obtain multi-source synchronized data.

[0025] Specifically, in a preferred embodiment of the present invention, performing multi-source data acquisition and preprocessing according to the flight mission parameters and the time synchronization configuration, and obtaining multi-source synchronized data specifically include the following steps: Perform multi-source data collection according to the flight mission parameters and the time synchronization configuration to obtain multi-source collected data; Determine the timestamps corresponding to the multi-source collected data and record the timestamp data; Performing noise filtering on the multi-source collected data to obtain multi-source filtered data; According to the timestamp data, a synchronization process of timestamp alignment is performed on the multi-source filtered data to obtain multi-source synchronized data.

[0026] In a preferred embodiment of the present invention, performing synchronization processing of timestamp alignment on the multi-source filtered data according to the timestamp data to obtain multi-source synchronized data specifically includes the following steps: Extract the hardware trigger moment of the main sensor as the reference point of the global time axis, generate a time series at fixed intervals, and obtain the global time axis sequence of the main sensor; Read the compensation amount and clock drift rate of each slave sensor from the slave sensor timestamp dynamic compensation parameter table; The first arrival time of the slave sensor data recorded when the system is started is used as the initial moment, and the original timestamp of the slave sensor plus the compensation amount of the slave sensor is added as the current timestamp; Based on the initial time and the current timestamp, the running time of the slave sensor since the initial time is calculated, and the clock drift rate that increases linearly with time is superimposed to obtain the corrected timestamp data of the slave sensor; When the master sensor hardware trigger moment arrives, the dynamic window width is calculated based on the current highest sampling frequency of the slave sensor. Based on the dynamic window width, the window width is expanded forward and backward along the global time axis sequence of the master sensor with the master sensor hardware trigger moment as the center to obtain the dynamic window range corresponding to the master sensor trigger moment. Traverse the filtered data of all slave sensors, extract the data points of the corrected timestamp data of the slave sensors whose timestamps fall within the dynamic window range, and obtain the data point set of the slave sensors within the dynamic window range; Calculate the absolute time difference between the corrected timestamp of each data point in the set of data points from the sensor within the dynamic window and the master clock; Based on the size of the absolute time difference, the exponential decay calculation is used to obtain the fusion weight. Multiple data points of the same sensor are weighted averaged according to the fusion weight to obtain multi-source synchronous data.

[0027] In an embodiment of the present invention, the hardware trigger moment of the master sensor is used as the global time axis benchmark, and the timestamp of the slave sensor is corrected in real time through fixed delay compensation and clock drift rate to eliminate the long-term error caused by crystal temperature drift. The window width is dynamically adjusted according to the sampling frequency of the slave sensor. For example, the window width of a 100Hz GNSS sensor is 5ms, while the window width of a 1000Hz lidar is 0.5ms, to avoid truncation of high-frequency sensor data or noise introduced by an excessively wide window of a low-frequency sensor. An exponential decay function is used to calculate the weight, and the smaller the time difference, the higher the weight of the data, to ensure alignment accuracy. Through the three-layer collaboration of "hardware benchmark + software optimization + closed-loop correction", the three core problems of frequency difference, clock drift, and data loss in multi-sensor synchronization are solved, providing high-precision and high-robustness time synchronization guarantee for the SLAM mobile measurement system.

[0028] Furthermore, the SLAM mobile measurement system data acquisition and processing method further includes the following steps: Step S103: Perform SLAM positioning on the multi-source synchronous data to construct a sparse point cloud map.

[0029] In an embodiment of the present invention, image feature extraction is performed on multi-source synchronous data (key points are extracted from the image, and inter-frame matching or optical flow tracking is performed), geometric feature data is recorded, and drone pose analysis is performed on the multi-source synchronous data (UAV pose is estimated by jointly using IMU and vision / laser data, and a nonlinear optimization algorithm is used to improve the estimation accuracy), the drone pose data is recorded, and then a sparse point cloud map is constructed based on the geometric feature data and the drone pose data (a preliminary sparse point cloud map is constructed based on the estimated pose back-projection feature points as the basis for global mapping).

[0030] Specifically, in a preferred embodiment provided by the present invention, performing SLAM positioning on the multi-source synchronous data and constructing a sparse point cloud map specifically includes the following steps: Performing image feature extraction on the multi-source synchronous data and recording geometric feature data; Performing drone posture analysis on the multi-source synchronous data and recording drone posture data; A sparse point cloud map is constructed based on the geometric feature data and the drone pose data.

[0031] In a preferred embodiment of the present invention, constructing a sparse point cloud map based on the geometric feature data and the drone pose data specifically includes the following steps: Based on the motion speed and angular velocity in the drone's posture data, the posture change rate is calculated. The current scene type is determined based on the posture change rate, and different sensor weights are assigned to the sensors based on the scene type. Scene types include dynamic scenes and static scenes. In static scenes, the lidar weight 0.7 and the camera weight 0.3 are read from the parameter table. In dynamic scenes, the IMU weight 0.6 and the lidar weight 0.4 are read. If a feature point is observed by multiple sensors (for example, both the lidar and the camera detect the same corner point), the weighted average is taken.

[0032] The motion blur coefficient is calculated based on the instantaneous angular velocity of the drone and the exposure time of the camera; Obtaining a feature matching score based on the temporal or spatial stability of each feature point in the geometric feature data; The feature matching score corresponding to each feature point in the geometric feature data is fused with the sensor weight, and the motion blur coefficient is subtracted to obtain a feature point list with reliability scores; Input the RGB image in the geometric feature data into the pre-trained lightweight semantic segmentation model to obtain pixel-level semantic labels; For areas with pixel-level semantic labels of "vehicle" or "pedestrian", the feature point motion vector is calculated based on the drone's motion direction; Compare the feature point motion vector with the drone's posture change. If the direction of the feature point motion vector is inconsistent with the drone's posture change, it is marked as a dynamic interference point. Filter the dynamic interference points in the feature point list with reliability scores to obtain a filtered static feature point list; The filtered static feature point list is used as the basic data source and is divided into uniform area grids of preset size according to the current drone field of view. Each uniform area grid covers the same physical range. The complexity is judged according to the distribution of semantic labels in the uniform area grid. According to the complexity, different numbers of filtered static feature points are retained in the corresponding uniform area grid. The filtered static feature points in the same grid whose spatial distance between feature points from different sensors is less than a preset spatial threshold are merged to obtain the optimized sparse point cloud map.

[0033] In this embodiment, confidence weights are assigned to different sensors based on the mission type (static / dynamic). For example, IMU data is more reliable in dynamic scenarios, while LiDAR data is more accurate in static scenarios. Furthermore, the ambiguity coefficient is calculated using the drone's angular velocity and camera exposure time to reduce the impact of visual feature errors during high-speed motion.

[0034] In addition, the low-computational-overhead semantic segmentation network can quickly identify dynamic objects such as vehicles and pedestrians in the image, compare the movement direction of feature points with the direction of change of the drone's posture, and eliminate abnormal moving points to achieve semantic-assisted dynamic object filtering.

[0035] By dividing the field of view into grids, the number of features retained in each grid is dynamically controlled based on semantic labels (such as buildings and ground), avoiding redundancy. Multi-source features at the same location (such as lidar corner points and visual corner points) are fused with coordinates and descriptors, skipping reconstruction of unstructured areas and improving feature uniqueness and recognition.

[0036] Furthermore, the SLAM mobile measurement system data acquisition and processing method further includes the following steps: Step S104: reconstructing a dense point cloud map based on the sparse point cloud map, and performing modeling processing on the dense point cloud map to generate a three-dimensional space map.

[0037] In an embodiment of the present invention, based on a sparse point cloud map, a dense point cloud map is reconstructed using structured light, depth map or stereo matching (the dense point cloud is reconstructed using structured light, depth map or stereo matching, and adjacent frame images and depth information are fused to improve reconstruction accuracy), and then the dense point cloud map is voxel filtered and outliers are removed, and the dense point cloud map is point cloud registered and merged (ICP or NDT algorithm is used for point cloud registration and merging), and then the processed point cloud is converted into a mesh model, and texture is superimposed to generate a three-dimensional space map.

[0038] Specifically, in a preferred embodiment of the present invention, reconstructing a dense point cloud map based on the sparse point cloud map, and performing modeling processing on the dense point cloud map to generate a three-dimensional space map specifically includes the following steps: Based on the sparse point cloud map, a dense point cloud map is reconstructed using structured light, depth map or stereo matching; Performing voxel filtering and outlier removal on the dense point cloud map; Performing point cloud registration and merging on the dense point cloud map; Transform the mesh model and overlay textures to generate a 3D space map.

[0039] In a preferred embodiment of the present invention, the reconstructing of a dense point cloud map based on the sparse point cloud map by using structured light, depth map or stereo matching specifically includes the following steps: Extract the semantic label of each feature point from the optimized sparse point cloud map and generate a semantic label distribution map; Define the reconstruction region priority according to the semantic label distribution map and obtain the region priority strategy table; Based on the regional priority strategy table, near-field high-resolution reconstruction, mid-field adaptive reconstruction, and far-field low-power reconstruction are performed to obtain near-field point cloud, mid-field point cloud, and far-field point cloud; The near-field high-resolution reconstruction is specifically as follows: projecting a coded grating pattern onto the target area, capturing the deformed grating image with a binocular camera, calculating the depth data of the deformed grating image based on the parallax of the left and right cameras to obtain an initial visual depth map, and using the initial visual depth map to generate a 0.5mm resolution point cloud to obtain a near-field high-precision point cloud. The midfield adaptive reconstruction is specifically done by adjusting the binocular camera spacing in real time according to the target distance, combining the initial visual depth map of multiple frames of the drone during flight, eliminating noise through weighted averaging to obtain the final visual depth map, and using the final visual depth map to generate a midfield adaptive point cloud. Far-field low-power reconstruction specifically involves using the lidar points in the optimized sparse point cloud map to generate a continuous surface through Bezier surface fitting. During the continuous surface generation process, the interpolation point density is limited to 10mm intervals to avoid over-refinement, resulting in a far-field low-density point cloud. Based on the semantic label distribution map, the near-field point cloud, mid-field point cloud, and far-field point cloud are aligned with the lidar points in the optimized sparse point cloud map, and then weighted fused with the optimized sparse point cloud map to obtain a dense point cloud after cross-modal optimization. Detect semantic label mutation areas based on the semantic label distribution map and obtain a semantic boundary mask map; Obtain original high-resolution RGB images from multi-source synchronized data and apply semantic boundary mask maps to obtain RGB images of boundary areas; The RGB image of the boundary area is input into the lightweight GAN network and upsampled to obtain a super-resolution boundary image; Sub-pixel disparity calculation is performed on the super-resolution image to generate a high-precision edge point cloud, and the boundary points in the dense point cloud after cross-modal optimization are replaced with the high-precision edge point cloud to obtain the final optimized dense point cloud map.

[0040] In this embodiment of the present invention, semantic labels (such as buildings and vegetation) directly influence the weighting of the first step: high-priority semantic regions (such as buildings) are assigned a higher visual weight (0.8) during fusion to preserve detail; low-priority regions (such as vegetation) receive a lower visual weight (0.4) to avoid over-optimization of non-critical areas. Furthermore, semantic labels help select high-confidence alignment reference points (such as building corners and road markings) in step 3, rather than randomly selecting points, thereby improving ICP registration accuracy.

[0041] Furthermore, through layered ICP alignment, we eliminate scale and position deviations caused by different reconstruction strategies, ensuring seamless splicing of multi-resolution point clouds. The near field relies on visual details, the far field relies on lidar stability, and the midfield acts as a transition layer. By weight fusion, we avoid model discontinuities caused by resolution mutations, giving full play to the respective advantages of multimodal data. The resulting cross-modal optimized point cloud has high global consistency, providing an accurate spatial benchmark for subsequent semantic edge super-resolution enhancement. The cross-modal optimized point cloud provides a coarse calibration point cloud, on which the semantic boundary is locally enhanced, greatly improving the accuracy of key areas such as the building-vegetation interface and improving the overall modeling accuracy. Furthermore, the SLAM mobile measurement system data acquisition and processing method further includes the following steps: Step S105 , performing loop detection and graph error optimization on the three-dimensional space map to generate an optimized three-dimensional map, and applying and displaying the three-dimensional space map.

[0042] In an embodiment of the present invention, loop closure detection is performed on the three-dimensional space map (using a loop closure detection algorithm to construct a graph optimization factor graph), and then the back-end optimization algorithm is used to eliminate the accumulated error, and the posture error is evaluated, and the map and trajectory are corrected (comparing the GNSS real trajectory with the SLAM estimated trajectory, evaluating the posture error, and correcting the map and trajectory based on the error), to generate an optimized three-dimensional map, and then determine the map application platform, and then apply the three-dimensional space map to display it on the map application platform.

[0043] Specifically, in a preferred embodiment of the present invention, performing loop detection and graph error optimization on the three-dimensional space map to generate an optimized three-dimensional map, and applying and displaying the three-dimensional space map specifically includes the following steps: Performing loop closure detection on the three-dimensional space map to eliminate accumulated errors; Evaluate pose errors and correct maps and trajectories; Generate optimized 3D maps; Determine the map application platform; On the map application platform, the three-dimensional space map is displayed as an application.

[0044] Further, Figure 2 The application architecture diagram of the system provided by the embodiment of the present invention is shown.

[0045] Among them, in another preferred embodiment provided by the present invention, a SLAM mobile measurement system data acquisition and processing system is applied to the above-mentioned SLAM mobile measurement system data acquisition and processing method, and the system includes a surveying and mapping task processing unit 101, a multi-source data acquisition unit 102, a sparse point cloud construction unit 103, a dense point cloud reconstruction unit 104 and a map application display unit 105, wherein: The surveying and mapping task processing unit 101 is used to receive the UAV surveying and mapping task, set the UAV flight mission parameters and the time synchronization configuration of multiple sensors, and initialize the SLAM mobile measurement system.

[0046] In an embodiment of the present invention, the surveying and mapping task processing unit 101 determines the target surveying and mapping area and the surveying and mapping task type by receiving the UAV surveying and mapping task, and then sets the UAV's flight mission parameters (including flight path, altitude, speed, and route overlap, etc.) according to the target surveying and mapping area, and sets the time synchronization configuration of multiple sensors according to the surveying and mapping task type (specifically: configuring the synchronization protocol of sensors such as cameras, LiDAR, IMU, GNSS, and providing a unified time and space reference), and initializes the SLAM mobile measurement system, initializes the UAV's position, attitude, and map status, and obtains key SLAM parameters.

[0047] The multi-source data acquisition unit 102 is used to perform multi-source data acquisition and pre-processing according to the flight mission parameters and the time synchronization configuration to obtain multi-source synchronized data.

[0048] In an embodiment of the present invention, the multi-source data acquisition unit 102 performs multi-source data acquisition in accordance with the flight mission parameters and time synchronization configuration, acquires multi-source acquired data (including multi-source data such as images, IMUs, and point clouds), determines the timestamps corresponding to the multi-source acquired data, records the timestamp data, and performs noise filtering on the multi-source acquired data to obtain multi-source filtered data. Furthermore, the multi-source filtered data is synchronized and timestamp-aligned according to the timestamp data to obtain multi-source synchronized data.

[0049] The sparse point cloud construction unit 103 is used to perform SLAM positioning on the multi-source synchronous data and construct a sparse point cloud map.

[0050] In an embodiment of the present invention, the sparse point cloud construction unit 103 performs image feature extraction on multi-source synchronous data (extracts key points in the image, performs inter-frame matching or optical flow tracking), records geometric feature data, and performs drone pose analysis on the multi-source synchronous data (uses IMU and vision / laser data to jointly estimate the drone pose, and adopts a nonlinear optimization algorithm to improve the estimation accuracy), records the drone pose data, and then constructs a sparse point cloud map based on the geometric feature data and the drone pose data (constructs a preliminary sparse point cloud map based on the estimated pose back-projection feature points as the basis for global mapping).

[0051] The dense point cloud reconstruction unit 104 is configured to reconstruct a dense point cloud map based on the sparse point cloud map, and perform modeling processing on the dense point cloud map to generate a three-dimensional space map.

[0052] In an embodiment of the present invention, the dense point cloud reconstruction unit 104 reconstructs a dense point cloud map based on the sparse point cloud map using structured light, depth map or stereo matching (using structured light, depth map or stereo matching to reconstruct the dense point cloud, and introducing the fusion of adjacent frame images and depth information to improve the reconstruction accuracy), then performs voxel filtering and outlier removal on the dense point cloud map, and performs point cloud registration and merging on the dense point cloud map (using ICP or NDT algorithm for point cloud registration and merging), and then converts the processed point cloud into a mesh model, and superimposes texture to generate a three-dimensional space map.

[0053] The map application display unit 105 is configured to perform loop detection and map error optimization on the three-dimensional space map, generate an optimized three-dimensional map, and display the three-dimensional space map in an application.

[0054] In an embodiment of the present invention, the map application display unit 105 performs loop detection on the three-dimensional space map (using a loop detection algorithm to construct a graph optimization factor graph), then eliminates the accumulated error through a back-end optimization algorithm, evaluates the posture error, corrects the map and trajectory (compares the GNSS real trajectory with the SLAM estimated trajectory, evaluates the posture error, and corrects the map and trajectory based on the error), generates an optimized three-dimensional map, and then determines the map application platform, and then displays the three-dimensional space map on the map application platform.

[0055] It should be understood that, although the various steps in the flow chart of each embodiment of the present invention are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0056] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When executed, the program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0057] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. The data acquisition and processing method of the SLAM mobile measurement system is characterized in that: The method specifically comprises the following steps: Receive UAV mapping tasks, set the UAV's flight mission parameters and multi-sensor time synchronization configuration, and initialize the SLAM mobile measurement system; Perform multi-source data acquisition and preprocessing according to the flight mission parameters and the time synchronization configuration to obtain multi-source synchronized data; Performing SLAM positioning on the multi-source synchronous data to construct a sparse point cloud map; Based on the sparse point cloud map, a dense point cloud map is reconstructed, and the dense point cloud map is modeled to generate a three-dimensional space map; Performing loop detection and graph error optimization on the three-dimensional space map to generate an optimized three-dimensional map, and applying and displaying the three-dimensional space map; The receiving of the UAV mapping task, setting the UAV flight mission parameters and the time synchronization configuration of the multi-sensor, and initializing the SLAM mobile measurement system specifically includes the following steps: Receive UAV mapping tasks, determine the target mapping area and mapping task type; Setting the flight mission parameters of the UAV according to the target mapping area; According to the type of surveying and mapping task, set the time synchronization configuration of multiple sensors; Initialize the SLAM mobile measurement system, initialize the drone's position, attitude, map status, and obtain key SLAM parameters.

2. The SLAM mobile measurement system data acquisition and processing method according to claim 1, wherein The step of setting the time synchronization configuration of multiple sensors according to the surveying and mapping task type specifically includes the following steps: Determine the type of surveying and mapping task, and select one of the sensors as the master sensor according to the surveying and mapping task type, the master sensor clock as the master clock, and the other sensors as slave sensors, the slave sensor clocks as slave clocks; Among them, the surveying and mapping task types include high-precision surveying and mapping, fast scanning, and the need to balance accuracy and speed. If the surveying and mapping task type is high-precision surveying and mapping, the IMU is selected as the master clock; if the surveying and mapping task type is fast scanning, the global clock of the lidar is selected as the master clock; if the surveying and mapping task type requires a balance between accuracy and speed, the priority score of each sensor clock is calculated, and the sensor clock with the highest priority score is selected as the master clock; According to the type of surveying and mapping task, different timestamp alignment modes are selected for alignment to obtain synchronization mode parameters and the corrected sensor timestamp table; Among them, different timestamp alignment modes include strict alignment mode and dynamic compensation mode. The strict alignment mode uses the phase-locked loop circuit of the FPGA chip to phase-lock other slave clock signals with the master clock and force the clock edges to be aligned; the dynamic compensation mode calculates the compensation amount based on the sensor's inherent delay parameters and transmission delay, quantizes the supplementary amount according to the master clock, and obtains the dynamic compensation parameter table of the slave sensor timestamp.

3. The SLAM mobile measurement system data acquisition and processing method according to claim 2, wherein The multi-source data acquisition and preprocessing according to the flight mission parameters and the time synchronization configuration to obtain multi-source synchronized data specifically includes the following steps: Perform multi-source data collection according to the flight mission parameters and the time synchronization configuration to obtain multi-source collected data; Determine the timestamps corresponding to the multi-source collected data and record the timestamp data; Performing noise filtering on the multi-source collected data to obtain multi-source filtered data; According to the timestamp data, a synchronization process of timestamp alignment is performed on the multi-source filtered data to obtain multi-source synchronized data.

4. The SLAM mobile measurement system data acquisition and processing method according to claim 3, wherein The step of performing synchronization processing of timestamp alignment on the multi-source filtered data according to the timestamp data to obtain multi-source synchronized data specifically includes the following steps: Extract the hardware trigger moment of the main sensor as the reference point of the global time axis, generate a time series at fixed intervals, and obtain the global time axis sequence of the main sensor; Read the compensation amount and clock drift rate of each slave sensor from the slave sensor timestamp dynamic compensation parameter table; The first arrival time of the slave sensor data recorded when the system is started is used as the initial moment, and the original timestamp of the slave sensor plus the compensation amount of the slave sensor is added as the current timestamp; Based on the initial time and the current timestamp, the running time of the slave sensor since the initial time is calculated, and the clock drift rate that increases linearly with time is superimposed to obtain the corrected timestamp data of the slave sensor; When the master sensor hardware trigger moment arrives, the dynamic window width is calculated based on the current highest sampling frequency of the slave sensor. Based on the dynamic window width, the window width is expanded forward and backward along the global time axis sequence of the master sensor with the master sensor hardware trigger moment as the center to obtain the dynamic window range corresponding to the master sensor trigger moment. Traverse the filtered data of all slave sensors, extract the data points of the corrected timestamp data of the slave sensors whose timestamps fall within the dynamic window range, and obtain the data point set of the slave sensors within the dynamic window range; Calculate the absolute time difference between the corrected timestamp of each data point in the set of data points from the sensor within the dynamic window and the master clock; Based on the size of the absolute time difference, the exponential decay calculation is used to obtain the fusion weight. Multiple data points of the same sensor are weighted averaged according to the fusion weight to obtain multi-source synchronous data.

5. The SLAM mobile measurement system data acquisition and processing method according to claim 4, wherein The performing SLAM positioning on the multi-source synchronous data to construct a sparse point cloud map specifically includes the following steps: Performing image feature extraction on the multi-source synchronous data and recording geometric feature data; Performing drone posture analysis on the multi-source synchronous data and recording drone posture data; A sparse point cloud map is constructed based on the geometric feature data and the drone pose data.

6. The SLAM mobile measurement system data acquisition and processing method according to claim 5, wherein: The step of constructing a sparse point cloud map based on the geometric feature data and the drone pose data specifically includes the following steps: Calculate the pose change rate based on the motion speed and angular velocity in the drone pose data, determine the current scene type based on the pose change rate, and assign different sensor weights to the sensors based on the scene type; The motion blur coefficient is calculated based on the instantaneous angular velocity of the drone and the exposure time of the camera; Obtain a feature matching score based on the temporal or spatial stability of each feature point in the geometric feature data; The feature matching score corresponding to each feature point in the geometric feature data is fused with the sensor weight, and the motion blur coefficient is subtracted to obtain a feature point list with reliability scores; Input the RGB image in the geometric feature data into the pre-trained lightweight semantic segmentation model to obtain pixel-level semantic labels; For areas with pixel-level semantic labels of "vehicle" or "pedestrian", the feature point motion vector is calculated based on the drone's motion direction; Compare the feature point motion vector with the drone's posture change. If the direction of the feature point motion vector is inconsistent with the drone's posture change, it is marked as a dynamic interference point. Filter the dynamic interference points in the feature point list with reliability scores to obtain a filtered static feature point list; The filtered static feature point list is used as the basic data source and is divided into uniform area grids of preset size according to the current drone field of view. Each uniform area grid covers the same physical range. The complexity is judged according to the distribution of semantic labels in the uniform area grid. According to the complexity, different numbers of filtered static feature points are retained in the corresponding uniform area grid. The filtered static feature points in the same grid whose spatial distance between feature points from different sensors is less than a preset spatial threshold are merged to obtain the optimized sparse point cloud map.

7. The SLAM mobile measurement system data acquisition and processing method according to claim 6, wherein: The process of reconstructing a dense point cloud map based on the sparse point cloud map and performing modeling on the dense point cloud map to generate a three-dimensional space map specifically includes the following steps: Based on the sparse point cloud map, a dense point cloud map is reconstructed using structured light, depth map or stereo matching; Performing voxel filtering and outlier removal on the dense point cloud map; Performing point cloud registration and merging on the dense point cloud map; Transform the mesh model and overlay textures to generate a 3D space map.

8. The SLAM mobile measurement system data acquisition and processing method according to claim 7, wherein: The method of reconstructing a dense point cloud map based on the sparse point cloud map by using structured light, depth map or stereo matching specifically includes the following steps: Extract the semantic label of each feature point from the optimized sparse point cloud map and generate a semantic label distribution map; Define the reconstruction region priority according to the semantic label distribution map and obtain the region priority strategy table; Based on the regional priority strategy table, near-field high-resolution reconstruction, mid-field adaptive reconstruction, and far-field low-power reconstruction are performed to obtain near-field point cloud, mid-field point cloud, and far-field point cloud; The near-field high-resolution reconstruction is specifically as follows: projecting a coded grating pattern onto the target area, capturing the deformed grating image with a binocular camera, calculating the depth data of the deformed grating image based on the parallax of the left and right cameras to obtain an initial visual depth map, and using the initial visual depth map to generate a 0.5mm resolution point cloud to obtain a near-field high-precision point cloud. The midfield adaptive reconstruction is specifically done by adjusting the binocular camera spacing in real time according to the target distance, combining the initial visual depth map of multiple frames of the drone during flight, eliminating noise through weighted averaging to obtain the final visual depth map, and using the final visual depth map to generate a midfield adaptive point cloud. Far-field low-power reconstruction specifically involves using the lidar points in the optimized sparse point cloud map to generate a continuous surface through Bezier surface fitting. During the continuous surface generation process, the interpolation point density is limited to 10mm intervals to avoid over-refinement, resulting in a far-field low-density point cloud. Based on the semantic label distribution map, the near-field point cloud, mid-field point cloud, and far-field point cloud are aligned with the lidar points in the optimized sparse point cloud map, and then weighted fused with the optimized sparse point cloud map to obtain a dense point cloud after cross-modal optimization. Detect semantic label mutation areas based on the semantic label distribution map and obtain a semantic boundary mask map; obtain the original high-resolution RGB image from multi-source synchronized data, and apply the semantic boundary mask map to obtain the RGB image of the boundary area; The RGB image of the boundary area is input into the lightweight GAN network and upsampled to obtain a super-resolution boundary image; Sub-pixel disparity calculation is performed on the super-resolution image to generate a high-precision edge point cloud, and the boundary points in the dense point cloud after cross-modal optimization are replaced with the high-precision edge point cloud to obtain the final optimized dense point cloud map.

9. The SLAM mobile measurement system data acquisition and processing method according to claim 8, wherein The performing loop detection and graph error optimization on the three-dimensional space map to generate an optimized three-dimensional map, and applying and displaying the three-dimensional space map specifically includes the following steps: Performing loop closure detection on the three-dimensional space map to eliminate accumulated errors; Evaluate pose errors and correct maps and trajectories; Generate optimized 3D maps; Determine the map application platform; On the map application platform, the three-dimensional space map is displayed as an application.

10. A SLAM mobile measurement system data acquisition and processing system, the system being applied to the SLAM mobile measurement system data acquisition and processing method according to any one of claims 1 to 9, characterized in that: The system includes a surveying and mapping task processing unit, a multi-source data acquisition unit, a sparse point cloud construction unit, a dense point cloud reconstruction unit, and a map application display unit, wherein: The mapping task processing unit is used to receive UAV mapping tasks, set the UAV flight mission parameters and multi-sensor time synchronization configuration, and initialize the SLAM mobile measurement system; A multi-source data acquisition unit, configured to perform multi-source data acquisition and preprocessing according to the flight mission parameters and the time synchronization configuration, and obtain multi-source synchronized data; A sparse point cloud construction unit, configured to perform SLAM positioning on the multi-source synchronous data and construct a sparse point cloud map; A dense point cloud reconstruction unit is used to reconstruct a dense point cloud map based on the sparse point cloud map, and perform modeling processing on the dense point cloud map to generate a three-dimensional space map; The map application display unit is used to perform loop detection and map error optimization on the three-dimensional space map, generate an optimized three-dimensional map, and display the three-dimensional space map in an application.

Images