A tunnel three-dimensional map construction method and system based on multi-source sensors

Abstract

The application provides a tunnel three-dimensional map construction method and system based on a multi-source sensor, belongs to the technical field of tunnel robot environment perception and intelligent surveying and mapping, and comprises the following steps: a multi-source heterogeneous sensor parameter adaptive mechanism is used to dynamically adjust multi-source heterogeneous sensor parameters and collect multi-source sensor data; the multi-source sensor data is synchronized and registered, and a fused point cloud is obtained through weighted fusion based on the synchronized and registered sensor data; a semantic segmentation model is used to identify semantic information of the fused point cloud, and a local map is obtained by optimizing a map label according to the semantic information; and a global tunnel three-dimensional map is obtained through fusion based on a plurality of local maps. The application provides a high-precision, strong-robustness and semantic-information-rich three-dimensional environment map for tunnel robot navigation.

Description

Technical Field

[0001] This invention belongs to the field of tunnel robot environmental perception and intelligent mapping technology, and particularly relates to a method and system for constructing a three-dimensional tunnel map based on multi-source sensors. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] As the scale and complexity of tunnel engineering projects gradually increase, tunnel construction and operation often face extreme environmental challenges. Tunnel robots, as an important piece of equipment to replace human labor in high-risk environments, have been widely used in tunnel inspection, structural testing, and emergency rescue scenarios. Their autonomous navigation capabilities are highly dependent on high-precision three-dimensional environmental maps.

[0004] Existing methods for constructing 3D tunnel maps primarily rely on a single sensor, which is prone to data loss or feature mismatch in environments with high dust concentrations and poor lighting conditions, leading to decreased map accuracy. Furthermore, traditional methods lack semantic understanding of the tunnel scene, making it difficult to meet the semantic obstacle avoidance and path planning requirements of robot navigation. Summary of the Invention

[0005] To overcome the shortcomings of the existing technologies, this invention proposes a method and system for constructing 3D tunnel maps based on multi-source sensors. This method addresses the problems of low accuracy, delayed updates, and semantic loss in traditional tunnel map construction methods, thereby providing high-precision, robust, and semantically rich 3D environmental maps for tunnel robot navigation. It is suitable for the real-time generation and dynamic updating of high-precision environmental maps for robot navigation in tunnel construction, inspection, and patrol scenarios.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, the present invention discloses a method for constructing a three-dimensional tunnel map based on multi-source sensors, comprising: A multi-source heterogeneous sensor parameter adaptive mechanism is adopted to dynamically adjust the parameters of the multi-source heterogeneous sensor and collect multi-source sensor data. The multi-source sensor data is synchronized and registered, and a fused point cloud is obtained by weighted fusion of the synchronized and registered sensor data. A semantic segmentation model is used to identify semantic information from the fused point cloud, and map labels are optimized based on the semantic information to obtain a local map. A global 3D map of the tunnel is obtained by fusing several local maps.

[0007] Secondly, this invention discloses a tunnel 3D map construction system based on multi-source sensors, comprising: The data acquisition module is used to dynamically adjust the parameters of the multi-source heterogeneous sensors and collect multi-source sensor data using a multi-source heterogeneous sensor parameter adaptive mechanism. The data fusion module is used to synchronize and register the multi-source sensor data, and to obtain a fused point cloud based on the weighted fusion of the synchronized and registered sensor data. The semantic segmentation module is used to identify semantic information from the fused point cloud using a semantic segmentation model, and to optimize map labels based on the semantic information to obtain a local map. The map generation module is used to obtain a global 3D map of the tunnel by fusing several local maps.

[0008] Thirdly, the present invention discloses an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, complete the steps of the above-mentioned method for constructing a tunnel 3D map based on multi-source sensors.

[0009] Fourthly, the present invention discloses a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the above-described method for constructing a three-dimensional tunnel map based on multi-source sensors.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention enables the sensor to acquire accurate and comprehensive data through dynamic adaptive sensor parameter adjustment and multimodal data fusion.

[0011] This invention first performs semantic segmentation on key structures within the tunnel through semantic annotation and optimization, assigning preliminary scene labels to the map to facilitate obstacle avoidance or path planning for robots; furthermore, a voting mechanism is used to optimize the labels and improve accuracy.

[0012] The cloud-edge collaborative map building process uses the edge to generate local maps and the cloud to generate and optimize global maps, which improves both the accuracy and speed of map generation, enabling tunnel robots to update accurate maps in real time.

[0013] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0014] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0015] Figure 1This is a flowchart of the tunnel 3D map construction method based on multi-source sensors described in Embodiment 1 of the present invention.

[0016] Figure 2 This is a schematic diagram of semantic segmentation and annotation of tunnel point clouds in Embodiment 1 of the present invention. Detailed Implementation

[0017] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0018] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0019] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0020] Example 1 In one or more embodiments, a method for constructing a 3D tunnel map based on multi-source sensors is disclosed, such as... Figure 1 As shown, it includes the following steps: Step S1: Dynamically adjust the parameters of the multi-source heterogeneous sensor and collect multi-source sensor data using a multi-source heterogeneous sensor parameter adaptive mechanism; Step S1-1: Construct a multi-source heterogeneous sensor array.

[0021] The tunnel robot integrates sensors such as lidar, binocular vision camera, temperature and humidity sensor, and dust concentration detection module to form a multi-source heterogeneous sensor array, which is powered by a dual power supply system.

[0022] By acquiring real-time dust concentration, humidity, and light intensity data in the tunnel using a multi-source heterogeneous sensor array, and dynamically adjusting sensor parameters such as lidar scanning frequency and visual camera exposure time, the integrity of data in the tunnel environment can be ensured.

[0023] Step S1-2: Establish an adaptive mechanism for parameters of multi-source heterogeneous sensors.

[0024] The emission power of the lidar is determined by combining dust concentration and humidity, and the expression is as follows:

[0025] In the formula, This represents the base power of the lidar. Dust concentration; Humidity; , These are empirical coefficients, whose values ​​are derived from fitting actual tunnel measurement data. By fixing the base power of the lidar, gradually increasing the dust concentration and humidity, recording the change curve of the effective echo rate power adjustment of the point cloud, and fitting the coefficients using the least squares method. The dust concentration threshold refers to the safe upper limit of dust concentration inside the tunnel. If this value is exceeded, enhanced ventilation or dust suppression measures must be initiated. The humidity threshold refers to the waterproof rating based on lidar; continuous exposure to this humidity level may cause motor condensation failure.

[0026] For binocular cameras, in low-light environments, infrared illumination is activated and the exposure time is extended.

[0027] In this embodiment, the illuminance threshold for low-light environments is set to less than 50 lux. The exposure extension time can be set based on experience.

[0028] Steps S1-3: Collect data from multiple sensor sources.

[0029] The tunnel construction site environment is harsh, especially for a period of time after blasting, when the concentration of water vapor and dust at the tunnel face is high, which seriously affects the accuracy of laser scanning and photogrammetry at the tunnel face. This invention obtains more accurate measurement data through environmental adaptive parameter adjustment.

[0030] Step S2: Synchronize and register the multi-source sensor data, and obtain a fused point cloud based on the weighted fusion of the synchronized and registered sensor data.

[0031] A real-time topology map is generated using the SLAM algorithm. Combined with prior tunnel structure information provided by the BIM model, spatiotemporal registration of laser point clouds and visual images is performed to obtain a synthetic point cloud. This synthetic point cloud (generated from the BIM model) is then fused with the measured point cloud to enhance the accuracy of feature extraction for key tunnel structures. Specifically, this includes: Step S2-1: Based on the hardware clock signal of the odometer encoder on the tunnel mobile robot body, align the data acquisition time of the LiDAR and vision camera.

[0032] Step S2-2: Input the aligned sensor features using the SLAM algorithm, output a synthetic point cloud, generate a robot pose and topology map, convert the predefined tunnel structure (such as pipe diameter, cable trough coordinates) in the BIM model into a synthetic point cloud, perform ICP registration with the measured point cloud, and solve for the optimal transformation matrix. The expression is:

[0033] in, This is the optimal transformation matrix; Used to suppress local distortions caused by noise; The three-dimensional coordinates of the i-th point in the synthesized point cloud; Let be the three-dimensional coordinates of the i-th point in the measured point cloud; This represents the number of points in the point cloud. A set of neighborhood point pairs defined on the point cloud of the synthetic model; This represents the normal vector at the i-th point in the synthetic model; This represents the normal vector at the j-th point in the measured model.

[0034] The optimal transformation matrix is ​​obtained by solving the problem. Its core function is to achieve precise spatial transformation from the synthetic point cloud to the measured point cloud coordinate system.

[0035] Step S2-3: Obtain the fused point cloud by weighted fusion of multi-source data.

[0036] The weights are dynamically assigned based on the confidence level of the sensors. If a sensor malfunction is detected, its weight is automatically reduced and a switchover to a backup device is triggered.

[0037] Sensor confidence The calculation is obtained through the fusion of multi-dimensional indicators, and the formula is as follows:

[0038] in, Let k be the kth evaluation index (normalized to 0~1). These are the weighting coefficients for the corresponding indicators. The main evaluation indicators include data quality indicators, environmental compatibility indicators, and equipment health indicators. Data quality indicators include point cloud density (LiDAR), image sharpness (camera), and signal-to-noise ratio (radar), etc. Environmental compatibility indicators are obtained based on environmental sensor data through a lookup table method. Historical consistency indicators represent the degree of matching between current data and historical fusion results. Equipment health indicators are derived from long-term monitored system status.

[0039] The weights of each sensor in data fusion are dynamically adjusted based on confidence level, and the calculation formula is as follows:

[0040] in, The basic weights of sensor j, The real-time confidence level is given by n, which represents the number of sensors involved in the fusion.

[0041] when When the confidence level is less than the set confidence threshold, the sensor weight is forcibly reset to zero, and the redundant sensor switching process is triggered to ensure that the system can continue to work stably even when the sensor fails.

[0042] Step S3: Use a semantic segmentation model to identify semantic information in the fused point cloud, and optimize the map labels based on the semantic information to obtain a local map.

[0043] Step S3-1: The semantic segmentation model adopts the Point Transformer deep learning model to process the fused point cloud obtained in step S2, automatically identify key elements in the tunnel environment, and add understandable semantic information to the map to obtain the initial segmentation result.

[0044] Key elements include static obstacles, dynamic targets, and functional structures (such as ventilation ducts and cable trays).

[0045] Step S3-2, Semantic Label Optimization: Combine the voting mechanism of semantic labels of neighboring grids to optimize the initial segmentation results and eliminate mislabeling caused by noise interference.

[0046] The entire fused point cloud space is divided into 0.1m×0.1m×0.1m cubic grids. Each grid contains several point cloud data points. The semantic labels of all points in each grid are counted. The labels with a proportion exceeding the first threshold (usually set to 70%) are selected as the final semantic labels of the grid. If the proportion of all labels does not reach the first threshold, it is marked as "area to be verified" and needs to be manually confirmed after subsequent close-range scanning by the tunnel robot.

[0047] Therefore, this invention addresses the issue of sporadic mislabeling that may occur during the processing by optimizing semantic tags, thereby eliminating noise interference.

[0048] Step S3-3: Store the semantic raster map using an octree structure. The semantic map of the point cloud after tunnel point cloud segmentation is as follows: Figure 2 As shown.

[0049] A lightweight semantic segmentation model is deployed locally on the robot to enable real-time updates of the local map and rapid marking of abnormal areas (such as landslides and water accumulation). The point cloud within a 10m radius is labeled in real time to update the local map.

[0050] Step S4: Obtain a global 3D tunnel map by fusing several local maps.

[0051] Edge-cloud collaborative map updates are adopted. Local map generation is performed by the edge end of the aforementioned tunnel robots. The local map is uploaded to the cloud control center. The cloud control center converts the local maps uploaded by each robot to the global coordinate system based on the timestamp and positioning information to obtain the global map.

[0052] Furthermore, if the same area is labeled with different semantics by different robots, the final label is determined by a weighted voting method based on confidence level. The optimized global map is then optimized through pose graph to eliminate accumulated errors and distributed to the navigation modules of each robot.

[0053] The confidence level of the semantic labels reported by each robot It is determined by the following factors:

[0054] in, The overall confidence level of the sensor is determined based on data quality and environmental compatibility. The historical annotation accuracy rate of the robot, determined based on manually verified records, This is the positioning accuracy factor determined by the inverse of the SLAM covariance matrix.

[0055] in The calculation formula is

[0056] In the formula, The average data quality metric is the data quality index for all main sensors (LiDAR, camera, radar) on robot k. Take the arithmetic mean; For the worst-case environmental metric, all environmental compatibility metrics are taken. The minimum value in; To determine the matching error term, the matching error is based on the current frame data and the local short-term map. calculate, This is the attenuation coefficient (usually taken as 1.0). The larger the error, the smaller this value.

[0057] For conflicting regions, the final semantic labels are determined according to the following rules:

[0058] Where Y is the final label. The semantic label reported by each robot, where c is one of all possible semantic label categories reported by all robots at the current conflict grid.

[0059] During execution, each robot uploads semantic labels and corresponding confidence levels for local maps. For each conflicting grid, the weighted sum of votes for each candidate label is calculated, and the label with the highest weighted vote is selected as the final semantic label. When the highest vote advantage is less than 10%, it is marked as "area to be verified" and the nearest robot is scheduled for verification.

[0060] Preferably, the generated map undergoes credibility assessment and parameter calibration.

[0061] A high-precision tunnel model was built using the UE5 engine to simulate changes in dust, humidity, and illumination, generating a simulated point cloud. The average error was calculated by comparing the measured point cloud with the simulated data.

[0062] In the formula, E is the average error; To simulate the coordinates of the i-th point in the point cloud; Here, represents the coordinates of the i-th point in the real point cloud; N represents the number of points in the point cloud. If the lidar error remains high, an automatic calibration procedure is triggered to correct the sensor's intrinsic parameters. Specifically, when the lidar error E > 0.2m, the sensor fusion weights in step S2-3 are adjusted using the gradient descent method.

[0063] This embodiment generates a complete, high-precision 3D semantic grid map by spatiotemporal alignment and global optimization of multi-robot data collected in the cloud, and supports dynamic obstacle trajectory prediction and navigation path correction.

[0064] Example 2 In one or more embodiments, a tunnel 3D map construction system based on multi-source sensors is disclosed, specifically including: The data acquisition module is used to dynamically adjust the parameters of the multi-source heterogeneous sensors and collect multi-source sensor data using a multi-source heterogeneous sensor parameter adaptive mechanism. The data fusion module is used to synchronize and register the multi-source sensor data, and to obtain a fused point cloud based on the weighted fusion of the synchronized and registered sensor data. The semantic segmentation module is used to identify semantic information from the fused point cloud using a semantic segmentation model, and to optimize map labels based on the semantic information to obtain a local map. The map generation module is used to obtain a global 3D map of the tunnel by fusing several local maps.

[0065] Example 3 This embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, they complete the steps of the above-described method for constructing a tunnel 3D map based on multiple source sensors.

[0066] Example 4 This embodiment provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the above-described method for constructing a three-dimensional tunnel map based on multi-source sensors.

[0067] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0069] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0070] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0071] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing a 3D tunnel map based on multi-source sensors, characterized in that, include: A multi-source heterogeneous sensor parameter adaptive mechanism is adopted to dynamically adjust the parameters of the multi-source heterogeneous sensor and collect multi-source sensor data. The multi-source sensor data is synchronized and registered, and a fused point cloud is obtained by weighted fusion of the synchronized and registered sensor data. A semantic segmentation model is used to identify semantic information from the fused point cloud, and map labels are optimized based on the semantic information to obtain a local map. A global 3D map of the tunnel is obtained by fusing several local maps. 2.The tunnel 3D map construction method based on multi-source sensors of claim 1, wherein, The multi-source heterogeneous sensor parameter adaptive mechanism includes: The emission power of the lidar is determined based on a combination of dust concentration and humidity. For binocular vision cameras, adjust infrared illumination and exposure time according to lighting conditions. 3.The tunnel 3D map construction method based on multi-source sensors of claim 1, wherein, The method for determining the transmission power of the lidar based on a combination of dust concentration and humidity is as follows: wherein is the laser radar base power; is the dust concentration; is the humidity; , is an empirical coefficient; is a dust concentration threshold value; is a humidity threshold value. 4.The tunnel 3D map construction method based on multi-source sensors of claim 1, wherein, The weighted fusion of synchronized and registered sensor data to obtain the fused point cloud includes dynamically assigning weights based on sensor confidence levels. If a sensor anomaly is detected, its weight is automatically reduced and a backup device switchover is triggered. Specifically: Sensor confidence The multi-dimensional index fusion calculation is obtained by the following calculation formula: wherein, is the kth evaluation index, is the weight coefficient of the corresponding index; The weights of each sensor in data fusion are dynamically adjusted based on confidence level, and the calculation formula is as follows: wherein, is the base weight of sensor j, is the real-time confidence, n is the number of sensors participating in fusion; When the confidence level of a sensor is less than the set confidence level threshold, the sensor weight is forcibly reset to zero, and the redundant sensor switching process is triggered. 5.The tunnel 3D map construction method based on multi-source sensors of claim 1, wherein, The process of optimizing map labels based on the semantic information to obtain a local map specifically involves: The entire fused point cloud space is divided into several cubic grids, each containing several point cloud data points. The semantic labels of all points in each grid are counted, and the labels with a proportion exceeding the first threshold are selected as the final semantic labels of that grid. If the proportion of all labels does not reach the first threshold, it is marked as "area to be verified" and awaits scanning confirmation. 6.The tunnel 3D map construction method based on multi-source sensors of claim 1, wherein, Edge-cloud collaborative map updates are adopted. Local map generation is performed by an edge-end execution system consisting of several tunnel robots. The local map is uploaded to the cloud control center. The cloud control center converts the local maps uploaded by each robot to the global coordinate system based on the timestamp and positioning information to obtain the global map. If the same area is labeled with different semantics by different robots, the final label is determined by a weighted vote based on confidence level. The optimized global map is then optimized through pose graph to eliminate accumulated errors and distributed to the navigation modules of each robot. 7.The tunnel 3D map construction method based on multi-source sensors of claim 1, wherein, The credibility assessment and parameter calibration of the global 3D tunnel map are performed as follows: A high-precision tunnel model was built based on the UE5 engine to simulate changes in dust, humidity, and light, and to generate a simulated point cloud. Compare the measured point cloud with the simulation data, calculate the average error, and if the average error is greater than the preset threshold, adjust the sensor fusion weights according to the gradient descent method. 8.A multi-source sensor based tunnel 3D map construction system, characterized in that, include: The data acquisition module is used to dynamically adjust the parameters of the multi-source heterogeneous sensors and collect multi-source sensor data using a multi-source heterogeneous sensor parameter adaptive mechanism. The data fusion module is used to synchronize and register the multi-source sensor data, and to obtain a fused point cloud based on the weighted fusion of the synchronized and registered sensor data. The semantic segmentation module is used to identify semantic information from the fused point cloud using a semantic segmentation model, and to optimize map labels based on the semantic information to obtain a local map. The map generation module is used to obtain a global 3D map of the tunnel by fusing several local maps.

9. An electronic device, comprising: It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, complete the tunnel 3D map construction method based on any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, complete the tunnel 3D map construction method based on multi-source sensors as described in any one of claims 1-7.

Images