Underground pipe gallery detection method based on robot and pipe gallery detection robot

Abstract

The invention relates to the technical field of intelligent environment detection, and particularly discloses an underground pipe gallery detection method based on a robot and a pipe gallery detection robot, and the method comprises the steps: obtaining the topological data of a pipe gallery through a multi-source sensing device, constructing a three-dimensional space model of the pipe gallery through the combination of a high-precision Beidou / UWB fusion positioning technology, and obtaining a three-dimensional space model of the pipe gallery; geometric parameters and initial defect distribution of the to-be-detected pipe gallery are marked; based on the three-dimensional space model, a pipe gallery detection robot is deployed, and multi-dimensional data of the pipe gallery environment and the equipment state are synchronously collected in the advancing process; according to the invention, robot inspection is introduced, so that personnel are prevented from being directly exposed in a dangerous environment, the safety risk of the inspection personnel is remarkably reduced, and the problem of high risk in manual inspection is effectively solved; meanwhile, through an automatic and intelligent robot inspection mode, the robot flexibly shuttles around the pipe gallery, all-directional and dead-corner-free detection of the pipe gallery is achieved, the inspection time is greatly shortened, and the detection comprehensiveness and accuracy are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent environmental detection, and in particular relates to a robot-based underground pipe gallery detection method and a pipe gallery detection robot. Background Art

[0002] Underground utility corridors, as crucial urban infrastructure, house a variety of pipelines, including electricity, communications, gas, and water supply and drainage. As the scale of urban underground utility corridor construction continues to expand, their operation and maintenance face significant challenges. Traditional manual inspections suffer from low efficiency and poor safety, while existing fixed sensor monitoring systems struggle to cover all areas within the corridor, particularly complex structures like branch pipes and elbows. Furthermore, sensor data formats vary from manufacturer to manufacturer, making it difficult to integrate and conduct in-depth analysis of multi-source data, creating data silos.

[0003] In recent years, both domestic and international efforts have begun exploring the use of mobile robots for tunnel inspections. Existing inspection robots are primarily categorized as tracked or wheeled, but numerous technical bottlenecks remain. Lack of adaptability of mobile platforms: Existing inspection robots mostly utilize tracked or wheeled structures, which have poor maneuverability and struggle to operate stably in tunnels with complex terrain (such as stagnant water, rubble, and fire door thresholds). While some robots possess some obstacle-crossing capabilities, their bulky structures make them inadequate for the narrow environments of branch tunnels. Low intelligence in inspection methods: Existing inspection methods primarily rely on single sensors (such as cameras or lidar), lacking multimodal fusion analysis of inspection data. This results in low defect recognition rates and high false positive rates. For example, in low-light and high-humidity environments, traditional vision algorithms struggle to accurately identify structural defects such as cracks and leaks. Furthermore, data acquisition and processing systems lack standardization, resulting in poor data compatibility between different devices and hindering unified management and intelligent decision-making. Low levels of standardization and industrialization: Currently, there is no unified technical specification for underground tunnel inspection robots. Equipment from different manufacturers exhibits significant differences in performance, interfaces, and data formats, hindering large-scale application. At the same time, the maintenance cost of the inspection robot is high and the lack of modular design leads to insufficient equipment availability.

[0004] Therefore, it is necessary to propose a robot-based underground pipeline corridor inspection method and a pipeline corridor inspection robot to solve the problems of "high risk + low efficiency" of manual inspection and "blank + one-sided" of intelligent inspection in the existing technology.

[0005] The above information disclosed in this background technology is only for enhancing understanding of the background technology of the present invention and therefore it may contain information that does not constitute the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0006] The purpose of the present invention is to provide a robot-based underground pipeline corridor detection method and a pipeline corridor detection robot to solve the problems of "high risk + low efficiency" of manual inspection proposed in the above-mentioned background technology (gas leakage, water accumulation and other hidden dangers in the pipeline corridor occur frequently, manual inspection takes 4 to 6 hours per kilometer, and it is necessary to face the risks of hypoxia and toxic gas in the confined space), and "blank + one-sided" of intelligent detection (35% of the existing pipeline corridors only deploy fixed sensors, which cannot cover branches and elbows, and the recognition rate of structural diseases is less than 20%).

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] The robot-based underground pipeline corridor detection method includes:

[0009] Multi-source sensing equipment is used to obtain the topological data of the pipeline corridor. Combined with high-precision BeiDou / UWB fusion positioning technology, a three-dimensional spatial model of the pipeline corridor is constructed, and the geometric parameters and initial defect distribution of the pipeline corridor to be inspected are marked.

[0010] Based on the three-dimensional spatial model, a pipeline corridor inspection robot is deployed to synchronously collect multi-dimensional data on the pipeline corridor environment and equipment status while moving. The multi-dimensional data, vehicle motion information, and spatial information are integrated to form a full-dimensional data fusion result covering vision, space, environment, and motion.

[0011] Based on the full-dimensional data fusion results, the pipeline corridor map is updated in real time through incremental SLAM technology, and a federated learning framework is used to fuse the historical database with the real-time data stream to perform dynamic digital twin modeling of the pipeline corridor status;

[0012] Based on the dynamic digital twin model, an improved YOLOv7-Transformer hybrid neural network is used to perform multi-scale identification of pipeline corridor defects. A multi-task learning model is used to perform defect classification, remaining life prediction, and structural reliability analysis in parallel to generate a risk level matrix.

[0013] Based on the risk level matrix, an augmented reality interactive platform is used to generate a pipeline corridor health status report. Blockchain technology is combined with trusted evidence storage and cross-terminal collaborative operations to generate the optimal maintenance strategy and trigger robots to perform local repairs or mark high-risk areas.

[0014] Preferably, the multi-source sensor equipment includes ground LiDAR, underground acoustic wave detectors and an inertial navigation system. When obtaining the tunnel topology data, the ground LiDAR is used to perform a three-dimensional scan of the ground and the tunnel entrance area, and the underground acoustic wave detector is used to perform acoustic wave reflection detection on the internal structure of the tunnel. During tunnel inspection, the inertial navigation system is used to assist in determining the position and posture information of the tunnel inspection robot in the tunnel.

[0015] Preferably, when the digital twin model is constructed, the three-dimensional space model is optimized using the full-dimensional data fusion result, and the real-time data of various sensors are associated with the optimized three-dimensional space model through incremental SLAM technology to replicate the pipeline corridor entity in the virtual space;

[0016] After establishing a dynamic digital twin model, the system used it to simulate and analyze tunnel operations, predict faults, and conduct emergency drills. The YOLOv7 model was used to extract low-level features from tunnel images, and multi-scale feature maps were used for target detection to identify defects of different sizes.

[0017] The Transformer network is introduced to combine the local features extracted by the YOLOv7 model with the global context to optimize the recognition of defects of different sizes. The self-attention mechanism is used to understand the relationship between various areas in the image, focus on the detailed features of the defective area, and locate the defects in the pipeline corridor.

[0018] Preferably, the multi-task learning model is used to perform defect classification, remaining life prediction, and structural reliability analysis in parallel, including the following aspects:

[0019] Adopt ResNet-Transformer based on attention mechanism for defect classification. The attention mechanism is introduced to focus on the characteristics of the defect area, ResNet is used to extract the deep features of the defect area, and the global information association of Transformer is combined to identify the defect type of the defect area.

[0020] The LSTM-GAN hybrid model is used for remaining life prediction. LSTM is used to process time series data to capture the temporal patterns of defect development, and GAN is combined with its generative adversarial capabilities to predict the remaining life of defect areas.

[0021] The reliability of the results is analyzed through the finite element simulation engine interface, which combines real-time data with the finite element model to dynamically analyze the reliability of the defect area.

[0022] Preferably, when generating the pipeline corridor health status report, the multi-dimensional data visualization information is integrated with the three-dimensional spatial model of the pipeline corridor for display, and the integrated display information is collaboratively shared among different departments in combination with the distributed ledger and encryption algorithm of the blockchain;

[0023] After generating the tunnel health status report, the maintenance cost, maintenance time, maintenance effect and impact on tunnel operation are comprehensively considered. The economic / safety trade-offs of different maintenance plans are simulated through digital twins to select the optimal maintenance strategy.

[0024] Pipeline corridor inspection robot, including:

[0025] A robot travel system, mounted at the lower end of the robot's main structure, employs suspended Mecanum wheels and an adaptive shock absorption system to enable omnidirectional movement and obstacle crossing.

[0026] A multimodal perception system, mounted on top of the robot's main structure, acquires multi-dimensional data about the tunnel's internal environment in real time through an omnidirectional environmental perception network comprised of three visual sensors, a lidar, a thermal imaging camera, a gyroscope, multiple ADC gas sensors, and temperature and humidity sensors.

[0027] The information processing unit is installed inside the main structure of the robot. The information processing unit integrates multi-dimensional data, vehicle body motion information, and spatial information to form a full-dimensional data fusion result covering vision, space, environment, and motion.

[0028] Preferably, the main structure of the robot adopts a carbon fiber reinforced acrylonitrile-styrene-acrylate composite material prepared by the fused deposition molding process as the main frame, combined with Ti-6Al-4V titanium alloy key load-bearing components formed by selective laser melting additive manufacturing technology, and is equipped with a multi-angle adjustment mechanical platform.

[0029] Preferably, the robot travel system further includes:

[0030] The hardware support layer is used to perform hardware signal transmission and motor drive control through the encoder interface, PWM waveform output interface, and push-pull IO interface;

[0031] The algorithm control layer is used to plan motion logic through control algorithms, analyze paths in combination with motion solutions, and monitor posture using gyroscope algorithms. The three work together to optimize motor drive parameters for robot motion control and posture stabilization.

[0032] Preferably, the multimodal perception system further includes:

[0033] The digital twin analysis module combines historical databases with real-time multi-dimensional data, retrieves information on the structure and layout of the pipeline corridor, and performs both dynamic and static analysis, eliminating interference from temporarily stored objects through reinforcement learning algorithms.

[0034] The temperature anomaly detection module is used to convert thermal imaging images into temperature data using the PMOD_Thermal32 thermal imaging submodule of the uncooled vanadium oxide focal plane array. A temperature-grayscale mapping model is established through blackbody radiation calibration. Based on the adaptive threshold segmentation and mean shift clustering algorithm, abnormal heat sources are located with an accuracy of ±2°C.

[0035] The composite obstacle avoidance module is used to plan obstacle avoidance paths based on lidar data through point cloud clustering and dynamic windowing. It also uses a deep vision network to extract features from RGB data and perform decimeter-level obstacle detection.

[0036] The edge computing module uses a lightweight YOLO model to perform real-time recognition of tools such as buckets and hammers, and maps the detection results to the robot coordinate system;

[0037] The hazardous gas detection module uses multiple ADC gas sensors to detect multiple gases, including CH4 (methane), H2S (hydrogen sulfide), and CO (carbon monoxide). If levels exceed the specified limit, an immediate alarm is triggered, initiating a linkage with the pipe corridor ventilation system. Temperature and humidity monitoring ranges from -20°C to 80°C, and humidity from 0% to 100% RH. Data is uploaded to the information processing unit in real time. It also interfaces with the following visual detection modules. For example, if excessive carbon dioxide or carbon monoxide levels are detected, or the ambient temperature is too high, the thermal imaging camera is automatically activated to locate high-temperature areas and determine whether a fire is present.

[0038] The rust detection module is used to construct HSV color space + LBP texture feature vectors, perform ISO 4628 rust grade assessment using XGBoost classifiers, and enhance the contrast of metal oxidation features using multispectral imaging;

[0039] The water accumulation detection module is used to segment the water accumulation area through the region generation algorithm, and the depth value is corrected by combining the RGB color threshold and the water body reflection characteristics to perform water volume estimation;

[0040] The instrument recognition module is used to design a cascade network structure. The digital meter is directly OCRed through CRNN+CTC decoding. The pointer meter uses Hough transform to detect the dial edge and pointer, combined with cosine similarity to match the angle-pressure mapping relationship in the training data.

[0041] Preferably, the information processing unit is further used for:

[0042] Run a deep learning model to perform target recognition on the collected images, identify equipment anomalies and leakage points, and fuse them with thermal imaging and thermal imaging camera information to form comprehensive visual information and extract visual feature data within the tunnel;

[0043] Combine LiDAR dot matrix information with comprehensive visual information to analyze the spatial structure of the tunnel and the distribution of obstacles, and construct spatial information for environmental perception and path planning;

[0044] The full-dimensional data fusion results are converted into data that can be understood and displayed, and the converted data is displayed to the user through the user interface and the operation instructions issued by the user are received.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] By introducing robotic inspections, the present invention avoids direct exposure of personnel to dangerous environments, significantly reduces the safety risks of inspectors, and effectively solves the "high-risk" problem faced in manual inspections. At the same time, through automated and intelligent robotic inspection methods, it flexibly shuttles throughout the pipeline corridor, achieving all-round, no-dead-angle detection of the pipeline corridor, greatly shortening the inspection time and significantly improving the comprehensiveness and accuracy of detection. By applying advanced detection technologies and algorithms, it accurately identifies structural defects in the pipeline corridor, helping to promptly discover and address potential problems and protect the safe operation of the pipeline corridor. In addition, by improving detection efficiency and accuracy, it can reduce safety accidents and maintenance costs caused by missed inspections and false detections, and the automated operation of the robots also reduces manpower input and reduces operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of the robot-based underground pipe gallery detection method of the present invention;

[0048] Figure 2 This is one of the structural diagrams of the pipe gallery inspection robot of the present invention;

[0049] Figure 3 This is the second structural diagram of the pipe gallery inspection robot of the present invention;

[0050] Figure 4 This is a motion control framework diagram of the pipe gallery inspection robot of the present invention;

[0051] Figure 5 This is a multimodal perception framework diagram of the pipeline corridor inspection robot of the present invention.

[0052] Explanation of the accompanying symbols: 1. Robot main structure; 2. Robot travel system; 3. Multimodal perception system; 4. Information processing unit; 5. Multi-angle adjustment mechanical platform. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] It should be noted that the drawings are schematic and not drawn to scale. For clarity and convenience, the relative sizes and proportions of parts shown in the drawings may be exaggerated or reduced in size. Any dimensions are illustrative only and are not intended to be limiting. Identical structures, elements, or components appearing in two or more drawings are denoted by the same reference numerals to indicate similar features.

[0055] Example 1:

[0056] See also Figure 1 As shown in FIG, the robot-based underground pipe gallery detection method includes:

[0057] Multi-source sensing equipment is used to obtain the topological data of the pipeline corridor. Combined with high-precision BeiDou / UWB fusion positioning technology, a three-dimensional spatial model of the pipeline corridor is constructed, and the geometric parameters and initial defect distribution of the pipeline corridor to be inspected are marked.

[0058] The multi-source sensor equipment includes ground LiDAR, underground acoustic wave detectors, and an inertial navigation system. When acquiring the tunnel topology data, the ground LiDAR is used to perform three-dimensional scanning of the ground and tunnel entrance area, and the underground acoustic wave detector is used to detect the internal structure of the tunnel by acoustic wave reflection. During the tunnel inspection, the inertial navigation system is used to assist in determining the position and posture information of the tunnel inspection robot within the tunnel.

[0059] Furthermore, multi-source sensing equipment combined with high-precision positioning technology efficiently and accurately acquires the corridor's three-dimensional topological data and geometric parameters, annotates initial defect distribution, and monitors the corridor's internal structural status in real time. Ground-based LiDAR scans the ground and entrance areas, while underground acoustic wave detectors detect reflections within the corridor. An inertial navigation system ensures the inspection robot's precise positioning and posture within the corridor, thereby improving the accuracy and efficiency of corridor inspections, reducing human intervention, and optimizing corridor maintenance and repair processes.

[0060] Based on the 3D spatial model, a pipeline corridor inspection robot is deployed to synchronously collect multi-dimensional data on the pipeline corridor environment and equipment status while moving. The robot integrates multi-dimensional data, vehicle motion information, and spatial information to form a full-dimensional data fusion result covering vision, space, environment, and motion.

[0061] Based on the full-dimensional data fusion results, the pipeline corridor map is updated in real time through incremental SLAM technology, and a federated learning framework is used to integrate historical databases with real-time data streams to perform dynamic digital twin modeling of the pipeline corridor status.

[0062] When building digital twins, the full-dimensional data fusion results are used to optimize the 3D spatial model. Incremental SLAM technology is then used to associate and map the real-time data from various sensors with the optimized 3D spatial model, replicating the pipeline corridor entity in the virtual space.

[0063] Furthermore, through full-dimensional data fusion and incremental SLAM technology, dynamic digital twin modeling of the corridor is achieved, corridor maps are updated in real time, and the 3D spatial model is optimized. By integrating multi-dimensional data such as vision, space, environment, and motion, precise monitoring and prediction of corridor status is achieved, enhancing the intelligent level of corridor management. Combining historical databases with real-time data streams, a virtual replica of the corridor entity is formed, providing data support for subsequent maintenance, repair, and optimization, significantly improving corridor management efficiency and decision-making accuracy.

[0064] After establishing a dynamic digital twin model, the system used it to simulate and analyze tunnel operations, predict faults, and conduct emergency drills. The YOLOv7 model was used to extract low-level features from tunnel images, and multi-scale feature maps were used for target detection to identify defects of different sizes.

[0065] The Transformer network is introduced to combine local features extracted by the YOLOv7 model with global context to optimize the recognition of defects of different sizes. The self-attention mechanism understands the relationship between various regions in the image, focusing on the detailed features of the defect area and locating defects in the pipeline corridor.

[0066] Furthermore, the YOLOv7 model is used to extract low-level features from tunnel images, and the Transformer network is used to optimize the recognition of defects of varying sizes, effectively improving defect detection accuracy. The self-attention mechanism helps focus on details in defect areas, enhancing the ability to accurately locate and analyze tunnel equipment status. This technology facilitates real-time monitoring of tunnel conditions, provides reliable fault warnings and emergency drill support, and enhances the intelligent management and maintenance of tunnels.

[0067] Based on the dynamic digital twin model, an improved YOLOv7-Transformer hybrid neural network is used to identify pipeline corridor defects at multiple scales. A multi-task learning model is then used to concurrently perform defect classification, remaining life prediction, and structural reliability analysis to generate a risk level matrix.

[0068] Adopt ResNet-Transformer based on attention mechanism for defect classification. The attention mechanism is introduced to focus on the characteristics of the defect area, ResNet is used to extract the deep features of the defect area, and the global information association of Transformer is combined to identify the defect type of the defect area.

[0069] The LSTM-GAN hybrid model is used for remaining life prediction. LSTM is used to process time series data to capture the temporal patterns of defect development, and GAN is combined with its generative adversarial capabilities to predict the remaining life of defect areas.

[0070] Conduct reliability analysis of results through the finite element simulation engine interface, combining real-time data with the finite element model to dynamically analyze the reliability of defect areas;

[0071] Furthermore, the introduction of an attention mechanism and ResNet-Transformer helps accurately identify defect types and optimize feature extraction. The LSTM-GAN hybrid model accurately predicts the remaining life of defects by capturing the changing patterns of time series data. Combined with a finite element simulation engine, results are analyzed to dynamically assess the reliability and risk level of the pipeline corridor. This technology can enhance maintenance decision support capabilities for pipeline corridor equipment, increase the accuracy of fault warnings and life predictions, and promote intelligent pipeline corridor management.

[0072] Based on the risk level matrix, an augmented reality interactive platform is used to generate a pipeline corridor health status report. Blockchain technology is combined with trusted evidence storage and cross-terminal collaborative operations to generate the optimal maintenance strategy and trigger robots to perform local repairs or mark high-risk areas.

[0073] When generating a pipeline corridor health report, the multi-dimensional data visualization information is integrated with the three-dimensional spatial model of the pipeline corridor. Combined with the blockchain's distributed ledger and encryption algorithm, the integrated display information can be collaboratively shared among different departments.

[0074] After generating the tunnel health status report, the maintenance cost, maintenance time, maintenance effect and impact on tunnel operation are comprehensively considered. The economic / safety trade-offs of different maintenance plans are simulated through digital twins to select the optimal maintenance strategy.

[0075] Furthermore, an augmented reality interactive platform and blockchain technology are used to generate pipeline corridor health status reports, combining multidimensional data visualization with three-dimensional spatial model presentations to ensure cross-departmental collaborative sharing and trusted evidence storage. Digital twin technology is used to simulate the economic and safety of different maintenance plans, select the optimal maintenance strategy, reduce maintenance costs, improve maintenance effectiveness, and ensure pipeline corridor operation stability. This technology enhances the intelligent management of pipeline corridors, optimizes maintenance decisions, and enables efficient cross-terminal collaboration and real-time information sharing, promoting precise maintenance and long-term stable operation of pipeline corridors.

[0076] Example 2:

[0077] See also Figure 2-Figure 5 As shown, the pipeline corridor inspection robot includes:

[0078] The robot's main structure 1 uses carbon fiber reinforced acrylonitrile-styrene-acrylate (ASA-CF) composite material prepared by fused deposition modeling as the main frame. This material system exhibits excellent specific strength (≥70MPa cm 3 / g) and significant cost advantages (approximately 40% lower than traditional metal components). For key load-bearing components and suspension systems, Ti-6Al-4V (TC4) titanium alloy components, formed using selective laser melting (SLM) additive manufacturing technology, are used. Their tensile strength reaches 900-1100 MPa. A multi-angle adjustable mechanical platform 5 is equipped to carry and adjust the position and orientation of three vision sensors.

[0079] This hybrid material manufacturing strategy achieves an optimized balance between structural performance and manufacturing cost. The ASA-CF composite material provides good specific stiffness and environmental tolerance (heat deformation temperature reaches 105°C), while the TC4 titanium alloy components formed by SLM additive manufacturing ensure the mechanical reliability of key connecting parts.

[0080] The robot's travel system 2, mounted at the bottom of the robot's main structure 1, innovatively utilizes suspended Mecanum wheels and an adaptive damping system for omnidirectional movement and obstacle navigability, improving maneuverability on complex terrain (including ponded water, gravel, slopes, and steps). Its compact size and flexible steering make it suitable for narrow branch pipeline corridors (widths less than 50 cm), increasing coverage by 100%.

[0081] The hardware support layer is used to perform hardware signal transmission and motor drive control through the encoder interface, PWM waveform output interface, and push-pull IO interface. After receiving the underlying interface signal, it drives the motor to operate and provide power for the robot's travel system.

[0082] The algorithm control layer is used to plan motion logic through control algorithms, analyze paths in combination with motion solutions, and monitor posture using gyroscope algorithms. The three work together to optimize motor drive parameters for robot motion control and posture stabilization.

[0083] This study combines an innovative four-independent suspension mechanism with a Mecanum wheel omnidirectional mobile platform. The robot's width can be controlled within 40 cm. Combined with high-damping shock absorbers, it achieves a 15 cm obstacle clearance height while maintaining omnidirectional mobility. This adapts to the space restrictions of more than 90% of domestic branch pipeline corridors. The modular quick-release design allows for maintenance and replacement of the suspension system within 10 minutes, significantly improving equipment availability.

[0084] A multimodal perception system 3 is installed on the upper end of the robot main structure 1. The multimodal perception system 3 obtains multi-dimensional data of the internal environment of the pipeline corridor in real time through an omnidirectional environmental perception network;

[0085] The all-round environmental perception network includes three visual sensors, lidar, thermal imaging camera, gyroscope, multi-channel ADC gas sensor and temperature and humidity sensor;

[0086] The digital twin analysis module is used to combine historical databases and real-time multi-dimensional data, retrieve information on the structure and layout of the pipeline corridor, and perform dynamic and static analysis simultaneously. It uses reinforcement learning algorithms to eliminate interference from temporarily stacked objects and make predictions about possible future situations based on the historical database, effectively issuing early warnings before problems occur to avoid further losses.

[0087] The temperature anomaly detection module is used to convert thermal imaging images into temperature data using the PMOD_Thermal32 thermal imaging submodule of the uncooled vanadium oxide focal plane array. A temperature-grayscale mapping model is established through blackbody radiation calibration. Based on adaptive threshold segmentation and mean shift clustering algorithms, it can locate abnormal heat sources with an accuracy of ±2°C. It supports temperature measurement from -40°C to 450°C and high-temperature area annotation at the 32×24 pixel level.

[0088] The composite obstacle avoidance module is used to plan obstacle avoidance paths based on lidar data through point cloud clustering and dynamic windowing. It also uses a deep vision network to extract features from RGB data and perform decimeter-level obstacle detection.

[0089] The edge computing module uses a lightweight YOLO model to perform real-time recognition of tools such as buckets and hammers, and maps the detection results to the robot coordinate system;

[0090] The hazardous gas detection module uses multiple ADC gas sensors to detect multiple gases, including CH4 (methane), H2S (hydrogen sulfide), and CO (carbon monoxide). If levels exceed the specified limit, an immediate alarm is generated, initiating a linkage to the pipe corridor ventilation system. Temperature and humidity monitoring ranges from -20°C to 80°C, and humidity from 0% to 100% RH. Data is uploaded to the information processing unit 4 in real time. It also interfaces with the following visual detection modules. For example, if excessive carbon dioxide or carbon monoxide levels are detected, or the ambient temperature is too high, the thermal imaging camera is automatically activated to locate high-temperature areas and determine whether a fire is present.

[0091] The rust detection module is used to construct HSV color space + LBP texture feature vectors, perform ISO 4628 rust grade assessment using XGBoost classifiers, and enhance the contrast of metal oxidation features using multispectral imaging;

[0092] The water accumulation detection module is used to segment the water accumulation area through the region generation algorithm, and the depth value is corrected by combining the RGB color threshold and the water body reflection characteristics to perform water volume estimation;

[0093] The instrument recognition module is used to design a cascade network structure. The digital meter uses CRNN+CTC decoding to directly perform OCR. The pointer meter uses Hough transform to detect the dial edge and pointer. Combined with cosine similarity to match the angle-pressure mapping relationship in the training data, it supports high-precision reading of 1 / 16 scale values.

[0094] The information processing unit 4 is installed inside the robot body structure 1. The information processing unit 4 integrates multi-dimensional data, vehicle body motion information, and spatial information to form a full-dimensional data fusion result covering vision, space, environment, and motion;

[0095] Run a deep learning model to perform target recognition on the collected images, identify equipment anomalies and leakage points, and integrate the information with thermal imaging and thermal imaging cameras to form comprehensive visual information, accurately extracting visual feature data within the pipeline corridor;

[0096] Combine LiDAR dot matrix information with comprehensive visual information to analyze the spatial structure of the tunnel and the distribution of obstacles, and construct spatial information for environmental perception and path planning;

[0097] The full-dimensional data fusion results are converted into data that can be understood and displayed, and the converted data is displayed to users through the user interface, allowing users to quickly understand the status of the pipeline corridor.

[0098] It is used to make decisions based on visual data and issue operation instructions through the user interface. After receiving the operation instructions, the robot uses control algorithms and other drive motors to perform corresponding actions, such as inspecting designated locations and testing equipment, thereby realizing human-computer interaction.

[0099] As can be seen from the above, the robot adopts a compact design and has excellent passing performance, which can flexibly shuttle through underground pipeline corridors and branch line environments with limited space. In view of the multi-section structural design characteristics of underground pipeline corridors (fire separation spacing layout), an innovative suspended mobile system is adopted, which can easily cross slopes and fire door thresholds to achieve barrier-free passage. Compared with traditional hoisting robots, it has more advantages in flexibility and adaptability. Equipped with a variety of sensors and cameras, and integrating multiple functions such as gas detection, temperature and humidity monitoring, and structural exploration, it can replace manual labor to complete complex inspection tasks in high-risk environments, effectively reduce the risk of accidents, and provide intelligent protection for the operation and maintenance safety of underground pipeline corridors.

[0100] Example 3:

[0101] Application example: Full-process operation of intelligent inspection of urban underground pipeline corridors

[0102] To improve the operation and maintenance of its underground utility corridors, a large city decided to introduce a robotic inspection method and robot to conduct comprehensive inspections of a 5-kilometer-long underground utility corridor within the city. The corridor houses a variety of pipelines, including power, communications, gas, and water supply and drainage. Traditional manual inspections were inefficient and posed safety risks.

[0103] 1. Application Objectives

[0104] (1) Achieve all-round, no-dead-angle detection of all areas in the pipeline corridor (including complex structures such as branches and elbows).

[0105] (2) Timely discover and locate structural defects and safety hazards in the tunnel, such as cracks, leakage, rust, etc.

[0106] (3) Generate a detailed report on the health status of the pipeline corridor to provide a scientific basis for subsequent repair and maintenance.

[0107] (4) Reduce inspection costs, improve inspection efficiency, and ensure the safe operation of the pipeline corridor.

[0108] 2. Implementation Steps

[0109] 1. Preliminary preparation

[0110] ① Equipment deployment: Deploy ground LiDAR, underground acoustic wave detectors, and inertial navigation systems at tunnel entrances and key locations. Simultaneously, prepare a tunnel inspection robot, including its main structure, travel system, multimodal perception system, and information processing unit.

[0111] ② 3D model construction: Use ground LiDAR and underground acoustic wave detectors to obtain the topological data of the pipeline corridor, combine with high-precision Beidou / UWB fusion positioning technology to build a 3D spatial model of the pipeline corridor, and mark the geometric parameters and initial defect distribution of the pipeline corridor to be inspected.

[0112] 2. Robot inspection

[0113] ①Robot entering the corridor: The corridor inspection robot is sent into the corridor through the entrance. The robot uses suspended Mecanum wheels combined with an adaptive shock absorption system to start autonomous inspections in the corridor.

[0114] ② Data Collection: As the robot moves, it uses a multimodal perception system to collect real-time, multi-dimensional data on the tunnel environment and equipment status. This includes visual images, LiDAR point clouds, thermal imaging data, gas concentrations (such as CH4, H2S, CO), temperature and humidity, etc. Simultaneously, it integrates vehicle motion and spatial information to generate a full-dimensional data fusion result.

[0115] 3. Dynamic digital twin modeling

[0116] ① Model optimization: Utilize the full-dimensional data fusion results to optimize the three-dimensional spatial model, and update the pipeline corridor map in real time through incremental SLAM technology.

[0117] ② Digital Twin Modeling: Using a federated learning framework, historical databases and real-time data streams are integrated to create dynamic digital twin models of the tunnel status. This model replicates the tunnel entity in a virtual space, enabling simulation analysis of tunnel operations, fault prediction, and emergency drills.

[0118] 4. Defect identification and risk assessment

[0119] ① Multi-scale defect identification: Based on the dynamic digital twin model, the improved YOLOv7-Transformer hybrid neural network is used to perform multi-scale identification of pipeline corridor defects, including cracks, leakage, rust, etc.

[0120] ② Multi-task learning analysis: Parallel execution of defect classification, remaining life prediction, and structural reliability analysis. Defect classification is performed using a ResNet-Transformer with an attention mechanism, while remaining life prediction is performed using an LSTM-GAN hybrid model. Reliability analysis of the results is performed through a finite element simulation engine interface.

[0121] ③ Risk level matrix generation: Generate a risk level matrix based on the analysis results, evaluate the safety status of each area in the tunnel, and mark high-risk areas.

[0122] 5. Health status reporting and maintenance strategy formulation

[0123] ① Health Status Report Generation: Utilizing an augmented reality interactive platform, a health status report is generated, integrating multi-dimensional data visualization with the corridor's three-dimensional spatial model. Blockchain technology is used for trusted evidence storage and cross-terminal collaboration, ensuring the authenticity and immutability of the information.

[0124] ② Optimal maintenance strategy formulation: Comprehensively consider maintenance costs, maintenance time, maintenance effects, and the impact on tunnel operations. Use digital twin simulation to balance the economic and safety trade-offs of different maintenance plans and select the optimal maintenance strategy.

[0125] 6. Perform maintenance tasks and follow-up monitoring

[0126] ① Local repair or marking: According to the established maintenance strategy, the robot is triggered to perform local repair tasks, such as using a robotic arm to perform simple repairs; or high-risk areas are marked to notify the manual maintenance team for further processing.

[0127] ②Continuous Monitoring: After maintenance is completed, the robot continues to monitor the corridor to ensure its safe operation. At the same time, the corridor's 3D spatial model and digital twin model are regularly updated to provide data support for subsequent operations and maintenance management.

[0128] 3. Application Effect

[0129] (1) Improve inspection efficiency: The robot can complete a comprehensive inspection of the pipeline corridor in a short time, which is much more efficient than the traditional manual inspection method.

[0130] (2) Reduce safety risks: It avoids direct exposure of personnel to dangerous environments and significantly reduces the safety risks of inspection personnel.

[0131] (3) Improve detection accuracy: Through multi-dimensional data fusion and advanced algorithm analysis, the accuracy and reliability of defect identification are improved.

[0132] (4) Optimize maintenance strategies: The maintenance strategies formulated based on digital twin simulation and risk assessment results are more scientific and reasonable, reducing maintenance costs and time.

[0133] Through this application example, the robot-based underground pipeline corridor detection method and the efficiency, safety and accuracy of the pipeline corridor detection robot in urban underground pipeline corridor inspection tasks were demonstrated, providing strong support for the operation and maintenance management of modern urban underground pipeline corridors.

[0134] An embodiment of the present invention further provides a computer-readable storage medium, on which is stored a program for any of the above-mentioned robot-based underground pipe gallery detection methods. When the program is executed by a processor, each process of the above-mentioned underground pipe gallery detection method embodiment is implemented, and the same technical effect is achieved. To avoid repetition, it is not described here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0135] In the description of this specification, the reference terms "one embodiment", "some embodiments", "examples", "specific examples" or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0136] The drawings of the embodiments disclosed in the present invention only involve structures related to the embodiments disclosed in the present invention. Other structures may refer to common designs. In the absence of conflicts, the same embodiment and different embodiments of the present invention may be combined with each other.

[0137] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0138] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A robot-based underground pipe gallery detection method, characterized in that: include: Multi-source sensing equipment is used to obtain the topological data of the pipeline corridor. Combined with high-precision BeiDou / UWB fusion positioning technology, a three-dimensional spatial model of the pipeline corridor is constructed, and the geometric parameters and initial defect distribution of the pipeline corridor to be inspected are marked. Based on the three-dimensional spatial model, a pipeline corridor inspection robot is deployed to synchronously collect multi-dimensional data on the pipeline corridor environment and equipment status while moving. The multi-dimensional data, vehicle motion information, and spatial information are integrated to form a full-dimensional data fusion result covering vision, space, environment, and motion. Based on the full-dimensional data fusion results, the pipeline corridor map is updated in real time through incremental SLAM technology, and a federated learning framework is used to fuse the historical database with the real-time data stream to perform dynamic digital twin modeling of the pipeline corridor status; Based on the dynamic digital twin model, an improved YOLOv7-Transformer hybrid neural network is used to perform multi-scale identification of pipeline corridor defects. A multi-task learning model is used to perform defect classification, remaining life prediction, and structural reliability analysis in parallel to generate a risk level matrix. Based on the risk level matrix, an augmented reality interactive platform is used to generate a pipeline corridor health status report. Blockchain technology is combined with trusted evidence storage and cross-terminal collaborative operations to generate the optimal maintenance strategy and trigger robots to perform local repairs or mark high-risk areas.

2. The robot-based underground pipe gallery detection method according to claim 1, characterized in that: The multi-source sensor equipment includes ground LiDAR, underground acoustic wave detectors and an inertial navigation system. When obtaining the topological data of the pipeline corridor, the ground LiDAR is used to perform a three-dimensional scan of the ground and the pipeline corridor entrance area, and the underground acoustic wave detector is used to perform acoustic wave reflection detection on the internal structure of the pipeline corridor. During the pipeline corridor inspection, the inertial navigation system is used to assist in determining the position and posture information of the pipeline corridor inspection robot in the pipeline corridor.

3. The robot-based underground pipe gallery detection method according to claim 2, characterized in that: When building digital twins, the three-dimensional space model is optimized using the full-dimensional data fusion results, and the real-time data of various sensors is associated with the optimized three-dimensional space model through incremental SLAM technology to replicate the pipeline corridor entity in the virtual space. After establishing a dynamic digital twin model, the system used it to simulate and analyze tunnel operations, predict faults, and conduct emergency drills. The YOLOv7 model was used to extract low-level features from tunnel images, and multi-scale feature maps were used for target detection to identify defects of different sizes. The Transformer network is introduced to combine the local features extracted by the YOLOv7 model with the global context to optimize the recognition of defects of different sizes. The self-attention mechanism is used to understand the relationship between various areas in the image, focus on the detailed features of the defective area, and locate the defects in the pipeline corridor.

4. The robot-based underground pipe gallery detection method according to claim 3, characterized in that: The multi-task learning model is used to perform defect classification, remaining life prediction, and structural reliability analysis in parallel, including the following aspects: The ResNet-Transformer based on the attention mechanism is used for defect classification. The attention mechanism is introduced to focus on the characteristics of the defect area, ResNet is used to extract the deep features of the defect area, and the global information association of the Transformer is combined to identify the defect type of the defect area. The LSTM-GAN hybrid model is used for remaining life prediction. LSTM is used to process time series data to capture the temporal patterns of defect development, and GAN is combined with its generative adversarial capabilities to predict the remaining life of defect areas. The reliability of the results is analyzed through the finite element simulation engine interface, which combines real-time data with the finite element model to dynamically analyze the reliability of the defect area.

5. The robot-based underground pipe gallery detection method according to claim 4, characterized in that: When generating the pipeline corridor health status report, the multi-dimensional data visualization information is integrated with the three-dimensional spatial model of the pipeline corridor for display, and the integrated display information is collaboratively shared among different departments by combining the distributed ledger and encryption algorithm of the blockchain; After generating the tunnel health status report, the maintenance cost, maintenance time, maintenance effect and impact on tunnel operation are comprehensively considered. The economic / safety trade-offs of different maintenance plans are simulated through digital twins to select the optimal maintenance strategy.

6. A pipe gallery inspection robot for executing the robot-based underground pipe gallery inspection method according to any one of claims 1 to 5, characterized in that: include: A robot travel system, mounted at the lower end of the robot's main structure, employs suspended Mecanum wheels and an adaptive shock absorption system to enable omnidirectional movement and obstacle crossing. A multimodal perception system, mounted on top of the robot's main structure, acquires multi-dimensional data about the tunnel's internal environment in real time through an omnidirectional environmental perception network comprised of three visual sensors, a lidar, a thermal imaging camera, a gyroscope, multiple ADC gas sensors, and temperature and humidity sensors. The information processing unit is installed inside the main structure of the robot. The information processing unit integrates multi-dimensional data, vehicle body motion information, and spatial information to form a full-dimensional data fusion result covering vision, space, environment, and motion.

7. The pipe gallery inspection robot according to claim 6, characterized in that: The main structure of the robot uses a carbon fiber reinforced acrylonitrile-styrene-acrylate composite material prepared by the fused deposition modeling process as the main frame, combined with Ti-6Al-4V titanium alloy key load-bearing components formed by selective laser melting additive manufacturing technology, and is equipped with a multi-angle adjustable mechanical platform.

8. The pipe gallery inspection robot according to claim 7, characterized in that: The robot travel system also includes: The hardware support layer is used to perform hardware signal transmission and motor drive control through the encoder interface, PWM waveform output interface, and push-pull IO interface; The algorithm control layer is used to plan motion logic through control algorithms, analyze paths in combination with motion solutions, and monitor posture using gyroscope algorithms. The three work together to optimize motor drive parameters for robot motion control and posture stabilization.

9. The pipe gallery inspection robot according to claim 8, characterized in that: The multimodal perception system further includes: The digital twin analysis module combines historical databases with real-time multi-dimensional data, retrieves information on the structure and layout of the pipeline corridor, and performs both dynamic and static analysis, eliminating interference from temporarily stored objects through reinforcement learning algorithms. The temperature anomaly detection module is used to establish a temperature-grayscale mapping model based on thermal imaging images through blackbody radiation calibration. It then locates abnormal heat sources with an accuracy of ±2°C based on adaptive threshold segmentation and mean shift clustering algorithms. The composite obstacle avoidance module is used to plan obstacle avoidance paths based on lidar data through point cloud clustering and dynamic windowing. It also uses a deep vision network to extract features from RGB data and perform decimeter-level obstacle detection. The rust detection module is used to construct HSV color space + LBP texture feature vectors, perform ISO4628 rust grade assessment using XGBoost classifier, and enhance the contrast of metal oxidation features using multispectral imaging; The water accumulation detection module is used to segment the water accumulation area through the region generation algorithm, and the depth value is corrected by combining the RGB color threshold and the water body reflection characteristics to perform water volume estimation; The instrument recognition module is used to design a cascade network structure. The digital meter is directly OCRed through CRNN+CTC decoding. The pointer meter uses Hough transform to detect the dial edge and pointer, combined with cosine similarity to match the angle-pressure mapping relationship in the training data.

10. The pipe gallery inspection robot according to claim 9, characterized in that: The information processing unit is further configured to: Run a deep learning model to perform target recognition on the collected images, identify equipment anomalies and leakage points, and fuse them with thermal imaging and thermal imaging camera information to form comprehensive visual information and extract visual feature data within the tunnel; Combine LiDAR dot matrix information with comprehensive visual information to analyze the spatial structure of the tunnel and the distribution of obstacles, and construct spatial information for environmental perception and path planning; The full-dimensional data fusion results are converted into data that can be understood and displayed, and the converted data is displayed to the user through the user interface and the operation instructions issued by the user are received.

Images