A multi-modal perception pumped storage self-flow drainage hole autonomous inspection robot control method and control system

Abstract

This application discloses a control method and control system for an autonomous inspection robot of pumped-storage gravity drainage tunnels, belonging to the field of autonomous inspection technology for pumped-storage gravity drainage tunnels. The method includes: generating an initial semantic cross-section by projecting visual features onto a laser point cloud; eliminating accumulated errors and matching absolute station numbers through a tightly coupled visual-laser-inertial navigation factor map to construct a precise semantic cross-section; subsequently, adaptively calling a composite execution head for dredging based on the siltation type and average thickness identified by semantic segmentation; and finally, calculating the thickness reduction rate by reversing and comparing the cross-section, achieving autonomous return upon reaching the target using a closed-loop iteration and benchmark freezing mechanism. This application achieves precise matching of the execution head based on the physical characteristics of siltation and dredging depth determined by thickness, combined with closed-loop quantitative control of dredging-re-inspection-correction, effectively avoiding ineffective operations and over-excavation, and greatly improving the precision and unmanned level of tunnel dredging.

Description

Technical Field

[0001] This application belongs to the field of autonomous inspection technology for pumped-storage gravity drainage tunnels, and specifically relates to a control method and control system for an autonomous inspection robot for pumped-storage gravity drainage tunnels with multimodal perception. Background Technology

[0002] The pumped storage power station has a long-distance gravity drainage tunnel with supported walls. It is dry during construction and flooded during operation. There is no signal throughout the tunnel. In daily life, personnel enter the tunnel to check the safety of the surrounding rock and the smoothness of drainage with their naked eyes and simple tools, and also clean up silt and fallen rocks. The only existing solution is to go into the cave with people and eyes: the inspectors carry searchlights, geological hammers, bamboo poles and cameras into the cave, walking, looking, knocking and taking pictures. When they find silt or fallen rocks, they put them in bags with shovels. After coming out of the cave, they compile a report based on their memory and photos. Existing technologies have the following drawbacks: a lack of understanding of the "embodied intelligence" dimension in purely manual solutions, including: Perceptual isolation: Image, sound, and water depth information are not related to each other and cannot form a unified semantic description; The positioning was rough: the readings were taken manually from the paint piles on the tunnel wall, and it was difficult to accurately return to the same cross section after re-measurement; Communication black box: No signal throughout the process, and the ground cannot be informed in time of emergencies; Limited mobility: Relying on walking, it is easy to slip and fall in deep water or thick silt, and there is no robotic platform to replace it; Inefficient dredging: manually shoveling and bagging the blockages is a small-scale, long-cycle operation, and the water flow quickly becomes blocked again during the operation period; Data gaps: Photos and records are stored in a scattered manner, and historical results cannot be automatically aligned and compared; Safety risks: Working alone in dark, slippery, oxygen-deficient, or hazardous gas environments can lead to delayed rescue responses; In summary, current technology remains at the stage of "human sensors," and the application of embodied intelligent robots in such long tunnels is still lacking.

[0003] Therefore, providing an integrated intelligent inspection and dredging robot system for the long-distance, signal-free, and alternating water flow / drying gravity drainage tunnels of pumped storage power stations, and directly solving the safety, efficiency, and data gap problems caused by purely manual walking inspections, is a technical problem that urgently needs to be solved in the current field. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a multimodal sensing autonomous inspection robot control method and control system for pumped storage gravity drainage tunnels. By providing an integrated intelligent inspection and dredging robot system for long-distance, signal-free, and alternating water flow / drying gravity drainage tunnels in pumped storage power stations, it directly solves the safety, efficiency, and data gap problems caused by purely manual walking inspections.

[0005] In a first aspect, this application provides a multimodal sensing control method for an autonomous inspection robot of a pumped-storage gravity-flow drainage tunnel, the method comprising, The target inspection robot is controlled to move in the pumped and gravity drainage tunnel. It uses a multimodal sensor array to simultaneously collect visual features and laser point clouds, and projects the visual features onto the laser point clouds to generate an initial semantic profile. The initial semantic cross-section is semantically segmented to identify the siltation type and average silt thickness. The initial semantic cross-section is then optimized using a vision-laser-inertial navigation tight coupling factor map based on inertial navigation data. Based on the optimization results, the initial semantic cross-section is matched with an absolute station number to generate an accurate semantic cross-section. Match the corresponding composite execution head tool according to the siltation type in the precise semantic section, and control the composite execution head tool to perform dredging operation on the section corresponding to the absolute station number based on the average silt thickness; The target inspection robot is controlled to reverse and the visual features and laser point cloud of the section corresponding to the absolute station number are re-collected to generate a re-inspection semantic section. The re-inspection semantic section is then compared with the precise semantic section to calculate the silt thickness reduction rate. When the silt thickness reduction rate meets the preset conditions, the re-inspection semantic section is frozen as a new historical benchmark, and the target inspection robot is controlled to autonomously return and exit the tunnel based on the absolute station number in the historical benchmark; when the silt thickness reduction rate does not meet the preset conditions, the composite execution head tool is controlled to perform dredging operations again until the regenerated re-inspection semantic section meets the preset conditions.

[0006] Furthermore, The controlled target inspection robot moves within the pumped-flow drainage tunnel, simultaneously acquiring visual features and laser point clouds using a multimodal sensor array, specifically including: A second pulse synchronization signal is generated based on a field-programmable gate array (FPGA) to trigger a solid-state lidar and a panoramic camera to simultaneously sample the walls of the pumped-flow drainage tunnel. The structure acoustic microphone array and millimeter-wave water level gauge are synchronously triggered to collect structure acoustic signals and water depth data, so that the timestamp error between the visual features, the laser point cloud, the structure acoustic signal and the water depth data is less than a preset error threshold.

[0007] Furthermore, The step of projecting the visual features onto the laser point cloud to generate an initial semantic profile specifically includes: Using the laser point cloud as a spatial reference, the visual features are projected onto the three-dimensional point cloud space using a pre-calibrated extrinsic calibration matrix to generate a colored point cloud; Based on the beamforming results of the structured acoustic microphone array, the position coordinates of the sidewall void source are located in the color point cloud, and the position coordinates are superimposed on the color point cloud to generate the initial semantic section.

[0008] Furthermore, The step of semantic segmentation of the initial semantic cross-section to identify the siltation type and average silt thickness specifically includes: The initial semantic profile is input into a pre-built lightweight transformer network, and the laser point cloud contained in the initial semantic profile is classified point by point to identify the various siltation types contained in the initial semantic profile. The average thickness of silt corresponding to the siltation type in the initial semantic cross-section is calculated based on the classification results.

[0009] Furthermore, The process of optimizing the initial semantic profile using a vision-laser-inertial navigation tight coupling factor map based on inertial navigation data, and matching absolute station numbers to the initial semantic profile based on the optimization results to generate an accurate semantic profile, specifically includes: Using the fixed steel mesh on the tunnel wall as a natural reflector, a joint factor map is constructed that includes visual reprojection error, registration error of the laser point cloud, and pre-integration error of the inertial navigation data. The six-degree-of-freedom pose of the target inspection robot at the initial semantic section is solved based on nonlinear optimization. The travel distance of the target inspection robot relative to the design axis is calculated based on the six degrees of freedom pose, and the travel distance is converted into the absolute station number by combining the preset start and end station numbers. The absolute station number is associated with and stored with the siltation type and average silt thickness to generate the precise semantic cross-section.

[0010] Furthermore, The step of matching the corresponding composite execution head tool according to the siltation type in the precise semantic section, and controlling the composite execution head tool to perform dredging operations on the section corresponding to the absolute station number based on the average silt thickness, specifically includes: The target inspection robot is controlled to perform obstacle clearing operations on the current section using the matched composite execution head tool, and the contact force is monitored in real time by a six-dimensional force sensor installed on the composite execution head tool. When the contact force does not exceed the preset threshold, the composite execution head tool continues to be controlled to perform obstacle clearing operations on the current section at the current feed speed, while the cumulative volume of discharged solids is calculated and updated in real time through the outlet flow meter. When the contact force exceeds the preset threshold, the composite execution head tool is controlled to retract and the obstacle clearing operation continues after the feed speed is reduced until the cumulative volume of discharged solids reaches the amount of silt estimated based on the average thickness of the silt, at which point the obstacle clearing stops.

[0011] Furthermore, The process of controlling the target inspection robot to reverse and re-acquiring the visual features and laser point cloud of the section corresponding to the absolute station number to generate a re-inspection semantic section specifically includes: The target inspection robot is controlled to reverse along the original driving path by a preset distance, and the visual features and laser point cloud of the section corresponding to the absolute station number are reacquired using the multimodal sensor array in the same acquisition method as the initial semantic section is generated. The reacquired visual features are projected onto the reacquired laser point cloud to generate the re-examined semantic profile.

[0012] Furthermore, The step of comparing the re-examined semantic cross-section with the precise semantic cross-section to calculate the silt thickness reduction rate specifically includes: The thickness of the dredged silt is extracted from the re-inspection semantic section, and the initial silt thickness before dredging is extracted from the precise semantic section. The silt thickness reduction rate is calculated based on the initial silt thickness and the re-inspected silt thickness.

[0013] Furthermore, When the silt thickness reduction rate does not meet the preset condition, record the number of times the current condition is not met, control the composite execution head tool to perform silt removal operation on the section corresponding to the absolute station number again and regenerate the re-inspection semantic section until the silt thickness reduction rate meets the preset condition or the number of silt removals reaches the preset target number threshold.

[0014] Secondly, based on the same inventive concept, this application provides a multimodal sensing autonomous inspection robot control system for pumped-flow drainage tunnels. The system includes: The initial scanning module controls the target inspection robot to move within the pumped and self-flowing drainage tunnel, and uses a multimodal sensor array to simultaneously collect visual features and laser point clouds, and projects the visual features onto the laser point clouds to generate an initial semantic profile. The precision inspection module performs semantic segmentation on the initial semantic cross-section to identify the siltation type and average silt thickness, and optimizes the initial semantic cross-section using a vision-laser-inertial navigation tight coupling factor map based on inertial navigation data. Based on the optimization results, it matches the initial semantic cross-section with absolute station numbers to generate a precise semantic cross-section. The obstacle removal module matches the corresponding composite execution head tool according to the siltation type in the precise semantic section, and controls the composite execution head tool to perform silt removal operations on the section corresponding to the absolute station number based on the average silt thickness; The re-inspection module controls the target inspection robot to reverse and re-collect the visual features and laser point cloud of the section corresponding to the absolute station number to generate a re-inspection semantic section, and compares the re-inspection semantic section with the precise semantic section to calculate the silt thickness reduction rate. The return module freezes the re-inspection semantic section as a new historical benchmark when the silt thickness reduction rate meets the preset conditions, and controls the target inspection robot to autonomously return and exit the tunnel based on the absolute station number in the historical benchmark; when the silt thickness reduction rate does not meet the preset conditions, it controls the composite execution head tool to perform dredging operations again until the regenerated re-inspection semantic section meets the preset conditions.

[0015] Thirdly, this application also provides an electronic device, including at least one processor and at least one memory electrically connected; The memory is electrically connected to the processor, wherein the memory stores instructions that can be executed by at least one of the processors, the instructions being executed by at least one of the processors to enable at least one of the processors to perform any of the multimodal sensing autonomous inspection robot control methods for pumped-flow drainage tunnels as described above.

[0016] Fourthly, this application also provides a computer storage medium, wherein a computer program is stored within the computer-readable storage medium; When the computer program is executed by the processor, it implements any of the multimodal sensing control methods for autonomous inspection robots of pumped-flow drainage tunnels as described above.

[0017] Fifthly, this application also provides a computer program product, which is stored in at least one storage medium; The computer program product includes several instructions to cause at least one electronic device to execute any of the multimodal sensing autonomous inspection robot control methods for pumped-storage drainage tunnels as described above.

[0018] Compared with the prior art, this application has the following advantages: 1. By generating second pulse signals, the timestamp errors of visual, laser, structural acoustic and water depth data are controlled within an extremely low threshold, eliminating the spatial misalignment of multimodal data; at the same time, the coordinates of the sidewall void source located by beamforming are superimposed on the visual-laser color point cloud, which not only achieves high-precision reconstruction of siltation morphology, but also realizes the fine-grained exploration of hidden defects (voids) in the tunnel. 2. A vision-laser-inertial navigation tightly coupled factor graph optimization mechanism was constructed, overcoming the challenge of high-precision positioning of absolute stationing in long-distance underground environments without GPS; by using the fixed steel mesh of the tunnel wall as a natural reflector, the six-degree-of-freedom pose was solved by integrating reprojection error, registration error and pre-integration error, which effectively suppressed the cumulative drift error during long-distance travel and achieved accurate correlation between cross-sectional data and absolute stationing of the project, providing a reliable benchmark for subsequent dredging operations; 3. An autonomous dredging control closed loop of "on-demand matching - force control to prevent jamming - closed-loop re-inspection" has been formed, which significantly improves the safety and intelligence level of dredging operations. Based on the semantic segmentation results, the composite execution head is adaptively matched. During the operation, a six-dimensional force sensor and flow meter are introduced to carry out force-position hybrid control, which effectively prevents jamming when encountering hard objects or over-excavation. The silt thickness reduction rate is calculated by re-inspecting the cross-section by reversing, and the deviation is repeatedly corrected by the benchmark freezing and number threshold mechanism, which completely eliminates the phenomenon of missed dredging and realizes high-quality dredging under unmanned conditions.

[0019] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating the control method of the multimodal sensing autonomous inspection robot for pumped-flow drainage tunnels according to an embodiment of this application is shown. Figure 2 This paper presents an overall system functional architecture diagram of the multimodal perception autonomous inspection robot control system for pumped-flow drainage tunnels, according to an embodiment of this application. Figure 3 A flowchart illustrating the underground operations according to an embodiment of this application is shown; Figure 4 A flowchart illustrating the multimodal perception and semantic mapping process according to an embodiment of this application is shown. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Figure 1 This paper illustrates a multimodal sensing control method for an autonomous inspection robot of a pumped-flow drainage tunnel according to an embodiment of this application. For example... Figure 1 As shown in the embodiment of this application, the multimodal sensing autonomous inspection robot control method for pumped-flow drainage tunnels includes the following steps: S1, control the target inspection robot to move in the pumped and self-flowing drainage tunnel, use a multimodal sensor array to synchronously collect visual features and laser point cloud, and project the visual features onto the laser point cloud to generate an initial semantic profile. S2, perform semantic segmentation on the initial semantic section to identify the siltation type and average silt thickness, and optimize the initial semantic section with visual-laser-inertial navigation tight coupling factor map by combining inertial navigation data. Based on the optimization results, match the absolute station number to the initial semantic section to generate an accurate semantic section. S3, match the corresponding composite execution head tool according to the siltation type in the precise semantic section, and control the composite execution head tool to perform dredging operation on the section corresponding to the absolute station number based on the average silt thickness; S4, control the target inspection robot to reverse and re-collect the visual features and laser point cloud of the section corresponding to the absolute station number to generate a re-inspection semantic section, and compare the re-inspection semantic section with the precise semantic section to calculate the silt thickness reduction rate. S5, when the silt thickness reduction rate meets the preset conditions, the re-inspection semantic section is frozen as a new historical benchmark, and the target inspection robot is controlled to autonomously return to the tunnel based on the absolute station number in the historical benchmark; when the silt thickness reduction rate does not meet the preset conditions, the composite execution head tool is controlled to perform dredging operation again until the regenerated re-inspection semantic section meets the preset conditions.

[0024] like Figure 3As shown, in the specific implementation process, with the concept of embodied intelligence as the core, five major functions are integrated into the target inspection robot body: multimodal sensing, edge computing, adaptive movement, composite execution, and ground-penetrating communication. This enables the target inspection robot to autonomously complete the closed loop of "perception-decision-dredging-data accumulation" even in drainage tunnels without public networks, lighting, and alternating water flow and dryness. The target inspection robot operates according to a five-step strategy of "rapid initial sweeping → fine inspection → obstacle removal → re-inspection → return to the warehouse". The edge box outside the tunnel only serves as a data mirror and emergency intervention node and does not rely on real-time remote control.

[0025] In the embodiments of this application, step S1 specifically includes: S111, a second pulse synchronization signal is generated based on the field programmable gate array to trigger the solid-state lidar and panoramic camera to simultaneously sample the wall of the pumped drainage tunnel; S112, synchronously trigger the structure acoustic microphone array and millimeter-wave water level gauge to collect structure acoustic signals and water depth data, so that the timestamp error between the visual features, the laser point cloud, the structure acoustic signal and the water depth data is less than a preset error threshold.

[0026] In the embodiments of this application, step S1 further includes: S121, using the laser point cloud as a spatial reference, the visual features are projected onto the three-dimensional point cloud space using a pre-calibrated extrinsic calibration matrix to generate a colored point cloud; S122, based on the beamforming result of the structured acoustic microphone array, locate the position coordinates of the sidewall void source in the color point cloud, and superimpose the position coordinates onto the color point cloud to generate the initial semantic section.

[0027] like Figure 4 As shown, in the specific implementation process, the operation and maintenance personnel select the daily inspection or flood season special mode on the plate at the tunnel entrance, and the edge box writes the tunnel section start and end chainage, historical digital twin cross section, and water level prediction curve into the target inspection robot NVM (Non-Volatile Memory). After the target inspection robot passes the self-inspection (battery, sensor zero bias, actuator head stroke), the edge box closes the laser fence, and the robot drives into the hole at low speed along the preset centerline to enter the "initial sweep" state. In the specific implementation process, the multimodal synchronous perception and semantic mapping process includes: Time synchronization: The FPGA (Field Programmable Gate Array) generates 1 PPS (1 Pulse Per Second) pulse to trigger the camera, solid-state LiDAR, structured acoustic microphone, millimeter-wave water level gauge, and IMU (Inertial Measurement Unit) to sample simultaneously, ensuring that the five-dimensional data timestamp error is <1 ms; Spatial registration: Based on the lidar point cloud, visual features are projected onto the three-dimensional point cloud through the extrinsic calibration matrix to form an RGB-Point (Red Green Blue Point Cloud); the structured acoustic microphone array locates the side wall void source through beamforming and superimposes it onto the same point cloud.

[0028] In the embodiments of this application, step S2 specifically includes: S211, the initial semantic profile is input into a pre-built lightweight transformer network to classify the laser point cloud contained in the initial semantic profile point by point to identify the various siltation types contained in the initial semantic profile. S212, and calculate the average thickness of silt of the corresponding siltation type in the initial semantic section based on the classification results.

[0029] In the embodiments of this application, step S2 further includes: S221, using the fixed steel mesh on the tunnel wall as a natural reflector, a joint factor map is constructed that includes visual reprojection error, registration error of the laser point cloud, and pre-integration error of the inertial navigation data; S222, the six-degree-of-freedom pose of the target inspection robot at the initial semantic section is solved based on nonlinear optimization.

[0030] S223, calculate the travel distance of the target inspection robot relative to the design axis based on the six-degree-of-freedom pose, and convert the travel distance into the absolute station number by combining the preset start and end station numbers; S224, The absolute station number is associated with the siltation type and the average silt thickness and stored to generate the precise semantic cross-section.

[0031] like Figure 4 As shown, in the specific implementation process, semantic segmentation: the edge GPU (Graphics Processing Unit) runs a lightweight Transformer network to classify the point cloud into six categories: "complete concrete, shotcrete mesh, seepage, shards, silt, and water accumulation", and calculates the volume of shards and the average thickness of silt. Odometer estimation: Optimization of the tight coupling factor map of vision-laser-IMU (Inertial Measurement Unit), using the steel mesh fixed in the tunnel wall as a natural reflector, positioning drift of 100 m is less than 3 cm, solving the slippage error of traditional encoders; Cross-section slicing: For every vehicle length (1 m) advanced, a semantic cross-section perpendicular to the design axis is generated and assigned an absolute station number, stored in the local SQLite (Structured Query Language Lite) database, and a summary packet is transmitted back via LoRa (Long Range) ground-penetrating link.

[0032] In the embodiments of this application, step S3 specifically includes: S31, control the target inspection robot to use the matched composite execution head tool to perform obstacle clearing operation on the current section, and monitor the contact force in real time through the six-dimensional force sensor installed on the composite execution head tool; S32, when the contact force does not exceed the preset threshold, the composite execution head tool continues to be controlled to perform obstacle clearing operation on the current section at the current feed speed, while the cumulative discharged solid volume is calculated and updated in real time through the outlet flow meter; S33, when the contact force exceeds the preset threshold, control the composite execution head tool to retract and continue the obstacle clearing operation after reducing the feed speed, until the cumulative volume of discharged solids reaches the amount of silt estimated based on the average thickness of the silt, and then stop the obstacle clearing.

[0033] In the specific implementation process, risk assessment is carried out: if five consecutive semantic cross-sections show blockage and water seepage with an area > 0.5 m², or CO concentration > 24 ppm, the target inspection robot automatically switches to the risk avoidance mode - retreats to a safe distance, activates the warning flash, and waits for instructions from outside the cave; Dredging Trigger: When the average thickness of the silt is greater than the set threshold and the water depth does not exceed the vehicle's air intake, the target inspection robot enters the dredging state. Local path adjustment: Model-Predictive Control (MMC) is used for rolling optimization. Based on the laser point cloud elevation map ahead, the swing arm is automatically raised and lowered and the track center distance is changed to ensure that the angle between the track and the wet and slippery ground is less than 10° to prevent sideslip.

[0034] In the specific implementation process, the adaptive obstacle removal and mud collection process includes: Execution Head Selection: 1. Soft mud: Lateral spiral mud collection tank + centrifugal slurry pump (20 m head) to discharge mud to the downstream water flow center through DN50 hose to avoid secondary sedimentation; 2. Rockfall: The rock is broken down to less than 1 / 3 of the pipe diameter by a front swing impact drill (0.8 J × 3000 rpm), and then swept into the suction port by a brush roller; 3. Exposed steel mesh: Abandon rotating tools and use high-pressure fine water jet (15 MPa) to rinse along the mesh to prevent the blade from getting tangled.

[0035] 4. Force control closed loop: A six-dimensional force sensor is installed on the flange of the actuator head. When the contact force is greater than 80 N or the torque is greater than 5 N·m, the robotic arm immediately retracts by 2 cm and reduces the feed speed by 50% to prevent jamming. 5. Sludge collection monitoring: The outlet flow meter calculates the volume of discharged solids in real time. When the cumulative value is approximately equal to the sludge volume estimated by the semantic model, the clearing process will automatically stop and the execution head will be raised to proceed to the re-inspection.

[0036] In the embodiments of this application, step S4 specifically includes: S411, control the target inspection robot to reverse a preset distance along the original driving path, and use the multimodal sensor array to reacquire the visual features and laser point cloud of the section corresponding to the absolute station number in the same acquisition method as the initial semantic section is generated; S412, the reacquired visual features are projected onto the reacquired laser point cloud to generate the re-examined semantic profile.

[0037] In the embodiments of this application, step S4 further includes: S421, extract the thickness of the re-inspected silt after dredging from the re-inspection semantic section, and extract the initial silt thickness before dredging from the precise semantic section; S422, the silt thickness reduction rate is calculated based on the initial silt thickness and the re-inspected silt thickness.

[0038] In the specific implementation process, the thickness of the re-inspected silt after dredging is extracted from the re-inspection semantic section, and the initial silt thickness before dredging is extracted from the precise semantic section. The ratio of the difference between the initial silt thickness and the re-inspection silt thickness to the initial silt thickness is calculated to obtain the silt thickness reduction rate.

[0039] In the embodiments of this application, step S5 specifically includes: When the silt thickness reduction rate does not meet the preset condition, record the number of times the current condition is not met, control the composite execution head tool to perform silt removal operation on the section corresponding to the absolute station number again and regenerate the re-inspection semantic section until the silt thickness reduction rate meets the preset condition or the number of silt removals reaches the preset target number threshold.

[0040] In the embodiments of this application, step S5 specifically further includes: The target inspection robot is controlled to perform reverse odometer calculation based on the absolute station number in the historical benchmark, and during the return journey, it receives the exit environment parameters broadcast by the outer edge box of the tunnel through the ground communication link. When the exit environment parameters meet the preset abnormal conditions, the target inspection robot is controlled to shut down the dredging mechanism and adjust the chassis attitude to drive towards the tunnel entrance with the target traction force.

[0041] In the specific implementation process, the target inspection robot reverses 2 m in place and rescans the same cross section. The network compares the difference in silt thickness before and after cleaning. If the reduction rate is ≥80%, it is judged as qualified; otherwise, the fine cleaning is restarted a second time, which can be repeated up to 3 times. After passing the test, the cross section status is marked as passed and frozen as a new historical benchmark. The target inspection robot travels in reverse according to the odometer. The edge box continuously broadcasts the outlet flow rate and CO real-time value through a ground-penetrating radio. If the outlet flow rate suddenly increases or the water level is abnormal, the target inspection robot immediately switches to high-speed return mode, shuts off the suction pump, lowers the swing arm, and drives towards the hole with maximum traction. After exiting the cave, the high-speed Wi-Fi 6 link automatically connects and transmits all semantic sections, high-definition images, and obstacle clearing videos to the edge box in batches. The edge box runs the SLAM (Simultaneous Localization and Mapping) backend, stitches the discrete cross sections into a complete three-dimensional mesh, performs a difference with the previous digital twin, generates a block expansion heat map and a siltation growth rate curve, and pushes it to the pumped storage control room to provide the maintenance team with a priority list for the next shutdown and maintenance.

[0042] like Figure 2 As shown, based on the same inventive concept, this application also provides a multimodal sensing autonomous inspection robot control system for pumped-flow drainage tunnels, corresponding to the above method. The system includes: The initial scanning module controls the target inspection robot to move within the pumped and self-flowing drainage tunnel, and uses a multimodal sensor array to simultaneously collect visual features and laser point clouds, and projects the visual features onto the laser point clouds to generate an initial semantic profile. The precision inspection module performs semantic segmentation on the initial semantic cross-section to identify the siltation type and average silt thickness, and optimizes the initial semantic cross-section using a vision-laser-inertial navigation tight coupling factor map based on inertial navigation data. Based on the optimization results, it matches the initial semantic cross-section with absolute station numbers to generate a precise semantic cross-section. The obstacle removal module matches the corresponding composite execution head tool according to the siltation type in the precise semantic section, and controls the composite execution head tool to perform silt removal operations on the section corresponding to the absolute station number based on the average silt thickness; The re-inspection module controls the target inspection robot to reverse and re-collect the visual features and laser point cloud of the section corresponding to the absolute station number to generate a re-inspection semantic section, and compares the re-inspection semantic section with the precise semantic section to calculate the silt thickness reduction rate. The return module freezes the re-inspection semantic section as a new historical benchmark when the silt thickness reduction rate meets the preset conditions, and controls the target inspection robot to autonomously return and exit the tunnel based on the absolute station number in the historical benchmark; when the silt thickness reduction rate does not meet the preset conditions, it controls the composite execution head tool to perform dredging operations again until the regenerated re-inspection semantic section meets the preset conditions.

[0043] In this embodiment, the multimodal sensing array consists of a solid-state lidar, a panoramic camera, a structured acoustic microphone, a millimeter-wave water level gauge, and an IMU—to achieve semantic mapping and risk identification. The tightly coupled positioning algorithm (FPGA+GPU) can ensure centimeter-level mileage in dark environments and is the core of autonomous return and cross-section remeasurement; The adaptive tracked chassis is equipped with a swing arm and adjustable center distance to ensure traction and obstacle crossing on wet and slippery concrete and local silt. The composite actuator interface is a unified flange + quick-change electro-hydraulic connector, which allows the three tools of spiral mud collection, impact drilling and water jet to be switched within 1 minute; Force control six-dimensional sensors can prevent tool jamming and robotic arm overload, and are the bottom line for safety to prevent crashes under unattended operation; LoRa-magnetic induction ground-penetrating communication device can still transmit critical status under zero public network conditions, meeting the needs of emergency intervention; Explosion-proof lithium battery pack: IP67 packaged, with built-in BMS balancing, it is the sole source of power for the entire device.

[0044] In this embodiment, the high-resolution thermal infrared camera can detect water seepage cracks with a temperature difference of 0.1 ℃ at an early stage, further improving the sensitivity of early warning of rockfall by 30%; CO / O2 / HS multi-gas sensors can detect toxic gas leaks in advance in areas with aging shotcrete anchors and high levels of humus in groundwater. Once all the external fiber optic cables are in place, the edge box GPU cluster can offload the semantic segmentation model from the robot to the edge box, reducing onboard power consumption by approximately 15%. The automatic charging station docks autonomously after the target inspection robot returns, and is fully charged in 2 hours, eliminating the need for manual plugging and unplugging of high-voltage connectors. Cloud-based digital twin platform: Data from multiple drainage tunnels is aggregated, enabling horizontal comparison of deterioration rates in different geological sections and achieving power plant-level life prediction.

[0045] This application completes the entire closed loop of "perception-decision-dredging-re-inspection-data accumulation" without relying on manual entry into the tunnel, upgrading long-distance drainage tunnel inspection from "high-risk walking" to "all-weather autonomous operation and maintenance".

[0046] In the specific implementation process, the following implementation case is provided: During the flood season operation period, a 3.6 km drainage tunnel was autonomously inspected and dredged while "with water"; Background: A pumped-storage power station has a gravity-flow drainage tunnel that is 3.6 km long, with a cross-section of 3 m × 3.5 m, lined with concrete, and supported by shotcrete and anchor bolts. During the flood season from July to September each year, the water depth is 0.2-0.6 m, and the flow velocity is 0.4-0.8 m / s. Walking to the site would take 6 hours and carries risks of falls and sudden water inrush. The power station aims to complete the inspection and dredging without reducing the generator load or emptying the tunnel. Preparation phase: The outer edge box reads the water level gauge of the upstream sump and the daily scheduling plan through Modbus-TCP, automatically generates a "water depth curve for the next 4 hours", matches it with the "operation-water level-flow velocity" three-dimensional template library, and obtains a safe operation window; Maintenance personnel can issue a "Flood Season Special" task with one click in the central control room: chainage K0+000-K3+600, focus on checking the two historical rockfall areas K1+800 and K2+900, and clear the silt if the silt thickness threshold is ≥15 cm; Robot power-on self-test: Battery SOC 98%, firmware version consistent, six-dimensional force sensor zero drift <0.3 N, all passed; Work process: ① Low-speed initial scan The robot enters the tunnel at a speed of 0.8 m / s, and a solid-state LiDAR and panoramic camera simultaneously collect data at 20 Hz. After FPGA timestamp alignment, the edge GPU segments the point cloud in real time and labels it with categories such as "water accumulation - concrete - silt - fallen blocks". ② Risk prediction A 0.7 m² section of rock was detected on the sidewall at K0+950, ​​accompanied by water seepage. The semantic network output a risk score of 0.82. The robot automatically retreated 5 m, activated its warning flash, and transmitted a "rock drop + water seepage + panoramic view" via LoRa. After the edge box operator confirmed safety, they pressed the "continue" button, and the robot resumed moving forward. ③ Precise inspection and positioning Upon reaching the historically hazardous section K1+800, with a water depth of 0.45 m, the tightly coupled visual-laser-IMU algorithm utilized the sidewall steel mesh as a natural reflective marker, achieving a drift of 2.1 cm over 100 m, meeting the centimeter-level remeasurement requirements. The system generated a semantic cross-section, displaying an average silt thickness of 18 cm, triggering the obstacle removal process. ④ Adaptive obstacle clearing The robot slowed down to 0.2 m / s, lowered its swing arm and tracks, and raised its body by 6 cm to avoid the water inlet being submerged. The quick-change composite head switches to the "spiral + slurry pump" mode, the side mud collection trough is inserted into the mud surface, the centrifugal pump has a head of 20 m, and the mud is directly discharged to the water flow in the center of the tunnel, and carried away by the flow rate of 0.6 m / s. A six-dimensional force sensor monitors in real time; if the contact force is greater than 75 N, the device will retract 2 cm to prevent jamming. The outlet flow meter discharged a cumulative 0.9 m³ of solids, with an error of less than 10% compared to the semantic estimate of 1.0 m³. The system deemed the discharge qualified and retracted the actuator head. ⑤ On-site re-inspection The robot reverses 2 meters, scans the same cross-section again, and the silt thickness is reduced to 3 cm, a reduction rate of 83%. It is marked "passed" and frozen as the new benchmark. ⑥ High-speed return After completing the full tunnel inspection, the edge box detected a sudden increase of 0.15 m in the outlet water level and immediately issued a "return" command. The robot shut down the suction pump, raised its swing arm, and drove towards the tunnel entrance at a full speed of 1.2 m / s against the wind and current. During the journey, the battery SOC dropped to 22%, triggering the second level of energy saving: reducing the number of lidar lines and turning off the supplementary lights, while still maintaining positioning accuracy. ⑦ Data Accumulation After exiting the tunnel, Wi-Fi 6 automatically connected, and within 10 minutes, all semantic sections, high-definition images, and obstacle clearing videos totaling 42GB were uploaded. The edge boxes were stitched into a three-dimensional mesh, and the difference was calculated with the previous digital twin to generate a "block expansion heat map" and a "siltation growth rate curve". The central control room determined that the tunnel volume increased by 5% after this dredging, and the drainage capacity during the flood season met the requirements, so no additional shutdown was required.

[0047] Results: Zero personnel entering the tunnel, avoiding risks of wading, rockfalls, and poisoning; single operation time is 3.5 hours, 40% shorter than manual inspection; centimeter-level retesting ensures that the quantification error of falling blocks is less than 5 mm, providing data support for subsequent support design; mud is carried away by water flow, eliminating the need for secondary transportation outside the tunnel, making it environmentally friendly and saving 50% of costs.

[0048] In the specific implementation process, we provide implementation case two: "dry inspection + pre-dredging" before the transition from the construction period to the operation period; Background: The same drainage tunnel needs to complete the final acceptance inspection before the unit is put into operation. At this time, there is basically no water in the tunnel, but the concrete blocks scattered during construction, shotcrete rebound material, and dust accumulation are 0.3-0.5 m thick. If water is injected directly, it may instantly block the pump outlet; the power station requires a "dry inspection + pre-dredging" to be completed before water is introduced, and an initial digital twin model to be established. Preparation phase: Select the "Dry Inspection" template for the edge box, turn off the water level prediction module, set the silt threshold to 5 cm (more stringent), and turn on the "Stone Crushing" function; The robot has been equipped with a triple composite head consisting of an impact drill, a brush roller, and a negative pressure suction, and the battery has been replaced with a high-energy pack (range > 4 hours). Work process: ① Full-section rapid scanning The robot travels at a speed of 1.5 m / s, and the lidar with a resolution of 1 cm acquires a panoramic view cloud, detecting three large concrete blocks with a volume greater than 0.2 m³ (located at K0+420, K2+100, and K3+050 respectively). ② Large pieces broken At K2+100, the robot first uses an impact drill with 0.8 J of energy to break the concrete blocks into particles smaller than 80 mm. Then, a brush roller pushes the fragments into a negative pressure suction port, which sucks them into a slag collection box (capacity 0.5 m³). Once the slag collection box is full, the robot automatically returns to the opening and pours the waste into a hopper using a tilting cylinder. This cycle is repeated twice to complete the removal of the crushed stone. ③ For fine particle suction and sweeping, the robot slows down to 0.6 m / s, and the brush roller uses 50 Hz micro-vibration + negative pressure 6 kPa to pick up the waste. The PM10 dust removal efficiency is >95%, ensuring that the original concrete surface is exposed on the tunnel wall, meeting the "dust-free" standard for final acceptance. ④ Initial model establishment After scanning the entire tunnel, the system generates the first version of the digital twin mesh, which includes information such as the cross-sectional profile per meter, concrete strength grade markings, and support shotcrete thickness, which will serve as a benchmark for comparison during future operation. ⑤ Verification of gas and temperature / humidity The robot's multi-gas sensors recorded the entire process, showing that CO inside the cave was <10 ppm, O2 >20.8%, and relative humidity was 65%, which met the safety requirements for subsequent limited operations by personnel, thus paving the way for the issuance of a safety certificate for the next stage of water supply operation. Results: Using only one robot, the entire tunnel's "dry inspection + gravel removal + fine particle dust removal" was completed in 4 hours, saving 3 days compared to the traditional "loader + manual labor" solution; the first version of the digital twin model allows for the quantification of changes in rockfall and siltation during subsequent operation to the millimeter level, achieving "traceable changes"; the modular design of the slag collection box prevents gravel from overflowing and undergoing secondary crushing, and the on-site civilized construction was approved by the supervisor in one go.

[0049] Based on the same inventive concept, this application also provides an electronic device. The electronic device of this application includes at least one processor and at least one memory electrically connected to each other. The memory is electrically connected to the processor, and the memory stores instructions executable by the at least one processor. These instructions are executed by the at least one processor to enable the at least one processor to perform the multimodal sensing autonomous inspection robot control method for pumped-flow drainage tunnels as described above.

[0050] It should be noted that the electrical connections between the various units mentioned above do not necessarily represent the connections between lines. Any indirect connection method can be applied to the embodiments of this application as long as it achieves the purpose of this application.

[0051] Based on the same inventive concept, this application also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the multimodal sensing autonomous inspection robot control method for pumped-flow drainage tunnels as described above.

[0052] Based on the same inventive concept, this application also provides a computer program product, which is stored in at least one storage medium; the computer program product includes several instructions to cause at least one computer device to execute the multimodal sensing autonomous inspection robot control method for pumped-storage drainage tunnels as described above.

[0053] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A control method for a multimodal sensing autonomous inspection robot for pumped-flow drainage tunnels, characterized in that, The visual features and laser point cloud of the target inspection robot during its movement in the pumped-flow drainage tunnel are collected synchronously using a multimodal sensor array, and the visual features are projected onto the laser point cloud to generate an initial semantic profile. The initial semantic cross-section is semantically segmented to identify the siltation type and average silt thickness. The initial semantic cross-section is then optimized using a vision-laser-inertial navigation tight coupling factor map based on inertial navigation data. Based on the optimization results, the initial semantic cross-section is matched with an absolute station number to generate an accurate semantic cross-section. Match the corresponding composite execution head tool according to the siltation type in the precise semantic section, and control the composite execution head tool to perform dredging operation on the section corresponding to the absolute station number based on the average silt thickness; The target inspection robot is controlled to reverse and the visual features and laser point cloud of the section corresponding to the absolute station number are re-collected to generate a re-inspection semantic section. The re-inspection semantic section is then compared with the precise semantic section to calculate the silt thickness reduction rate. When the silt thickness reduction rate meets the preset conditions, the re-inspection semantic section is frozen as a new historical benchmark, and the target inspection robot is controlled to autonomously return and exit the tunnel based on the absolute station number in the historical benchmark. When the silt thickness reduction rate does not meet the preset conditions, the composite execution head tool is controlled to perform dredging operations again until the regenerated re-inspection semantic section meets the preset conditions.

2. The method of claim 1, wherein, The controlled target inspection robot moves within the pumped-flow drainage tunnel, simultaneously acquiring visual features and laser point clouds using a multimodal sensor array, specifically including: A second pulse synchronization signal is generated based on a field-programmable gate array (FPGA) to trigger a solid-state lidar and a panoramic camera to simultaneously sample the walls of the pumped-flow drainage tunnel. The structure acoustic microphone array and millimeter-wave water level gauge are synchronously triggered to collect structure acoustic signals and water depth data, so that the timestamp error between the visual features, the laser point cloud, the structure acoustic signal and the water depth data is less than a preset error threshold.

3. The method of claim 2, wherein, The step of projecting the visual features onto the laser point cloud to generate an initial semantic profile specifically includes: Using the laser point cloud as a spatial reference, the visual features are projected onto the three-dimensional point cloud space using a pre-calibrated extrinsic calibration matrix to generate a colored point cloud; Based on the beamforming results of the structured acoustic microphone array, the position coordinates of the sidewall void source are located in the color point cloud, and the position coordinates are superimposed on the color point cloud to generate the initial semantic section.

4. The method of claim 1, wherein, The step of semantic segmentation of the initial semantic cross-section to identify the siltation type and average silt thickness specifically includes: The initial semantic profile is input into a pre-built lightweight transformer network, and the laser point cloud contained in the initial semantic profile is classified point by point to identify the various siltation types contained in the initial semantic profile. The average thickness of silt corresponding to the siltation type in the initial semantic cross-section is calculated based on the classification results.

5. The method of claim 1, wherein, The optimization of the initial semantic profile using a vision-laser-inertial navigation tight coupling factor map incorporating inertial navigation data specifically includes: Using the fixed steel mesh on the tunnel wall as a natural reflector, a joint factor map is constructed that includes visual reprojection error, registration error of the laser point cloud, and pre-integration error of the inertial navigation data. The six-degree-of-freedom pose of the target inspection robot at the initial semantic section is solved based on nonlinear optimization.

6. The method of claim 5, wherein, Based on the optimization results, absolute station numbers are matched to the initial semantic profile to generate accurate semantic profiles, specifically including: The travel distance of the target inspection robot relative to the design axis is calculated based on the six degrees of freedom pose, and the travel distance is converted into the absolute station number by combining the preset start and end station numbers. The absolute station number is associated with and stored with the siltation type and average silt thickness to generate the precise semantic cross-section.

7. The method of claim 1, wherein, The step of matching the corresponding composite execution head tool according to the siltation type in the precise semantic section, and controlling the composite execution head tool to perform dredging operations on the section corresponding to the absolute station number based on the average silt thickness, specifically includes: The target inspection robot is controlled to perform obstacle clearing operations on the current section using the matched composite execution head tool, and the contact force is monitored in real time by a six-dimensional force sensor installed on the composite execution head tool. When the contact force does not exceed the preset threshold, the composite execution head tool continues to be controlled to perform obstacle clearing operations on the current section at the current feed speed, while the cumulative volume of discharged solids is calculated and updated in real time through the outlet flow meter. When the contact force exceeds the preset threshold, the composite execution head tool is controlled to retract and the obstacle clearing operation continues after the feed speed is reduced until the cumulative volume of discharged solids reaches the amount of silt estimated based on the average thickness of the silt, at which point the obstacle clearing stops.

8. The method of claim 1, wherein, The process of controlling the target inspection robot to reverse and re-acquiring the visual features and laser point cloud of the section corresponding to the absolute station number to generate a re-inspection semantic section specifically includes: The target inspection robot is controlled to reverse along the original driving path by a preset distance, and the visual features and laser point cloud of the section corresponding to the absolute station number are reacquired using the multimodal sensor array in the same acquisition method as the initial semantic section is generated. The reacquired visual features are projected onto the reacquired laser point cloud to generate the re-examined semantic profile.

9. The method of claim 1, wherein, The step of comparing the re-examined semantic cross-section with the precise semantic cross-section to calculate the silt thickness reduction rate specifically includes: The thickness of the dredged silt is extracted from the re-inspection semantic section, and the initial silt thickness before dredging is extracted from the precise semantic section. The silt thickness reduction rate is calculated based on the initial silt thickness and the re-inspected silt thickness.

10. The method of claim 1, wherein, When the silt thickness reduction rate does not meet the preset condition, record the number of times the current condition is not met, control the composite execution head tool to perform silt removal operation on the section corresponding to the absolute station number again and regenerate the re-inspection semantic section until the silt thickness reduction rate meets the preset condition or the number of silt removals reaches the preset target number threshold.

11. The method of claim 1, wherein, Based on the absolute station number in the historical benchmark, the odometer is reversed, and during the return journey, the exit environment parameters broadcast by the outer edge box of the tunnel are received through the ground communication link. When the exit environment parameters meet the preset abnormal conditions, the target inspection robot is controlled to close the dredging mechanism and adjust the chassis attitude to drive towards the tunnel entrance with the target traction force.

12. A multimodal sensing autonomous inspection robot control system for pumped-flow drainage tunnels, characterized in that, The system includes: The initial scanning module uses a multimodal sensor array to synchronously collect visual features and laser point clouds of the target inspection robot during its movement in the pumped and self-flowing drainage tunnel, and projects the visual features onto the laser point cloud to generate an initial semantic profile. The precision inspection module performs semantic segmentation on the initial semantic cross-section to identify the siltation type and average silt thickness, and optimizes the initial semantic cross-section using a vision-laser-inertial navigation tight coupling factor map based on inertial navigation data. Based on the optimization results, it matches the initial semantic cross-section with absolute station numbers to generate a precise semantic cross-section. The obstacle removal module matches the corresponding composite execution head tool according to the siltation type in the precise semantic section, and controls the composite execution head tool to perform silt removal operations on the section corresponding to the absolute station number based on the average silt thickness; The re-inspection module controls the target inspection robot to reverse and re-collect the visual features and laser point cloud of the section corresponding to the absolute station number to generate a re-inspection semantic section, and compares the re-inspection semantic section with the precise semantic section to calculate the silt thickness reduction rate. The return module freezes the re-inspection semantic section as a new historical benchmark when the silt thickness reduction rate meets the preset conditions, and controls the target inspection robot to autonomously return and exit the tunnel based on the absolute station number in the historical benchmark; when the silt thickness reduction rate does not meet the preset conditions, it controls the composite execution head tool to perform dredging operations again until the regenerated re-inspection semantic section meets the preset conditions.

13. An electronic device, characterized in that, Includes at least one processor and at least one memory electrically connected; The memory is electrically connected to the processor, wherein the memory stores instructions executable by at least one of the processors, the instructions being executed by at least one of the processors to enable at least one of the processors to perform the multimodal sensing autonomous inspection robot control method for pumped-flow drainage tunnels as described in any one of claims 1-11.

14. A computer storage medium, characterized in that, The computer-readable storage medium stores a computer program. When the computer program is executed by the processor, it implements the multimodal sensing autonomous inspection robot control method for pumped-flow drainage tunnels as described in any one of claims 1-11.

15. A computer program product, characterized in that, The computer program product is stored in at least one storage medium; The computer program product includes several instructions to cause at least one electronic device to execute the multimodal sensing autonomous inspection robot control method for pumped-storage drainage tunnels as described in any one of claims 1-11.

Images