Intelligent sensing robot system for tunneling face

Abstract

This application provides an intelligent sensing robot system for tunneling faces, belonging to the field of underground robots. The system includes a slide rail fixed to the top of the tunneling roadway where the tunneling machine is located, with a liftable hanging robot mounted on the slide rail. Fixed targets are also spaced along the slide rail. The hanging robot is equipped with a vision sensor, lidar, event camera, and inertial measurement unit. Infrared targets are installed on both sides of the tunneling machine, and a strapdown inertial navigation system is also installed on the tunneling machine. The tunneling machine and the hanging robot interact through a two-way interactive collaborative sensing mechanism to achieve continuous positioning of the tunneling machine. Furthermore, by fusing multi-source sensor data from the hanging robot, real-time digital construction of the tunneling roadway's three-dimensional perception is achieved. This application solves the problems of insufficient sensing stability and poor real-time performance of existing technologies, enabling accurate and stable perception of the tunneling machine's position and attitude.

Description

Technical Field

[0001] This application relates to the field of downhole robot technology, and in particular to an intelligent sensing robot system for tunneling faces. Background Technology

[0002] Currently, intelligent sensing and positioning technologies for tunneling faces have become core means to improve mining efficiency and safety. Domestic and international research institutions and enterprises have achieved a series of representative results in this field, laying a preliminary foundation for intelligent tunneling. Domestically, one university proposed a pose perception technology based on three laser beams, another developed a visual target recognition method, and a company introduced dust adaptive technology to improve perception robustness. In addition, many research institutes and universities have also promoted the development of this field through different technical paths such as autonomous navigation, multi-sensor fusion, and vision-inertial integration. International research focuses primarily on high-precision laser sensing and machine vision systems, improving the accuracy of pose estimation in complex environments by optimizing optical path design and image algorithms.

[0003] Despite the diversity of technical approaches and the fact that some systems (such as LiDAR SLAM and multi-sensor fusion positioning) have achieved centimeter-level positioning in structured roadways, suppressed cumulative errors in inertial navigation, and improved all-weather operation capabilities, significant limitations still exist. Existing solutions generally face severe challenges from the extreme underground environment, including GPS rejection, complex roadway structures, electromagnetic interference, dust pollution, and multi-source interference such as moisture and vibration. This results in insufficient sensing stability, poor real-time performance of algorithms, high system costs, and the need for frequent calibration, making it difficult to meet the actual requirements of continuous, high-precision, and robust positioning for underground tunneling equipment in coal mines. Summary of the Invention

[0004] To address the aforementioned technical issues, this application proposes an intelligent sensing robot system for tunneling faces.

[0005] The technical solution adopted in this application is: an intelligent sensing robot system for tunneling face, including a slide rail fixed to the top of the tunneling roadway where the tunneling machine is located, a liftable hanging robot installed on the slide rail, fixed targets with known coordinates set at intervals on the tunneling roadway, and a visual sensor, a lidar, an event camera and an inertial measurement unit mounted on the hanging robot.

[0006] Infrared targets are installed on both sides of the tunneling machine. The tunneling machine is also equipped with a strapdown inertial navigation system. The tunneling machine and the rail-mounted robot interact through a two-way interactive collaborative perception mechanism to achieve continuous positioning of the tunneling machine. The tunneling machine also achieves real-time digital construction of three-dimensional perception of the tunneling roadway by integrating multi-source sensor data from the rail-mounted robot.

[0007] A self-healing digital twin model is deployed on the monitoring terminal of the tunneling machine. The self-healing digital twin model is used to generate a three-dimensional digital twin covering the surrounding rock of the tunnel, the tunneling machine, and the support structure.

[0008] Furthermore, the system also includes acoustic sensors fixed to the body of the tunneling machine and a micro-vibration sensor array fixed to the surrounding rock and floor of the tunnel. These acoustic sensors and micro-vibration sensor array serve as redundant sensing channels and are introduced when the lidar and vision sensors are severely degraded to sense the position of the tunneling machine.

[0009] Furthermore, the system also constructs a personnel location tracking system through positioning base stations and intelligent visual monitoring networks deployed in the tunnels, as well as positioning devices worn by workers, to achieve real-time proactive perception and prediction of personnel and processes in the tunnels.

[0010] Furthermore, the system also includes magnetic wall-mounted robots, wheeled / tracked ground robots, and hovering drones, which together with the rail-mounted robots form a distributed perception network at the tunneling face. Each robot shares its pose, point cloud, and environmental parameters in real time through a wireless self-organizing network to form a global map.

[0011] Furthermore, the visual sensor can accurately identify the tunneling machine and its spatial location in low-light or dusty environments by recognizing the infrared targets installed on both sides of the tunneling machine. Combined with the machine's strapdown inertial navigation system, it forms a complete positioning and perception system.

[0012] Meanwhile, during the positioning process, the visual sensor and the lidar work together. The visual sensor is used to identify and lock the feature points of the infrared target, while the lidar provides accurate distance measurement. The two types of data are fused with the fuselage strapdown inertial navigation data and then optimized by Kalman filtering.

[0013] When the tunneling machine enters the observation range of the fixed target, the vision sensor will automatically capture the fixed target as a correction marker and perform reverse calibration.

[0014] Furthermore, the real-time digital construction of three-dimensional perception of the tunneling roadway is achieved by integrating multi-source sensor data from the rail-mounted robot, including:

[0015] a. Sensor spatiotemporal alignment: A weighted approach is used to achieve spatiotemporal alignment of multi-source sensors;

[0016] b. Prior geometric constraints: Use prior geometric constraints to correct outliers in the roadway geometric model, minimizing the distance between the point cloud and the points on the roadway geometric model;

[0017] c. The final output is a continuous 3D model of the tunnel.

[0018] Furthermore, the two-way interactive collaborative sensing mechanism includes:

[0019] Airborne sensor data feedback: The tunneling machine transmits its onboard inertial navigation data, vibration data, and operating status data in real time;

[0020] Rail-mounted robot reverse feedback: The rail-mounted robot returns the real-time modeled spatial point cloud and positioning information to the tunneling machine, forming a complementary relationship;

[0021] Closed-loop correction mechanism: The positioning result of the tunneling machine is corrected by the rail-mounted robot, and the point cloud modeling of the rail-mounted robot is based on the posture of the tunneling machine body, thus achieving dual verification;

[0022] Collaborative optimization algorithm: Introducing a "mutual verification filter" based on Bayesian inference to cross-validate and fuse multi-source data from tunneling machines and rail-mounted robots.

[0023] Furthermore, the personnel location tracking system introduces a spatiotemporal data analysis model to integrate personnel trajectories, equipment operating status, and environmental parameters in multiple dimensions, forming a predictive mechanism for process evolution.

[0024] Furthermore, the system uses multi-sensor fusion technology to achieve accurate identification and positioning of anchor holes and anchor bolts / cables, and combines roadway surface displacement monitoring data to establish a correlation model between support effect and surrounding rock deformation.

[0025] Furthermore, the functionality of the self-healing digital twin model includes:

[0026] Breakpoint completion: When point cloud data is lost, historical modeling data, surrounding rock mechanics prediction model and roadway geometric priors are called to quickly complete the breakpoint;

[0027] Predictive Correction: When sensors are temporarily unavailable, the self-healing digital twin model uses a Kalman-Transformer-based time-series prediction model to infer the evolution trend of tunnel morphology.

[0028] Adaptive fusion: When the sensed data is recovered, the new data and the completed data are fused through differential allocation and residual minimization algorithms to ensure continuous and seamless connection;

[0029] Anomaly self-correction: Identifies "false point clouds" caused by dust or vibration, and automatically corrects them through a triple constraint of history, prediction, and geometry;

[0030] The construction of a digital twin model can be abstracted as follows:

[0031] ;

[0032] in, Indicates time Digital twin model of the tunnel; It is a geometric prior set; These are the operating parameters for the tunneling equipment; For surrounding rock monitoring data; The state of the support structure; Modeling and evolution functions for twins.

[0033] The advantages of this application over the prior art are as follows:

[0034] Real-time digital construction of tunnels: Using lidar and vision sensors, the working tunnels are continuously scanned to generate high-precision 3D point cloud maps and digital models in real time, dynamically reflecting changes in tunnel morphology.

[0035] Real-time monitoring of cutting and shaping: By accurately tracking the position and posture of the tunneling machine's cutting head, and combining it with the real-time constructed roadway model, the degree of conformity between the actual cutting section and the design section is dynamically evaluated, providing a basis for cutting quality control and process optimization.

[0036] Tunnel deformation monitoring: By periodically scanning and comparing with historical models, it automatically identifies deformation signs such as convergence and delamination of the tunnel roof and walls, providing early warning information for tunnel stability assessment and support decisions.

[0037] Real-time monitoring of personnel and work processes: Utilizing visual perception (infrared binocular cameras, event cameras) and spatial positioning technology, the system can identify the location and activity status of personnel within the work area in real time, and automatically identify the execution status of key work processes (such as cutting, machine shutdown, drill rod replacement, etc.).

[0038] Automated support assistance and evaluation:

[0039] Support Anchor Bolt / Anchor Hole Identification and Positioning: In automated support operations, the system can accurately identify the location of installed anchor bolts or anchor holes to be constructed, providing precise positioning guidance for the support robotic arm.

[0040] Support condition monitoring and evaluation: Combining visual inspection and spatial data, the system automatically identifies the installation status of anchor trays (such as whether they are close to the rock surface and whether the angle is compliant), the mesh laying status, etc., and conducts a preliminary automated evaluation of the support construction quality.

[0041] Working face space medium perception: The robot platform can be flexibly equipped with dust concentration sensors, gas (methane) sensors and other hazardous gas sensors, and can move or stay in the roadway as needed to monitor, so as to realize the fine and dynamic perception of dust distribution, gas concentration and other harmful gases in the working space, and provide key data for environmental safety and health protection.

[0042] This intelligent sensing robot system, through innovative platform design, advanced multi-source information fusion navigation and positioning technology, and rich integrated sensing functions, effectively overcomes the challenge of precise positioning of tunneling machines in complex and harsh environments. It achieves reliable, real-time, and multi-dimensional sensing of all elements of the tunneling face, including personnel, machine, environment, and pipes. It provides indispensable "eyes" and "brain" for intelligent tunneling operations in coal mines, serving as a key sensing infrastructure for achieving the goals of reduced manpower, unmanned operation, and intelligent management of tunneling faces. This is of great significance for improving tunneling efficiency, ensuring operational safety, optimizing support quality, and achieving refined management. Attached Figure Description

[0043] The following description, in conjunction with the accompanying drawings, further illustrates this application:

[0044] Figure 1 This is a schematic diagram of the system structure provided in the embodiments of this application;

[0045] Figure 2 A flowchart illustrating continuous positioning via multi-sensor fusion provided in this application embodiment;

[0046] Figure 3 A flowchart illustrating the real-time digital construction process of three-dimensional perception of tunnels provided in this application embodiment;

[0047] Figure 4 A flowchart of real-time active perception and prediction of personnel and processes in tunneling roadways provided in this application embodiment;

[0048] Figure 5 A flowchart illustrating the precise identification and positioning of anchor holes and anchor bolts / cables in tunneling support, and the perception of support status, provided in this application embodiment.

[0049] Figure 6 A system overall workflow diagram provided for embodiments of this application;

[0050] In the diagram: 1 is the strapdown inertial navigation system, 2 is the infrared target, 3 is the tunneling machine, and 4 is the rail-mounted robot. Detailed Implementation

[0051] In underground tunneling operations in coal mines, there are generally harsh working conditions such as extremely high dust concentration at the working face, easy slippage of the tunneling machine tracks under complex floor conditions, and strong mechanical vibrations generated during equipment operation. These factors seriously interfere with the accurate and stable perception of the position and attitude of the tunneling machine (i.e., the "difficulty in machine positioning"), becoming one of the key bottlenecks restricting the improvement of the level of intelligence in tunneling operations.

[0052] To address the aforementioned core challenges and deeply integrate with the operational characteristics of coal mine tunneling equipment, this application proposes an innovative rail-mounted intelligent sensing robot system for tunneling faces. This system abandons the traditional approach of relying on sensors mounted on the tunneling machine itself, instead employing a rail-mounted mobile platform independent of the tunneling machine. This platform is equipped with an advanced multi-source sensor array, including an infrared binocular camera as a visual sensor, a high-resolution LiDAR, a high dynamic range event camera, and a high-precision inertial measurement unit (IMU), thus forming the rail-mounted robot. By deeply integrating information from these heterogeneous sensors, the system constructs an autonomous navigation system with bidirectional calibration capabilities for tracking and positioning. This navigation system not only overcomes the environmental impacts of high dust, low illumination, and strong vibration, continuously and stably tracking the absolute position and relative attitude of the tunneling machine, but also significantly improves the robustness and accuracy of positioning through cross-validation and closed-loop feedback of multi-sensor information, providing a reliable spatial reference for subsequent intelligent operations.

[0053] like Figures 1 to 6 As shown, the intelligent sensing robot system of this application mainly implements the following functions:

[0054] (1) High-precision continuous positioning information perception of tunneling equipment

[0055] In terms of high-precision continuous positioning of the tunneling equipment, the system employs rail-mounted and multi-sensor fusion technology to achieve precise tracking of the position and attitude of the tunneling machine 3. The rail-mounted robot 4 is fixed on an extendable rail at the top of the tunnel, equipped with a high-precision lidar and a vision sensor with deep learning capabilities. It can move along the tunnel axis and flexibly adjust its observation height. The vision sensor identifies the infrared targets 2 mounted on both sides of the tunneling machine 3, accurately identifying the tunneling machine 3 and its spatial orientation even in low-light or dusty environments. Combined with the machine's strapdown inertial navigation system 1, this forms a complete positioning and perception system. During positioning, the vision sensor and lidar work together. The former identifies and locks onto feature points of the infrared targets 2 on the machine, while the latter provides precise distance measurement. The two types of data are fused with the machine's strapdown inertial navigation data and optimized using Kalman filtering, effectively eliminating multi-source measurement errors. Simultaneously, fixed targets with known coordinates and physical markings are set at intervals along the tunneling roadway. When the tunneling machine 3 enters the observation range of the fixed targets, the vision sensor automatically captures the fixed targets as correction markers, thereby quickly, stably, and accurately identifying the correction representation from the complex background. Simultaneously, by comparing the difference between the "observed value" and the "true value," reverse calibration is performed, the current system error is calculated, and immediate correction is made, ensuring centimeter-level positioning accuracy even under complex conditions such as rapid movement and temporary obstruction. Furthermore, the system incorporates an intelligent track extension and coordinate system transformation mechanism. During track extension, the vision sensor automatically identifies joint marks to complete coordinate reference switching, achieving seamless connection of the positioning system during tunneling. The overall architecture ensures the continuity, reliability, and long-term stability of positioning, and through vision-assisted anomaly detection and automatic alarm functions, the system can continuously provide a high-precision spatial reference for the automated control and digital management of tunneling equipment in harsh environments.

[0056] The system uses multi-sensor fusion to achieve continuous positioning, and its estimated position and attitude can be expressed as:

[0057] ;

[0058] in, Indicates the time of the tunneling machine The estimated position and attitude; This is laser ranging data; Target features identified by visual sensors; For inertial navigation data of the fuselage; This is a Kalman filter fusion operator. The formula embodies a weighted fusion mechanism of multi-source information, including laser, vision, and inertial sensors, thus maintaining centimeter-level positioning accuracy even under complex conditions such as dust and obstruction.

[0059] Furthermore, the system constructs a redundant sensing system in addition to optical and inertial navigation, including:

[0060] Acoustic positioning: By analyzing the echo, attenuation, and multipath characteristics of low-frequency sound waves emitted by acoustic sensors as they propagate in the tunnel, a geometric model of the tunnel and a 3D position estimate of the tunnel boring machine are established, maintaining sensing capabilities even in situations with extremely high dust density and complete failure of optical equipment.

[0061] Seismic wave sensing: Microseismic sensor arrays are deployed in the surrounding rock and floor slab. The vibration waves generated by the operation of the tunnel boring machine 3 are used as the excitation source. The real-time position of the tunnel boring machine 3 is inverted by propagation delay and waveform characteristics.

[0062] To further enhance the system's robustness under extreme conditions, redundant sensing channels based on acoustic / seismic waves were introduced. When the lidar and visual sensors severely degrade, the acoustic sensors utilize the echoes and time-of-flight (TOF) of low-frequency sound waves for localization.

[0063] ;

[0064] in, Represents the spatial coordinates (x, y, z) of the rail-mounted robot (or tunneling machine) to be solved; Indicates the first The spatial coordinates of a sound source (emitter or known fixed point); This indicates the speed of sound (approximately 340 m / s, depending on temperature and air pressure). This represents the measured sound wave propagation delay (the time it takes for a sound wave to travel from the sound source to the microphone). Point To the sound source The geometric distance.

[0065] Simultaneously, a microseismic sensor array is deployed in the surrounding rock and floor slab. The relative time difference between the arrival of the seismic wave at two different sensors is accurately measured using the cross-correlation method. This provides the most fundamental and crucial input data for the subsequent "wave velocity inversion" positioning algorithm. Even under extreme conditions where optical and acoustic sensors fail, the positioning capability of the tunnel boring machine 3 is maintained by analyzing the vibration, thus ensuring the high robustness and continuity of the entire system. Its mathematical expression is as follows:

[0066] ;

[0067] in: Indicates the first Vibration signals collected by a seismic wave sensor; For time delay variables; This represents the time delay when the correlation between two signals is at its maximum.

[0068] And combined with wave speed Inverting the position of tunnel boring machine 3. This is achieved through adaptive weight fusion. It can dynamically switch between optical / acoustic / seismic wave channels to ensure the continuity and fault tolerance of the system.

[0069] (2) Real-time digital construction of three-dimensional perception of tunnels

[0070] In terms of 3D perception and digital modeling, the system integrates multiple sensors, including infrared binocular cameras, LiDAR, event cameras, and IMUs, to form a high-precision perception system adapted to the complex working conditions of coal mines. The infrared binocular cameras employ active infrared imaging, enabling stable acquisition of depth images even under conditions of no light and high dust levels; the LiDAR provides high-precision point clouds; the event cameras capture dynamic edge information with microsecond-level latency, compensating for blurring caused by low light and rapid movement; and the IMU ensures the continuity of the trajectory and high-frequency updates of attitude information.

[0071] To achieve unified processing of multi-source heterogeneous data, the system adopts a continuous temporal modeling framework based on SE(3) B-splines, mapping the outputs of multiple sensors to a unified trajectory and combining it with IMU for motion compensation. Simultaneously, an adaptive weighting mechanism is introduced to dynamically adjust the weights of each sensor based on dust concentration, illuminance, and vibration intensity, ensuring an adaptive fusion effect. Under high dust conditions, the system further combines a radiative transfer physical model with a neural network residual compensation framework to form a physical-neural hybrid compensation model, significantly reducing extinction and scattering interference and achieving digital modeling of dust distribution. Event cameras and lidar achieve sub-millisecond time synchronization through cross-modal alignment, ensuring accurate modeling in rapid vibration and dynamic scenarios. During the optimization phase, the system simultaneously utilizes the sparse dynamic features of the event camera and the dense geometric constraints of the lidar / infrared binocular camera to jointly improve modeling robustness. The final output 3D model also incorporates roadway geometric priors and surrounding rock mechanics evolution models for closed-loop correction, enabling prediction of roadway morphology and correction of anomalous point clouds. The overall solution breaks through the perception bottleneck under conditions of high dust, low light and strong vibration, and can maintain centimeter-level accuracy and continuity for a long time, providing a robust and real-time three-dimensional digital tunnel model for tunneling operations.

[0072] The system achieves high-precision 3D perception and digital modeling of coal mine tunnels through multi-source fusion of infrared binocular cameras, LiDAR, event cameras, and IMUs. Its core innovations include a unified spatiotemporal reference, a concise physical-neural hybrid compensation model, and a closed-loop prior geometric constraint. The underlying algorithm principles are explained below.

[0073] a. Sensor spatiotemporal alignment

[0074] Data from different sensors are not synchronized in time and have different noise characteristics. Weighted spatiotemporal alignment is employed.

[0075] ;

[0076] in, Indicates the moment after fusion A unified coordinate or state; Indicates the first Observations from individual sensors (such as point cloud, depth, inertial navigation); The sensor weights are dynamically adjusted based on their noise variance and stability to satisfy... .

[0077] b. Prior geometric constraints, mainly those concerning the roadway's prior geometric constraints. The roadway cross-section is mostly close to an ellipse or arch. The roadway geometric model is corrected using these prior geometric constraints.

[0078] ;

[0079] in, Represents a set of sensor point clouds; Represents a point on the geometric model of the tunnel; Represents the geometric prior set of the tunnel (such as ellipse, circular arc, rectangular arch).

[0080] The goal of the above formula is to make the points on the point cloud and the tunnel geometry model... The distance is the smallest;

[0081] Constraints: It must belong to the geometric prior set of the roadway. (For example, ellipse, arch, etc.).

[0082] c. Fusion modeling output

[0083] The final output is a continuous 3D model of the tunnel:

[0084] ;

[0085] in, Represents a dynamically updated 3D model of the tunnel; This represents the point cloud / surface after compensation and geometric constraint correction; This represents the union operation, used to merge point clouds at different times into a unified three-dimensional model of the tunnel.

[0086] The core architecture of the physics-neural hybrid compensation model consists of two progressive and complementary layers, like a precisely coordinated "double filter," achieving deviation correction from coarse to fine. The first layer is the physical mechanism compensation layer, based on known natural laws, to perform "basic correction" for sensor data. Faced with diffuse dust, it uses the laws of light radiation transmission and atmospheric scattering principles to analyze the attenuation difference between the emitted and received light intensity, thus deducing the deviation between the true distance and the measured value. Simultaneously, addressing the blurring problem of visual images, it combines the correlation between ambient light intensity and pixel distribution to initially restore image clarity, clearing obstacles for subsequent target identification and feature extraction. For the strong vibrations generated by the tunneling machine 3, the model constructs an error model for the inertial measurement unit based on the power characteristics of vibration noise. By analyzing the relationship between vibration frequency, amplitude, and positioning drift, it effectively suppresses the cumulative error of inertial data, solidifying the foundation for positioning stability.

[0087] If the physical mechanism compensation layer addresses the systematic interference that is "traceable," then the second layer, the neural network residual correction layer, specializes in dealing with unsystematic deviations that are "unpredictable." The underground environment is constantly changing; uneven dust particle size distribution, sudden vibrations and shocks, and abrupt changes in localized lighting are common occurrences. A simple physical model cannot fully cover all these dynamic disturbances. At this point, the neural network leverages its adaptive learning advantage, using physically compensated sensor data and real-time environmental parameters (such as dust concentration and vibration frequency) as input. By learning the deviation patterns under numerous different interference scenarios, it accurately captures residual errors not covered by the physical model. Finally, by superimposing the physical compensation results with the residual output of the neural network, accurate and reliable sensing data is obtained, providing reliable support for core tasks such as tunnel modeling and tunnel boring machine positioning.

[0088] As a powerful tool for anti-interference in downhole intelligent sensing, the physical-neural hybrid compensation model offers multiple advantages. It overcomes the limitations of poor generalization of single physical models and the difficulty in understanding the "black box" nature of pure neural networks. The clear mechanism of the physical compensation layer ensures interpretability and debuggability in engineering applications, while the adaptive capability of the neural network residual correction layer allows the model to flexibly adapt to dynamically changing downhole environments without frequent reconstruction. Simultaneously, the lightweight architecture ensures the model's real-time response capability, meeting the high-frequency sensing requirements of downhole sensors and significantly improving data accuracy without affecting the overall system efficiency.

[0089] (3) Collaborative perception mechanism of the track-mounted robot:

[0090] Traditional sensing methods mostly involve unidirectional observation between the tracked robot 4 and the tunneling machine 3. This application proposes a bidirectional interactive collaborative sensing mechanism:

[0091] The tunneling machine 3 transmits state variables such as IMU, load, and vibration back in real time, while the rail-mounted robot 4 feeds back the three-dimensional point cloud and positioning results, forming a closed loop.

[0092] Airborne sensor data feedback: Tunneling machine 3 transmits its onboard inertia, vibration, cutting load and other operating status data in real time.

[0093] Rail-mounted robot 4 provides feedback: Rail-mounted robot 4 returns the real-time modeled spatial point cloud and positioning information to tunneling machine 3, forming a complementary relationship.

[0094] Closed-loop correction mechanism: The positioning result of tunneling machine 3 is corrected by rail-mounted robot 4, and the point cloud modeling of rail-mounted robot 4 is based on the posture of tunneling machine 3, thus achieving dual verification.

[0095] Collaborative optimization algorithm: Introducing a "mutual verification filter" based on Bayesian inference to cross-validate and fuse multi-source data from tunneling machine 3 and rail-mounted robot 4, significantly reducing the cumulative error caused by single-source bias.

[0096] This two-way interactive collaborative sensing mechanism enables "machine-to-machine mutual calibration," which not only improves positioning accuracy and modeling stability but also ensures continuous system operation even when any platform fails.

[0097] The system uniformly models the states of "tunneling machine-rail-mounted robot-environment" and assigns state vectors. Defined as:

[0098] ;

[0099] in, Indicates the location of the tunneling machine; Indicates the attitude (angle) of the tunneling machine; Indicates the position of the track-mounted robot; Represents the global map (point cloud / model).

[0100] Cross-validation and fusion of multi-source data are performed using mutual-validation Bayesian filtering and Kalman fusion:

[0101] ;

[0102] in, Represents the probability of the system state after a given observation; This indicates the sensor observations (IMU, load, etc.) of the tunneling machine. This represents the sensor observations (point cloud map, absolute positioning) of the rail-mounted robot. This indicates the state prediction at the previous moment.

[0103] The positioning robustness is improved by two-way mutual verification between the tunneling machine 3 and the rail-mounted robot 4.

[0104] Kalman filtering is used to achieve the optimal balance between prediction and measurement.

[0105] ;

[0106] in, Indicates time The predicted state; Indicates the actual observed value; This represents the observation model (which maps states to observations).

[0107] This two-way interactive collaborative sensing mechanism effectively reduces single-source drift and improves long-term robustness.

[0108] (4) Real-time proactive perception and prediction of personnel and processes in tunneling roadways

[0109] In terms of personnel and process perception and prediction, the system achieves intelligent management of the operation process through multi-source perception technology. Relying on precise positioning base stations and intelligent visual monitoring networks deployed in the tunnels, combined with positioning devices worn by workers, the system can build a comprehensive personnel location tracking system and perceive personnel activity status in real time. Based on deep learning-based computer vision algorithms, the system can automatically identify the execution of typical processes in tunneling operations, including cutting, shutdown, and drill pipe replacement, and analyze and record the compliance of process execution. On this basis, the system innovatively introduces a multi-source spatiotemporal fusion model, which integrates personnel trajectories, equipment operating status, and environmental parameters in multiple dimensions to form a predictive mechanism for process evolution, enabling early prediction of operation progress trends and identification of potential delay risks. When abnormal working conditions are detected, the system automatically generates early warning prompts to ensure the continuity and safety of tunneling operations. The system architecture adopts an edge computing and cloud collaboration approach, with underground intelligent terminals ensuring real-time response and local decision-making capabilities, while the cloud platform provides support for complex in-depth analysis and historical data mining. The accompanying mobile application terminal provides on-site managers with intuitive and convenient operation monitoring tools, enabling integrated support for personnel management, process analysis, and progress forecasting.

[0110] Process perception and prediction adopts a multi-source spatiotemporal fusion model, the expression of which is:

[0111] ;

[0112] in, Indicating a future moment The predicted results of the process status or work progress; For the trajectory data of the workers; This refers to equipment operating status information; For environmental parameters (gas, energy consumption, dust, etc.); This is a spatiotemporal fusion prediction model. The formula embodies the multi-dimensional data fusion of "personnel + equipment + environment", enabling early prediction of process progress trends and potential risks.

[0113] (5) Accurate identification and positioning of anchor holes and anchor bolts / anchor cables in tunneling roadways, and perception of support conditions.

[0114] In terms of full-process monitoring and guidance of support operations, the system achieves accurate identification and positioning of anchor holes and anchor bolts / cables through multi-sensor fusion technology. The combination of high-precision lidar and visual sensors constructs a 3D point cloud model of the tunnel surface and automatically identifies anchor hole locations and completes 3D coordinate calibration using deep learning algorithms, thus providing sub-centimeter-level positioning guidance for the anchoring robotic arm. Through feature matching and point cloud registration technology, the system can continuously track anchor hole positions at different construction stages, avoiding identification errors caused by tunneling progress or construction disturbances. In the support quality inspection stage, the system integrates multispectral imaging and ultrasonic testing technologies to automatically identify and assess the condition of installed anchor bolts / cables, analyze the anchoring agent filling status and prestress loss, and promptly detect support defects. Combined with tunnel surface displacement monitoring data, the system further establishes a correlation model between support effectiveness and surrounding rock deformation, providing a scientific basis for support parameter optimization and process improvement. In terms of architecture, the system adopts an edge computing model, deploying intelligent analysis terminals underground to achieve real-time data processing and feedback. It also features an intuitive operation guidance interface that dynamically displays anchor hole positioning deviations and support quality levels. When abnormal support conditions are detected, the system automatically generates reinforcement or process adjustment suggestions to help operators make quick decisions, forming a closed-loop management process of "monitoring-analysis-guidance," effectively ensuring the safety and standardization of roadway support operations.

[0115] The support quality evaluation can be modeled as follows:

[0116] ;

[0117] in, Indicates the evaluation indicators for support quality; The point cloud coordinates obtained by laser scanning are used for anchor hole location calibration; Image features for industrial cameras or multispectral imaging; The ultrasonic detection signal is used to analyze the anchoring agent filling status and prestress loss. For roadway surface displacement monitoring data; This is a multimodal fusion and quality analysis function. The formula embodies the fusion analysis approach of "point cloud + image + ultrasound + displacement monitoring," enabling automated evaluation of support deviations and quality.

[0118] (6) Self-healing digital twin model of tunneling equipment-environment integration

[0119] In terms of building a digital twin model that integrates equipment and environment, the system takes a unified spatiotemporal benchmark as its core and integrates multi-source heterogeneous data such as geological exploration data, tunneling equipment operating parameters, and environmental monitoring information in real time to generate a three-dimensional digital twin covering elements such as roadway surrounding rock, tunneling machine 3, and support structure, thereby realizing synchronous mapping and interaction between physical space and virtual space.

[0120] The functions of the self-healing digital twin model include:

[0121] Breakpoint completion: When point cloud data is lost, historical modeling data, surrounding rock mechanics prediction models, and roadway geometric priors are invoked for rapid completion.

[0122] Predictive Correction: When sensors are temporarily unavailable, the self-healing digital twin model uses a Kalman-Transformer-based time-series prediction model to infer the evolution trend of tunnel morphology.

[0123] Adaptive fusion: When the sensed data is recovered, the new data and the supplementary data are fused through differential allocation standardization and residual minimization algorithms to ensure continuous and seamless connection.

[0124] Anomaly self-correction: It can identify "false point clouds" caused by dust or vibration and automatically correct them through a redundant model (history-prediction-geometric triple constraints).

[0125] The self-healing digital twin model proposed in this application breaks through the limitation of existing digital twins where "data interruption means modeling interruption", ensuring the continuity and reliability of the twin.

[0126] It can maintain model continuity even when point cloud is lost or sensors fail. The history completion formula is:

[0127] ;

[0128] in, This represents the model generated by historical interpolation; A model representing past moments; This represents the interpolation function.

[0129] Prediction correction uses Transformer:

[0130] ;

[0131] in, This represents the predicted future model; Represents historical time-series data; This represents a deep learning model.

[0132] At the same time, AI can be used to predict model morphology, avoiding data loss that could cause "gap".

[0133] The final fusion is represented as:

[0134] ;

[0135] in, Represents the global map; This represents the weighting coefficient, which is dynamically adjusted based on the sensor's health status. This represents real-time sensor data.

[0136] And when real data is insufficient, it is supplemented by historical and forecast results.

[0137] Geometric prior constraints are used to correct outliers and maintain the rationality of the digital twin model:

[0138] ;

[0139] in, This represents the point cloud collected by the sensor; Represents geometric curves or surfaces; Represents the geometric prior set (such as the cross-section of an arched or elliptical tunnel).

[0140] The construction of a digital twin model can be abstracted as follows:

[0141] ;

[0142] in, Indicates time Digital twin model of the tunnel; It is a geometric prior set, including geological exploration data; These are the operating parameters for the tunneling equipment; This includes monitoring data for the surrounding rock, such as deformation and stress. The state of the support structure; This provides a modeling and evolution function for twins. The formula embodies a multi-source fusion mechanism of "geology + equipment + surrounding rock + support," enabling dynamic synchronization and interaction between physical and virtual spaces.

[0143] (7) Multi-agent distributed cooperative perception

[0144] In addition to the rail-mounted robot 4, this application introduces multiple types of small robots (wall-mounted, ground-based, and unmanned aerial vehicles) to form a distributed sensing network.

[0145] Multiple types of small robots: including magnetic wall-mounted robots, wheeled / tracked ground robots and hovering drones, forming a multi-dimensional deployment on the tunnel face.

[0146] Distributed sensing network: Each robot shares pose, point cloud and environmental parameters in real time through a wireless self-organizing network (Mesh).

[0147] Each robot first performs SLAM locally:

[0148] ;

[0149] in, Indicates the first A local map generated by a robot; Indicates the first One robot; This represents the sensor input data.

[0150] Then, the local maps of the multiple robots are globally stitched together:

[0151] ;

[0152] in, Represents the global map; Indicates the first The coordinate transformation matrix of a local map generated by a robot.

[0153] And optimize the objective function using ADMM:

[0154]

[0155] in, This represents the objective function to be optimized. Indicates with robots Other nearby robot groups; Represents robots Position in global coordinates.

[0156] Minimize map stitching error by minimizing the location differences between neighbors.

[0157] Multiple robots can adaptively divide tasks based on task rewards, avoiding repetition or omission of areas, thereby achieving a "three-dimensional, full-coverage, and fault-tolerant" group perception system, which significantly enhances operational safety and full-space perception capabilities.

[0158] Task allocation and fault tolerance: Different robots are automatically assigned tasks based on their positions (such as top plate monitoring, dust detection, and anchor hole identification). Even if a single robot fails, the system can still rely on the remaining nodes to complete the overall task.

[0159] Task allocation employs a reinforcement learning strategy:

[0160] ;

[0161] in, Represents the policy function (represents the state) Take action below (probability) The reward function is represented, taking into account coverage, redundancy, and energy consumption. This represents the expected reward value.

[0162] The system employs a lightweight modeling approach to ensure real-time rendering and interactive performance even in large-scale tunnel scenarios. Simultaneously, by collecting real-time tunneling parameters and surrounding rock response data, it constructs an equipment-surrounding rock interaction model, dynamically simulating the disturbance impact of tunneling operations on the surrounding rock under different geological conditions and predicting tunnel deformation trends. At the equipment level, the system also introduces a performance degradation model, dynamically assessing the lifespan and status of key components based on real-time operating data, supporting preventative maintenance and spare parts management. To achieve full-cycle digital management and control, the system adopts a distributed architecture design, constructing an operational database covering the design, construction, and operation and maintenance phases, and developing a data intelligent analysis engine capable of multi-dimensional visualization and trend prediction of the tunneling process. The digital twin platform not only supports real-time monitoring of tunnel status and virtual verification of construction plans but also enables emergency simulations in the event of unforeseen circumstances, thus providing strong support for intelligent scheduling and decision support of tunneling operations, achieving refined management and intelligent control throughout the entire tunnel lifecycle.

[0163] The intelligent sensing robot system proposed in this application solves the following core problems existing in the application of current intelligent sensing and positioning technologies in tunneling faces in real underground environments:

[0164] Addressing the issue of sensing failure in complex and harsh environments: Existing positioning methods, whether single-source or simple fusion methods using vision, laser, or radio frequency sensors, suffer from a sharp drop in accuracy and poor stability under conditions such as the absence of GPS, dense dust, abnormal lighting, severe electromagnetic interference, and significant multipath effects in underground environments. This application overcomes the challenge of sensor data distortion, degradation, and even failure under multi-source interference coupling environments, achieving highly robust continuous pose sensing.

[0165] The real-time performance and engineering application challenges of multi-sensor fusion systems: Existing fusion algorithms are computationally complex and rely on high-cost hardware (such as fiber optic inertial navigation), making it difficult to meet the real-time processing requirements of dynamic operations in tunneling equipment. Furthermore, these systems suffer from frequent calibration, poor compatibility, and difficulty in expansion and integration. This application constructs a multi-sensor fusion architecture that is computationally balanced, loosely coupled, and easily deployed, improving system real-time performance and reducing hardware costs and maintenance requirements while ensuring centimeter-level positioning accuracy.

[0166] Drift and fault tolerance issues in positioning systems during long-term operation: To address the significant cumulative error of inertial navigation and the tendency of visual sensors / LiDAR to lose frames in dynamic occlusion and harsh environments, this application designs an intelligent fusion algorithm with online correction, adaptive weight adjustment and fault diagnosis capabilities (bidirectional interactive collaborative perception mechanism, multi-sensor spatiotemporal alignment and environmental adaptive fusion, redundant perception channels and multi-agent distributed cooperation), ensuring that the system maintains high accuracy and high reliability under long-term, high-vibration and strong occlusion conditions.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A tunneling face intelligent perception robot system, characterized in that: It includes a slide rail fixed to the top of the tunnel where the tunneling machine is located, a liftable hanging robot mounted on the slide rail, fixed targets with known coordinates set at intervals on the tunnel, and a vision sensor, lidar, event camera and inertial measurement unit mounted on the hanging robot. Infrared targets are installed on both sides of the tunnel boring machine (TBM), and a strapdown inertial navigation system is also installed on the TBM. The TBM and the rail-mounted robot interact through a two-way interactive collaborative sensing mechanism to achieve continuous positioning of the TBM. Furthermore, by fusing multi-source sensor data from the rail-mounted robot, real-time digital construction of the tunnel's three-dimensional perception is achieved, including: a. Sensor spatiotemporal alignment: A weighted approach is used to achieve spatiotemporal alignment of multi-source sensors; b. Prior geometric constraints: Use prior geometric constraints to correct outliers in the roadway geometric model, minimizing the distance between the point cloud and the points on the roadway geometric model; c. The final output is a continuous 3D model of the tunnel; The two-way interactive collaborative sensing mechanism includes: Airborne sensor data feedback: The tunneling machine transmits its onboard inertial navigation data, vibration data, and operating status data in real time; Rail-mounted robot reverse feedback: The rail-mounted robot returns the real-time modeled spatial point cloud and positioning information to the tunneling machine, forming a complementary relationship; Closed-loop correction mechanism: The positioning result of the tunneling machine is corrected by the rail-mounted robot, and the point cloud modeling of the rail-mounted robot is based on the posture of the tunneling machine body, thus achieving dual verification; Collaborative optimization algorithm: Introducing "mutual verification filtering" based on Bayesian inference to cross-validate and fuse multi-source data from tunneling machines and rail-mounted robots; A self-healing digital twin model is deployed on the monitoring terminal of the tunneling machine. The self-healing digital twin model is used to generate a three-dimensional digital twin covering the surrounding rock of the tunnel, the tunneling machine, and the support structure. The functions of the self-healing digital twin model include: Breakpoint completion: When point cloud data is lost, historical modeling data, surrounding rock mechanics prediction model and roadway geometric priors are called to quickly complete the breakpoint; Predictive Correction: When sensors are temporarily unavailable, the self-healing digital twin model uses a Kalman-Transformer-based time-series prediction model to infer the evolution trend of tunnel morphology. Adaptive fusion: When the sensed data is recovered, the new data and the completed data are fused through differential allocation and residual minimization algorithms to ensure continuous and seamless connection; Anomaly self-correction: Identifies "false point clouds" caused by dust or vibration, and automatically corrects them through a triple constraint of history, prediction, and geometry; The construction of a digital twin model can be abstracted as follows: ； wherein, represents a time instant of the roadway digital twin model; is a set of geometric priors; is a tunneling equipment operating parameter; is a surrounding rock monitoring data; is a support structure state; is a twin modeling and evolution function.

2. The intelligent perception robot system for tunneling working face according to claim 1, characterized in that: The system also includes acoustic sensors fixed to the body of the tunneling machine and a micro-vibration sensor array fixed to the surrounding rock and floor of the tunnel. These acoustic sensors and micro-vibration sensor array serve as redundant sensing channels and are introduced when lidar and vision sensors are severely degraded to sense the position of the tunneling machine.

3. The intelligent perception robot system for tunneling working face according to claim 2, characterized in that: The system also constructs a personnel location tracking system through positioning base stations and intelligent visual monitoring networks deployed in the tunnels, as well as positioning devices worn by workers, to achieve real-time proactive perception and prediction of personnel and processes in the tunnels.

4. The intelligent perception robot system for tunneling working face according to claim 2 or 3, characterized in that: The system also includes magnetic wall-mounted robots, wheeled / tracked ground robots, and hovering drones, which together with the rail-mounted robots form a distributed sensing network at the tunneling face. Each robot shares its pose, point cloud, and environmental parameters in real time through a wireless self-organizing network to form a global map.

5. The intelligent perception robot system for tunneling working face according to any one of claims 1-3, characterized in that: The visual sensor identifies the tunneling machine and its spatial location by recognizing the infrared targets installed on both sides of the machine body in low light or dusty environments. Combined with the machine body's strapdown inertial navigation system, it forms a complete positioning and perception system. Meanwhile, during the positioning process, the visual sensor and the lidar work together. The visual sensor is used to identify and lock the feature points of the infrared target, while the lidar provides accurate distance measurement. The two types of data are fused with the fuselage strapdown inertial navigation data and then optimized by Kalman filtering. When the tunneling machine enters the observation range of the fixed target, the vision sensor will automatically capture the fixed target as a correction marker and perform reverse calibration.

6. The intelligent perception robot system for tunneling working face according to claim 3, characterized in that: The personnel location tracking system introduces a spatiotemporal data analysis model to integrate personnel trajectories, equipment operating status, and environmental parameters in multiple dimensions, forming a predictive mechanism for process evolution.

7. The intelligent perception robot system for tunneling working face according to claim 4, characterized in that: The system uses multi-sensor fusion technology to accurately identify and locate anchor holes and anchor bolts / cables, and combines roadway surface displacement monitoring data to establish a correlation model between support effect and surrounding rock deformation.

Images