A tunnel monitoring system based on autonomous robots

Abstract

The present application relates to the technical field of tunnel defect monitoring, in particular to a tunnel monitoring system based on autonomous robots, which integrates image acquisition and ground penetrating radar devices through a multi-module collaborative autonomous robot system, realizes autonomous inspection and multi-modal data acquisition inside the tunnel, and improves defect detection efficiency and continuity. Compared with the traditional image recognition method, the radar scan map and the characteristic image are fused, the feature extraction and defect reasoning are carried out through the deep learning model, the type and extension area of the structural hidden defect are identified, and the identification ability of the deep structure anomaly is enhanced. At the same time, combined with the defect type and extension prediction, the robot is guided to perform supplementary scanning to ensure data integrity. Further, through the cause classification analysis of a plurality of complete defect detection data, the intelligent closed-loop monitoring from defect identification to cause diagnosis is realized, which effectively reduces the monitoring cost and improves the accuracy and automation level of the tunnel structure disease cause judgment.

Description

Technical Field

[0001] This invention relates to the field of tunnel defect monitoring technology, and more particularly to a tunnel monitoring system based on an autonomous robot. Background Technology

[0002] Tunnels operate in complex environments characterized by high humidity, high loads, and frequent vibrations, making them highly susceptible to structural defects such as cracks, water seepage, and spalling. Failure to detect and address these defects promptly can lead to operational risks and even safety accidents. Therefore, conducting regular inspections and risk assessments of tunnel internal structures has become a crucial means of infrastructure safety maintenance. Currently, commonly used tunnel monitoring methods rely heavily on manual inspections using handheld cameras and radar equipment, often delivered via ladders to high points within the tunnel walls, resulting in high inspection costs.

[0003] Existing technologies automate image acquisition and surface defect identification by setting up mobile platforms on tunnel tracks. However, these solutions only identify surface images and cannot accurately identify hidden defects inside the tunnel structure (such as voids in the lining layer, steel corrosion, and crack propagation paths). This results in incomplete defect data acquisition and a lack of effective data support for the analysis of defect causes, leading to low accuracy in defect cause analysis. Summary of the Invention

[0004] To address the above problems, this invention provides a tunnel monitoring system based on an autonomous robot.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A tunnel monitoring system based on an autonomous robot includes: a real-time monitoring module, a defect location module, a path planning module, a defect scanning module, a defect tracking module, and a result output module;

[0007] The real-time monitoring module is used to acquire characterizing images of the tunnel's inner wall in real time.

[0008] The defect location module is used to identify defect locations based on tunnel characterization image data to obtain defect location coordinate data.

[0009] The path planning module is used to perform constraint optimization calculations on the defect point coordinate data and obstacle distribution data based on the defect point coordinate data and real-time obstacle distribution data to obtain a path planning strategy, and guide the robot to the defect point according to the path planning strategy.

[0010] The defect scanning module is used to control the robot to perform ground-penetrating radar scanning on defect locations and simultaneously acquire defect characterization images. It obtains GPR perspective radar scan images and defect characterization data, and uses a deep learning model to perform defect inference to obtain defect type and extended prediction data.

[0011] The defect tracking module is used to guide the robot to perform supplementary ground-penetrating radar scans based on defect type and extended prediction data to obtain complete defect detection data.

[0012] The results output module is used to predict the causes of defects based on complete defect detection data and generate tunnel monitoring results.

[0013] Furthermore, the real-time acquisition of characterization images of the tunnel inner wall includes:

[0014] The robot is controlled to autonomously inspect and travel along the tunnel wall along a preset path, and simultaneously captures images through a camera installed on the robot body to obtain raw image frames;

[0015] Image enhancement, brightness equalization, and noise suppression are performed on the original image frames to obtain a characterization image of the tunnel inner wall.

[0016] Furthermore, the defect location module is used to perform the following steps:

[0017] Based on the representational image of the tunnel inner wall, image classification and region localization processing are performed through a convolutional neural network to identify abnormal regions and generate a candidate set of abnormal regions containing anomaly type labels, confidence scores and image bounding boxes.

[0018] Based on the candidate set of abnormal regions and the pose information of the robot at the time of image acquisition, the image bounding box is spatially reprojected and distortion corrected to obtain the corresponding three-dimensional spatial coordinates of the abnormal region.

[0019] Based on the three-dimensional spatial coordinates of the abnormal area, clustering filtering is performed on each abnormal point to eliminate redundant points and low-confidence interference points, thereby obtaining the coordinate data of the defect points.

[0020] Furthermore, the path planning module is used to perform the following steps:

[0021] Based on the coordinate data of the defect points and the current position and posture data of the robot, the spatial positions of the defect points and obstacles are uniformly mapped to generate a two-dimensional tunnel wall map containing the distribution of defect points and obstacles.

[0022] Based on the positional relationship between the two-dimensional tunnel wall map and the defect points on the unfolded map, a heuristic search algorithm is used to plan the path from the robot's current position to each defect point, resulting in a set of candidate paths that satisfy obstacle avoidance constraints.

[0023] Based on the path length, number of direction changes, and obstacle avoidance distance in the candidate path set, the paths are sorted and evaluated to determine the current optimal path planning strategy.

[0024] Based on the path planning strategy, navigation instructions are generated to control the robot to reach the designated defect point along the path.

[0025] Furthermore, the deep learning model is used to perform the following steps:

[0026] Spatial resolution alignment and time stamp synchronization processing are performed on the GPR perspective radar scan image and defect characterization image data synchronously acquired by the robot at the defect point to obtain a structurally unified fused input data pair.

[0027] Based on the fused input data pair, the radar image and the representation image are concatenated into a unified tensor input in the channel dimension and input into the encoder network. By sharing the convolution kernel, radar penetration features and image surface texture features are extracted simultaneously to obtain a shallow fused semantic feature representation.

[0028] Gradient direction calculation and structural tensor analysis are performed based on the shallow fusion semantic feature representation to extract the main direction of image texture to form a direction weighting matrix. In the feature propagation process, the feature response of the radar channel is adjusted by directional convolution to obtain the deep fusion feature map.

[0029] Based on the deep fusion feature map, defect classification prediction and defect extension region segmentation are performed respectively to obtain defect type labels and defect extension region mask maps.

[0030] Furthermore, the deep learning model is trained through the following steps:

[0031] The GPR perspective radar scan image of complete defects collected in history is divided into multiple segments in spatial order. Defect label mapping is performed on each segment to obtain a training sample set containing radar segment images, defect characterization image data and mapped defect label pairs.

[0032] Based on each pair of radar segment images and defect characterization image data in the training sample set, spatial registration and channel stitching are performed to obtain a unified input tensor, which is then input into the encoder network to obtain the deep fusion feature map generated during training.

[0033] Based on the deep fusion feature map generated during training, defect type labels and defect extension region masks are predicted respectively. A loss function containing type error, region error and cross-segment continuity error is constructed based on the mapped defect label pairs. Backpropagation optimization is performed based on the loss value calculated according to the loss function.

[0034] Furthermore, the loss function is as follows:

[0035] ;

[0036] in, This is the loss value; The cross-entropy loss function; It is the intersection-union ratio function; The true label for the defect type; Predicted labels for defect types; A true mask image of the defect extension region; A predicted mask for the defect extension region; and These represent the deep fusion feature vectors of the i-th and j-th radar sub-segments under the same defect label, respectively. The cosine similarity function; , and These are the weighting coefficients.

[0037] Furthermore, the step of performing gradient direction calculation and structural tensor analysis based on the shallow fusion semantic feature representation to extract the principal directions of image texture to form a direction weighting matrix includes the following steps:

[0038] Based on the image channels in the shallow fusion semantic feature representation, the Sobel operator convolution process is performed on the image region to calculate the gradient values ​​in the horizontal and vertical directions, and obtain the gradient magnitude map and gradient direction map of each pixel.

[0039] Based on the gradient direction map, the gradient directions in the local neighborhood of each pixel are weighted and averaged to construct a structural tensor field describing the distribution of the main directions of the region.

[0040] Based on the structure tensor field, the principal direction angle information is extracted for each pixel, and a pixel-level orientation mapping map is constructed for the entire image to obtain the image texture principal direction map;

[0041] Based on the main orientation map of image texture, spatial orientation weighted convolution processing is performed on the deep radar feature map to construct an orientation weighted matrix.

[0042] Furthermore, the defect tracking module is used to perform the following steps:

[0043] Based on the defect type labels and defect extension area mask map output during the defect scanning stage, combined with the robot's current position and attitude information, the defect prediction starting area is marked on the two-dimensional tunnel wall map, and an extension direction search window is set.

[0044] Within the extended direction search window, the robot is controlled to move segment by segment along the defect prediction extension direction, and simultaneously performs ground penetrating radar scanning and image acquisition operations to obtain continuous GPR perspective radar images and defect image sequences until the current defect prediction area continuously expands to the preset maximum length or no defect signal is detected in the current frame. Then, the extension tracking is terminated, and the complete defect detection data under the tracked path is output.

[0045] Furthermore, the step of predicting the cause of a defect based on complete defect detection data includes the following steps:

[0046] Based on several complete defect detection data and corresponding defect type labels in the tunnel, tunnel defect description data containing information on defect types and spatial distribution in the tunnel is constructed.

[0047] Spatial distribution feature extraction is performed on the defect description data in the tunnel to quantify the distribution density, extension morphology and aggregation characteristics of defects in the tunnel wall, and to obtain the defect distribution feature vector.

[0048] The defect distribution map is input into the graph neural network model. Through node feature aggregation and graph structure learning, global defect distribution features are extracted and classification is performed to predict the corresponding defect cause label and its confidence score.

[0049] Tunnel monitoring results are generated based on defect cause labels and their confidence scores.

[0050] The beneficial effects of this invention are as follows: This invention utilizes a multi-module collaboratively controlled autonomous wall-climbing robot platform, integrating image acquisition devices and ground-penetrating radar equipment. This enables high-frequency inspections and multi-modal data acquisition within tunnels, significantly improving the efficiency and continuity of structural defect coverage detection. Compared to traditional surface defect identification methods based solely on images, this invention employs a deep learning model to fuse and reason about ground-penetrating radar scans and synchronously acquired defect characterization images. This process considers both radar penetration features and image texture features, achieving accurate identification of both latent structural defect types and spatial extension areas, enhancing the system's ability to analyze deep structural anomalies in tunnels. Simultaneously, by combining defect type and extension prediction results, the robot is controlled to perform extended scanning of defect areas, ensuring the integrity of defect data. Furthermore, spatial structural modeling is performed on the complete defect detection data, extracting defect distribution feature vectors and performing causal classification reasoning. Based on multi-dimensional features such as defect type, distribution density, and extension morphology, the causes of defects are predicted, achieving closed-loop monitoring from problem discovery to intelligent diagnosis. This effectively reduces monitoring costs and improves the accuracy and automation level of determining the causes of tunnel structural defects. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the structure of a tunnel monitoring system based on an autonomous robot according to the present invention.

[0052] Figure 2 This is a flowchart of the execution steps of the deep learning model in this invention. Detailed Implementation

[0053] Please see Figure 1-2As shown, the present invention relates to a tunnel monitoring system based on an autonomous robot, comprising: a real-time monitoring module, a defect location module, a path planning module, a defect scanning module, a defect tracking module, and a result output module;

[0054] The real-time monitoring module is used to acquire characterizing images of the tunnel's inner wall in real time.

[0055] The defect location module is used to identify defect locations based on tunnel characterization image data to obtain defect location coordinate data.

[0056] The path planning module is used to perform constraint optimization calculations on the defect point coordinate data and obstacle distribution data based on the defect point coordinate data and real-time obstacle distribution data to obtain a path planning strategy, and guide the robot to the defect point according to the path planning strategy.

[0057] The defect scanning module is used to control the robot to perform ground-penetrating radar scanning on defect locations and simultaneously acquire defect characterization images. It obtains GPR perspective radar scan images and defect characterization data, and uses a deep learning model to perform defect inference to obtain defect type and extended prediction data.

[0058] The defect tracking module is used to guide the robot to perform supplementary ground-penetrating radar scans based on defect type and extended prediction data to obtain complete defect detection data.

[0059] The results output module is used to predict the causes of defects based on complete defect detection data and generate tunnel monitoring results.

[0060] It should be noted that this system can be set up on a cloud server or a terminal device connected to the robot. The robot itself is a wall-climbing robot using the principle of vacuum negative pressure adsorption, equipped with a high-definition industrial camera, a ground-penetrating radar (GPR) array, attitude sensors, and an edge computing unit, supporting stable operation and multi-source data collaborative processing in complex tunnel environments. The system first uses a real-time monitoring module to control the camera to continuously acquire images of the tunnel wall at a fixed frame rate and transmits them in real time to the edge computing unit. Image enhancement, brightness correction, and noise filtering are performed to obtain stable tunnel characterization images. The defect localization module uses a convolutional neural network (CNN) model to perform anomaly detection and region recognition on the characterization images, generating a set of candidate boxes containing the location, category label, and confidence level of the abnormal regions. This is then combined with the robot's pose information to perform spatial reprojection, converting it into three-dimensional coordinates of the defect points. The path planning module maps these defect coordinates and synchronously acquired obstacle point cloud data to a two-dimensional tunnel wall unfolded map. Using an A* heuristic search algorithm, a multi-objective path planning map is constructed, calculating the shortest obstacle avoidance path to guide the robot to each defect point efficiently. After reaching the target area, the defect scanning module controls the GPR radar to perform high-resolution scanning of the defect location along a set trajectory, simultaneously acquiring characterizing images. These radar images and image data are spatially registered and channel-fused before being input into a deep fusion neural network to extract cross-modal features, infer the defect type, and predict its spatial extension direction. If the model prediction indicates a significant extension trend, the defect tracking module sets a directional search window based on the starting area, guiding the robot to perform iterative radar scanning and image acquisition along the extension direction until the extension signal terminates or covers the entire area. Finally, the results output module constructs a spatial graph structure from the complete defect scan data and its distribution pattern, inputting it into a graph neural network for causative classification inference. Combining feature aggregation and edge weight modeling, it outputs causative labels and confidence scores, forming a structured tunnel monitoring result report. This achieves a closed-loop intelligent analysis process from defect discovery and tracking to causative diagnosis. Unlike existing multimodal branching processing technologies, this invention uses a shared convolutional kernel mechanism in a unified network to collaboratively encode image texture and radar structural features. It also constructs a directional weighted matrix based on the main texture direction of the image to adjust the radar feature response direction, achieving deep fusion and information enhancement of radar and visual information. This results in higher latent defect recognition capabilities, particularly suitable for detecting early-stage, invisible defects such as cracks and cavities. During the defect tracking stage, a directional search window is constructed based on the defect type and predicted mask image. Combined with changes in radar reflection intensity and image anomaly features, the scanning path direction is dynamically adjusted, making the tracking process adaptive and significantly improving the coverage integrity and boundary accuracy of defect extension range detection.Through collaborative innovation in hardware integration, data synchronization, deep model structure design, tracking control logic, and causal reasoning models, a tunnel structure monitoring system with high adaptability and intelligent analysis capabilities has been built.

[0061] Furthermore, the real-time acquisition of characterization images of the tunnel inner wall includes:

[0062] The robot is controlled to autonomously inspect and travel along the tunnel wall along a preset path, and simultaneously captures images through a camera installed on the robot body to obtain raw image frames;

[0063] Image enhancement, brightness equalization, and noise suppression are performed on the original image frames to obtain a characterization image of the tunnel inner wall.

[0064] In some embodiments, the real-time acquisition of images representing the tunnel wall combines autonomous robot navigation control with image quality optimization algorithms to ensure a high-precision input data foundation for subsequent defect identification tasks. First, the robot's autonomous positioning in the tunnel environment is achieved through a positioning module integrated into the robot chassis (e.g., IMU + wheel encoder + LiDAR fusion). Following a preset spiral scanning path or wall-adhering curve travel path, the robot is driven to move at a constant speed along the tunnel wall. An image acquisition device, installed at the front or side of the robot, possesses automatic exposure and image stabilization capabilities. During robot movement, it synchronously acquires image data at a fixed frame rate, forming a sequence of original image frames. For the acquired original image frames, the system first performs image enhancement processing based on histogram equalization to improve the discernibility of low-light areas and enhance the overall image contrast and structural texture clarity. Then, to address the issue of uneven illumination during different acquisition time periods, an adaptive brightness mapping function is used to perform local brightness reconstruction on the image, ensuring the balance of the image's grayscale distribution. Furthermore, a composite noise suppression algorithm based on bilateral filtering and non-local means is used to smooth out high-frequency noise points in the image caused by environmental dust and moisture reflection.

[0065] Furthermore, the defect location module is used to perform the following steps:

[0066] Based on the representational image of the tunnel inner wall, image classification and region localization processing are performed through a convolutional neural network to identify abnormal regions and generate a candidate set of abnormal regions containing anomaly type labels, confidence scores and image bounding boxes.

[0067] Based on the candidate set of abnormal regions and the pose information of the robot at the time of image acquisition, the image bounding box is spatially reprojected and distortion corrected to obtain the corresponding three-dimensional spatial coordinates of the abnormal region.

[0068] Based on the three-dimensional spatial coordinates of the abnormal area, clustering filtering is performed on each abnormal point to eliminate redundant points and low-confidence interference points, thereby obtaining the coordinate data of the defect points.

[0069] In some embodiments, firstly, the tunnel inner wall representation image processed by the image enhancement module is input into a pre-trained convolutional neural network model (such as ResNet-50 or EfficientNet), which has been optimized through transfer learning on a tunnel defect image dataset. During forward inference, the network extracts multi-scale texture features of the image through layer-by-layer convolution, batch normalization, and ReLU activation function, and finally completes the discrimination and localization of abnormal regions in the image at the output layer. The model output includes the abnormality type label (such as crack, seepage, peeling, etc.) of each detection box, the corresponding confidence score, and the coordinate parameters of the image bounding box, forming a candidate set of abnormal regions. Subsequently, in order to map the image coordinate information to the real space coordinates, spatial reprojection processing is performed on the candidate bounding boxes based on the pose data (including translation vector and planar pose angle) at the time of robot image acquisition. Specifically, combining the camera intrinsic parameter matrix and extrinsic parameter calibration parameters, the pinhole camera model is used to perform distortion correction and 3D projection transformation on the four corner points of the bounding box in the image plane, obtaining the set of 3D spatial coordinate points of the abnormal region in the local coordinate system of the tunnel, thus solving the positional error caused by image projection distortion. Finally, to further improve the accuracy and representativeness of the defect location data, the above spatial coordinate point set was clustered using the DBSCAN clustering algorithm to remove isolated points and low-confidence redundant detection points. During the clustering process, spatial distance and label similarity were used as clustering constraints, and adjacent high-density outliers were merged to generate the final defect center point coordinates.

[0070] Furthermore, the path planning module is used to perform the following steps:

[0071] Based on the coordinate data of the defect points and the current position and posture data of the robot, the spatial positions of the defect points and obstacles are uniformly mapped to generate a two-dimensional tunnel wall map containing the distribution of defect points and obstacles.

[0072] Based on the positional relationship between the two-dimensional tunnel wall map and the defect points on the unfolded map, a heuristic search algorithm is used to plan the path from the robot's current position to each defect point, resulting in a set of candidate paths that satisfy obstacle avoidance constraints.

[0073] Based on the path length, number of direction changes, and obstacle avoidance distance in the candidate path set, the paths are sorted and evaluated to determine the current optimal path planning strategy.

[0074] Based on the path planning strategy, navigation instructions are generated to control the robot to reach the designated defect point along the path.

[0075] In some embodiments, the spatial coordinates of the defect location are first mapped to the robot's current position and posture information, projecting the positions of the defect and obstacles onto the same two-dimensional coordinate system to construct a two-dimensional tunnel map reflecting the relationship between the defect location and the obstacle distribution. On this map, spatial accessibility is represented by a grid, with the defect location as the target location and obstacle areas marked as impassable units. Then, based on this map, an A or D Lite heuristic search algorithm is used for path planning. The algorithm introduces a cost function during the search process, comprehensively evaluating the length of the current path, the number of direction changes, and the minimum distance to the obstacle, thereby generating multiple candidate paths that satisfy obstacle avoidance constraints. To select the optimal path, the system further weights and scores the candidate paths, prioritizing paths that are shorter, have fewer turns, and are farther from obstacles. After determining the optimal path, it is smoothed and converted into continuous navigation commands to guide the robot to the designated defect location according to a preset movement strategy.

[0076] Furthermore, the deep learning model is used to perform the following steps:

[0077] Spatial resolution alignment and time stamp synchronization processing are performed on the GPR perspective radar scan image and defect characterization image data synchronously acquired by the robot at the defect point to obtain a structurally unified fused input data pair.

[0078] Based on the fused input data pair, the radar image and the representation image are concatenated into a unified tensor input in the channel dimension and input into the encoder network. By sharing the convolution kernel, radar penetration features and image surface texture features are extracted simultaneously to obtain a shallow fused semantic feature representation.

[0079] Gradient direction calculation and structural tensor analysis are performed based on the shallow fusion semantic feature representation to extract the main direction of image texture to form a direction weighting matrix. In the feature propagation process, the feature response of the radar channel is adjusted by directional convolution to obtain the deep fusion feature map.

[0080] Based on the deep fusion feature map, defect classification prediction and defect extension region segmentation are performed respectively to obtain defect type labels and defect extension region mask maps.

[0081] It should be noted that, firstly, spatial resolution alignment and timestamp synchronization are performed on the two types of image data synchronously acquired by the robot at the defect location. Specifically, a bilateral interpolation algorithm is used to upsample the low-resolution radar scan image to match the resolution of the defect representation image. Simultaneously, by reading the timestamps of the image acquisition module and radar triggering, a nearest neighbor matching strategy is used to temporally pair image frames, thereby constructing structurally consistent input image pairs. Next, the registered GPR radar image and the representation image are concatenated along the channel dimension to construct a unified input tensor. This tensor is then input into a convolutional encoder designed with shared weights. In the convolutional encoder, penetration features are extracted from the radar image channels, and texture and deformation features are extracted from the representation image channels, fusing the output to form a shallow semantic feature map. This stage employs a multi-scale convolutional kernel structure, combined with BN layers and ReLU activation functions to ensure the robustness and non-linear expressiveness of feature extraction. Gradient direction estimation and structural tensor construction are performed based on the image channels in the shallow semantic feature map. The Sobel operator is used to calculate the gradients in the horizontal and vertical directions, and a principal direction tensor field is constructed based on the gradient distribution within a local window. The direction weighting matrix extracted from the tensor field is used to perform directional convolution adjustment on the radar feature map. Specifically, this is achieved by rotating the convolution kernel weights based on the principal direction angle θ, and combining an attention mechanism to enhance the feature response of regions with significant directional consistency, thereby generating a deep fused feature map. Finally, the model branches this deep fused feature map to two task networks to perform defect classification prediction and defect extension region segmentation, respectively. The classification branch outputs defect type labels through fully connected layers and a softmax activation function, while the segmentation branch gradually restores spatial resolution through deconvolution and skip connection modules, outputting defect extension prediction results in the form of a mask image. The two task networks share the backbone feature representation, and a multi-task joint loss function is used for optimization during the training phase to ensure a joint improvement in classification and segmentation accuracy. This model significantly enhances the system's intelligent identification and representation capabilities of latent defects in complex tunnel structures.

[0082] Furthermore, the deep learning model is trained through the following steps:

[0083] The GPR perspective radar scan image of complete defects collected in history is divided into multiple segments in spatial order. Defect label mapping is performed on each segment to obtain a training sample set containing radar segment images, defect characterization image data and mapped defect label pairs.

[0084] Based on each pair of radar segment images and defect characterization image data in the training sample set, spatial registration and channel stitching are performed to obtain a unified input tensor, which is then input into the encoder network to obtain the deep fusion feature map generated during training.

[0085] Based on the deep fusion feature map generated during training, defect type labels and defect extension region masks are predicted respectively. A loss function containing type error, region error and cross-segment continuity error is constructed based on the mapped defect label pairs. Backpropagation optimization is performed based on the loss value calculated according to the loss function.

[0086] In some embodiments, to expand the training sample size of the deep learning model and improve the representativeness of the sample distribution, the system performs spatial slicing on historically acquired complete GPR perspective radar scan images. Specifically, the original radar image is segmented into equal-length segments according to spatial order, and a sliding window mechanism is introduced to generate a sequence of sub-segment images with a high overlap rate. This significantly increases the amount of training data and the sample distribution density of defect boundary regions without introducing additional sampling costs. This strategy can effectively improve the model's discrimination ability in transition boundaries and weak signal regions. To ensure that each sub-segment sample has a clear learning objective, the sub-segment image is paired with synchronously acquired image data based on the corresponding timestamp and pose data. At the same time, the complete defect label is cropped into local labels for each sub-segment through spatial mapping, constructing a unified label mapping relationship. In this way, each sample consists of a triplet of "radar image sub-segment + representation image + matching label", which not only ensures the consistency of sample labels but also improves the model's generalization learning ability for local feature patterns. Compared to existing technologies, this sample expansion method does not rely on additional data acquisition. Instead, it extracts richer training information from the original data through algorithmic data segmentation and label reconstruction, effectively improving the model's performance in modeling complex defect types and spatial extension patterns. In the sample preprocessing stage, the system performs spatial registration on each pair of radar images and representation image data, using scale normalization, edge alignment, and timestamp synchronization to ensure consistency in both spatial and temporal dimensions. Subsequently, the processed image data is concatenated into a unified tensor along the channel dimension, forming a multi-channel fusion input. This input tensor is fed into a shared encoder network, which employs a multi-scale residual convolution structure and a spatial attention module to jointly extract local defect features and radar penetration pattern features, outputting a deep fusion feature map. During training, the system constructs a dual-branch output module. The first branch predicts the defect type, using a fully connected classifier to output a softmax-activated class probability distribution. The second branch predicts the mask for the defect extension region, restoring image resolution through upsampling and skip connections to generate a pixel-level mask map. In terms of loss function design, the system introduces three types of loss terms: the first is cross-entropy loss, which is used to constrain the classification error between the predicted defect type label and the real label; the second is IoU (Intersection over Union) loss, which is used to measure the overlap between the masked prediction region and the real defect region; and the third is cross-segment continuity loss, which uses cosine similarity to maximize the semantic consistency of deep feature vectors from different radar sub-segments under the same defect label, thereby improving the prediction coherence of the defect extension region.

[0087] Furthermore, the loss function is as follows:

[0088] ;

[0089] in, This is the loss value; The cross-entropy loss function; It is the intersection-union ratio function; The true label for the defect type; Predicted labels for defect types; A true mask image of the defect extension region; A predicted mask for the defect extension region; and These represent the deep fusion feature vectors of the i-th and j-th radar sub-segments under the same defect label, respectively. The cosine similarity function; , and These are the weighting coefficients.

[0090] Specifically, the loss function comprehensively considers the error in defect type discrimination, the error in segmenting extended regions, and the feature consistency error of the same type of defect in different radar sub-segments. The overall structure consists of three main indicators, used to supervise classification accuracy, region matching degree, and cross-segment feature coherence. The first term is the cross-entropy loss function for defect types, which quantifies the difference in probability distribution between the predicted category label and the true label output by the model, and is used to measure the accuracy of the model in the defect classification task, ensuring that the model can effectively distinguish different types of structural defects. The second term is the intersection-over-union (IoU) loss term for the defect extension region, which is used to evaluate the degree of spatial overlap between the predicted defect mask map and the true defect region. The smaller this loss term is, the more accurately the model fits the defect boundary, and the better it can characterize the true form and spatial extension trend of structural anomalies such as cracks and cavities. The third term is the cross-segment feature consistency loss term, which introduces cosine similarity as a consistency index for the feature representation of the same defect in multiple radar sub-segment images, and imposes a penalty constraint on the angle relationship between feature vectors, encouraging the model to maintain semantic feature consistency when processing continuous sub-segment data, and enhancing the model's cognitive stability of defect extension paths. Ultimately, the loss function is a weighted combination of three sub-items that correspond to the training objectives of classification accuracy, region fit, and cross-segment consistency, respectively. During the training process, the model parameters are continuously optimized through backpropagation to achieve high-precision defect identification and cause prediction capabilities by integrating multimodal data.

[0091] Furthermore, the step of performing gradient direction calculation and structural tensor analysis based on the shallow fusion semantic feature representation to extract the principal directions of image texture to form a direction weighting matrix includes the following steps:

[0092] Based on the image channels in the shallow fusion semantic feature representation, the Sobel operator convolution process is performed on the image region to calculate the gradient values ​​in the horizontal and vertical directions, and obtain the gradient magnitude map and gradient direction map of each pixel.

[0093] Based on the gradient direction map, the gradient directions in the local neighborhood of each pixel are weighted and averaged to construct a structural tensor field describing the distribution of the main directions of the region.

[0094] Based on the structure tensor field, the principal direction angle information is extracted for each pixel, and a pixel-level orientation mapping map is constructed for the entire image to obtain the image texture principal direction map;

[0095] Based on the main orientation map of image texture, spatial orientation weighted convolution processing is performed on the deep radar feature map to construct an orientation weighted matrix.

[0096] It should be noted that, firstly, for the image channels in the shallow fusion semantic feature representation, the Sobel operator is used to perform convolution operations on the image region, calculating the gray-level gradient of each pixel in the horizontal (Gx) and vertical (Gy) directions respectively. By combining the gradients in these two directions, the gradient magnitude map (representing edge intensity) and gradient direction map (representing the direction angle of the texture edge) of each pixel are calculated. Subsequently, based on the gradient direction map, to enhance the spatial continuity and regional consistency of the texture direction, a weighted average of the direction vectors is performed on the local neighborhood of each pixel to form a region principal direction description tensor. This tensor not only preserves the pixel-level edge response but also characterizes the dominant direction pattern of the texture field through the statistical covariance structure of the direction, thus forming a continuous and direction-aware structural tensor field. Next, the principal direction angle of each pixel is extracted from the structural tensor field and mapped to a standard direction vector, and a pixel-level direction mapping map, i.e., the image texture principal direction map, is constructed using this. This map marks the spatial direction distribution pattern of the entire image in the texture dimension in the form of angle information. Finally, the image texture main direction map is used as guiding information and introduced into the deep radar feature map processing stage to perform a spatially weighted convolution operation. This operation introduces a direction weight adjustment term on the basis of the traditional convolution kernel, making the convolution response more perceptive in the main direction and suppressing it in non-main directions. This constructs a set of direction weight matrices to adjust the expression of radar features in spatial directions, improving the model's sensitivity and analytical ability to structural defect edges and crack propagation directions. The innovation of this process lies in opening up the directional channel between image texture features and radar penetration features. By constructing a cross-modal information interaction strategy through a main direction guidance mechanism, it significantly enhances the model's ability to recognize the directional features of defect structures.

[0097] Furthermore, the defect tracking module is used to perform the following steps:

[0098] Based on the defect type labels and defect extension area mask map output during the defect scanning stage, combined with the robot's current position and attitude information, the defect prediction starting area is marked on the two-dimensional tunnel wall map, and an extension direction search window is set.

[0099] Within the extended direction search window, the robot is controlled to move segment by segment along the defect prediction extension direction, and simultaneously performs ground penetrating radar scanning and image acquisition operations to obtain continuous GPR perspective radar images and defect image sequences until the current defect prediction area continuously expands to the preset maximum length or no defect signal is detected in the current frame. Then, the extension tracking is terminated, and the complete defect detection data under the tracked path is output.

[0100] Specifically, firstly, based on the defect type label and corresponding defect extension region mask map output by the defect scanning module, combined with the robot's current spatial position and attitude data, the starting region of the identified defect is located on the previously constructed 2D tunnel wall map. A directional extension direction search window is then constructed at its edge based on the mask's principal direction information. The orientation of this window is determined by the defect principal direction vector extracted from the deep learning model, guiding the spatial direction of subsequent radar detection. Next, the robot controls its movement path to advance segment by segment along this direction, based on the direction information of the extension direction search window. In each advancement segment, the robot simultaneously performs GPR (Geometry Perspective) radar scanning and image acquisition operations, recording the radar penetration image and surface defect image of the current segment, forming a continuous defect image-radar data pair sequence. Each segment's acquisition not only includes the deep structural state of the current defect region but also considers the impact of pose changes on the geometric consistency of the image and radar data, and automatically calibrates and registers them using image and radar acquisition timestamps and spatial coordinates. During tracking, each newly acquired segment is assessed for the presence of a significant defect structure response. The judgment criterion is constructed based on the continuity of the mask image output by the model, the characteristics of radar echo energy changes, and the confidence threshold. If the detection results maintain a stable structural extension trend in several consecutive segments, the system continues to advance in the current direction; if the current frame begins to lack obvious defect responses (e.g., the mask image confidence is lower than the set threshold or the radar signal waveform changes tend to stabilize), or the advancing distance exceeds the set maximum extension length limit, it is determined that the defect has terminated its extension, and the tracking process ends. Finally, the continuous GPR radar images and image sequences collected along the main direction of defect extension are output as a complete dataset of the defect region.

[0101] Furthermore, the step of predicting the cause of a defect based on complete defect detection data includes the following steps:

[0102] Based on several complete defect detection data and corresponding defect type labels in the tunnel, tunnel defect description data containing information on defect types and spatial distribution in the tunnel is constructed.

[0103] Spatial distribution feature extraction is performed on the defect description data in the tunnel to quantify the distribution density, extension morphology and aggregation characteristics of defects in the tunnel wall, and to obtain the defect distribution feature vector.

[0104] The defect distribution map is input into the graph neural network model. Through node feature aggregation and graph structure learning, global defect distribution features are extracted and classification is performed to predict the corresponding defect cause label and its confidence score.

[0105] Tunnel monitoring results are generated based on defect cause labels and their confidence scores.

[0106] In some embodiments, firstly, complete detection data samples output by multiple defect tracking modules are collected and combined with their corresponding defect type labels to construct tunnel defect description data containing the spatial distribution of defects inside the tunnel. This data includes not only the spatial location of defects (such as a set of coordinate points), extension length, and direction vector, but also integrates information such as the type of each defect, inference confidence, and scanning coverage, forming a graph node and edge structure for graph structure learning. Subsequently, spatial distribution feature extraction processing is performed on the defect description data. By unfolding the entire tunnel into a two-dimensional wall mapping map and dividing it into several equally spaced grid cells, the number, area, and directional consistency of defects within each cell are quantified to extract their distribution density features; and by measuring the linear or nonlinear extension trajectory of defects, their extension morphological features are extracted. Combined with the proximity and overlap ratio between multiple defects, their aggregation characteristic index is calculated, thereby forming a set of defect distribution feature vectors for learning. Next, the constructed defect spatial structure map is input into a graph neural network model (GNN). In this model, each graph node represents a defect region, the node features include the spatial feature vectors extracted above, and the edge weights are constructed based on the spatial adjacency, type similarity, and temporal correlation between defects. The GNN model performs node feature aggregation and graph structure encoding learning through a message-passing mechanism, continuously integrating global structural information in multiple iterations to generate a global distribution context vector representation for each defect. Finally, based on the output vector representation of the graph neural network, it performs a multi-class classification task, outputting the defect cause label (e.g., uneven stress, construction defects, hydrological erosion, etc.) and its confidence score for each group of defects. This result not only characterizes the representational attributes of individual defects but also integrates their distribution semantic relationships in the structural space. Ultimately, the system generates a tunnel monitoring result report based on the above prediction results, providing a basis for subsequent structural reinforcement and operation and maintenance strategies.

[0107] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A tunnel monitoring system based on autonomous robots, characterized in that, The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. ； wherein, is a loss value; is a cross-entropy loss function; is a Jaccard index function; is a true label of a defect type; is a predicted label of a defect type; is a true mask map of a defect extension area; is a predicted mask map of a defect extension area; and respectively represent deep fusion feature vectors of an i-th and a j-th radar sub-section under the same defect label; is a cosine similarity function; , and are weighting coefficients. 2.The tunnel monitoring system based on autonomous robot of claim 1, wherein, The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. 3.The tunnel monitoring system based on autonomous robot of claim 2, wherein, The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. 4.The tunnel monitoring system based on autonomous robot of claim 3, wherein, The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection and repair method based on real-time monitoring and deep learning. The application relates to a tunnel defect detection Based on the defect point coordinate data and the current position and posture data of the robot, unified mapping processing is performed on the spatial positions of the defect points and obstacles to generate a two-dimensional tunnel wall map containing the distribution of the defect points and obstacles; Based on the two-dimensional tunnel wall map and the position relationship of the defect points on the development map, a heuristic search algorithm is used to plan paths from the current position of the robot to each defect point, obtaining a candidate path set that satisfies the obstacle avoidance constraint; According to the path length, direction transformation times and obstacle avoidance distance in the candidate path set, sorting and evaluation are performed to determine the current optimal path planning strategy; According to the path planning strategy, navigation instructions are generated to control the robot to reach the specified defect point according to the path. 5.The tunnel monitoring system based on autonomous robot of claim 1, wherein, The deep learning model is used to perform the following steps: The GPR perspective radar scan and defect representation image data collected by the robot at the defect point are subjected to spatial resolution alignment and timestamp synchronization processing to obtain a structure-unified fusion input data pair; Based on the fusion input data pair, the radar image and the representation image are spliced in the channel dimension into a unified tensor input, which is input into the encoder network to simultaneously extract radar penetration features and image surface texture features through shared convolution kernels, obtaining a shallow fusion semantic feature representation; According to the shallow fusion semantic feature representation, gradient direction calculation and structure tensor analysis are performed to extract the main direction of the image texture to form a direction weighting matrix, and the feature response of the radar channel is directionally convolved and adjusted during feature propagation to obtain a deep fusion feature map; Based on the deep fusion feature map, defect classification prediction processing and defect extension region segmentation processing are respectively performed to obtain a defect type label and a defect extension region mask map. 6.The tunnel monitoring system based on autonomous robot of claim 5, wherein, The gradient direction calculation and structure tensor analysis based on the shallow fusion semantic feature representation include the following steps: Based on the image channel in the shallow fusion semantic feature representation, Sobel operator convolution processing is performed on the image region to calculate the gradient values in the horizontal and vertical directions, obtaining a gradient amplitude map and a gradient direction map for each pixel point; According to the gradient direction map, the gradient directions in the local neighborhood of each pixel are weighted and averaged to construct a structure tensor field describing the main direction distribution of the region; Based on the structure tensor field, the main direction angle information of each pixel is extracted, and a pixel-level direction mapping map is constructed for the entire image to obtain an image texture main direction map; Based on the image texture main direction map, spatial direction weighted convolution processing is performed on the deep radar feature map to construct a direction weighting matrix. 7.The tunnel monitoring system based on autonomous robot of claim 1, wherein, The defect tracking module is used to perform the following steps: Based on the defect type label and the defect extension region mask map output in the defect scanning stage, combined with the current position and posture information of the robot, a defect prediction starting region is marked on the two-dimensional tunnel wall map, and an extension direction search window is set. Within the search window in the extension direction, the robot is controlled to move along the defect prediction extension direction in segments, and simultaneously performs ground penetrating radar scanning and image acquisition operations to obtain a continuous GPR perspective radar map and a sequence of defect images, until the current defect prediction area continuously extends to a preset maximum length or no defect signal is detected in the current frame, then the extension tracking is terminated, and complete defect detection data under the tracked path is output. 8.The tunnel monitoring system based on autonomous robot of claim 1, wherein, The defect cause prediction according to the complete defect detection data includes the following steps: Based on a plurality of complete defect detection data and corresponding defect type labels in the tunnel, tunnel defect description data containing defect type and spatial distribution information in the tunnel is constructed; Spatial distribution feature extraction is performed on the tunnel defect description data to quantify the distribution density, extension morphology and aggregation characteristics of the defects in the tunnel wall to obtain a defect distribution feature vector; The defect distribution feature vector is input into a graph neural network model to extract global defect distribution features and perform classification through node feature aggregation and graph structure learning, and a corresponding defect cause label and its confidence score are predicted; A tunnel monitoring result is generated according to the defect cause label and its confidence score.

Images