Method and system for detecting water inrush and mud inrush disasters in tunnel

Abstract

The invention provides a method and system for detecting water inrush and mud inrush disasters in a tunnel, and the method comprises the steps: recognizing the structure type and scale parameters of a tunnel section through an unmanned plane, determining a flight path according to a recognition result, and collecting a thermal imaging image and a vision field image; executing a preprocessing operation and performing diffuse reflection interference identification and suppression by adopting a RefNet network pair; performing feature extraction on the thermal imaging image and the vision field image based on a YOLOv8 backbone network; feature weight fusion is carried out by using a cross-modal adaptive fusion attention mechanism; and a YOLOv8 neck network and a detection head are adopted to carry out feature analysis and water and mud inrush disaster detection respectively. The method can adapt to various complex environments in the tunnel and select a corresponding image acquisition mode, can avoid feature confusion of a vision field image and a thermal imaging image, and greatly improves the detection accuracy of water and mud inrush disasters in the tunnel.

Description

Technical Field

[0001] This invention belongs to the field of tunnel water and mud inrush detection technology, and particularly relates to a method and system for detecting water and mud inrush disasters in tunnels. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Water and mud inrushes are the most common geological problems encountered during underground engineering construction, accounting for about one-third of tunnel accidents. With the vigorous development of infrastructure construction, tunnel engineering is constantly facing new challenges, and water and mud inrush disasters are becoming increasingly severe.

[0004] Currently, various methods have been developed for detecting sudden water and mud inrush disasters, but these methods generally have some technical problems, such as: (1) When collecting images, using drones is a common method. However, due to the special nature of the tunnel environment (which may have different cross sections such as horseshoe, circle, and matrix), the drone can only fly along the middle area of ​​the tunnel. Therefore, it is far away from the area being photographed and it is difficult to maintain a reasonable distance for image collection, resulting in low quality of the collected images.

[0005] (2) After acquiring images, existing methods generally extract image features directly or increase image contrast through ordinary preprocessing. However, images acquired inside tunnels differ from those in other scenes. Due to uneven lighting inside tunnels, reflection interference is prone to occur in areas with strong light, especially the diffuse reflection interference of infrared / visible light from the tunnel walls (such as reflection from metal supports, water film refraction, etc.), which makes it easy to confuse thermal imaging and visible light image features. Existing methods cannot distinguish between real disasters and reflected noise in such situations; and the recognition effect is poor in low light environments. Therefore, the detection effect of water and mud inrush disasters inside tunnels is not ideal. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention provides a method and system for detecting water and mud inrush disasters in tunnels. It can adapt to various complex environments in tunnels and select appropriate image acquisition methods. At the same time, it can avoid feature confusion between field-of-view images and thermal imaging images, greatly improving the accuracy of detecting water and mud inrush disasters in tunnels.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of the present invention provides a method for detecting water and mud inrush disasters in tunnels.

[0008] A method for detecting water and mud inrush disasters in tunnels includes: The drone was used to inspect the tunnel and identify the structural type and dimensional parameters of the tunnel cross section. Based on the identification results, the flight path of the drone in the tunnel was determined and thermal imaging images and field-of-view images of the tunnel environment and thermal imaging conditions were collected. The thermal imaging image and the field of view image are labeled and preprocessing operations are performed; A YOLOv8-derived convolutional network, RefNet, is used to identify and suppress diffuse reflection interference in preprocessed thermal imaging images and field-of-view images. The YOLOv8-based backbone network extracts features from the preprocessed thermal imaging image and the field of view image respectively; and uses a cross-modal adaptive fusion attention mechanism to fuse the feature extraction results of the backbone network with feature weights to obtain a fused weighted image feature map. The neck network of YOLOv8 is used to perform feature analysis on the image feature map after fusion weights, and the feature analysis results are output to the detection head of YOLOv8 to determine whether water inrush or mud inrush disaster has occurred in the tunnel.

[0009] Furthermore, the drone is equipped with a control chip, which embeds the intelligent cross-section recognition network Tunnel-Net, the YOLOv8 network architecture, and its derived convolutional network RefNet.

[0010] Furthermore, the intelligent cross-section recognition network Tunnel-Net includes a geometric feature encoding module, a classification and regression module, and a binocular visual depth estimation module, and the binocular visual depth estimation module contains IMU data; wherein, the geometric feature encoding module and the classification and regression module cooperate to identify the structural type of the tunnel cross-section; the binocular visual depth estimation module is used to determine the flight risk of the UAV and provide data support for collision warning and obstacle avoidance.

[0011] Furthermore, the YOLOv8-derived convolutional network RefNet is used for diffuse reflection interference identification and suppression, including: constructing a reflection noise database, extracting reflection feature vectors from thermal imaging images and field-of-view images based on the derived convolutional network RefNet, and performing real-time feature matching with the constructed reflection noise feature database through a fast matching algorithm to achieve diffuse reflection interference identification; generating an adaptive reflection mask for the identified diffuse reflection interference to achieve diffuse reflection interference suppression.

[0012] Furthermore, the backbone network based on YOLOv8 performs feature extraction on the thermal imaging image and the field of view image respectively, including: performing feature extraction on the thermal imaging image and the field of view image at different scales based on the convolution modules and C2f modules of different orders in the backbone network, so as to obtain thermal imaging images and field of view images with multiple scales respectively.

[0013] Furthermore, the cross-modal adaptive fusion attention mechanism is based on a cross-modal feature fusion architecture constructed by a cross-modal tensor product attention interaction module and an adaptive modal fusion weight module.

[0014] Furthermore, a cross-modal adaptive fusion attention mechanism is used to fuse the feature extraction results of the backbone network, including: constructing a modal interactive attention mechanism based on tensor product attention, flattening the enhanced thermal imaging feature map and the intramodal enhancement map of visible light in the field of view into token form; using two independent fully connected networks to generate a set of low-rank tensor factors for the thermal imaging feature map and the field of view feature map in token form, respectively, to construct the query, key and value in tensor product form for calculating the cross-modal attention matrix.

[0015] A second aspect of the present invention provides a system for detecting water and mud inrush disasters in tunnels.

[0016] A system for detecting water and mud inrush disasters in tunnels includes: The UAV inspection image acquisition module is configured to: use a UAV to inspect inside the tunnel and identify the structural type and scale parameters of the tunnel cross section; determine the flight path of the UAV inside the tunnel based on the identification results, and acquire thermal imaging images and field-of-view images of the tunnel environment and thermal imaging conditions. The preprocessing module is configured to: annotate the thermal imaging image and the field-of-view image, and perform preprocessing operations; The diffuse reflection interference identification and suppression module is configured to use the YOLOv8 derived convolutional network RefNet to identify and suppress diffuse reflection interference in the preprocessed thermal imaging image and field of view image; The cross-modal adaptive fusion module is configured to: extract features from the preprocessed thermal imaging image and the field of view image using a YOLOv8-based backbone network; and fuse the feature extraction results of the backbone network using a cross-modal adaptive fusion attention mechanism to obtain a fused weighted image feature map. The water and mud inrush disaster identification module is configured to: use the neck network of YOLOv8 to perform feature analysis on the feature map of the image after fusion weights, and output the feature analysis results to the detection head of YOLOv8 to determine whether a water and mud inrush disaster has occurred in the tunnel. A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a method for detecting water and mud inrush disasters in tunnels as described in the first aspect of the present invention.

[0017] The fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a method for detecting water and mud inrush disasters in tunnels as described in the first aspect of the present invention.

[0018] The above one or more technical solutions have the following beneficial effects: (1) In this invention, when acquiring images, a UAV is used to inspect the tunnel and identify the structural type and scale parameters of the tunnel cross-section. Based on the identification results, the flight path of the UAV in the tunnel is determined, and thermal imaging images and field-of-view images under the tunnel environment and thermal imaging conditions are acquired. By determining the structural type of the tunnel cross-section, the UAV can better adapt to changes in the tunnel cross-section, significantly improving the detection coverage in tunnels with variable cross-sections, and with zero collision accidents. Therefore, this invention can maintain a reasonable distance for image acquisition, and the quality of the acquired images is higher than that of the prior art.

[0019] (2) After acquiring the images, this invention first annotates the thermal imaging images and the field of view images and performs preprocessing operations; then, it uses RefNet, a derivative convolutional network of YOLOv8, to identify and suppress diffuse reflection interference in the preprocessed thermal imaging images and the field of view images; only then does it begin feature extraction from the images. Therefore, even in situations with strong reflection environments (such as steel plate supported tunnels) within the tunnel, it can still accurately identify real disasters and reflected noise, thereby significantly reducing the false detection rate. Therefore, compared with existing technologies, this invention greatly improves the accuracy of detecting water and mud inrush disasters within tunnels.

[0020] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a flowchart of a method for detecting water and mud inrush disasters in tunnels according to Embodiment 1 of the present invention.

[0023] Figure 2 This is a schematic diagram of the structure of the YOLOv8 network in Embodiment 1 of the present invention.

[0024] Figure 3 This is a flowchart of the combined weight output of two different types of images in Embodiment 1 of the present invention.

[0025] Figure 4 This is a schematic diagram of the MFF module in the YOLOv8 neck network in Embodiment 1 of the present invention.

[0026] Figure 5 This is a schematic diagram of the branch structure of the YOLOv8 backbone network in Embodiment 1 of the present invention. Detailed Implementation

[0027] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0028] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0029] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0030] Example 1 This embodiment discloses a method for detecting water and mud inrush disasters in tunnels.

[0031] like Figure 1 As shown, a method for detecting water and mud inrush disasters in tunnels includes: Step S1: Use a drone to inspect inside the tunnel and identify the structural type and dimensional parameters of the tunnel cross section; determine the flight path of the drone inside the tunnel based on the identification results, and collect thermal imaging images and field-of-view images of the tunnel environment and thermal imaging conditions. Step S2: Annotate the thermal imaging image and the field of view image, and perform preprocessing operations; Step S3: Use RefNet, a derivative convolutional network of YOLOv8, to identify and suppress diffuse reflection interference in the preprocessed thermal imaging image and field of view image; Step S4: The YOLOv8-based backbone network extracts features from the preprocessed thermal imaging image and the field of view image respectively; and uses a cross-modal adaptive fusion attention mechanism to fuse the feature extraction results of the backbone network with feature weights to obtain the image feature map with fused weights. Step S5: Use the neck network of YOLOv8 to perform feature analysis on the image feature map after fusion weights, and output the feature analysis results to the detection head of YOLOv8 to determine whether a water inrush or mud inrush disaster has occurred in the tunnel.

[0032] Based on the above process, this invention can adapt to various complex environments within tunnels and select appropriate image acquisition methods; simultaneously, it can avoid feature confusion between field-of-view images and thermal imaging images, greatly improving the accuracy of detecting water and mud inrush disasters within tunnels. To facilitate understanding of the technical solution of this invention, the specific implementation methods of this invention will be further explained and described below.

[0033] In step S1, a drone is used to inspect the tunnel and identify the structural type and scale parameters of the tunnel cross section; based on the identification results, the flight path of the drone in the tunnel is determined, and thermal imaging images and field-of-view images under the tunnel environment and thermal imaging conditions are collected.

[0034] Using an infrared-guided field-of-view (dual-spectrum) device mounted on a drone, tunnel inspections for water and mud inrushes were conducted under low-light conditions, acquiring thermal and field-of-view images of the tunnel environment and under thermal imaging conditions. By collecting a large number of images of water and mud inrush events taken in low-light tunnel environments and under thermal imaging conditions, including but not limited to data from different types of tunnels and different lighting conditions, preparation was made for training an algorithm (YOLOv8 network model) suitable for the infrared-guided field-of-view (dual-spectrum) device. This allows for the subsequent use of... Figure 2 The YOLOv8 network shown after training performs water and mud inrush disaster detection based on newly acquired tunnel environment and thermal imaging images and field-of-view images under thermal imaging conditions.

[0035] Before using a drone to collect images, an initial planned path is pre-configured for the drone. The drone then performs inspections based on the configured initial planned path, records the locations of target points that meet the acquisition criteria, and collects images. Specifically, the drone operation steps are as follows: Place the drone at the starting point and obtain the initial point positioning information; start the drone and conduct inspections according to the initially planned path; during the inspection, the drone focuses on checking abnormal areas based on the infrared thermal imaging characteristics of the surrounding rock mass; during the inspection, determine whether there is a sudden water or mud outburst based on the thermal imaging image and the field of view image, and if so, record the target point location and upload the image; after completing the inspection, return to the initial path to continue the inspection, and return to the initial point after the inspection is completed.

[0036] Furthermore, to address the issues of insufficient detection coverage, missed detection of seepage targets, and high collision risks associated with existing UAV path planning methods in tunnels with varying cross-sectional structures, and to make UAV image acquisition more suitable for tunnel environments, this invention optimizes the dynamic path control of UAVs for adaptive inspection of complex tunnel structures. Specifically, the UAV used is equipped with a control chip, which embeds the intelligent cross-section recognition network Tunnel-Net, the YOLOv8 network architecture, and its derived convolutional network RefNet. It is understood that the selected control chip only needs to be able to achieve the functions required by this invention, and this embodiment does not impose specific restrictions on the specific model of the control chip.

[0037] When a drone is acquiring actual images, the following judgment process is included: 1) Tunnel cross-section identification.

[0038] The UAV used is based on the intelligent cross-section recognition network Tunnel-Net to identify cross-sections in the tunnel. The network takes real-time 3D point cloud or point cloud projection map collected by LiDAR as input to determine the structural type and corresponding scale parameters of the current tunnel cross section.

[0039] Specifically, the intelligent tunnel cross-section recognition network Tunnel-Net integrates a geometric feature encoding module, a classification and regression module, and a binocular vision depth estimation module, with IMU data included in the binocular vision depth estimation module. The geometric feature encoding module and the classification and regression module work together to identify the structural type of the tunnel cross-section (e.g., horseshoe, circular, rectangular, trapezoidal) and its corresponding scale parameters (e.g., top curvature radius, wall height, bottom width, etc.). Furthermore, Tunnel-Net, as a lightweight neural network module for tunnel cross-section structure recognition, is actually built on the YOLOv8 framework. It can receive LiDAR point cloud projection maps or depth maps as input and output the current tunnel cross-section morphology category (horseshoe, rectangular, circular, trapezoidal, etc.) and its lateral width, height, and other scale parameters. Its backbone structure can use MobileNet or YOLOv8-n, with added point cloud encoding layers and scale regression branches, ultimately outputting a set of structural parameters for path adaptive control.

[0040] 2) Collision warning and obstacle avoidance.

[0041] To address the high-risk factors of tunnel environments, such as confined space, complex and uneven walls, and asymmetrical cross-sectional structures, the intelligent cross-section recognition network Tunnel-Net integrates a binocular vision depth estimation module. This module includes inertial measurement unit (IMU) data to construct a spatial relative ranging model. This model can estimate the minimum wall distance of a UAV in any direction in real time. It combines the current velocity vector field to determine the flight risk level, providing data support for collision warning and obstacle avoidance.

[0042] As an optional embodiment, if the minimum wall distance is detected... (Adjustable threshold) will immediately trigger the "emergency hovering mechanism", suspend the flight mission and issue an obstacle avoidance signal, and at the same time restart the path reconstruction module to restore the original inspection path after obstacle avoidance, so as to ensure flight stability and equipment safety.

[0043] In step S2, the thermal imaging image and the field of view image are labeled, and preprocessing operations are performed.

[0044] First, precise bounding boxes were added to the numerous thermal imaging images and field-of-view images collected under low-light tunnel environments and thermal imaging conditions to ensure that areas of water and mud inrush in both types of images could be accurately marked.

[0045] Subsequently, preprocessing operations are performed on the labeled thermal imaging images and field-of-view images. These preprocessing operations include image contrast enhancement, lighting correction, and denoising. Specifically, for low-light environments, image enhancement techniques can be used to simulate darker or more complex lighting conditions, improving the model's adaptability to such environments. For example, histogram adaptive equalization maps pixel values ​​in low-contrast areas to a wider range, thereby improving overall contrast. By improving image contrast and clarity, details of sudden water or mud inrushes can be better identified under low-light conditions. Power-law grayscale transformation is then performed to correct images with poor lighting conditions. Finally, mean filtering is used to smooth and denoise the images, filtering out random noise caused by sensor and environmental factors while preserving important features such as edges, textures, and structures as much as possible, making subsequent disaster detection more accurate.

[0046] In step S3, the YOLOv8 derived convolutional network RefNet is used to identify and suppress diffuse reflection interference in the preprocessed thermal imaging image and field of view image.

[0047] To address the problem of severe diffuse reflection interference from the tunnel wall surface on infrared and visible light (such as reflection from metal supports and light refraction from water films), which easily leads to confusion between thermal imaging and field-of-view images, such as... Figure 3As shown, this invention employs the YOLOv8-derived convolutional network RefNet to identify and suppress diffuse reflection interference in preprocessed thermal imaging images and view images. Specifically, it constructs a reflection noise database, extracts reflection feature vectors from the thermal imaging and view images using RefNet, and performs real-time feature matching with the constructed reflection noise feature database using a fast matching algorithm to achieve diffuse reflection interference identification. An adaptive reflection mask is then generated for the identified diffuse reflection interference to achieve diffuse reflection interference suppression. This can be achieved through the following methods: Step S3-1: Construct a reflection database Reflectance spectral data of tunnel walls made of different materials, such as concrete, steel plates, and rock, were collected under varying humidity conditions (e.g., dry, slightly humid, moderately humid, and high humidity). As an optional embodiment, infrared and visible spectrometers can be used to collect reflectance data of these materials in simulated laboratory environments and within actual tunnels, with at least 50 data samples collected for each material to form a reflectance spectral feature dataset. Subsequently, data preprocessing (e.g., normalization and denoising) was performed to clean the data and construct a standardized reflectance noise feature library for accurate identification of reflectance interference areas in subsequent images.

[0048] Step S3-2, Real-time Reflection Recognition A lightweight convolutional neural network, RefNet, is introduced to perform real-time detection and recognition of bright regions in thermal imaging and field-of-view images. RefNet employs depthwise separable convolution and multi-scale dilated convolution structures, which improves the model's inference speed and computational efficiency, meeting the requirements of real-time applications. In practice, for an input image, RefNet first locates the bright regions; then, it extracts the reflection feature vectors of these bright regions and performs real-time feature matching with a reflection noise feature library using a fast matching algorithm (such as Euclidean distance or cosine similarity algorithm) to quickly and accurately determine whether the bright region belongs to reflection interference.

[0049] Understandably, RefNet takes thermal imaging images or viewport images as input and employs depthwise separable convolution stacking and multi-scale dilated convolution structures to achieve rapid localization and confidence prediction of highly reflective regions in an image. In practical application, the output is represented as a reflective mask.

[0050] Step S3-3: Dynamic Mask Generation and Suppression For the real-time identified reflection interference areas, an adaptive reflection mask is automatically generated. The specific generation steps include: based on the reflection interference confidence score output by RefNet, areas exceeding a set threshold are marked as interference areas; a pixel-level continuous value mask is generated, with mask values ​​between 0 and 1. Higher confidence scores result in mask values ​​closer to 1, accurately reflecting the reflection interference intensity of different areas. The mask is then applied to the thermal imaging feature map. With view feature map The modal features after mask modulation are obtained, namely: ; ; in, This represents the modal characteristics of the thermal imaging feature map after mask modulation. This represents the modal features of the view feature map after mask modulation; This represents the mask confidence level of the network output, and its value ranges from (0,1). This represents the wall depth feature map obtained through lidar.

[0051] In step S4, the YOLOv8-based backbone network extracts features from the preprocessed thermal imaging image and the field of view image respectively; and the cross-modal adaptive fusion attention mechanism is used to fuse the feature extraction results of the backbone network with feature weights to obtain the image feature map with fused weights.

[0052] like Figure 5 As shown, the YOLOv8 backbone network can be structurally divided into two branches. Specifically, the C2f-t and C2f-v modules add channel attention and spatial attention feature extraction modules for thermal imaging images and the field of view images, respectively, to allow for different processing of the two types of images based on their different features. Thermal imaging images are generally smooth with weak texture details, so orthogonal channel compression and average pooling channel compression are chosen. Channel attention mainly focuses on the importance of different channels (i.e., feature types), while the important features (temperature information) of thermal imaging images are evenly distributed. Channel attention can more effectively identify and strengthen the overall features representing temperature change trends. The field of view image has rich texture and detail information, including clear contours, edges, and local features. The distinction between the target and the background often manifests in obvious differences in local spatial features (such as edges and shapes). Spatial attention is selected using max pooling and average pooling compression. Spatial attention can directly locate important spatial regions, effectively emphasizing local features and edge information, strengthening the spatial location information of the target, and enabling the network to capture the target region more accurately. The SPPF module is used to expand the receptive field and fuse multi-scale contextual information. Two types of intramodal augmentation maps with four scales were obtained by performing convolutions and feature extractions at different numbers.

[0053] A YOLOv8-based backbone network extracts features from the preprocessed thermal imaging image and the field of view image, specifically by using convolutional modules and C2f modules of varying orders within the backbone network to extract features at different scales, resulting in thermal imaging and field of view images with multiple scales. Subsequently, a cross-modal adaptive fusion attention mechanism is used to fuse the feature extraction results of the backbone network (i.e., CAFA modal fusion) to obtain a weighted image feature map. The cross-modal adaptive fusion attention mechanism is implemented based on a cross-modal feature fusion architecture constructed from a cross-modal tensor product attention interaction module and an adaptive modal fusion weight module.

[0054] A cross-modal adaptive fusion attention mechanism is used to fuse feature weights in the feature extraction results of the backbone network. This includes: constructing a modal interactive attention mechanism based on tensor product attention, flattening the enhanced thermal imaging feature map and the intramodal enhancement map of visible light in the field of view into token form; and using two independent fully connected networks to generate a set of low-rank tensor factors for the token-form thermal imaging feature map and the field of view feature map, respectively, to construct the query, key, and value in tensor product form for calculating the cross-modal attention matrix. Specifically: First, a modal interactive attention mechanism based on tensor product attention is constructed. The enhanced thermal imaging feature map and the visible light intramodal enhancement map are flattened into token form, i.e., a two-dimensional matrix of HW×C, denoted as follows: and Two independent fully connected networks (MLPs) are used to respectively... and Generate a set of low-rank tensor factors to construct a query, key, and value in tensor product form, i.e.: ; ; ; Where Q, K, and V are the query, key, and value in the form of a tensor product, respectively. and These are the context-dependent and channel-dependent factor vectors extracted from the fully connected network, respectively. R is the tensor product, and R is the tensor rank. r Represents the dimension of a tensor, and modifies it. r It allows control over the fineness of expression and computational complexity, i.e. r The larger the size, the stronger the expressive power and the more detailed the features captured.

[0055] The above formula uses the enhanced thermal imaging feature map To construct query Q, using the enhanced visible light feature map To construct the key K and value V, this indicates that the attention is primarily based on infrared images, with visible light images used to supplement texture details. The construction method can be changed if necessary.

[0056] Then, the cross-modal attention matrix is ​​calculated using tensor product attention, i.e.: ; in, A Represents the cross-modal attention matrix; d The dimensions of Q and K are used to ensure the numerical stability of attention weight calculation.

[0057] The value vectors are then weighted and aggregated using the attention matrix to obtain the fused token representation, i.e.: ; The merged flattened features Reconstructed into a fused feature map with the same spatial dimensions as the original input, i.e.: ; in, Reshape () indicates that the flattened token is reconstructed into a fused feature map corresponding to the original input, R represents the tensor rank, C represents the number of channels, H represents the image height, and W represents the image width.

[0058] The augmented features of each modality are summed with their cross-modal fusion features, and then global average pooling is performed to obtain the semantic representation, i.e.: ; ; in, For the semantic representation of infrared images, For the semantic representation of the field of view image, GAP () represents the global average pooling operation. This formula indicates that a fusion residual structure is used in the generation of modal gating weights. That is, the original modal features and cross-modal fusion features are weighted and summed before the global average pooling operation is performed. This operation not only preserves the independent information of the original modality, but also introduces fusion semantic guidance, which improves the modality perception capability and selective discrimination accuracy of the weights.

[0059] Therefore, the weights of two modalities can be learned through a fully connected network, namely: ; in, () represents the Sigmoid activation function, used to map the output of the fully connected layer to the [0,1] interval to generate a normalized weight representation. Therefore, the final feature map of the fused weights can be represented as: .

[0060] In step S5, the YOLOv8 neck network is used to perform feature analysis on the image feature map after fusion weights, and the feature analysis results are output to the YOLOv8 detection head to determine whether a water inrush or mudslide disaster has occurred in the tunnel. That is, the optimized YOLOv8 neck network further mines and fuses the feature map of the input fusion weights, and outputs it to four detection heads for judgment.

[0061] like Figure 2 As shown, in this embodiment, the optimized YOLOv8 neck network includes: a multilayer feature focusing (MFF) module, a connection module Concat, a convolution module Conv, a feature extraction module C2f, and an upsampling module. The multilayer feature focusing module focuses multi-scale information, the connection module Concat stitches feature maps from different sources, the convolution module Conv extracts features and aligns dimensions, and the upsampling module Upsamples deep, small-sized feature maps back to shallower dimensions.

[0062] YOLOv8's neck network focuses the four input multi-scale feature maps through the MFF module, enriching the feature map information and addressing the issues of low semantic validity in shallow layers and poor spatial validity in deep layers, while reducing feature redundancy. The two feature maps, now fused with multi-scale information, are then distributed to various scales for separate extraction. Information exchange is achieved through upsampling and subsampling, and the feature map at each scale is mined multiple times.

[0063] like Figure 4 As shown, the MFF module still includes several sub-modules: a convolutional module (Conv), an upsampling module (Upsample), a downsampling module (Downsample), a connection module (Concat), and a TPA module. The Conv module is used for feature extraction and dimension alignment; the Upsample and Downsample modules are used for feature map resizing; and the Concat module is used to stitch together feature maps from different sources. The TPA module integrates a tensor product attention mechanism. Unlike the CAFA module, Q, K, and V all originate from the fused feature map output by CAFA, instead of being partly derived from the infrared image and partly from the view image.

[0064] MFF stands for Multi-Scale Feature Focusing Module. Its input consists of feature maps at four scales. After upsampling and downsampling, the features converge to the same intermediate scale. Global correlation is established through tensor product attention, followed by depthwise separable convolution. After convolution, feature information is superimposed through element-wise addition. The output from the convolutional layer before the multi-kernel DWConv module is directly added to the output of DWConv via element-wise residual connections. This approach effectively preserves mid-level semantic features while enhancing the ability of multi-scale depthwise separable convolution to model local pattern diversity. The output of the multi-scale feature focusing module is a feature map that integrates features from all four scales.

[0065] The MFF module introduced a Tensor Product Attention (TPA) mechanism to capture inter-scale and intra-channel interactions between multi-scale fused features. By projecting features as low-rank tensor factors and modeling global correlations, TPA enables selective semantic alignment between different feature levels, thereby effectively enhancing cross-scale consistency and representation accuracy.

[0066] The feature maps, synthesized from four scales, are output to four detection heads for evaluation. To adapt to the optimization requirements of multimodal and multi-scale fusion network structures, this invention designs an adaptive loss function module with multi-branch joint supervision based on the traditional target detection loss function. This module enhances multimodal semantic alignment and target region focusing capabilities, thus better suiting the characteristics of water and mud inrush disaster detection. Specifically, the loss function is: ; in, For the first i Scale-weighted parameters for each detection head; For the first i The bounding box regression loss of each detection head uses CIoU as the metric to supervise the position, shape and centroid offset between the predicted box and the ground box, ensuring that the box fits the water or mud inrush area closely. For the first i The target confidence loss of each detection head is used to determine whether each predicted box contains a valid target region. Binary cross-entropy is used for supervision to suppress false alarms and background false alarms. For the first i Classification loss of each detection head; This is a hyperparameter used to control the degree of involvement of the auxiliary loss; This is a modality stabilization loss used to constrain the feature representations from thermal and visible light images to have similar semantic orientations after fusion, preventing modality bias or information imbalance. Specifically, L2 distance or cosine similarity loss can be used to constrain the attention maps or global average pooling features of the two modalities. To address the potential misalignment between the attention map generated in the TPA module and the actual water and mud inrush areas, this attention alignment loss is introduced to ensure that attention is focused on the true target area. KL divergence or BCE loss is commonly used to supervise the fitting of the attention heatmap to the region labeling map.

[0067] The above parameters can be adjusted by testing mAP and F1 scores on the validation set to select the optimal combination. This process outputs the water and mud inrush situation of the detected area, including the location of the inrush, confidence level, and disaster category. When water and mud inrush is detected in a certain area, it will be marked with a bounding box to indicate the presence of an anomaly.

[0068] Example 2 This embodiment discloses a system for detecting water and mud inrush disasters in tunnels.

[0069] A system for detecting water and mud inrush disasters in tunnels includes: The UAV inspection image acquisition module is configured to: use a UAV to inspect inside the tunnel and identify the structural type and scale parameters of the tunnel cross section; determine the flight path of the UAV inside the tunnel based on the identification results, and acquire thermal imaging images and field-of-view images of the tunnel environment and thermal imaging conditions. The preprocessing module is configured to: annotate the thermal imaging image and the field-of-view image, and perform preprocessing operations; The diffuse reflection interference identification and suppression module is configured to use the YOLOv8 derived convolutional network RefNet to identify and suppress diffuse reflection interference in the preprocessed thermal imaging image and field of view image; The cross-modal adaptive fusion module is configured to: extract features from the preprocessed thermal imaging image and the field of view image using a YOLOv8-based backbone network; and fuse the feature extraction results of the backbone network using a cross-modal adaptive fusion attention mechanism to obtain a fused weighted image feature map. The water and mud inrush disaster identification module is configured to: use the neck network of YOLOv8 to perform feature analysis on the feature map of the image after fusion weights, and output the feature analysis results to the detection head of YOLOv8 to determine whether a water and mud inrush disaster has occurred in the tunnel. Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0070] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for detecting water and mud inrush disasters in tunnels as described in Embodiment 1 of this disclosure.

[0071] Example 4 The purpose of this embodiment is to provide an electronic device.

[0072] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a method for detecting water and mud inrush disasters in tunnels as described in Embodiment 1 of this disclosure.

[0073] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0074] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0075] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for detecting water and mud inrush disasters in tunnels, characterized in that, include: Drones were used to inspect tunnels and identify the structural type and dimensional parameters of tunnel cross sections. Based on the identification results, the flight path of the UAV in the tunnel is determined, and thermal imaging images and field-of-view images of the tunnel environment and thermal imaging conditions are collected. The thermal imaging image and the field of view image are labeled and preprocessing operations are performed; A YOLOv8-derived convolutional network, RefNet, is used to identify and suppress diffuse reflection interference in preprocessed thermal imaging images and field-of-view images. The YOLOv8-based backbone network extracts features from the preprocessed thermal imaging image and the field of view image respectively; and uses a cross-modal adaptive fusion attention mechanism to fuse the feature extraction results of the backbone network with feature weights to obtain a fused weighted image feature map. The neck network of YOLOv8 is used to perform feature analysis on the image feature map after fusion weights, and the feature analysis results are output to the detection head of YOLOv8 to determine whether water inrush or mud inrush disaster has occurred in the tunnel.

2. The method for detecting water and mud inrush disasters in tunnels as described in claim 1, characterized in that, The drone used is equipped with a control chip, which embeds the intelligent cross-section recognition network Tunnel-Net, the YOLOv8 network architecture and its derived convolutional network RefNet.

3. The method for detecting water and mud inrush disasters in tunnels as described in claim 2, characterized in that, The intelligent cross-section recognition network Tunnel-Net includes a geometric feature encoding module, a classification and regression module, and a binocular visual depth estimation module, and the binocular visual depth estimation module contains IMU data; wherein, the geometric feature encoding module and the classification and regression module work together to identify the structural type of the tunnel cross-section; the binocular visual depth estimation module is used to determine the flight risk of the UAV and provide data support for collision warning and obstacle avoidance.

4. The method for detecting water and mud inrush disasters in tunnels as described in claim 1, characterized in that, The diffuse reflection interference is identified and suppressed using RefNet, a convolutional network derived from YOLOv8, including: A reflection noise database is constructed, and reflection feature vectors in thermal imaging images and field-of-view images are extracted based on the derived convolutional network RefNet. Real-time feature matching is performed with the constructed reflection noise feature database through a fast matching algorithm to achieve diffuse reflection interference identification. An adaptive reflection mask is generated for the identified diffuse reflection interference to achieve diffuse reflection interference suppression.

5. The method for detecting water and mud inrush disasters in tunnels as described in claim 1, characterized in that, The YOLOv8-based backbone network performs feature extraction on both thermal imaging images and viewport images, including: Based on the convolutional modules and C2f modules of different orders in the backbone network, features of different scales are extracted from the thermal imaging image and the field of view image to obtain thermal imaging images and field of view images with multiple scales respectively.

6. The method for detecting water and mud inrush disasters in tunnels as described in claim 1, characterized in that, The cross-modal adaptive fusion attention mechanism is based on a cross-modal feature fusion architecture constructed by a cross-modal tensor product attention interaction module and an adaptive modal fusion weight module.

7. The method for detecting water and mud inrush disasters in tunnels as described in claim 1, characterized in that, The feature extraction results of the backbone network are fused using a cross-modal adaptive fusion attention mechanism, including: A modal interactive attention mechanism based on tensor product attention is constructed, which flattens the enhanced thermal imaging feature map and the intramodal enhancement map of visible light in the field of view into token form. Two independent fully connected networks are used to generate a set of low-rank tensor factors for the thermal imaging feature map and the field of view feature map in token form, respectively, to construct the query, key and value in tensor product form for calculating the cross-modal attention matrix.

8. A system for detecting water and mud inrush disasters in tunnels, characterized in that, include: The drone inspection image acquisition module is configured to: use drones to inspect inside the tunnel and identify the structural type and scale parameters of the tunnel cross section; Based on the identification results, the flight path of the UAV in the tunnel is determined, and thermal imaging images and field-of-view images of the tunnel environment and thermal imaging conditions are collected. The preprocessing module is configured to: annotate the thermal imaging image and the field-of-view image, and perform preprocessing operations; The diffuse reflection interference identification and suppression module is configured to use the YOLOv8 derived convolutional network RefNet to identify and suppress diffuse reflection interference in the preprocessed thermal imaging image and field of view image; The cross-modal adaptive fusion module is configured to: extract features from the preprocessed thermal imaging image and the field of view image using a YOLOv8-based backbone network; and fuse the feature extraction results of the backbone network using a cross-modal adaptive fusion attention mechanism to obtain a fused weighted image feature map. The water and mud inrush disaster identification module is configured to: use the neck network of YOLOv8 to perform feature analysis on the feature map of the image after fusion weights, and output the feature analysis results to the detection head of YOLOv8 to determine whether a water and mud inrush disaster has occurred in the tunnel.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by a processor, the program implements the steps of a method for detecting water and mud inrush disasters in tunnels as described in any one of claims 1-7.

10. An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the method for detecting water and mud inrush disasters in tunnels as described in any one of claims 1-7.

Images