Real-time semantic segmentation tunnel back break monitoring method and system

Abstract

The invention relates to a real-time semantic segmentation tunnel over-excavation and under-excavation monitoring method and system, and the method comprises the steps: obtaining high-resolution images of different excavation surface regions in a tunnel construction process suitable for excavation under different excavation conditions and geological backgrounds, and marking over-excavation and under-excavation region labels; performing data cleaning and enhancement, obtaining a plurality of tunnel over-break and under-break images with the resolution of 512 * 1024 after data cleaning and enhancement processing, and forming a target data set; constructing an FP-Former semantic segmentation deep neural network, training the FP-Former semantic segmentation deep neural network by using images in the target data set, monitoring an over-excavation and under-excavation area, and outputting a prediction probability graph; and carrying out binarization processing on the output prediction probability graph, marking the pixel with the probability greater than 0.5 as an over-excavation area, and otherwise, marking the pixel with the probability greater than 0.5 as an under-excavation area. According to the method, the features of different scales can be effectively extracted, so that the segmentation precision is improved, good performance and efficiency balance is shown when the method is operated on GPU equipment, and large-scale data processing becomes possible.

Description

Technical Field

[0001] The present invention relates to the field of intelligent construction technology, and in particular to a real-time semantic segmentation tunnel over-excavation and under-excavation monitoring method and system. Background Art

[0002] Tunnel engineering plays a vital role in modern infrastructure construction, especially in urban transportation, subway systems and water conservancy projects. However, in the actual construction process, over-excavation and under-excavation often occur. Traditional tunnel excavation methods rely on manual operation and are limited by the experience and judgment of construction workers. When faced with emergencies, construction workers may make wrong decisions, resulting in over-excavation or under-excavation. This phenomenon may not only lead to safety hazards of tunnel structures, but also increase the complexity and cost of subsequent maintenance.

[0003] Overbreak usually refers to the situation where the actual amount of earth excavated exceeds the design requirements during the excavation process. This situation may lead to a decrease in the stability of the tunnel wall and increase the risk of collapse. Underbreak means failure to excavate to the predetermined depth or width, which may lead to insufficient tunnel capacity and affect subsequent equipment installation and operational safety. During the construction process, if there is a lack of effective real-time monitoring methods, the construction team may not be able to understand the changes in the excavation progress and status in a timely manner, and miss the best time to correct the problem, which increases the risk of overbreak and underbreak. Therefore, in order to ensure the overall quality and safety of tunnel projects, real-time and accurate monitoring of the state of the excavation surface and identification of overbreak and underbreak have become the key to ensuring the quality of tunnel construction.

[0004] In recent years, deep learning and computer vision technologies have made remarkable progress in various fields, especially in image processing and analysis. Among them, semantic segmentation, as an important image processing task, has been widely studied and applied.

[0005] Real-time semantic segmentation technology provides strong support for tunnel construction monitoring, and can quickly and accurately analyze images obtained in the tunnel. Through deep learning algorithms, the system can automatically identify over-excavation and under-excavation areas in the image and clearly mark potential construction problems. This precise recognition capability allows construction personnel to obtain feedback in the first time and understand the construction progress and safety status in a timely manner. Once the system detects over-excavation or under-excavation, construction personnel can quickly take corresponding corrective measures, thereby significantly reducing safety risks. This not only ensures the safety of construction personnel, but also helps maintain the structural integrity of the tunnel and avoid greater losses caused by the expansion of problems. In addition, the real-time monitoring and feedback mechanism improves construction efficiency, allowing construction teams to respond quickly in a dynamically changing environment and optimize construction plans and resource allocation.

[0006] Existing semantic segmentation uses convolutional neural networks (CNN) and its various variants, such as U-Net and DeepLab, etc. Although good results have been achieved, due to its large number of parameters and slow processing speed, it cannot be well deployed on the mobile terminal, and its real-time segmentation accuracy and speed need to be improved. Based on the above problems, the present invention proposes a real-time semantic segmentation tunnel over-excavation and under-excavation monitoring method that can be more rapid and more accurately quantified, which not only improves the accuracy of real-time detection of tunnel over-excavation and under-excavation, but also can provide the fastest feedback, so that engineers can have a deep understanding of the health status of the tunnel, identify potential problems in a timely manner, and implement more effective maintenance strategies in a timely manner. Summary of the invention

[0007] The purpose of the present invention is to overcome the shortcomings of the above technologies and provide an intelligent real-time semantic segmentation tunnel over-break and under-break monitoring method and system, which can monitor the over-break and under-break earthwork area of ​​the tunnel in real time.

[0008] To achieve the above object, the technical solution of the present invention is:

[0009] In a first aspect, the present invention provides a real-time semantic segmentation tunnel over-digging and under-digging monitoring method, characterized in that the monitoring method comprises the following contents:

[0010] Step 1: Obtain high-resolution images of different excavation surface areas during tunnel construction that are suitable for excavation under different excavation conditions and geological backgrounds, and mark the over-excavation and under-excavation area labels;

[0011] Step 2: Data cleaning and enhancement

[0012] 2.1 Data cleaning:

[0013] ①Mark the images with a clarity variance lower than 10 as blurred images and delete them, and retain the images with a clarity variance not lower than 10;

[0014] ② Calculate the hash value of the image, delete the images with the same hash value, and keep only one image with the same hash value to ensure that each image has a unique hash value;

[0015] ③ Delete invalid images containing useless information to ensure that the remaining images are related to the mining conditions and over-mining and under-mining states. The invalid images include completely black screens, completely white screens, or other irrelevant backgrounds;

[0016] 2.2 Data enhancement is carried out in the following steps:

[0017] ① Rotation: Randomly rotate the image with a rotation angle between 30° and +30° to generate images at multiple angles to enhance the model's adaptability to different directions;

[0018] ② Zoom: Randomly zoom the image in a range of 50%-200% to simulate shooting at different distances;

[0019] ③ Cropping: Randomly crop a part of the image and crop the image resolution to 512×1024;

[0020] ④ Color adjustment: adjust the brightness, contrast, and saturation of the image to simulate images under different lighting conditions;

[0021] ⑥Adding noise: Adding random noise or simulating sensor noise to the image to improve the robustness of the model under low-quality images or interference conditions;

[0022] ⑦Flip: Randomly flip the image horizontally or vertically;

[0023] After data cleaning and enhancement, several tunnel over-break and under-break images with a resolution of 512×1024 are obtained to form the target data set.

[0024] Step 3: Build FP-Former semantic segmentation deep neural network

[0025] The FP-Former semantic segmentation deep neural network includes a trunk block, a residual block, a deep dual-resolution module, a two-layer superimposed FPformer module, a deep aggregation pyramid pool module and a pixel-level classification head; the trunk block is composed of two 3×3 convolutional layers in series, each of which includes a 3×3 convolution, a batch normalization operation and a RELU operation; the residual block is composed of four consecutive basic residual blocks in series, the input of the residual block is the output of the trunk block, the output of the residual block is a high-resolution feature map N, and the high-resolution feature map N is reduced in dimension by step convolution to obtain a low-resolution feature map n; the deep dual-resolution module includes a high-resolution branch and a low-resolution branch, and the high-resolution feature map N and the low-resolution feature map n enter the high-resolution branch and the low-resolution branch of the deep dual-resolution module respectively; the deep dual-resolution module outputs a low-resolution feature f and a high-resolution feature F; then enters the two-layer superimposed FPformer module to output a high-resolution feature O and a low-resolution feature o;

[0026] The low-resolution feature o enters the deep aggregation pyramid pooling module to extract context information. The output feature information processed by the deep aggregation pyramid pooling module is fused with the high-resolution feature O to obtain an output feature map with a step size of 8;

[0027] Finally, the output feature map with a stride of 8 is passed to the pixel-level classification head to predict dense semantic labels and output the predicted probability map;

[0028] Step 4. Use the images in the target data set to train the FP-Former semantic segmentation deep neural network, use the trained FP-Former semantic segmentation deep neural network to monitor the over-excavation and under-excavation areas, and output the predicted probability map; binarize the output predicted probability map, set the threshold to 0.5, and mark the pixels with a probability greater than 0.5 as over-excavation areas, otherwise as under-excavation areas, and then obtain the area of ​​the over-excavation or under-excavation areas respectively for engineering monitoring.

[0029] Furthermore, the pixel-level classification head consists of a 3×3 convolutional layer and a 1×1 convolutional layer, and both convolutional layers include batch normalization and ReLU activation functions.

[0030] Furthermore, the deep dual-resolution module includes a high-resolution branch, a low-resolution branch and a bilateral fusion unit, the high-resolution branch includes two 3×3 convolution operations, two batch normalization operations, and uses ReLU as the activation function; the high-resolution feature map N is sequentially subjected to a 3×3 convolution operation, a batch normalization operation, a ReLU function, a 3×3 convolution operation and a batch normalization operation to extract feature information, and at the same time, the high-resolution feature map N is residually connected with the output result of the last batch normalization operation, and the feature after the residual connection is recorded as N1; the number of channels of the two 3×3 convolution operations is 128;

[0031] The low-resolution branch has the same structure as the high-resolution branch, which also includes two 3×3 convolution operations, two batch normalizations, ReLU as the activation function and residual connection, but the number of channels of the two 3×3 convolution operations is 256, and the residual connection is also used. The input of the low-resolution branch is the low-resolution feature map n, and the feature after the residual connection is recorded as n1;

[0032] The bilateral fusion unit includes high-to-low fusion and low-to-high fusion. For high-to-low fusion, N1 is downsampled by a 3×3 convolution sequence with a step size of 2 and a channel number of 256, and then batch normalized and concatenated with n1 to obtain n2;

[0033] For low-to-high fusion, n1 is first compressed by a 1×1 convolution with 128 channels, batch normalized, upsampled by bilinear interpolation, and then concatenated and fused with N1 to obtain N2;

[0034] n2 is activated by ReLU and upsampled, and then fused with N2 after ReLU activation to obtain a new high-resolution feature map F.

[0035] At the same time, n2 is processed with the ReLU activation function and performs stride convolution dimensionality reduction to obtain a new low-resolution feature map f. At this time, the step size of the stride convolution is 16.

[0036] Furthermore, the FPformer module has a low-resolution branch and a high-resolution branch. In the low-resolution branch, the new low-resolution feature map f is passed through the GPU-friendly attention module to capture the high-level global context. After the attention map output by the GPU-friendly attention module is fused with the new low-resolution feature map f, the fused feature map enters a KAN module and undergoes a 3×3 convolution operation to extract features, and then is residually connected with the input of the KAN module to obtain the feature map X l ;

[0037] In the high-resolution branch, a cross-attention module is used, and the new high-resolution feature map F is combined with the feature map X l After splicing and fusion, we get the feature map X h , feature map X h As the Q branch input cross attention module, the feature map X l After convolution and pooling, the feature map X is obtained. c , feature map X c As the K and V branches input into the cross-attention module, the cross-attention module processes and outputs the attention map, which is then combined with the feature map X h Perform residual connection, then extract features through two 3×3 stacked convolutional layers, and perform residual connection on the inputs of the two 3×3 stacked convolutional layers to obtain feature map F1;

[0038] Feature map F1 and feature map X l After splicing, the low-resolution feature o is obtained as the output of the low-resolution branch; the low-resolution feature o is then upsampled and fused with the feature map F1 to generate the high-resolution feature O as the output of the high-resolution branch.

[0039] Furthermore, the GPU-friendly attention module is expressed as:

[0040]

[0041] Among them, Δ=0.05∈(-0.1.0.1) is the correction coefficient, which is used to adjust the GDN output; α=0.5 is the weight parameter, which is used to adjust V g The influence of; GFA represents GPU-friendly attention operation; GDN is the grouped double normalization operation; X represents the input feature; is the learnable parameter in the GPU-friendly attention module; M g =M×H, where M is the parameter dimension, d is the feature dimension, H is the number of heads of the multi-head attention mechanism, and T represents the matrix transpose;

[0042] The cross-attention module is expressed as:

[0043]

[0044] Among them, β=1 is the weight parameter used to adjust V c The influence of; CA is cross-attention module operation; X h , X l Represent the feature maps on the high-resolution branch and the low-resolution branch respectively, It is a set of matrix operations including splitting, permutation and reshaping, which is applied to the feature map X c After processing, the inputs of the K and V branches are recorded as K c , V c ;d h represents the feature dimension of the high-resolution branch. At the same time, the feature map X c By X l It is calculated by the function θ composed of the pooling layer and the convolution layer; X c The size of the space is expressed from X l The number of tokens generated.

[0045] Further, the deep aggregation pyramid pooling module includes a pooling operation with a step size of 2 and a core of 5, a pooling operation with a step size of 4 and a core of 9, a pooling operation with a step size of 8 and a core of 17, and a global average pooling operation;

[0046] The outputs after the three pooling operations and the global average pooling operation are processed by 1×1 convolution and upsampling operations respectively, with a total of four branches; at the same time, the low-resolution feature o is processed by 1×1 convolution to obtain the output y1, y1 and the output of the first branch are processed by dilated convolution with a dilation rate of 4 to obtain the output y2, y2 and the output of the second branch are processed by dilated convolution with a dilation rate of 8 to obtain the output y3, y3 and the output of the third branch are processed by dilated convolution with a dilation rate of 12 to obtain the output y4, y4 and the output of the fourth branch are processed by dilated convolution with a dilation rate of 16 to obtain the output y5; y1, y2, y3, y4, and y5 are spliced, processed by 1×1 convolution, and then spliced ​​with y1 to obtain the output of the deep aggregation pyramid pooling module.

[0047] Furthermore, the Adam W optimizer was used to train the FP-Former semantic segmentation deep neural network. During training, the initial learning rate was 0.0004 and the weight decay was 0.0125. A poly learning strategy with a power of 0.9 was used to reduce the learning rate, and 484 epochs and a batch size of 12 were set.

[0048] Furthermore, the area of ​​the over-excavation or under-excavation area is ratioed to the total area of ​​the tunnel to obtain the area percentage, and an area percentage threshold is set. When the area percentage of the detected over-excavation or under-excavation area exceeds the set area percentage threshold, an early warning is immediately triggered. Different levels of early warnings are set according to the severity of the over-excavation or under-excavation area.

[0049] ① Warning level: Slight over-excavation or under-excavation, the excavator's excavation posture needs to be corrected;

[0050] ② Alarm level: moderate over-excavation or moderate under-excavation, immediate measures must be taken to re-excavate the under-excavated part and adjust the excavation posture;

[0051] ③ Emergency level: Severe over-excavation or under-excavation requires stopping operations and conducting in-depth investigations, resetting the excavation plan, and correcting the deviation;

[0052] If the area percentage does not exceed the threshold range, the tunnel quality is qualified; otherwise, an early warning and feedback are given if the quality is unqualified.

[0053] Further, check the area percentage. If the over-digging area percentage is within [3%, 5%] of the total area or the under-digging area percentage is within [2%-3%] of the total area, set it to a warning level; if the over-digging area percentage is within (5%-10%] of the total area or the under-digging area percentage is within (3%-5%] of the total area, set it to an alarm level; if the over-digging area percentage exceeds 10% of the total area or the under-digging area percentage exceeds 5% of the total area, set it to an emergency level; if the remaining area ratios are not within the above range, they are considered qualified.

[0054] In a second aspect, the present invention provides a real-time semantic segmentation tunnel over-excavation and under-excavation monitoring system, the system comprising the following modules:

[0055] Data collection module: Use drones, laser scanners or high-definition cameras to obtain high-resolution images during tunnel construction to ensure coverage of different excavation surface areas, and combine soil sensors and pressure sensors to record real-time excavation depth and soil properties;

[0056] Data annotation module: annotate the collected images, clearly distinguish the over-excavated and under-excavated image areas and make accurate labels;

[0057] Data cleaning and enhancement module: removes blurry, duplicate or invalid images, and performs image preprocessing using rotation, scaling, cropping and color adjustment, adding noise and flipping operations;

[0058] Image segmentation module: FP-Former semantic segmentation deep neural network is established for image segmentation, and connected with data annotation module and data cleaning and enhancement module to obtain processed images, so as to train FP-Former semantic segmentation deep neural network to detect and quantify the predicted probability map of tunnel over-break and under-break areas;

[0059] Over-break and under-break area measurement module: Binarize the predicted probability map, set the threshold to 0.5, mark pixels with a probability greater than 0.5 as over-break areas, otherwise mark pixels as under-break areas, calculate the area of ​​over-break and under-break areas, and compare the area of ​​over-break or under-break areas with the total area of ​​the tunnel to obtain the area percentage, and set the area percentage threshold;

[0060] Early warning feedback module: connected with the over-excavation and under-excavation area measurement module, according to the detected area percentage, timely early warning and feedback to the background system on the degree of tunnel over-excavation and under-excavation. The background system automatically issues an alarm and prompts relevant personnel to take measures.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] In view of the inadequacy of tunnel over-excavation and under-excavation monitoring accuracy and real-time performance, the present invention proposes to use FP-Former semantic segmentation deep neural network for image semantic segmentation. The FPformer module combines the advantages of feature pyramid network and can effectively extract features of different scales, thereby improving segmentation accuracy. When the method of the present invention is run on GPU-type devices, it shows a good balance between performance and efficiency, making large-scale data processing possible.

[0063] The method of the present invention fully considers the use of global context information. By using the cross-attention mechanism, the network can combine global information while analyzing local features, thereby enhancing the ability to recognize over-excavated and under-excavated areas of tunnels, and allowing the network to dynamically focus on important features when processing images, thereby improving the accuracy of semantic segmentation.

[0064] In practical applications, especially in tunnel construction environments, over-excavation and under-excavation areas can be detected quickly and accurately, ensuring efficient real-time monitoring. By quantifying these areas with high precision and analyzing the percentage of over-excavation and under-excavation areas, construction personnel can obtain feedback in a timely manner, thereby taking corresponding measures to ensure the safety and quality of tunnel construction. The method of the present invention not only improves the efficiency of monitoring, but also provides an effective technical means for tunnel quality assurance, and can play an important role in complex construction environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a schematic diagram of the structure of the FP-Former semantic segmentation deep neural network in the present invention;

[0066] Figure 2 It is a schematic diagram of the structure of the deep dual-resolution module in the FP-Former semantic segmentation deep neural network of the present invention;

[0067] Figure 3 It is a schematic diagram of the structure of the FPformer module in the FP-Former semantic segmentation deep neural network of the present invention;

[0068] Figure 4 It is a schematic diagram of the structure of the deep aggregation pyramid pool module in the FP-Former semantic segmentation deep neural network of the present invention;

[0069] Figure 5 It is a flow chart of the real-time semantic segmentation tunnel over- and under-digging monitoring system in the present invention. DETAILED DESCRIPTION

[0070] Compared with the prior art, the present invention adopts the FP-Former semantic segmentation deep neural network and uses the FPformer module to achieve a better compromise between performance and efficiency on GPU-type devices for semantic segmentation tasks; at the same time, without losing efficiency, it makes full use of the global context and deeply uses attention to improve semantic segmentation, thereby achieving high-precision and rapid monitoring of the over- and under-digging areas of the tunnel, and can accurately and efficiently detect and quantify the over- and under-digging categories, providing an effective means to ensure the quality of the tunnel.

[0071] The steps of a real-time semantic segmentation tunnel over-excavation and under-excavation monitoring method of the present invention are:

[0072] 1. Dataset Preparation

[0073] 1. Sensor data: Combined with soil sensors, pressure sensors and other equipment, real-time excavation depth and soil properties are recorded. The type, density and permeability of soil are closely related to tunneling operations. The properties of the soil are recorded in real time to ensure that the selected area is suitable for excavation.

[0074] 2. Image acquisition: Use industrial cameras or high-definition video equipment to obtain high-resolution images during tunnel construction to ensure that different excavation surface areas can be collected.

[0075] 2. Data Annotation

[0076] 1. Over- and under-digging annotation: Use image annotation tools (such as LabelMe, VGG Image Annotator, etc.) to manually annotate the over- and under-digging areas to ensure that each image has accurate labels.

[0077] 2. Diversity considerations: Ensure that the labeled samples cover different mining conditions and geological backgrounds to improve the generalization ability of the model.

[0078] 3. Data Cleaning and Enhancement

[0079] 1. Data cleaning:

[0080] ① Use image clarity indicators (such as calculating Laplace variance) to evaluate the clarity of each image. Images with a clarity variance lower than 10 will be marked as blurry images and subsequently deleted.

[0081] ② Calculate the hash value of the image to detect duplicate images. The same hash value means that these images are duplicates, and then delete the duplicate images to ensure that each image in the dataset is unique.

[0082] ③ Check whether the image contains useless information, such as completely black screen, white screen or other irrelevant background, which is an invalid image. Then delete these invalid images to ensure that the remaining images are related to the mining conditions and over-mining status.

[0083] Through the above operations, blurred, duplicated or invalid images are removed to ensure the quality of the dataset.

[0084] 2 Data enhancement is carried out in the following steps:

[0085] ① Rotation: Randomly rotate the image (rotation angle between 30° and +30°) to generate images at multiple angles to enhance the model’s adaptability to different directions.

[0086] ② Zoom: Randomly zoom the image (zoom range is between 50%-200%) to simulate shooting at different distances.

[0087] ③ Cropping: Randomly crop a part of the image and crop the image resolution to 512×1024.

[0088] ④ Color adjustment: adjust the brightness, contrast, saturation, etc. of the image to simulate images under different lighting conditions.

[0089] ⑥ Add noise: Add random noise (such as Gaussian noise) or simulated sensor noise to the image to improve the robustness of the model under low-quality images or interference conditions.

[0090] ⑦Flip: Randomly flip the image horizontally or vertically.

[0091] Through data cleaning and enhancement, the quality and diversity of the data set can be effectively improved, providing a more reliable and rich sample basis for subsequent model training, thereby improving the performance and effect of the model in practical applications.

[0092] 4. Divide the Dataset

[0093] After data cleaning and enhancement, the above images were processed to obtain 10,000 tunnel over- and under-break images with a resolution of 512×1024.

[0094] The target dataset R for training the FP-Former semantic segmentation deep neural network is constructed with images after data cleaning and enhancement. The target dataset R is divided into training set, validation set and test set in a ratio of 7:2:1 for the training and validation of the FP-Former semantic segmentation deep neural network.

[0095] 5. Building FP-Former semantic segmentation deep neural network

[0096] The FP-Former semantic segmentation deep neural network, such as Figure 1 As shown, it includes a trunk block, a residual block, a deep double-resolution module, a two-layer superimposed FPformer module, a deep aggregation pyramid pool module and a pixel-level classification head; the trunk block is composed of two 3×3 convolutional layers in series, and each 3×3 convolutional layer includes a 3×3 convolution, a batch normalization operation and a RELU operation; the residual block is composed of four consecutive basic residual blocks in series, the input of the residual block is the output of the trunk block, and the output of the residual block is a high-resolution feature map N. The high-resolution feature map N is reduced in dimension by step convolution to obtain a low-resolution feature map n; the deep double-resolution module includes a high-resolution branch and a low-resolution branch, and the high-resolution feature map N and the low-resolution feature map n enter the high-resolution branch and the low-resolution branch of the deep double-resolution module respectively; the deep double-resolution module outputs a low-resolution feature f and a high-resolution feature F; then enters the two-layer superimposed FPformer module, and outputs a high-resolution feature O and a low-resolution feature o;

[0097] The low-resolution feature o enters the deep aggregation pyramid pooling module to extract context information. The output feature information processed by the deep aggregation pyramid pooling module is fused with the high-resolution feature O to obtain an output feature map with a step size of 8.

[0098] Finally, the output feature map with a stride of 8 is passed to the pixel-level classification head to predict dense semantic labels and output the predicted probability map.

[0099] In this embodiment, the pixel-level classification head consists of a 3×3 convolution layer and a 1×1 convolution layer. The convolution layers both contain batch normalization, and the activation function is ReLU.

[0100] The specific process is: the input image starts with a backbone block consisting of two 3×3 convolutional layers, and then passes through 4 consecutive basic residual blocks to obtain a high-resolution feature map N, which can extract enough local information required for the high-resolution feature map for preprocessing. Then the high-resolution feature map N and the low-resolution feature map n are sent to the deep dual-resolution module respectively to realize the feature exchange between the high-resolution branch and the low-resolution branch, and use the output of the low-resolution branch to improve the semantic representation of the high-resolution feature. The required low-resolution feature map n is obtained by reducing the dimension of the high-resolution feature map N through step-length convolution. At this time, the step length of the low-resolution feature branch is 8, and the step length of the high-resolution feature branch is always maintained at 8.

[0101] The following is a detailed description of the deep dual-resolution module:

[0102] The deep dual-resolution module is as follows Figure 2 As shown in the figure, it contains high-resolution branches, low-resolution branches, and bilateral fusion units. The high-resolution branch contains two 3×3 convolution operations and two batch normalization operations, and uses ReLU as the activation function. The high-resolution feature map N is sequentially subjected to 3×3 convolution operations, batch normalization operations, ReLU functions, 3×3 convolution operations, and batch normalization operations to extract feature information. At the same time, the high-resolution feature map N is residually connected with the output result of the last batch normalization operation, and the feature after the residual connection is recorded as N1; the number of channels of the two 3×3 convolution operations is 128.

[0103] The low-resolution branch has the same structure as the high-resolution branch, and also contains two 3×3 convolution operations, two batch normalizations, and uses ReLU as the activation function. The only difference is that the number of channels of the two 3×3 convolution operations is changed to 256, and residual connections are also used. The input of the low-resolution branch is the low-resolution feature map n, and the feature after residual connection is recorded as n1;

[0104] The bilateral fusion unit includes fusing the high-resolution branch to the low-resolution branch (high-to-low fusion) and fusing the low-resolution branch to the high-resolution branch (low-to-high fusion). For high-to-low fusion, N1 is downsampled by a 3×3 convolution sequence (with a step size of 2 and a number of channels of 256), and then batch normalized and concatenated with n1 to obtain n2.

[0105] For low-to-high fusion, n1 is first compressed by 1×1 convolution, batch normalized, upsampled by bilinear interpolation, and then concatenated and fused with N1 to obtain N2;

[0106] n2 uses the ReLU activation function, and after upsampling, it is fused again with N2 processed by the ReLU activation function to obtain a new high-resolution feature map F;

[0107] At the same time, n2 is processed with the ReLU activation function and performs stride convolution dimensionality reduction to obtain a new low-resolution feature map f. At this time, the step size of the stride convolution is 16.

[0108] The following is a detailed description of the FPformer module:

[0109] The FPformer module is as follows Figure 3 As shown, it is also a dual-resolution branch.

[0110] In the low-resolution branch, the new low-resolution feature map f is passed through a GPU-friendly attention module to capture high-level global context. The GPU-friendly attention module can be expressed as:

[0111]

[0112] Among them, Δ=0.05∈(-0.1.0.1) is the correction coefficient, which is used to adjust the GDN output, and α=0.5 is the weight parameter, which is used to adjust V g GFA represents the GPU-friendly attention operation; GDN is the grouped double normalization operation; X represents the input feature, is the learnable parameter in the GPU-friendly attention module, M g =M×H, where M is the parameter dimension, d is the feature dimension, H is the number of heads of the multi-head attention mechanism, and T represents the matrix transpose.

[0113] After the attention map output by the GPU-friendly attention module is fused with the new low-resolution feature map f, the fused feature map enters a KAN module and undergoes a 3×3 convolution operation to extract features, and then performs a residual connection with the input of the KAN module to obtain the feature map X l .

[0114] In the high-resolution branch, a cross-attention module is used, and the new high-resolution feature map F is combined with the feature map X l After splicing and fusion, we get the feature map X h , as the Q branch input cross-attention module, while the feature map X l After convolution and pooling, the feature map X is obtained. c , as the input of K and V branches to the cross-attention module, after being processed by the cross-attention module, the attention map is output, and the attention map is then residually connected with the input of the Q branch. After that, two 3×3 stacked convolutional layers are used to extract features, and batch normalization is used. The activation function uses the ReLU function, and the residual connection is also used to obtain the feature map F1;

[0115] The high-level global context learned from the low-resolution branch is broadcast to each high-resolution pixel, making full use of the high-level semantic information learned from the low-resolution branch, and using a staircase layout to feed more representative features in the low-resolution branch into the cross-attention module. The cross-attention module can be expressed as:

[0116]

[0117] Among them, β=1 is the weight parameter used to adjust V c The influence of CA is the cross-attention module operation; X h , X l Represent the feature maps on the high-resolution branch and the low-resolution branch respectively, It is a set of matrix operations including splitting, permutation and reshaping, which is applied to the feature map X c After processing, the inputs of the K and V branches are recorded as K c ,V c ;d h represents the feature dimension of the high-resolution branch. At the same time, the feature map X c By X l It is calculated by the function θ composed of the pooling layer and the convolution layer. c The size of the space is expressed from X l The number of tokens generated.

[0118] Feature map F1 and feature map X l After splicing, the low-resolution feature o is obtained as the output of the low-resolution branch; the low-resolution feature o is then upsampled and fused with the feature map F1 to generate the high-resolution feature O as the output of the high-resolution branch; at this time, the step size of the low-resolution feature branch is 32.

[0119] The following is a detailed description of the deep aggregation pyramid pooling module:

[0120] The deep aggregation pyramid pooling module is as follows Figure 4 As shown, it includes pooling operations with a step size of 2 and a kernel of 5, a pooling operation with a step size of 4 and a kernel of 9, a pooling operation with a step size of 8 and a kernel of 17, and a global average pooling operation (the kernel is H×W, H is the height of the input image, and W is the width of the input image);

[0121] The outputs after the three pooling operations and the global average pooling operation are processed by 1×1 convolution and upsampling operations respectively, with a total of four branches; at the same time, the low-resolution feature o is processed by 1×1 convolution to obtain the output y1, y1 and the output of the first branch are processed by dilated convolution with a dilation rate of 4 to obtain the output y2, y2 and the output of the second branch are processed by dilated convolution with a dilation rate of 8 to obtain the output y3, y3 and the output of the third branch are processed by dilated convolution with a dilation rate of 12 to obtain the output y4, y4 and the output of the fourth branch are processed by dilated convolution with a dilation rate of 16 to obtain the output y5; y1, y2, y3, y4, and y5 are spliced, processed by 1×1 convolution, and then spliced ​​with y1 to obtain the output of the deep aggregation pyramid pooling module.

[0122] After the low-resolution feature o is input, a large pooling kernel with an exponential step size is used to generate feature maps with image resolutions of 1 / 128, 1 / 256, and 1 / 512. The input feature map and image-level information generated by the global average pooling are utilized. It is not enough to use a single 3×3 or 1×1 convolution to mix all multi-scale context information here. First, the feature map is upsampled, and then more 3×3 dilated convolutions are used, using different dilation rates to receive different degrees of receptive fields, aiming to capture multi-scale context information while fusing context information of different scales in a hierarchical residual manner. In addition, a 1×1 projection shortcut is added for easy optimization. In this module, the context extracted by the larger pooling kernel is integrated with the deeper information flow, and the multi-scale nature is formed by integrating different depths with different sizes of pooling kernels. Finally, all feature maps are connected and compressed using 1×1 convolutions.

[0123] 6. Training and Calculation

[0124] 7000, 2000, and 1000 finely annotated images in the target dataset R are used for training, validation, and testing, respectively. The Adam W optimizer is used for training, with an initial learning rate of 0.0004 and a weight decay of 0.0125. A poly learning strategy with a power of 0.9 is used to reduce the learning rate. The model is trained on 4 V100 GPUs using 484 epochs (about 120K iterations), a batch size of 12, and syncBN batch normalization. When the training model reaches convergence, that is, the model training gradient is close to 0.

[0125] Measurement of over-excavation and under-excavation areas: Binarize the predicted probability map output by the model, set the threshold to 0.5, and mark pixels with a probability greater than 0.5 as over-excavation areas, otherwise as under-excavation areas.

[0126] The area of ​​the over-excavation or under-excavation region can be obtained by counting the number of pixels within it:

[0127] S=NxAp

[0128] N is the number of pixels in the area, A p is the actual area corresponding to each pixel. If the image resolution is R pixels / meter (that is, there are R pixels per meter), then

[0129]

[0130] And the area of ​​each connected region i is:

[0131] S i =N i xA p

[0132] The total area is:

[0133]

[0134] Where k is the number of regions.

[0135] The annotated tunnel over-excavation and under-excavation images to be detected are pre-input into the FP-Former semantic segmentation deep neural network for quantitative analysis to obtain the predicted probability map of the over-excavation and under-excavation area, and then the area of ​​the over-excavation and under-excavation area is obtained. The area percentage is obtained by comparing the area of ​​the over-excavation or under-excavation area with the total area of ​​the tunnel, and the area percentage threshold is set. When the area percentage of the detected over-excavation or under-excavation area exceeds the set area percentage threshold, the system immediately triggers an early warning. According to the severity of the over-excavation or under-excavation area, different levels of early warning are set, such as:

[0136] ① Warning level: Slight over-excavation or under-excavation, the excavator's excavation posture needs to be corrected.

[0137] ②Alarm level: Moderate over-excavation or moderate under-excavation. Immediate measures must be taken to re-excavate the under-excavated part and adjust the excavation posture.

[0138] ③ Emergency level: Severe over-excavation or under-excavation requires stopping operations and conducting in-depth investigations, resetting the excavation plan, and correcting the deviation.

[0139] If the threshold value is not exceeded, the tunnel quality is qualified; otherwise, an early warning and feedback will be given if the quality is unqualified.

[0140] Specifically, check the area percentage. If the over-excavation area percentage is within [3%, 5%] of the total area or the under-excavation area percentage is within [2%-3%] of the total area, set it to a warning level; if the over-excavation area percentage is within (5%-10%] of the total area or the under-excavation area percentage is within (3%-5%] of the total area, set it to an alarm level; if the over-excavation area percentage exceeds 10% of the total area or the under-excavation area percentage exceeds 5% of the total area, set it to an emergency level; if the remaining area ratios are not within the above range, they are considered qualified.

[0141] This paper compares the FP-Former semantic segmentation deep neural network with other advanced methods, compares the segmentation effect of tunnel over-excavation and under-excavation areas, and evaluates it on the above-constructed dataset (containing noisy images). mIoU and FPS are used as performance and efficiency metrics, respectively, and FPS is measured on a single RTX 2080Ti without tensor acceleration by default. The results of the experiment on the test set are shown in Table 1 below:

[0142] Table 1

[0143]

[0144] The present invention uses the FP-Former semantic segmentation deep neural network as the segmentation model of the target image, which greatly improves the accuracy of recognition and quantification, enabling the system to achieve real-time high-accuracy quantization, with FPS above 190 and mIoU above 80%. Multi-scale hole convolution combined with layered residuals receives different feature information from multiple angles and performs feature fusion by combining contextual information, which ensures the quality of feature information while solving the problem of information loss and the robustness of the model to noise in complex scenarios, making the model more stable and reliable. And by introducing the KAN module, the model can effectively reduce the risk of overfitting, while greatly reducing the amount of calculation and the number of parameters, while improving the efficiency and speed of the model. While maintaining relatively high performance, the size and computing requirements of the model can be significantly reduced, especially in the industrial environment, while maintaining high-precision segmentation performance, it also meets the needs of high real-time performance.

[0145] As an embodiment, the present invention provides a real-time semantic segmentation tunnel over-break and under-break monitoring system, which is specially used to detect tunnel over-break and under-break and complete quantitative analysis thereof, and specifically includes the following modules:

[0146] Data collection module: Use drones, laser scanners or high-definition cameras to obtain high-resolution images during tunnel construction to ensure coverage of different excavation surface areas, and combine with soil sensors, pressure sensors and other equipment to record real-time excavation depth and soil properties;

[0147] Data annotation module: annotate the collected images, clearly distinguish the over-excavated and under-excavated image areas and make accurate labels;

[0148] Data cleaning and enhancement module: removes blurry, duplicate or invalid images to ensure the quality of the data set. At the same time, it increases sample diversity and improves the robustness of the model through techniques such as rotation, scaling, cropping and color adjustment;

[0149] Image segmentation module: FP-Former semantic segmentation deep neural network is established for image segmentation, and connected with data annotation module and data cleaning and enhancement module to obtain processed images, so as to train FP-Former semantic segmentation deep neural network to detect and quantify the predicted probability map of tunnel over-break and under-break areas;

[0150] Over-break and under-break area measurement module: Binarize the predicted probability map, set the threshold to 0.5, mark pixels with a probability greater than 0.5 as over-break areas, otherwise mark pixels as under-break areas, calculate the area of ​​over-break and under-break areas, and compare the area of ​​over-break or under-break areas with the total area of ​​the tunnel to obtain the area percentage, and set the area percentage threshold;

[0151] Early warning feedback module: connected with the over-excavation and under-excavation area measurement module, according to the detected area percentage, timely early warning and feedback to the background system on the degree of tunnel over-excavation and under-excavation. The background system automatically issues an alarm and prompts relevant personnel to take measures.

[0152] The real-time tunnel over-break and under-break area monitoring system of the present invention is highly customizable and can be used in a variety of different working environments, and can provide real-time feedback to workers so that safety measures can be adjusted in a timely manner. It has wide applicability and practicality and can be used for safety monitoring in a variety of working environments, including tunnels, subway light rails, roads and bridges, and underground pipelines. In practical applications, the present invention provides a new idea and solution for problems such as earthwork over-break and under-break area detection and real-time feedback in a variety of workplaces, further promoting the development of this field, with great significance and broad application prospects, and also provides strong support and guarantee for the development of tunnel over-break and under-break monitoring technology.

[0153] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A real-time semantic segmentation tunnel over-digging and under-digging monitoring method, characterized in that The monitoring method includes the following contents: Step 1: Obtain high-resolution images of different excavation surface areas during tunnel construction that are suitable for excavation under different excavation conditions and geological backgrounds, and mark the over-excavation and under-excavation area labels; Step 2: Data cleaning and enhancement 2.1 Data cleaning: ①Mark the images with a clarity variance lower than 10 as blurred images and delete them, and retain the images with a clarity variance not lower than 10; ② Calculate the hash value of the image, delete the images with the same hash value, and keep only one image with the same hash value to ensure that each image has a unique hash value; ③ Delete invalid images containing useless information to ensure that the remaining images are related to the mining conditions and over-mining and under-mining states. The invalid images include completely black screens, completely white screens, or other irrelevant backgrounds. 2.2 Data enhancement is carried out in the following steps: ① Rotation: Randomly rotate the image with a rotation angle between 30° and +30° to generate images at multiple angles to enhance the model's adaptability to different directions; ② Zoom: Randomly zoom the image in a range of 50%-200% to simulate shooting at different distances; ③ Cropping: Randomly crop a part of the image and crop the image resolution to 512×1024; ④ Color adjustment: adjust the brightness, contrast, and saturation of the image to simulate images under different lighting conditions; ⑥Adding noise: Adding random noise or simulating sensor noise to the image to improve the robustness of the model under low-quality images or interference conditions; ⑦Flip: Randomly flip the image horizontally or vertically; After data cleaning and enhancement, several tunnel over-break and under-break images with a resolution of 512×1024 are obtained to form the target data set. Step 3: Build FP-Former semantic segmentation deep neural network The FP-Former semantic segmentation deep neural network includes a trunk block, a residual block, a deep dual-resolution module, a two-layer superimposed FPformer module, a deep aggregation pyramid pool module and a pixel-level classification head; the trunk block is composed of two 3×3 convolutional layers in series, each of which includes a 3×3 convolution, a batch normalization operation and a RELU operation; the residual block is composed of four consecutive basic residual blocks in series, the input of the residual block is the output of the trunk block, the output of the residual block is a high-resolution feature map N, and the high-resolution feature map N is reduced in dimension by step convolution to obtain a low-resolution feature map n; the deep dual-resolution module includes a high-resolution branch and a low-resolution branch, and the high-resolution feature map N and the low-resolution feature map n enter the high-resolution branch and the low-resolution branch of the deep dual-resolution module respectively; the deep dual-resolution module outputs a low-resolution feature f and a high-resolution feature F; then enters the two-layer superimposed FPformer module to output a high-resolution feature O and a low-resolution feature o; The low-resolution feature o enters the deep aggregation pyramid pooling module to extract context information. The output feature information processed by the deep aggregation pyramid pooling module is fused with the high-resolution feature O to obtain an output feature map with a step size of 8; Finally, the output feature map with a stride of 8 is passed to the pixel-level classification head to predict dense semantic labels and output the predicted probability map; Step 4. Use the images in the target data set to train the FP-Former semantic segmentation deep neural network, use the trained FP-Former semantic segmentation deep neural network to monitor the over-excavation and under-excavation areas, and output the predicted probability map; binarize the output predicted probability map, set the threshold to 0.5, and mark the pixels with a probability greater than 0.5 as over-excavation areas, otherwise as under-excavation areas, and then obtain the area of ​​the over-excavation or under-excavation areas respectively for engineering monitoring.

2. The monitoring method according to claim 1, characterized in that: The pixel-level classification head consists of a 3×3 convolutional layer and a 1×1 convolutional layer, and both convolutional layers contain batch normalization and ReLU activation functions.

3. The monitoring method according to claim 1, characterized in that: The deep dual-resolution module includes a high-resolution branch, a low-resolution branch and a bilateral fusion unit. The high-resolution branch includes two 3×3 convolution operations and two batch normalization operations, and uses ReLU as the activation function. The high-resolution feature map N is sequentially subjected to a 3×3 convolution operation, a batch normalization operation, a ReLU function, a 3×3 convolution operation and a batch normalization operation to extract feature information. At the same time, the high-resolution feature map N is residually connected with the output result of the last batch normalization operation, and the feature after the residual connection is recorded as N1. The number of channels for both 3×3 convolution operations is 128; The low-resolution branch has the same structure as the high-resolution branch, which also includes two 3×3 convolution operations, two batch normalizations, ReLU as the activation function and residual connection, but the number of channels of the two 3×3 convolution operations is 256, and the residual connection is also used. The input of the low-resolution branch is the low-resolution feature map n, and the feature after the residual connection is recorded as n1; The bilateral fusion unit includes high-to-low fusion and low-to-high fusion. For high-to-low fusion, N1 is downsampled by a 3×3 convolution sequence with a step size of 2 and a channel number of 256, and then batch normalized and concatenated with n1 to obtain n2; For low-to-high fusion, n1 is first compressed by a 1×1 convolution with 128 channels, batch normalized, upsampled by bilinear interpolation, and then concatenated and fused with N1 to obtain N2; n2 is activated by ReLU and upsampled, and then fused with N2 after ReLU activation to obtain a new high-resolution feature map F. At the same time, n2 is processed with the ReLU activation function and performs stride convolution dimensionality reduction to obtain a new low-resolution feature map f. At this time, the step size of the stride convolution is 16.

4. The monitoring method according to claim 1, characterized in that: The FPformer module has a low-resolution branch and a high-resolution branch. In the low-resolution branch, the new low-resolution feature map f is passed through the GPU-friendly attention module to capture the high-level global context. After the attention map output by the GPU-friendly attention module is fused with the new low-resolution feature map f, the fused feature map enters a KAN module and undergoes a 3×3 convolution operation to extract features, and then is residually connected with the input of the KAN module to obtain the feature map X l ; In the high-resolution branch, a cross-attention module is used, and the new high-resolution feature map F is combined with the feature map X l After splicing and fusion, we get the feature map X h , feature map X h As the Q branch input cross attention module, the feature map X l After convolution and pooling, the feature map X is obtained. c , feature map X c As the K and V branches input into the cross-attention module, the cross-attention module processes and outputs the attention map, which is then combined with the feature map X h Perform residual connection, then extract features through two 3×3 stacked convolutional layers, and perform residual connection on the inputs of the two 3×3 stacked convolutional layers to obtain feature map F1; Feature map F1 and feature map X l After splicing, the low-resolution feature o is obtained as the output of the low-resolution branch; the low-resolution feature o is then upsampled and fused with the feature map F1 to generate the high-resolution feature O as the output of the high-resolution branch.

5. The monitoring method according to claim 4, characterized in that: The GPU-friendly attention module is represented as: Among them, Δ=0.05∈(-0.1.0.1) is the correction coefficient, which is used to adjust the GDN output; α=0.5 is the weight parameter, which is used to adjust V g The influence of; GFA represents GPU-friendly attention operation; GDN is the grouped double normalization operation; X represents the input feature; is the learnable parameter in the GPU-friendly attention module; M g =M×H, where M is the parameter dimension, d is the feature dimension, H is the number of heads of the multi-head attention mechanism, and T represents the matrix transpose; The cross-attention module is expressed as: Among them, β=1 is the weight parameter used to adjust V c The influence of; CA is cross-attention module operation; X h , X l Represent the feature maps on the high-resolution branch and the low-resolution branch respectively, It is a set of matrix operations including splitting, permutation and reshaping, which is applied to the feature map X c After processing, the inputs of the K and V branches are recorded as K c , V c ;d h represents the feature dimension of the high-resolution branch. At the same time, the feature map X c By X l It is calculated by the function θ composed of the pooling layer and the convolution layer; X c The size of the space is expressed from X l The number of tokens generated.

6. The monitoring method according to claim 1, characterized in that: The deep aggregation pyramid pooling module includes a pooling operation with a step size of 2 and a core of 5, a pooling operation with a step size of 4 and a core of 9, a pooling operation with a step size of 8 and a core of 17, and a global average pooling operation; The outputs after the three pooling operations and the global average pooling operation are processed by 1×1 convolution and upsampling operations respectively, with a total of four branches; at the same time, the low-resolution feature o is processed by 1×1 convolution to obtain the output y1, y1 and the output of the first branch are processed by dilated convolution with a dilation rate of 4 to obtain the output y2, y2 and the output of the second branch are processed by dilated convolution with a dilation rate of 8 to obtain the output y3, y3 and the output of the third branch are processed by dilated convolution with a dilation rate of 12 to obtain the output y4, y4 and the output of the fourth branch are processed by dilated convolution with a dilation rate of 16 to obtain the output y5; y1, y2, y3, y4, and y5 are spliced, processed by 1×1 convolution, and then spliced ​​with y1 to obtain the output of the deep aggregation pyramid pooling module.

7. The monitoring method according to claim 1, characterized in that: The Adam W optimizer is used to train the FP-Former semantic segmentation deep neural network. During training, the initial learning rate is 0.0004 and the weight decay is 0.0125. A poly learning strategy with a power of 0.9 is used to reduce the learning rate. 484 epochs and a batch size of 12 are set.

8. The monitoring method according to claim 1, characterized in that: The area of ​​the over-excavation or under-excavation area is compared with the total area of ​​the tunnel to obtain the area percentage, and the area percentage threshold is set. When the area percentage of the detected over-excavation or under-excavation area exceeds the set area percentage threshold, an early warning is triggered immediately. Different levels of early warnings are set according to the severity of the over-excavation or under-excavation area. ① Warning level: Slight over-excavation or under-excavation, the excavator's excavation posture needs to be corrected; ② Alarm level: moderate over-excavation or moderate under-excavation, immediate measures must be taken to re-excavate the under-excavated part and adjust the excavation posture; ③ Emergency level: Severe over-excavation or under-excavation requires stopping operations and conducting in-depth investigations, resetting the excavation plan, and correcting the deviation; If the area percentage does not exceed the threshold range, the tunnel quality is qualified; otherwise, an early warning and feedback are given if the quality is unqualified.

9. The monitoring method according to claim 7, characterized in that: Check the area percentage. If the over-digging area percentage is within [3%, 5%] of the total area or the under-digging area percentage is within [2%-3%] of the total area, set it to warning level; if the over-digging area percentage is within (5%-10%] of the total area or the under-digging area percentage is within (3%-5%] of the total area, set it to alarm level; if the over-digging area percentage exceeds 10% of the total area or the under-digging area percentage exceeds 5% of the total area, set it to emergency level; if the remaining area ratios are not within the above range, they are considered qualified.

10. A real-time semantic segmentation tunnel over-excavation and under-excavation monitoring system, characterized in that: The system includes the following modules: Data collection module: Use drones, laser scanners or high-definition cameras to obtain high-resolution images during tunnel construction to ensure coverage of different excavation surface areas, and combine soil sensors and pressure sensors to record real-time excavation depth and soil properties; Data annotation module: annotate the collected images, clearly distinguish the over-excavated and under-excavated image areas and make accurate labels; Data cleaning and enhancement module: removes blurry, duplicate or invalid images, and performs image preprocessing using rotation, scaling, cropping and color adjustment, adding noise and flipping operations; Image segmentation module: FP-Former semantic segmentation deep neural network is established for image segmentation, and connected with data annotation module and data cleaning and enhancement module to obtain processed images, so as to train FP-Former semantic segmentation deep neural network to detect and quantify the predicted probability map of tunnel over-break and under-break areas; Over-break and under-break area measurement module: Binarize the predicted probability map, set the threshold to 0.5, mark pixels with a probability greater than 0.5 as over-break areas, otherwise mark pixels as under-break areas, calculate the area of ​​over-break and under-break areas, and compare the area of ​​over-break or under-break areas with the total area of ​​the tunnel to obtain the area percentage, and set the area percentage threshold; Early warning feedback module: connected with the over-excavation and under-excavation area measurement module, according to the detected area percentage, timely early warning and feedback to the background system on the degree of tunnel over-excavation and under-excavation. The background system automatically issues an alarm and prompts relevant personnel to take measures.

Images