Automatic segmentation method for double-U-shaped tunnel lining leakage water disease image

Abstract

The invention provides an automatic segmentation method for a double-U-shaped tunnel lining leakage water disease image. The method comprises the following steps: inputting a tunnel lining leakage water disease image into a first U-shaped segmentation network to obtain a first segmentation result; wherein the first U-shaped segmentation network comprises a first encoder and a first decoder, the first encoder comprises a BTransX block, the BTransX block is a TransX block with improved attention, the first decoder comprises a DConvSTA block, and the DConvSTA block is a dilated convolution block with improved attention; inputting the tunnel lining leakage water disease image and the first segmentation result into a second U-shaped segmentation network to obtain a segmentation result of the tunnel lining leakage water disease image; wherein the second U-type segmentation network comprises a second encoder and a second decoder, and each of the second encoder and the second decoder comprises a DConvSTA block. According to the invention, the accuracy of tunnel lining leakage water disease image segmentation can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular, to an automatic segmentation method for images of leakage diseases in double U-shaped tunnel linings. Background Art

[0002] The image segmentation algorithm for tunnel lining diseases can not only classify and locate leakage, but also accurately segment the specific morphological features of leakage, which has great application value for improving the detection of leakage diseases in tunnel linings. The two-stage segmentation model of first detection and then segmentation is widely adopted because it can achieve pixel-level disease segmentation. Although the multi-stage method improves the detection accuracy to a certain extent, its disadvantages are high model complexity and slow running speed, which hinder the engineering application of leakage disease segmentation in tunnel lining disease detection.

[0003] Compared with the two-stage method, the single-stage algorithm for leakage diseases in tunnel linings can achieve a better balance between accuracy and efficiency. Currently, Transformer models the long-term dependence relationship of data through the self-attention mechanism and is widely used in the field of image segmentation. However, the Transformer method has deficiencies in capturing local regions of leakage diseases, resulting in insufficient accuracy in segmenting images of leakage diseases in tunnel linings. Summary of the Invention

[0004] An embodiment of the present invention provides an automatic segmentation method for images of leakage diseases in double U-shaped tunnel linings to solve the problem of low accuracy in segmenting images of leakage diseases in tunnel linings.

[0005] In a first aspect, an embodiment of the present invention provides an automatic segmentation method for images of leakage diseases in double U-shaped tunnel linings, including:

[0006] Inputting an image of a leakage disease in a tunnel lining into a first U-shaped segmentation network to obtain a first segmentation result; wherein, the first U-shaped segmentation network includes a first encoder and a first decoder, the first encoder includes a BTransX block, and the BTransX block is a TransX block with improved attention, and the first decoder includes a DConvSTA block, and the DConvSTA block is a dilated convolutional block with improved attention;

[0007] Inputting the image of the leakage disease in the tunnel lining and the first segmentation result into a second U-shaped segmentation network to obtain a segmentation result of the image of the leakage disease in the tunnel lining; wherein, the second U-shaped segmentation network includes a second encoder and a second decoder, and both the second encoder and the second decoder include DConvSTA blocks.

[0008] In a possible implementation manner, the BTransX block includes a TransX block and a BWA attention module connected in sequence;

[0009] The BWA attention module includes two convolutional layers filled with 1 connected in sequence, a 1×1 convolutional layer, and a Sigmoid activation function. The output of the Sigmoid activation function is multiplied by the original feature map input to the BWA attention module and then added together to obtain the output of the BWA attention module.

[0010] In a possible implementation, the DConvSTA block includes two dilated convolutional layers filled with 1 and with a dilation rate of 1 connected in sequence, and a triple attention module.

[0011] In a possible implementation, the triple attention module includes three branches. Each branch includes a Z-Pool layer, two depthwise strip convolutional layers, and a Sigmoid activation function connected in sequence. The output of the Sigmoid activation function in each branch is multiplied by the original feature map input to that branch to obtain the output of that branch. The outputs of each branch are averaged and aggregated to obtain the output of the triple attention module.

[0012] In a possible implementation, the first U-shaped segmentation network further includes a SEASPP block. The first encoder includes a first BTransX block, a second BTransX block, and a third BTransX block connected in sequence. The second encoder includes a first DConvSTA block, a second DConvSTA block, and a third DConvSTA block connected in sequence;

[0013] The first encoder is connected to the first decoder through the SEASPP block, and each BTransX block is skip-connected to the DConvSTA block in the corresponding stage;

[0014] The SEASPP block includes a dilated spatial pyramid pooling module and a SE module connected in sequence.

[0015] In a possible implementation, the second U-shaped segmentation network further includes a dilated spatial pyramid pooling module. The second encoder includes four DConvSTA blocks connected in sequence. The second decoder includes four DConvSTA blocks connected in sequence;

[0016] The second encoder is connected to the second decoder through the dilated spatial pyramid pooling module, and each DConvSTA block in the second encoder is skip-connected to the DConvSTA block in the corresponding stage of the second decoder.

[0017] In a possible implementation, each BTransX block of the first encoder is skip-connected to the DConvSTA block in the corresponding stage of the second decoder.

[0018] In a second aspect, an embodiment of the present invention provides an automatic segmentation device for the image of the leakage disease of the double U-shaped tunnel lining, including:

[0019] The first segmentation module is configured to input the image of the leakage disease of the tunnel lining into the first U-shaped segmentation network to obtain a first segmentation result. The first U-shaped segmentation network includes a first encoder and a first decoder. The first encoder includes BTransX blocks, and the BTransX block is a TransX block with improved attention. The first decoder includes DConvSTA blocks, and the DConvSTA block is a dilated convolutional block with improved attention.

[0020] The second segmentation module is configured to input the image of the leakage disease of the tunnel lining and the first segmentation result into the second U-shaped segmentation network to obtain the segmentation result of the image of the leakage disease of the tunnel lining. The second U-shaped segmentation network includes a second encoder and a second decoder, and both the second encoder and the second decoder include DConvSTA blocks.

[0021] In a third aspect, an embodiment of the present invention provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method described in the first aspect or any possible implementation manner of the first aspect above are implemented.

[0022] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the steps of the method described in the first aspect or any possible implementation manner of the first aspect above are implemented.

[0023] An embodiment of the present invention provides an automatic segmentation method for the image of the leakage disease of the double U-shaped tunnel lining. By constructing a DoubleUNet structure combining Transformer and CNN through two U-shaped segmentation networks, the advantage that the convolutional operation in the CNN segmentation model is good at extracting local features of the leakage area is retained, and the details of the leakage area can be effectively captured in different directions. At the same time, the inherent global self-attention mechanism of Transformer is used to make up for the defect that the receptive field of the CNN model is limited and it is difficult to obtain the global context information of the leakage area, so as to effectively extract the global and local information of the leakage area and effectively improve the accuracy of the segmentation of the image of the leakage disease of the tunnel lining. Description of the Drawings

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0025] Figure 1 It is the implementation flowchart of an automatic segmentation method for the image of leakage disease of a double U-shaped tunnel lining provided by an embodiment of the present invention;

[0026] Figure 2 It is the structural schematic diagram of the BTransX block provided by an embodiment of the present invention;

[0027] Figure 3 It is the structural schematic diagram of the BWA attention module provided by an embodiment of the present invention;

[0028] Figure 4 It is the structural schematic diagram of the DConvSTA block provided by an embodiment of the present invention;

[0029] Figure 5 It is the structural schematic diagram of the triple attention module provided by an embodiment of the present invention;

[0030] Figure 6 It is the structural schematic diagram of the SEASPP block provided by an embodiment of the present invention;

[0031] Figure 7 It is the structural schematic diagram of the CBDoubleUNet provided by an embodiment of the present invention;

[0032] Figure 8A It is the original image in the dataset provided by an embodiment of the present invention;

[0033] Figure 8B It is the segmentation result obtained based on CACDU-Net provided by an embodiment of the present invention;

[0034] Figure 8C It is the segmentation result obtained based on UNet provided by an embodiment of the present invention;

[0035] Figure 8D It is the segmentation result obtained by an automatic segmentation method for the image of leakage disease of a double U-shaped tunnel lining provided by an embodiment of the present invention;

[0036] Figure 8E It is the original image in the dataset provided by another embodiment of the present invention;

[0037] Figure 8F It is the segmentation result obtained based on CACDU-Net provided by another embodiment of the present invention;

[0038] Figure 8G It is the segmentation result obtained based on UNet provided by another embodiment of the present invention;

[0039] Figure 8H It is the segmentation result obtained by an automatic segmentation method for the image of leakage disease of a double U-shaped tunnel lining provided by another embodiment of the present invention;

[0040] Figure 9 It is a schematic structural diagram of an automatic segmentation device for the image of the leakage disease of a double U-shaped tunnel lining provided by an embodiment of the present invention;

[0041] Figure 10 It is a schematic diagram of a terminal provided by an embodiment of the present invention. Specific Embodiments

[0042] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present invention. However, those skilled in the art should clearly understand that the present invention can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present invention.

[0043] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will be described through specific embodiments with reference to the accompanying drawings.

[0044] The tunnel lining disease image segmentation algorithm can not only classify and locate the leakage, but also accurately segment the specific morphological features of the leakage, which has great application value for improving the detection of the leakage disease of the tunnel lining. In recent years, many fully supervised leakage disease image segmentation methods have emerged. Huang et al. first introduced the fully convolutional network (FCN) and proposed a two-stream disease segmentation algorithm to segment the tunnel lining crack disease and the leakage disease. Subsequently, a two-stage tunnel lining leakage disease image segmentation method based on Mask R-CNN emerged. The two-stage segmentation model of first detecting and then segmenting is widely adopted because it can achieve pixel-level disease segmentation. Although the multi-stage method improves the detection accuracy to a certain extent, its disadvantages are high model complexity and slow running speed, which hinder the engineering application of the leakage disease segmentation in the tunnel lining disease detection.

[0045] Compared with the two-stage method, the single-stage algorithm for the leakage disease of tunnel lining can achieve a better balance between accuracy and efficiency. Although a leakage disease segmentation method proposed by Huang et al. based on the FCN architecture is a single-stage method, it does not consider the global context information that plays an important role, so the resulting segmentation accuracy is low; and it is easily interfered by facilities such as pipes, holes, cables, brackets, etc. To solve this problem, Li et al. improved the DeepLabv3+ algorithm to detect the leakage disease of tunnel lining. Specifically, an efficient channel attention mechanism (ECANet) was added to the encoder-decoder part of DeepLabv3+ to further effectively improve the segmentation accuracy. Wang et al. first proposed replacing the multi-branch standard convolution of the encoder with two strip convolutions of the same depth, and this method is more in line with the actual characteristics of the leakage disease. Feng et al. developed dozens of U-shaped semantic segmentation models by combining UNet and UNet++ with six classification convolutional neural networks. The experimental results show that increasing the number of layers of UNet can enable the network to learn more representative features, and skip connections play a crucial role in achieving accurate segmentation of leakage disease images. From this experimental conclusion, to overcome the defects of the multi-stage method with slow running speed and the single-stage method with low accuracy, the DoubleUNet with two stacked UNet structures gives a good inspiration. However, it is not appropriate to directly apply it to the segmentation of tunnel lining leakage images. Due to the interference of gravity factors, the DoubleUNet of pure convolutional neural network (CNN) is not conducive to extracting leakage areas with complex shapes due to the inherent limitations of convolutional operations.

[0046] Recently, Transformer has been widely used in the field of image segmentation by modeling the long-term dependence relationship of data through the self-attention mechanism. Aiming at the problem of leakage disease image segmentation, Zhao et al. first proposed a model composed of a multi-level Transformer encoder and an adaptive multi-task decoder. Compared with other methods, the Transformer used in this network module takes into account the global context information, and the decoder pays more attention to detecting the edges of the leakage areas, and can accurately segment the leakage areas and the surrounding wet stain areas in the tunnel lining leakage images. However, the Transformer method has deficiencies in capturing the local areas of leakage diseases. Therefore, the present invention constructs a DoubleUNet structure combining Transformer and CNN, which can effectively extract the global and local information of the leakage areas, and will effectively improve the accuracy of tunnel lining leakage disease image segmentation.

[0047] See Figure 1 , which shows the implementation flowchart of an automatic segmentation method for double U-shaped tunnel lining leakage disease images provided by an embodiment of the present invention, and is described in detail as follows:

[0048] Step 101: Input the image of the leakage disease of the tunnel lining into the first U-shaped segmentation network to obtain the first segmentation result. Among them, the first U-shaped segmentation network includes a first encoder and a first decoder. The first encoder includes BTransX blocks, and the BTransX block is a TransX block with improved attention. The first decoder includes DConvSTA blocks, and the DConvSTA block is a dilated convolutional block with improved attention.

[0049] In this embodiment, the convolutional operation used in the CNN segmentation model is good at extracting the local features of the leakage area and can effectively capture the details of the leakage area in different directions. However, due to its limited receptive field, it is difficult to obtain the global context information of the leakage area. Therefore, using a pure CNN network structure is not conducive to extracting the leakage area with a relatively dispersed distribution and complex shape.

[0050] Different from CNN, Transformer performs well in extracting global information and modeling long-distance dependencies due to its inherent global self-attention mechanism, which can effectively make up for the deficiencies of convolutional operations in these aspects. In addition, increasing the number of layers of U-Net can enable the network to learn more representative features, which is of great significance for achieving accurate segmentation of leakage images in complex environments. Therefore, in this embodiment, a dual U-shaped segmentation network CBDoubleUNet combining Transformer and CNN is proposed, and the TransX block is used as the encoder component of the first U-shaped segmentation network, so that the first part of the dual U-shaped segmentation network has the advantages of Transformer.

[0051] Step 102: Input the image of the leakage disease of the tunnel lining and the first segmentation result into the second U-shaped segmentation network to obtain the segmentation result of the image of the leakage disease of the tunnel lining. Among them, the second U-shaped segmentation network includes a second encoder and a second decoder, and both the second encoder and the second decoder include DConvSTA blocks.

[0052] In this embodiment, both the encoder and the decoder of the second U-shaped segmentation network are composed of DConvSTA blocks. After learning the global features of the leakage image with the help of Transformer in Step 101, the local details of the image are further explored through the CNN network, so that the segmentation network can better ensure the integrity of the segmentation area while making full use of the global context information, avoiding false detection and missed detection in the complex tunnel environment, and thus improving the accuracy of the segmentation result.

[0053] In the test embodiment of the present invention, a DoubleUNet structure combining Transformer and CNN is constructed through two U-shaped segmentation networks, retaining the advantage that the convolutional operation in the CNN segmentation model is good at extracting local features of the water seepage area, and can effectively capture the details of the water seepage area in different directions. At the same time, the inherent global self-attention mechanism of Transformer is used to make up for the defect that the receptive field of the CNN model is limited and it is difficult to obtain the global context information of the water seepage area, and can effectively extract the global and local information of the water seepage area, effectively improving the accuracy of image segmentation of water seepage diseases in tunnel linings.

[0054] In a possible implementation, the BTransX block includes a TransX block and a BWA attention module connected in sequence;

[0055] The BWA attention module includes two convolutional layers filled with 1 connected in sequence, a 1×1 convolutional layer, and a Sigmoid activation function. The output of the Sigmoid activation function is multiplied and added to the original feature map input to the BWA attention module to obtain the output of the BWA attention module.

[0056] In this embodiment, the TransX block can endow the network with strong inductive bias and an effectively enlarged receptive field through the dual-dynamic TokenMixer (D-Mixer) that aggregates global information and local information, and is good at obtaining the entire water seepage area in the water seepage image. In order to enhance the local detail extraction ability of the TransX block in the water seepage area and avoid the problem of blurred water seepage edges, a lightweight attention (BWA) module with good boundary perception ability is proposed in this embodiment and integrated into the TransX block, which is named the BTransX block. The structure of the BTransX block is as Figure 2 shown, and the structure of the BWA attention module is as Figure 3 shown. Therefore, the BTransX block can better model the characteristics of the entire water seepage area in the water seepage image and improve its ability to capture the boundary details of the water seepage area.

[0057] In this embodiment, the proposed BWA attention module is composed of two 3×3 convolutional layers filled with 1, a 1×1 convolutional layer, and a Sigmoid activation function. Among them, the convolutional operation filled with 1 can make the convolutional kernel cover all input pixels, making the feature extraction effect of each pixel point in the feature map more uniform, ensuring that the edge pixels in the water seepage area can be fully utilized, so as to retain more water seepage boundary information. The final attention weight is generated through the Sigmoid activation function and multiplied by the input feature to generate the attention feature. At the same time, a residual connection is also introduced. The following equation gives the specific formula of the proposed BWA attention module.

[0058]

[0059] Among them, x represents the original feature map input to the BWA attention module, and σ represents the Sigmoid function.

[0060] In a possible implementation, the DConvSTA block includes two dilated convolutional layers with a padding of 1 and a dilation rate of 1 connected in sequence and a triple attention module.

[0061] In this embodiment, as Figure 4 shown, the DConvSTA module includes two dilated convolutions with a padding of 1 and a dilation rate of 1 and the proposed triple attention module. The convolution with a padding of 1 in the DConvSTA module makes full use of the image edge information, and the dilated convolution with a dilation rate of 1 can expand the receptive field. Combining the two can obtain more detailed characteristics of the seepage water image without losing resolution and introducing additional parameters, and can effectively avoid the appearance of holes in the segmented seepage water area.

[0062] In a possible implementation, the triple attention module includes three branches. Each branch includes a Z-Pool layer, two depthwise strip convolutions, and a Sigmoid activation function connected in sequence. The output of the Sigmoid activation function in each branch is multiplied by the original feature map input to the branch to obtain the output of the branch. The outputs of each branch are averaged and aggregated to obtain the output of the triple attention module.

[0063] In this embodiment, the triple attention module (Triplet Attention) can establish cross-dimensional dependence relationships between different channels and between channels and space, enabling the network to automatically judge the importance of different feature channels during the training process, thereby effectively selecting the features of the seepage water area in the seepage water image.

[0064] Considering that the actual seepage water area of the tunnel lining presents a strip shape due to the influence of gravity, in order to enhance the segmentation ability of the CBDoubleUNet for the strip-shaped seepage water, in this embodiment, a Stripe Triplet Attention module is proposed based on Triplet Attention. The structure is as Figure 5 shown. First, the channel dimension of each layer is reduced to two dimensions through the Z-Pool layer, enabling the layer to retain rich feature representations while reducing the computational load. Then, two depthwise strip convolutions are used to approximate the standard depthwise convolution with a large kernel, where the kernel size is 7. The attention weights are generated through Sigmoid, and the original shape is restored to complete the operation of one branch. Finally, the outputs of the three branches are aggregated using an average operation.

[0065] In a possible implementation, the first U-shaped segmentation network further includes a SEASPP block, the first encoder includes a first BTransX block, a second BTransX block, and a third BTransX block connected in sequence, and the second encoder includes a first DConvSTA block, a second DConvSTA block, and a third DConvSTA block connected in sequence;

[0066] The first encoder is connected to the first decoder through the SEASPP block, and each BTransX block is skip-connected to the DConvSTA block in the corresponding stage;

[0067] The SEASPP block includes an atrous spatial pyramid pooling module and a SE module connected in sequence.

[0068] In this embodiment, the bottleneck layer in the U-Net structure plays a crucial role in achieving accurate leakage water segmentation. In this embodiment, the Figure 6 shown SEASPP module is embedded between the encoder and decoder to better capture the leakage water area. Convolution kernels with different dilation rates in the atrous spatial pyramid pooling (ASPP) module can capture multi-scale information. The convolution kernels with large dilation rates are used for segmenting large-area leakage water areas, and the convolution kernels with small dilation rates help segment small-area leakage water areas. After the SE module is added after the ASPP, the multi-scale features captured by the ASPP module can be recalibrated to filter out the feature maps that significantly contribute to the leakage water segmentation.

[0069] In a possible implementation, the second U-shaped segmentation network further includes an atrous spatial pyramid pooling module, the second encoder includes four DConvSTA blocks connected in sequence, and the second decoder includes four DConvSTA blocks connected in sequence;

[0070] The second encoder is connected to the second decoder through the atrous spatial pyramid pooling module, and each DConvSTA block in the second encoder is skip-connected to the DConvSTA block in the corresponding stage of the second decoder.

[0071] In this embodiment, the overall structure of the CBDoubleUNet composed of the first U-shaped segmentation network and the second U-shaped segmentation network is as Figure 7As shown in the figure, the whole is mainly composed of two stacked U-Net structures, which mainly include four parts: BTransXBlock, DConvSTA Block, banded triple attention and SEASPP module, wherein the serial number of the DConvSTA Block in the first decoder, the second encoder and the second decoder only represents the serial number of each DConvSTABlock in each encoder / decoder, and does not mean that the parameters of each DConvSTA1 and each DConvSTA2 are exactly the same, and the size of each BTransX and each DConvSTA block can be set and adjusted according to actual conditions.

[0072] CBDoubleUNet first uses Transformer to learn the global features of the water leakage image, and then uses the CNN network to deeply explore the local details of the image, so that the segmentation network can fully utilize the global context information while better ensuring the integrity of the segmented area, avoiding false detection and missed detection of tunnel lining water leakage areas in complex tunnel environments. The rest of the encoder and decoder of CBDoubleUNet are composed of DConvSTA modules.

[0073] In a possible implementation, each BTransX block of the first encoder is jump-connected to a DConvSTA block of a corresponding stage in the second decoder.

[0074] In this embodiment, if Figure 7 As shown, the feature maps output by each stage of the first encoder are respectively input into each stage of the second decoder, and together with the feature maps output by each stage of the second encoder constitute the input of the second decoder, so that the second decoder can capture and fuse global features and local detail features.

[0075] In a specific embodiment, in order to evaluate the effect of the automatic segmentation method of double U-shaped tunnel lining water leakage disease image proposed in this embodiment, a comparative experiment was conducted using a public tunnel lining water leakage image segmentation dataset. The dataset contains 3,300 images of water leakage inside the tunnel lining under complex environments. These leakage areas have obvious differences in morphology, including block, vertical strips or horizontal strips. The resolution of these images is unified to 256×256 pixels. During the training phase, the leakage dataset is further divided into a training set and a test set in a ratio of 7:3. In order to reduce the model's dependence on certain specific attributes in the image and to build consistency for the same input under different data perturbations, a pixel-level data enhancement strategy (randomly changing the image hue, randomly reducing the color depth, and applying Gaussian blur) is used for training.

[0076] The software environment of this experiment is: Windows 11 operating system, Pytorch 2.0.1 deep learning framework, and Python 3.9 programming language. The hardware environment is the Lenovo Intelligent Supercomputing Platform (LiCO), and the GPU used on this platform is Tesla V100S. The fixed learning rate is 1e-4, and the batch size is set to 6.

[0077] The experimental progress of this model is as follows: First, an ablation experiment of CBDoubleUNet was conducted. Using the DoubleUNet segmentation model as the Baseline, components such as the proposed BTransX block, SEASPP, and DConvSTA were gradually added. Table 1 shows the indicators at each stage of the experiment. It can be seen from Table 1 that there is a slight improvement in the IoU indicator after adding TransX compared to the Baseline. Replacing TransX with the BTransX block proposed in the present invention results in a more obvious improvement in the IoU and Dice indicators. Finally, after replacing the ordinary convolutions of the remaining encoders and decoders with DConvSTA that incorporates the strip triple attention mechanism, the IoU and Dice indicators are further significantly improved, increasing by 1.28% and 0.75% respectively.

[0078] Table 1

[0079]

[0080]

[0081] In addition, the CBDoubleUNet segmentation model was also compared with other methods in terms of Params, FLOPs, and segmentation metrics. The comparison data is shown in Table 2. It can be seen from the figure that CBDoubleUNet achieved the best results in almost all segmentation metrics and was at a medium level in terms of Params and FLOPs. Its number of parameters was lower than that of SwinUNet and UTNet, which also use Transformer, and its computational cost was relatively close to that of the UNet model. Thus, it can be seen that the CBDoubleUNet proposed in the present invention can achieve relatively high segmentation metrics with fewer computational costs and parameters.

[0082] Table 2

[0083] Model Pr Recall Dice IoU Acc Params FLOPs SwinUNet 0.9284 0.9314 0.9324 0.8741 0.9733 41.38 8.70 DeepLabv3+ 0.9364 0.9392 0.9406 0.8884 0.9760 5.81 6.60 UNet 0.9474 0.9386 0.9453 0.8968 0.9779 6.28 16.46 UTNet 0.9548 0.9447 0.9469 0.8996 0.9784 29.40 49.49 CACDU-Net 0.9508 0.9445 0.9474 0.9007 0.9787 38.34 22.89 CBDoubleUNet 0.9554 0.9544 0.9546 0.9137 0.9815 20.77 18.00

[0084] The results of the qualitative comparison of the method proposed in the present invention with other methods are as Figure 8A - Figure 8H shown, where Figure 8A is the original image 1 in the dataset, Figure 8B is the segmentation result of the original image 1 obtained based on CACDU-Net, Figure 8Cis the segmentation result of the original image 1 obtained based on UNet, Figure 8D is the segmentation result of the original image 1 obtained based on the method provided by the present invention, Figure 8E is the original image 2 in the dataset, Figure 8F is the segmentation result of the original image 2 obtained based on CACDU-Net, Figure 8G is the segmentation result of the original image 2 obtained based on UNet, Figure 8H is the segmentation result of the original image 2 obtained based on the method provided by the present invention. Overall, the method proposed by the present invention shows more accurate segmentation of the leakage area than other methods. Observation Figure 8A - Figure 8D , it can be found that CBDoubleUNet can more completely segment the defective area; observation Figure 8E - Figure 8H , it can be found that compared with other segmentation models, CBDoubleUNet can not only better exclude pipeline interference, but also has stronger boundary perception ability, and can capture more leakage details by making full use of boundary information.

[0085] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution is prior or subsequent. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0086] The following is the device embodiment of the present invention. For the details not described in detail, reference can be made to the corresponding method embodiments above.

[0087] Figure 9 The structural schematic diagram of an automatic segmentation device for leakage diseases of double U-shaped tunnel linings provided by an embodiment of the present invention is shown. For the convenience of description, only the parts related to the embodiments of the present invention are shown and are described in detail as follows:

[0088] As Figure 9 shown, an automatic segmentation device 9 for leakage diseases of double U-shaped tunnel linings includes:

[0089] A first segmentation module 91, configured to input the leakage disease image of the tunnel lining into the first U-shaped segmentation network to obtain a first segmentation result; wherein, the first U-shaped segmentation network includes a first encoder and a first decoder, the first encoder includes a BTransX block, and the BTransX block is a TransX block with improved attention, and the first decoder includes a DConvSTA block, and the DConvSTA block is a dilated convolutional block with improved attention;

[0090] The second segmentation module 92 is configured to input the tunnel lining leakage disease image and the first segmentation result into a second U-shaped segmentation network to obtain a segmentation result of the tunnel lining leakage disease image. The second U-shaped segmentation network includes a second encoder and a second decoder, and both the second encoder and the second decoder include DConvSTA blocks.

[0091] In a possible implementation, the BTransX block includes a TransX block and a BWA attention module connected in sequence.

[0092] The BWA attention module includes two convolutional layers with padding of 1, a 1×1 convolutional layer, and a Sigmoid activation function connected in sequence. The output of the Sigmoid activation function is multiplied and added to the original feature map input to the BWA attention module to obtain the output of the BWA attention module.

[0093] In a possible implementation, the DConvSTA block includes two dilated convolutional layers with padding of 1 and dilation rate of 1 and a triple attention module connected in sequence.

[0094] In a possible implementation, the triple attention module includes three branches. Each branch includes a Z-Pool layer, two depthwise strip convolutional layers, and a Sigmoid activation function connected in sequence. The output of the Sigmoid activation function in each branch is multiplied by the original feature map input to the branch to obtain the output of the branch. The outputs of the respective branches are averaged and aggregated to obtain the output of the triple attention module.

[0095] In a possible implementation, the first U-shaped segmentation network further includes a SEASPP block. The first encoder includes a first BTransX block, a second BTransX block, and a third BTransX block connected in sequence. The second encoder includes a first DConvSTA block, a second DConvSTA block, and a third DConvSTA block connected in sequence.

[0096] The first encoder is connected to the first decoder through the SEASPP block, and each BTransX block is skip-connected to the DConvSTA block in the corresponding stage.

[0097] The SEASPP block includes an atrous spatial pyramid pooling module and a SE module connected in sequence.

[0098] In a possible implementation, the second U-shaped segmentation network further includes an atrous spatial pyramid pooling module. The second encoder includes four DConvSTA blocks connected in sequence. The second decoder includes four DConvSTA blocks connected in sequence.

[0099] The second encoder is connected to the second decoder through an atrous spatial pyramid pooling module, and each DConvSTA block in the second encoder is skip-connected to the DConvSTA block in the corresponding stage of the second decoder.

[0100] In a possible implementation, each BTransX block of the first encoder is skip-connected to the DConvSTA block in the corresponding stage of the second decoder.

[0101] In the embodiment of the present invention, a DoubleUNet structure combining Transformer and CNN is constructed through two U-shaped segmentation networks, which retains the advantage that the convolutional operation in the CNN segmentation model is good at extracting local features of the water seepage area, can effectively capture details of the water seepage area in different directions, and at the same time uses the inherent global self-attention mechanism of Transformer to make up for the defect that the CNN model has a limited receptive field and is difficult to obtain global context information of the water seepage area, and can effectively extract global and local information of the water seepage area, effectively improving the accuracy of image segmentation of water seepage diseases in tunnel linings.

[0102] Figure 10 It is a schematic diagram of the terminal provided by the embodiment of the present invention. As Figure 10 shown, the terminal 10 of this embodiment includes: a processor 100, a memory 1001, and a computer program 1002 stored in the memory 1001 and executable on the processor 100. When the processor 100 executes the computer program 1002, the steps in each of the above embodiments of a method for automatically segmenting images of water seepage diseases in double U-shaped tunnel linings are implemented, such as Figure 1 the steps 101 to 102 shown. Alternatively, when the processor 100 executes the computer program 1002, the functions of each module / unit in each of the above device embodiments are implemented, such as Figure 9 the functions of the modules / units 91 to 92 shown.

[0103] Exemplarily, the computer program 1002 can be divided into one or more modules / units, and the one or more modules / units are stored in the memory 1001 and executed by the processor 100 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 1002 in the terminal 10. For example, the computer program 1002 can be divided into Figure 9 the modules / units 91 to 92 shown.

[0104] The terminal 10 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal 10 may include, but is not limited to, a processor 100 and a memory 1001. Those skilled in the art can understand that Figure 10 These are merely examples of the terminal 10 and do not constitute a limitation on the terminal 10. It may include more or fewer components than those shown in the figure, or combine certain components, or have different components. For example, the terminal may also include input / output devices, network access devices, a bus, etc.

[0105] The so-called processor 100 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0106] The memory 1001 may be an internal storage unit of the terminal 10, such as the hard disk or memory of the terminal 10. The memory 1001 may also be an external storage device of the terminal 10, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal 10. Further, the memory 1001 may also include both the internal storage unit and the external storage device of the terminal 10. The memory 1001 is used to store the computer program and other programs and data required by the terminal. The memory 1001 may also be used to temporarily store data that has been output or is to be output.

[0107] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0108] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0109] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in the form of hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0110] In the embodiments provided by the present invention, it should be understood that the disclosed device / terminal and method can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For example, the division of the module or unit is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0111] The unit described as a separated component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0112] In addition, in each embodiment of the present invention, each functional unit may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0113] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present invention, it may also be completed by instructing relevant hardware through a computer program. The computer program may be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various embodiments of a method for automatically segmenting images of water seepage and leakage diseases in a double-U-shaped tunnel lining can be implemented. Among them, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0114] The above-mentioned embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. An automatic image segmentation method for the leakage disease of a double-U-shaped tunnel lining, characterized in that, Including: Input the image of the leakage disease of the tunnel lining into the first U-shaped segmentation network to obtain the first segmentation result; wherein, the first U-shaped segmentation network includes a first encoder and a first decoder, the first encoder includes a BTransX block, the BTransX block is a TransX block with improved attention, the first decoder includes a DConvSTA block, and the DConvSTA block is a dilated convolutional block with improved attention; Input the image of the leakage disease of the tunnel lining and the first segmentation result into the second U-shaped segmentation network to obtain the segmentation result of the image of the leakage disease of the tunnel lining; wherein, the second U-shaped segmentation network includes a second encoder and a second decoder, and both the second encoder and the second decoder include the DConvSTA block.

2. The automatic segmentation method for the image of the leakage disease of a double-U-shaped tunnel lining according to claim 1, wherein The BTransX block includes a TransX block and a BWA attention module connected in sequence; The BWA attention module includes two convolutional layers with padding of 1, a 1×1 convolutional layer, and a Sigmoid activation function connected in sequence. The output of the Sigmoid activation function is multiplied by the original feature map input to the BWA attention module and then added to obtain the output of the BWA attention module.

3. The automatic segmentation method for the image of the leakage disease of a double U-shaped tunnel lining according to claim 1, characterized in that The DConvSTA block includes two dilated convolutional layers with padding of 1 and a dilation rate of 1 and a triple attention module connected in sequence.

4. An automatic image segmentation method for the leakage disease of a double U-shaped tunnel lining according to claim 3, characterized in that, The triple attention module includes three branches. Each branch includes a Z-Pool layer, two depthwise strip convolutional layers, and a Sigmoid activation function connected in sequence. The output of the Sigmoid activation function in each branch is multiplied by the original feature map input to the branch to obtain the output of the branch. The outputs of each branch are averaged and aggregated to obtain the output of the triple attention module.

5. An automatic image segmentation method for leakage diseases of a double-U-shaped tunnel lining according to claim 1, characterized in that The first U-shaped segmentation network further includes a SEASPP block. The first encoder includes a first BTransX block, a second BTransX block, and a third BTransX block connected in sequence. The second encoder includes a first DConvSTA block, a second DConvSTA block, and a third DConvSTA block connected in sequence; The first encoder is connected to the first decoder through the SEASPP block, and each BTransX block is skip-connected to the DConvSTA block in the corresponding stage; The SEASPP block includes an atrous spatial pyramid pooling module and a SE module connected in sequence.

6. The automatic segmentation method for the image of the leakage disease of a double U-shaped tunnel lining according to claim 5, characterized in that, The second U-shaped segmentation network further includes an atrous spatial pyramid pooling module. The second encoder includes four DConvSTA blocks connected in sequence. The second decoder includes four DConvSTA blocks connected in sequence; The second encoder is connected to the second decoder through the atrous spatial pyramid pooling module, and each DConvSTA block in the second encoder is skip-connected to the DConvSTA block in the corresponding stage in the second decoder.

7. An automatic image segmentation method for the leakage disease of a double U-shaped tunnel lining according to claim 6, characterized in that, Each BTransX block of the first encoder is skip-connected to the DConvSTA block in the corresponding stage of the second decoder.

8. An automatic image segmentation device for the leakage disease of a double-U-shaped tunnel lining, characterized in that, Including: The first segmentation module is used to input the image of the leakage disease of the tunnel lining into the first U-shaped segmentation network to obtain the first segmentation result; wherein, the first U-shaped segmentation network includes a first encoder and a first decoder, the first encoder includes BTransX blocks, and the BTransX block is a TransX block with improved attention, and the first decoder includes DConvSTA blocks, and the DConvSTA block is a dilated convolutional block with improved attention; The second segmentation module is used to input the image of the leakage disease of the tunnel lining and the first segmentation result into the second U-shaped segmentation network to obtain the segmentation result of the image of the leakage disease of the tunnel lining; wherein, the second U-shaped segmentation network includes a second encoder and a second decoder, and both the second encoder and the second decoder include DConvSTA blocks.

9. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7 above.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 7 above.

Images