Shield tunnel water leakage image segmentation method suitable for low-light environment, storage medium and equipment

Abstract

The invention discloses a shield tunnel water leakage image segmentation method suitable for a low-light environment, a storage medium and equipment. The shield tunnel water leakage image segmentation method suitable for the low-light environment comprises the following steps: acquiring a real-time sRGB image of a shield tunnel in the low-light environment; performing inverse operation on the obtained real-time sRGB image to obtain a pseudo RAW format image; and inputting the obtained pseudo RAW format image into a pre-trained instance segmentation model, and segmenting to obtain a water leakage disease area image of the shield tunnel. According to the invention, the image of the water leakage disease area of the shield tunnel can be quickly segmented in an extremely low light environment, and water leakage disease detection is realized.

Description

Technical Field

[0001] The invention relates to a shield tunnel water leakage image segmentation method, storage medium and equipment suitable for low-light environment, belonging to the technical field of tunnel engineering. Background Art

[0002] The urban underground transportation system utilizes underground space to effectively alleviate the congestion of highway traffic and improve travel efficiency. As the main form of subway tunnel construction, shield tunnels often face a series of problems including water leakage, settlement deformation, lining damage and corrosion due to factors such as uneven strata, construction quality, and surrounding engineering disturbances.

[0003] Among the above problems, the disaster caused by water leakage in the shield tunnel is particularly serious. Water leakage in the subway shield tunnel will cause changes in the shield tunnel structure and cause problems such as the fall-off of shield tunnel components, tunnel collapse, and short circuit of electrical equipment, which will seriously affect the normal operation of the train, endanger the personal safety of passengers, and cause huge economic and property losses.

[0004] At present, the construction focus of subway shield tunnels is gradually shifting from traditional construction to intelligent and rapid maintenance and monitoring. Traditional shield tunnel leakage detection methods mainly include visual inspection, fiber optic sensor detection and infrared thermal imaging detection. Visual inspection is the most basic method. The inspectors enter the tunnel and observe the leakage traces on the surface of the tunnel with their naked eyes, which has high labor costs. In addition, since the underground light is very dim, the observation results are easily affected by subjective factors, and it is difficult to ensure the accuracy of the detection. Fiber optic sensors are an advanced detection method that detects leakage diseases through the high sensitivity of sensors to changes in the external environment. However, the high cost and complex maintenance of fiber optic sensors limit their widespread application. Infrared thermal imaging is suitable for large-area leakage detection, but it is easily affected by the external environment and its accuracy is affected.

[0005] With the development of artificial intelligence technology, segmentation and detection algorithms based on deep learning have shown remarkable performance in the field of image processing. These algorithms can automatically identify areas of shield tunnel leakage and determine their severity, which significantly improves the efficiency of detection and brings great changes to the intelligent monitoring of subway tunnel diseases.

[0006] In this field, many scholars have made important contributions. Various algorithms in existing research have significantly improved the effect of water leakage segmentation on specific data sets. However, while the existing models have high detection accuracy, they often have large computational complexity and slow inference speed, which cannot meet the actual needs of fast and efficient detection. Existing studies have selected water leakage diseases under normal lighting as research objects. These images are collected under the intervention of a large number of lighting equipment, and the low-light engineering environment of subway shield tunnels is not fully considered. When the existing models process low-light tasks, the image features of the water leakage area will degenerate, the shallow features of the image will be polluted by noise, and the semantic response of the deep features will be reduced. In low-light environments, the model often misses the subtle features of the image and focuses on other areas polluted by noise, which will have an adverse effect on the actual detection task. The model is trained using a dataset collected under normal lighting conditions, and performs poorly in subsequent image segmentation tasks in low-light environments, and lacks generalization.

[0007] Using instance segmentation models for detection is one of the current mainstream methods. However, existing instance segmentation models complete segmentation tasks in high-exposure environments and are not suitable for the actual low-light environment inside tunnels. Traditional low-light detection methods require image enhancement and denoising algorithm preprocessing when dealing with low-light tasks, which is computationally complex. In addition, the current mainstream instance segmentation models have a large number of parameters and a slow segmentation speed. Summary of the invention

[0008] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a shield tunnel water leakage image segmentation method, storage medium and device suitable for low light environment, which can quickly segment the image of the water leakage disease area of ​​the shield tunnel in an extremely low light environment to realize water leakage disease detection. To achieve the above purpose, the present invention is implemented by adopting the following technical solutions:

[0009] In a first aspect, the present invention provides a shield tunnel water leakage image segmentation method suitable for low light environment, comprising:

[0010] Get real-time sRGB images of shield tunnels in low-light environments;

[0011] Perform an inverse operation on the acquired real-time sRGB image to obtain a pseudo RAW format image;

[0012] The obtained pseudo RAW format image is input into the pre-trained instance segmentation model to segment the leakage disease area image of the shield tunnel.

[0013] In combination with the first aspect, optionally, performing an inverse operation on the acquired real-time sRGB image to obtain a pseudo RAW format image includes:

[0014] The acquired real-time sRGB image is contrast adjusted by using an inverse change curve of a preset tone mapping curve to obtain a first adjusted image, wherein the inverse change curve of the preset tone mapping curve is expressed by the following formula:

[0015] ,

[0016] in, The contrast of the image after adjustment is the contrast of the image before adjustment;

[0017] The light intensity of the first adjusted image is adjusted by using an inverse transformation of gamma correction to obtain a second adjusted image, wherein the inverse transformation of gamma correction is expressed by the following formula:

[0018] ,

[0019] in, is the light intensity value of the corrected image, γ is the gamma value, is the light intensity value of the image before correction, The value is The constant of

[0020] Performing color correction on the second adjusted image using a preset color correction matrix to obtain an image in a standard color space;

[0021] Performing white balance adjustment on the image in the standard color space according to the light source of the shooting environment to obtain a third adjusted image;

[0022] The third adjusted image is digitally gained to obtain a gained image; the intensity values ​​of the third adjusted image and the gained image follow the exponential distribution of the following formula:

[0023] ,

[0024] Where p is the probability distribution of image intensity values, is the intensity value of the color image after gain, is the intensity value of the color image;

[0025] The bilinear interpolation algorithm is used to downsample each color channel in the gained image to obtain a pseudo RAW format image.

[0026] In combination with the first aspect, optionally, the pre-trained instance segmentation model is obtained by training a pre-built instance segmentation model using a pseudo RAW format image;

[0027] The pre-built instance segmentation model is improved by taking YOLOv8 as the benchmark model. The constructed instance segmentation model includes a feature extraction network for extracting features from an input image, a neck network for transmitting and enhancing features, and a lightweight string, parallel structure segmentation and detection head for detecting and segmenting different feature information.

[0028] In combination with the first aspect, optionally, the feature extraction network includes a first stage, a second stage, a third stage and a fourth stage;

[0029] The first stage includes two layers of space-to-depth convolution modules and one layer of star-structured feature fusion module in sequence;

[0030] The second stage includes one layer of convolution module and one layer of star-structured feature fusion module in sequence;

[0031] The third stage includes one layer of convolution module and one layer of star-structured feature fusion module in sequence;

[0032] The fourth stage includes, in sequence, a layer of convolution module, a layer of star-structured feature fusion module, and a layer of spatial pyramid fast pooling module.

[0033] In combination with the first aspect, optionally, the space-to-depth convolution module includes a space-to-depth layer and a non-strided convolution layer,

[0034] The image X of size H×W×C is input into the depth layer and downsampled by 2 times to obtain four images of size × ×C sub-feature maps; four sub-feature maps of the same size are concatenated in the channel dimension, and the output size is × ×4C intermediate feature map X';

[0035] The intermediate feature map X' is input into the non-strided convolutional layer to obtain a size of × ×C’s feature map X″;

[0036] The non-strided convolution layer contains C filters, the convolution kernel is 3, and the stride is 1.

[0037] In combination with the first aspect, optionally, the star-shaped feature fusion module includes 2 depth-separable convolutional layers, 2 1x1 convolutional layers, 2 parallel 1x1 convolutional layers and an SE attention module,

[0038] The input feature map X1 is processed by a depth-separable convolution layer to perform a single convolution operation on the channel dimension and two parallel 1x1 convolution layers to perform linear transformation of the features, and then two linearly transformed feature maps X2 are output;

[0039] The two linearly transformed feature maps X2 are element-wise multiplied to obtain feature map X3;

[0040] After feature map X3 is processed by one 1x1 convolution layer and one depth-separable convolution layer, feature map X4 is obtained;

[0041] The input feature map X1 is processed by a 1x1 convolution layer and the SE attention module to obtain the feature map X5;

[0042] The feature map X4 and the feature map X5 are fused.

[0043] In combination with the first aspect, optionally, the lightweight serial and parallel structure segmentation and detection head includes a detection head and a segmentation head;

[0044] The detection head includes a serial part of the detection head and a parallel part of the detection head, wherein the serial part of the detection head includes a serial multi-scale context aggregation attention module and three 3×3 grouped convolutional layers, and the parallel part of the detection head includes two parallel 1×1 convolutional layers;

[0045] The detection head performs the following actions:

[0046] The feature map obtained by the neck network processing is input into the multi-scale context aggregation attention module to extract the global context information of the image, integrate the multi-scale features, and obtain the multi-scale feature map, which is expressed by the following formula:

[0047] ,

[0048] in, The feature map input to the multi-scale contextual aggregation attention module, is a 1×1 convolution operation, is the i-th complementary banded convolution branch, is a depth-wise separable convolution, is the input feature of the module, The multi-scale feature map output by the multi-scale context aggregation attention module;

[0049] The multi-scale feature map passes through three 3×3 grouped convolutional layers, the number of channels of the multi-scale feature map is evenly divided into g groups, and the convolution operation is performed independently on each group to obtain the processed feature map;

[0050] The processed feature map is input into the parallel part of the detection head, and a 1×1 convolutional layer is used to process the category classification task to obtain the category information of the detection target; a 1×1 convolutional layer is used to process the detection frame position regression task to obtain the position of the detection frame and the position information of the target; wherein the position information of the target is used to obtain the bounding box of the target, and the category information of the detection target is used to obtain the semantic label corresponding to the target category;

[0051] The segmentation head includes two 3×3 convolutional layers and one 1×1 convolutional layer in series;

[0052] The segmentation head processes the mask prediction task to obtain mask information of the segmentation target; the mask information of the segmentation target is used to convert the processed feature map into a segmentation mask;

[0053] Based on the category information of the detected target, the location information of the target and the mask information of the segmented target, the image of the leakage disease area of ​​the shield tunnel is obtained.

[0054] In combination with the first aspect, optionally, the 3×3 grouped convolution layer includes a 3×3 convolution layer, a batch normalization layer, and a Gaussian error linear unit activation function,

[0055] The feature map of size H×W×C is input into a 3×3 convolution, and the number of channels is evenly divided into g groups to obtain g groups of size H×W× Sub-feature graph of ;

[0056] G dimensions H×W× After the sub-feature map is normalized by the batch normalization layer, the Gaussian error linear unit activation function is input for nonlinear feature capture, and the output feature map is of size H×W×C.

[0057] In a second aspect, the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the shield tunnel water leakage image segmentation method suitable for low-light environments described in the first aspect are implemented.

[0058] In a third aspect, the present invention provides a computer device, comprising:

[0059] Memory, for storing computer programs / instructions;

[0060] A processor is used to execute the computer program / instructions to implement the steps of the shield tunnel water leakage image segmentation method suitable for low-light environment described in the first aspect.

[0061] Compared with the prior art, the shield tunnel water leakage image segmentation method, storage medium and device provided by the embodiment of the present invention for low-light environment have the following beneficial effects:

[0062] The present invention obtains a real-time sRGB image of a shield tunnel in a low-light environment; performs an inverse operation on the obtained real-time sRGB image to obtain a pseudo RAW format image; the present invention uses the inverse operation to process the real-time sRGB image to obtain a pseudo RAW format image with more detailed features;

[0063] The present invention can input the obtained pseudo RAW format image into a pre-trained instance segmentation model to segment and obtain the image of the water leakage disease area of ​​the shield tunnel; the instance segmentation model provided by the present invention can quickly segment the image of the water leakage disease area of ​​the shield tunnel in an extremely low light environment to achieve water leakage disease detection;

[0064] The instance segmentation model of the present invention includes a feature extraction network for extracting features from an input image, and the first stage of the feature extraction network includes two layers of space-to-depth convolution modules and one layer of feature fusion module with a star structure in sequence; the present invention sets up a space-to-depth convolution module, which can reduce the detail loss in the feature extraction process, reduce the information loss problem caused by the instance segmentation model being affected by the low-light environment, optimize the instance segmentation model's processing ability for the subtle details of the leaking area image, improve the model recall rate, and avoid missed detection; the present invention sets up a feature fusion module with a star structure, which can enhance the feature fusion ability of the image in a dark environment without significantly increasing the computational burden, and at the same time increase the working efficiency of the instance segmentation model; the present invention sets up an SE attention module, which can enhance the feature expression ability of the neural network, enable the convolution layer to focus on the extraction and enhancement of local features, realize multi-scale processing and effective fusion of features, and enhance the instance segmentation model's ability to capture and understand complex background information;

[0065] The instance segmentation model of the present invention includes a lightweight string, parallel structure segmentation and detection head for detecting and segmenting different feature information; the present invention designs a lightweight string, parallel structure segmentation and detection head, which greatly reduces the number of parameters of the model and maintains a high level of segmentation accuracy, so that the instance segmentation model has a faster computing speed and meets the actual engineering rapid detection needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a flow chart of a shield tunnel water leakage image segmentation method suitable for low-light environment in Example 1 of the present invention;

[0067] Figure 2 It is a flowchart of the inverse operation in a shield tunnel water leakage image segmentation method suitable for low-light environment in Embodiment 1 of the present invention;

[0068] Figure 3It is a structural schematic diagram of a feature extraction network in a shield tunnel water leakage image segmentation method suitable for low-light environment in Example 1 of the present invention;

[0069] Figure 4 It is a schematic diagram of the structure of a space-to-depth convolution module in a shield tunnel water leakage image segmentation method suitable for a low-light environment in Example 1 of the present invention;

[0070] Figure 5 It is a structural schematic diagram of a feature fusion module of a star structure in a shield tunnel water leakage image segmentation method suitable for a low-light environment in Example 1 of the present invention;

[0071] Figure 6 It is a structural schematic diagram of a lightweight serial and parallel structure segmentation and detection head in a shield tunnel water leakage image segmentation method suitable for low-light environment in Example 1 of the present invention. DETAILED DESCRIPTION

[0072] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0073] Embodiment 1:

[0074] like Figure 1 As shown, this embodiment provides a shield tunnel water leakage image segmentation method suitable for low light environment, including:

[0075] Get real-time sRGB images of shield tunnels in low-light environments;

[0076] Perform an inverse operation on the acquired real-time sRGB image to obtain a pseudo RAW format image;

[0077] The obtained pseudo RAW format image is input into the pre-trained instance segmentation model to segment the leakage disease area image of the shield tunnel.

[0078] The specific implementation steps are as follows.

[0079] Step 1: Acquire real-time sRGB images of the shield tunnel in low-light environment.

[0080] Step 2: If Figure 2 As shown, the acquired real-time sRGB image is subjected to an inverse operation to obtain a pseudo RAW format image.

[0081] Step 2.1: Use an inverse change curve of a preset tone mapping curve to perform contrast adjustment on the acquired real-time sRGB image to obtain a first adjusted image.

[0082] This step defines a specific tone mapping curve to achieve nonlinear transformation of the image.

[0083] The mathematical expression of the tone mapping curve is expressed as follows:

[0084] ,

[0085] in, is the contrast of the image before adjustment;

[0086] The inverse change curve of the preset tone mapping curve is expressed by the following formula:

[0087] ,

[0088] in, The contrast of the adjusted image.

[0089] Step 2.2: Use the inverse transformation of gamma correction to adjust the light intensity of the first adjusted image to obtain a second adjusted image.

[0090] This step maps the linear light intensity to a new value through a nonlinear transformation to better reflect the visual characteristics of the human eye.

[0091] The gamma correction transformation is expressed as follows:

[0092]

[0093] Then the inverse transform of gamma correction is expressed as follows:

[0094] ,

[0095] in, is the light intensity value of the corrected image, γ is the gamma value, is the light intensity value of the image before correction, The value is The constant.

[0096] Specifically, the gamma value is used to define the degree of nonlinear transformation, and the selection of the gamma value involves adapting to different lighting conditions and image content characteristics.

[0097] Step 2.3: Use a preset color correction matrix to perform color correction on the second adjusted image to obtain an image in a standard color space.

[0098] Corrects the color balance according to the light source in the shooting environment to make the image colors more natural.

[0099] In this embodiment, the color correction matrix is ​​a CCM matrix.

[0100] Step 2.4: Perform white balance adjustment on the image in the standard color space according to the light source of the shooting environment to obtain a third adjusted image.

[0101] In this embodiment, white balance adjustment is performed by using a lookup table.

[0102] Step 2.5: Perform digital gain on the third adjusted image to obtain a gained image.

[0103] This step involves adjusting the intensity values ​​of the images, where the gain value for each image is automatically determined by the camera's built-in auto-exposure algorithm.

[0104] The intensity values ​​of the third adjusted image and the image after gain follow the exponential distribution of the following formula:

[0105] ,

[0106] Where p is the probability distribution of image intensity values, is the intensity value of the color image after gain, is the intensity value of the color image.

[0107] This step can match a common exponential distribution for the sample means of the two groups of images, thereby achieving uniform adjustment of image intensity.

[0108] Step 2.6: Use a bilinear interpolation algorithm to downsample each color channel in the gained image to obtain a pseudo RAW format image.

[0109] In this embodiment, the gained image is demosaiced.

[0110] In each 2x2 Bayer array, the green channel data is obtained by averaging two green pixels and converted into a three-channel pseudo RAW image. Although the images are three-channel, they retain the high bit depth and rich scene information of RAW images to a certain extent, which is beneficial for instance segmentation tasks in low-light environments.

[0111] This step can convert monochrome pixels to full-color pixels to obtain high-quality color images.

[0112] like Figure 2 As shown, this embodiment not only obtains the real-time sRGB image of the shield tunnel in a low-light environment, but also obtains the real-time sRGB image of the position in a normal light environment. Taking the real-time sRGB image of the position in a normal light environment as a reference, it is verified that this embodiment uses the inverse operation to process the real-time sRGB image, which can effectively enhance the image quality under low-light conditions, restore the original detail features of the image, and obtain a pseudo RAW format image with more detail features.

[0113] This embodiment also avoids the model calculation burden and a large amount of algorithm training time caused by introducing pre-processing steps such as traditional image enhancement algorithms and denoising algorithms.

[0114] Step 3: Input the obtained pseudo RAW format image into the pre-trained instance segmentation model to segment the shield tunnel leakage area image

[0115] The pre-trained instance segmentation model is obtained by training a pre-built instance segmentation model using pseudo RAW format images.

[0116] The pre-built instance segmentation model is improved based on YOLOv8 as the baseline model.

[0117] The pre-built instance segmentation model includes a feature extraction network for extracting features from the input image, a neck network for transferring and enhancing features, and a lightweight string, parallel structure segmentation and detection head for different feature information for detection and segmentation.

[0118] Specifically, Figure 3 As shown, the feature extraction network includes the first stage, the second stage, the third stage and the fourth stage;

[0119] The first stage includes two layers of spatial to deep convolution modules and one layer of star-structured feature fusion module;

[0120] The second stage includes one layer of convolution module and one layer of star-structured feature fusion module in sequence;

[0121] The third stage includes one layer of convolution module and one layer of star-structured feature fusion module in sequence;

[0122] The fourth stage includes 1 layer of convolution module, 1 layer of star-structured feature fusion module and 1 layer of spatial pyramid fast pooling module.

[0123] like Figure 3 As shown, the feature dimension extracted by the space-to-depth convolution module in the first stage is a one-dimensional feature dimension.

[0124] like Figure 4 As shown, the spatial-to-depth convolution module in the first stage includes a spatial-to-depth layer and a non-strided convolution layer.

[0125] In this embodiment, the non-strided convolution layer contains C filters, the convolution kernel is 3, and the stride is 1.

[0126] The image X of size H×W×C is input into the depth layer and downsampled by 2 times to obtain four sub-feature maps of size H / 2×W / 2×C. The four sub-feature maps of the same size are concatenated in the channel dimension to output an intermediate feature map X' of size H / 2×W / 2×4C.

[0127] The intermediate feature map X’ is input into the non-strided convolutional layer to obtain a feature map X″ of size H / 2×W / 2×C.

[0128] Compared with traditional convolution operations, the spatial-to-depth convolution module retains a higher degree of information within the channel, and therefore can effectively improve the ability to extract small targets and low-resolution features.

[0129] In this embodiment, the first stage of the feature extraction network uses a 2-layer space-to-depth convolution module to replace the strided convolution and pooling in the YOLOv8 benchmark model, which can improve the model's segmentation effect on low-resolution water leakage disease images and small targets in low-light environments, and reduce the loss of details in the feature extraction process; reduce the information loss problem of the instance segmentation model due to the influence of low-light environments, optimize the instance segmentation model's ability to process subtle details of images in leaking areas, improve the model recall rate, and avoid missed detections.

[0130] like Figure 5 As shown in Figure 1, the star-structured feature fusion module includes 2 depth-wise separable convolutional layers, 2 1x1 convolutional layers, 2 parallel 1x1 convolutional layers, and a SE attention module.

[0131] The input feature map X1 is processed by a depth-separable convolution layer to perform a single convolution operation on the channel dimension and two parallel 1x1 convolution layers to perform linear transformation of the features, and then two linearly transformed feature maps X2 are output.

[0132] The two linearly transformed feature maps X2 are element-wise multiplied to obtain feature map X3 (this step is the core of the star structure). The multiplication operation nonlinearly fuses the features of different subspaces, enhancing the representation ability of the features and maintaining the efficiency of the calculation.

[0133] After feature map X3 is processed by one 1x1 convolution layer and one depth-wise separable convolution layer, feature map X4 is obtained.

[0134] The input feature map X1 is processed by a 1x1 convolution layer and the SE attention module to obtain the feature map X5.

[0135] The feature map X4 and the feature map X5 are fused.

[0136] This embodiment sets up a feature fusion module with a star structure, which can enhance the feature fusion capability of images in dark environments without significantly increasing the computational burden, while increasing the working efficiency of the instance segmentation model.

[0137] like Figure 5 As shown in the figure, feature map X1 is processed by a 1x1 convolutional layer to obtain feature map X1' with a size of H×W×C. Feature map X1' is input into the SE attention module and processed in two ways. The first process is processed in turn by and After that, the first result is obtained, and the second result is processed by After that, the second result is obtained. The first result is applied to the second result, and the feature map X5 with a size of H×W×C is output.

[0138] The principle of the SE attention module is to compress the spatial dimension of the feature map through global average pooling, generate channel descriptors, learn the importance weights of each channel through the fully connected layer, and finally apply these weights to the original feature map to achieve feature recalibration.

[0139] This embodiment sets up an SE attention module, which can enhance the feature expression ability of the neural network, enable the convolution layer to focus on the extraction and enhancement of local features, realize multi-scale processing and effective fusion of features, and improve the instance segmentation model's ability to capture and understand complex background information.

[0140] Feature map X4 and feature map X5 are fused to obtain feature map X6, and X6 undergoes a series of operations through the neck network to obtain feature maps X7, X8 and X9.

[0141] Reducing the number of model parameters to achieve fast segmentation of leaking images and maintaining a certain level of segmentation accuracy is the key to practical engineering detection applications. The segmentation detection head of YOLOv8 adopts a decoupled head design without anchor frames, using independent branches to focus on different feature information of classification, detection, and segmentation. The existing detection head uses a traditional parallel structure, and the classification regression and detection regression tasks use two 3×3 convolutions and one 1×1 convolution to extract corresponding features. It is not difficult to see that the traditional large-kernel convolution operation will lead to an exponential increase in the number of model parameters, which will inevitably affect the speed of fast segmentation and detection tasks.

[0142] like Figure 6 As shown, this embodiment provides a lightweight serial and parallel structure segmentation and detection head, including a detection head and a segmentation head.

[0143] In this embodiment, there are three lightweight serial and parallel structure segmentation and detection heads, which process feature map X7, feature map X8 and feature map X9 respectively.

[0144] The detection head includes a serial part of the detection head and a parallel part of the detection head. The serial part of the detection head includes a serial multi-scale context aggregation attention module and three 3×3 grouped convolutional layers. The parallel part of the detection head includes two parallel 1×1 convolutional layers.

[0145] For any detection head in the lightweight serial, parallel structure segmentation and detection head, perform the following actions:

[0146] Step a: Input the feature map obtained by the neck network processing into the multi-scale context aggregation attention module to extract the global context information of the image, integrate the multi-scale features, and obtain the multi-scale feature map, which is expressed by the following formula:

[0147] ,

[0148] in, The feature map input to the multi-scale contextual aggregation attention module, is a 1×1 convolution operation, is the i-th complementary banded convolution branch, is a depth-wise separable convolution, is the input feature of the module, Multi-scale feature maps output by the attention module for multi-scale context aggregation.

[0149] The multi-scale context aggregation attention module can extract the global context information of the image and integrate multi-scale features. It is worth noting that the multi-scale context aggregation attention module is also a lightweight attention mechanism.

[0150] Specifically, Figure 6 As shown in the figure, the multi-scale contextual aggregation attention module includes three components, namely, deep convolution, multi-branch deep banded convolution and a 1×1 ordinary convolution. First, the deep separable convolution is responsible for aggregating the underlying features in the local area, and capturing the local detail information of the low-light image through the deep network structure. Then, the multi-branch deep banded convolution is used to simulate the large-core deep convolution and complete the extraction and fusion of multi-scale information. Each branch has a convolution kernel of different sizes. In this embodiment, the kernel sizes are 7, 11 and 21 respectively. A pair of n×1 and 1×n convolutions are used to simulate the n×n two-dimensional large-core convolution to capture the different scale features of the leaky image area, cleverly solve the computational load problem brought by the large-core convolution, and meet the lightweight design requirements of the model. Finally, a 1×1 ordinary convolution is used to simulate the relationship between different branches, and its output is used as the attention weight to enhance the expressiveness of the feature map.

[0151] Step b: The multi-scale feature map passes through three 3×3 grouped convolutional layers to evenly divide the number of channels of the multi-scale feature map into g groups, and perform convolution operations on each group independently to obtain the processed feature map.

[0152] Specifically, Figure 6 As shown, the 3×3 grouped convolution layer includes a 3×3 convolution layer, a batch normalization layer, and a Gaussian error linear unit activation function.

[0153] The feature map of size H×W×C is input into a 3×3 convolution, and the number of channels is evenly divided into g groups to obtain g sub-feature maps of size H×W×C / g.

[0154] After g sub-feature maps of size H×W×C / g are normalized by the batch normalization layer, the Gaussian error linear unit activation function is input for nonlinear feature capture, and a feature map of size H×W×C is output.

[0155] In this embodiment, the calculation amount formula of the 3×3 grouped convolution layer is provided as follows:

[0156] ,

[0157] in, is the number of input feature map channels, is the number of output feature map channels. is the input feature map size. is the convolution kernel size, which is 3 in this embodiment. is the number of groups, which is 16 in this embodiment.

[0158] The calculation formula of the traditional convolutional layer is as follows:

[0159] .

[0160] From the above two formulas, it can be seen that the computational amount of the 3×3 grouped convolution layer accounts for 1 / g of the computational amount of the traditional convolution. By comparative calculation, using the 3×3 grouped convolution layer operation provided by this embodiment, the computational amount of the final single serial and parallel detection head is 1 / 8 of the original parallel detection head, indicating that the 3×3 grouped convolution layer provided by this embodiment significantly improves the computational efficiency of the model.

[0161] Step c: The processed feature map is input into the parallel part of the detection head, and a 1×1 convolutional layer is used to process the category classification task to obtain the category information of the detected target; a 1×1 convolutional layer is used to process the detection box position regression task to obtain the position of the detection box and the position information of the target.

[0162] Specifically, the location information of the target is used to obtain a bounding box of the target, and the category information of the detected target is used to obtain a semantic label corresponding to the target category.

[0163] For any segmentation head in a lightweight serial and parallel structure segmentation and detection head, it includes two serial 3×3 convolutional layers and one 1×1 convolutional layer.

[0164] The segmentation head handles the mask prediction task and obtains the mask information of the segmented target.

[0165] Specifically, the mask information of the segmentation target is used to convert the processed feature map into a segmentation mask.

[0166] Therefore, based on the category information of the detected target, the location information of the target and the mask information of the segmented target, the image of the leakage disease area of ​​the shield tunnel is obtained.

[0167] This embodiment designs a lightweight serial and parallel structure segmentation and detection head, which greatly reduces the number of model parameters and maintains a high level of segmentation accuracy, so that the instance segmentation model has a faster computing speed and meets the actual engineering rapid detection needs.

[0168] Embodiment 2:

[0169] This embodiment compares the segmentation performance of a shield tunnel water leakage image segmentation method suitable for a low-light environment provided in Embodiment 1 with the existing segmentation method.

[0170] The segmentation performance comparison was completed under the same hardware and software configuration, and the hyperparameters were strictly kept consistent. The hardware and software configurations are shown in Table 1. The operating system is Windows 10, the CPU is Intel(R) Core(TM) i7-12700K, and the GPU model is Nvidia GeForce RTX3070Ti, using CUDA11.8 for acceleration.

[0171] Table 1 Hardware and software configuration

[0172]

[0173] The segmentation performance is evaluated using precision, recall, and average precision, and the parameter count and computational effort FLOPs (G) are used for lightweight evaluation.

[0174] Accuracy , expressed by the following formula:

[0175] ,

[0176] in, Indicates the accuracy, represents true positive samples (leakage), Indicates a false positive sample (false detection).

[0177] Recall , expressed by the following formula:

[0178] ,

[0179] in, represents the recall rate, represents true positive samples (leakage), Indicates false negatives (real leak areas that are not detected or mistakenly detected as negative).

[0180] Average Precision , expressed by the following formula:

[0181] ,

[0182] in, represents the average precision, Represents the total number of categories for the segmentation task, It is expressed as the average precision (AP) of a segmentation category. In this embodiment, there is only one category, water leakage.

[0183] In the following It is expressed as the AP value when the IoU threshold reaches 50%. It represents the average value of 10 APs when the IoU threshold ranges from 50% to 95% with a step size of 0.05.

[0184] Under the same test set, a shield tunnel water leakage image segmentation method suitable for low-light environment provided in Example 1 is compared with the existing segmentation method.

[0185] Existing segmentation methods use mainstream models in the segmentation field, including Mask R-CNN model, YOLACT model, SOLO series models, YOLO series models or Hybrid Task Cascade model.

[0186] YOLACT, SOLO series, and YOLO series are single-stage instance segmentation models.

[0187] In this embodiment, the YOLO series selects the YOLOv5n model, the YOLOv6 model, the YOLOv8n model, and the YOLOv9c model.

[0188] In this embodiment, the SOLO series selects the SOLOv1 model and the SOLOv2 model.

[0189] Mask R-CNN is a two-stage instance segmentation model.

[0190] Hybrid Task Cascade is an instance segmentation model with a cascade structure.

[0191] In addition, this embodiment also selects an instance segmentation model (recorded as a composite model in Table 5) obtained by combining the Retinexformer low-light enhancement algorithm with the YOLOv8 instance segmentation model. The instance segmentation model uses traditional sRGB images after low-light enhancement for model training.

[0192] The performance comparison of different types of instance segmentation models is shown in Table 5.

[0193]

[0194] It can be seen from Table 5 that the shield tunnel water leakage image segmentation method suitable for low-light environment provided in Example 1 surpasses the comprehensive performance of the single-stage instance segmentation model YOLACT model, SOLO series model and YOLO series model as well as previous models.

[0195] Considering that the YOLO series updates and iterations have enhanced the performance of historical versions, this embodiment compares Embodiment 1 with a higher version of YOLOv9c. Experiments show that YOLOv9c performs well in indicators such as mAP50 and mAP50-95, but the model parameters and calculation amount are large, and the accuracy is improved at the expense of calculation speed.

[0196] Compared with the traditional two-stage instance segmentation network, the shield tunnel water leakage image segmentation method suitable for low-light environment provided in Example 1 has obvious advantages in terms of accuracy and computational complexity.

[0197] The instance segmentation model (recorded as a composite model in Table 5) obtained by combining the Retinexformer low-light enhancement algorithm with the YOLOv8 instance segmentation model has a low-light detection channel that surpasses the model proposed in this paper in some performance indicators, but it is not advantageous in terms of the number of model parameters and the amount of computation, and the training speed is slow, which is not friendly to resource-constrained hardware devices.

[0198] In summary, the shield tunnel water leakage image segmentation method suitable for low-light environment provided in this embodiment 1 can ensure lightweight while quickly segmenting the image of the water leakage disease area of ​​the shield tunnel in an extremely low-light environment to achieve water leakage disease detection.

[0199] Embodiment 3:

[0200] This embodiment provides a computer-readable storage medium having a computer program / instruction stored thereon, characterized in that when the computer program / instruction is executed by a processor, the steps of the shield tunnel water leakage image segmentation method suitable for low-light environments described in Example 1 are implemented.

[0201] Embodiment 4:

[0202] This embodiment provides a computer device, characterized by comprising:

[0203] Memory, for storing computer programs / instructions;

[0204] A processor is used to execute the computer program / instructions to implement the steps of the shield tunnel water leakage image segmentation method suitable for low-light environment described in Example 1.

[0205] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0206] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0207] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0208] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0209] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A shield tunnel water leakage image segmentation method suitable for low light environment, characterized in that: include: Get real-time sRGB images of shield tunnels in low-light environments; Perform an inverse operation on the acquired real-time sRGB image to obtain a pseudo RAW format image; The obtained pseudo RAW format image is input into the pre-trained instance segmentation model to segment the leakage disease area image of the shield tunnel.

2. The shield tunnel water leakage image segmentation method suitable for low light environment according to claim 1 is characterized in that: The performing an inverse operation on the acquired real-time sRGB image to obtain a pseudo RAW format image includes: The acquired real-time sRGB image is contrast adjusted by using an inverse change curve of a preset tone mapping curve to obtain a first adjusted image, wherein the inverse change curve of the preset tone mapping curve is expressed by the following formula: ， in, The contrast of the adjusted image; The light intensity of the first adjusted image is adjusted by using an inverse transformation of gamma correction to obtain a second adjusted image, wherein the inverse transformation of gamma correction is expressed by the following formula: ， in, is the light intensity value of the corrected image, γ is the gamma value, is the light intensity value of the image before correction, The value is The constant of Performing color correction on the second adjusted image using a preset color correction matrix to obtain an image in a standard color space; Performing white balance adjustment on the image in the standard color space according to the light source of the shooting environment to obtain a third adjusted image; The third adjusted image is digitally gained to obtain a gained image; the intensity values ​​of the third adjusted image and the gained image follow the exponential distribution of the following formula: ， Where p is the probability distribution of image intensity values, is the intensity value of the color image after gain, is the intensity value of the color image; The bilinear interpolation algorithm is used to downsample each color channel in the gained image to obtain a pseudo RAW format image.

3. The shield tunnel water leakage image segmentation method suitable for low light environment according to claim 1 is characterized in that: The pre-trained instance segmentation model is obtained by training a pre-built instance segmentation model using a pseudo RAW format image; The pre-built instance segmentation model is improved by taking YOLOv8 as the benchmark model. The constructed instance segmentation model includes a feature extraction network for extracting features from an input image, a neck network for transmitting and enhancing features, and a lightweight string, parallel structure segmentation and detection head for detecting and segmenting different feature information.

4. The shield tunnel water leakage image segmentation method according to claim 3 is suitable for low light environment, characterized in that: The feature extraction network includes a first stage, a second stage, a third stage and a fourth stage; The first stage includes two layers of space-to-depth convolution modules and one layer of star-structured feature fusion module in sequence; The second stage includes one layer of convolution module and one layer of star-structured feature fusion module in sequence; The third stage includes one layer of convolution module and one layer of star-structured feature fusion module in sequence; The fourth stage includes, in sequence, a layer of convolution module, a layer of star-structured feature fusion module, and a layer of spatial pyramid fast pooling module.

5. The shield tunnel water leakage image segmentation method suitable for low light environment according to claim 4 is characterized in that: The space-to-depth convolution module includes a space-to-depth layer and a non-strided convolution layer. The image X of size H×W×C is input into the depth layer and downsampled by 2 times to obtain four images of size × ×C sub-feature maps; four sub-feature maps of the same size are concatenated in the channel dimension, and the output size is × ×4C intermediate feature map X'; The intermediate feature map X' is input into the non-strided convolutional layer to obtain a size of × ×C’s feature map X″; The non-strided convolution layer contains C filters, the convolution kernel is 3, and the stride is 1.

6. The shield tunnel water leakage image segmentation method suitable for low light environment according to claim 4, characterized in that: The star-shaped feature fusion module includes 2 depth-separable convolutional layers, 2 1x1 convolutional layers, 2 parallel 1x1 convolutional layers and an SE attention module. The input feature map X1 is processed by a depth-separable convolution layer to perform a single convolution operation on the channel dimension and two parallel 1x1 convolution layers to perform linear transformation of the features, and then two linearly transformed feature maps X2 are output; The two linearly transformed feature maps X2 are element-wise multiplied to obtain feature map X3; After feature map X3 is processed by one 1x1 convolution layer and one depth-separable convolution layer, feature map X4 is obtained; The input feature map X1 is processed by a 1x1 convolution layer and the SE attention module to obtain the feature map X5; The feature map X4 and the feature map X5 are fused.

7. The shield tunnel water leakage image segmentation method suitable for low light environment according to claim 4 is characterized in that: The lightweight serial and parallel structure segmentation and detection head includes a detection head and a segmentation head; The detection head includes a serial part of the detection head and a parallel part of the detection head, wherein the serial part of the detection head includes a serial multi-scale context aggregation attention module and three 3×3 grouped convolutional layers, and the parallel part of the detection head includes two parallel 1×1 convolutional layers; The detection head performs the following actions: The feature map obtained by the neck network processing is input into the multi-scale context aggregation attention module to extract the global context information of the image, integrate the multi-scale features, and obtain the multi-scale feature map, which is expressed by the following formula: ， in, The feature map input to the multi-scale contextual aggregation attention module, is a 1×1 convolution operation, is the i-th complementary banded convolution branch, is a depth-wise separable convolution, is the input feature of the module, The multi-scale feature map output by the multi-scale context aggregation attention module; The multi-scale feature map passes through three 3×3 grouped convolutional layers, the number of channels of the multi-scale feature map is evenly divided into g groups, and the convolution operation is performed independently on each group to obtain the processed feature map; The processed feature map is input into the parallel part of the detection head, and a 1×1 convolutional layer is used to process the category classification task to obtain the category information of the detection target; a 1×1 convolutional layer is used to process the detection frame position regression task to obtain the position of the detection frame and the position information of the target; wherein the position information of the target is used to obtain the bounding box of the target, and the category information of the detection target is used to obtain the semantic label corresponding to the target category; The segmentation head includes two 3×3 convolutional layers and one 1×1 convolutional layer in series; The segmentation head processes the mask prediction task to obtain mask information of the segmentation target; the mask information of the segmentation target is used to convert the processed feature map into a segmentation mask; Based on the category information of the detected target, the location information of the target and the mask information of the segmented target, the image of the leakage disease area of ​​the shield tunnel is obtained.

8. The shield tunnel water leakage image segmentation method suitable for low light environment according to claim 7, characterized in that: The 3×3 grouped convolutional layer includes a 3×3 convolutional layer, a batch normalization layer, and a Gaussian error linear unit activation function. The feature map of size H×W×C is input into a 3×3 convolution, and the number of channels is evenly divided into g groups to obtain g groups of size H×W× Sub-feature graph of ; G dimensions H×W× After the sub-feature map is normalized by the batch normalization layer, the Gaussian error linear unit activation function is input for nonlinear feature capture, and the output feature map is of size H×W×C.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps of the shield tunnel water leakage image segmentation method suitable for low-light environment described in any one of claims 1 to 7 are implemented.

10. A computer device, characterized in that: include: Memory, for storing computer programs / instructions; A processor is used to execute the computer program / instructions to implement the steps of the shield tunnel water leakage image segmentation method suitable for low-light environment described in any one of claims 1-7.

Images