An underwater pipeline defect detection method based on an improved UNet model

Abstract

The application relates to the technical field of deep learning, and provides an underwater pipeline defect detection method based on an improved UNet model, which comprises the following steps: in a first branch, a prior distribution network for feature extraction is constructed based on an improved UNet network framework, original underwater pipeline images are subjected to feature extraction and reconstruction, and pipeline defect feature extraction images are generated; in a second branch, a posterior distribution network for underwater image enhancement is constructed based on the improved UNet network framework, the original underwater pipeline images and reference images are subjected to feature enhancement to generate pipeline defect feature enhancement images; and the pipeline defect feature extraction images and the pipeline defect feature enhancement images are subjected to enhancement statistical transformation through Pr modules, Po modules and AdaIN modules to generate pipeline defect recognition images. The application solves different problems existing in the enhancement and defect detection of underwater pipeline images, improves the feature expression capability of the network model, and finally has the advantages of good generalization capability, good robustness, high enhancement and defect detection efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, in particular to an underwater pipeline defect detection method based on an improved UNet model. BACKGROUND

[0002] Underwater pipelines are an important part of ocean oil and gas exploration and marine engineering, and they bear the function of energy transportation. With the continuous development of ocean exploration, the scale of submarine pipeline systems is gradually expanding. These pipelines face various complex environmental challenges when they are operated for a long time under the sea. Due to long-term exposure to harsh marine environments, underwater pipelines are affected by a series of factors such as corrosion, wear, mechanical damage, and marine organism attachment. The cumulative effect of these factors can cause the pipeline to develop cracks, dents, perforations, and other defects, which may in turn lead to serious safety problems such as energy leakage or pipeline rupture. The above problems not only cause economic losses, but also can cause serious pollution to the marine ecological environment. Therefore, how to effectively detect, maintain, and manage submarine pipelines has become a research topic of great concern in the field of modern marine engineering and energy transportation.

[0003] Traditional submarine pipeline detection methods, such as sonar and laser scanning, can identify pipeline defects to some extent, but these methods have relatively low resolution and are susceptible to noise interference. At the same time, these methods usually require complex post-processing and manual intervention, which is inefficient and difficult to meet the real-time detection requirements of modern marine engineering. In order to ensure the normal operation of the pipeline, innovative technical upgrading detection means are needed to ensure the long-term safe operation of the pipeline.

[0004] Therefore, research on deep learning-based methods has received widespread attention. For example, neural networks can quantitatively identify submarine pipeline defects, and although these methods have high accuracy, their performance may vary under different environmental and conditions. The environment of submarine pipelines is complex and variable, and factors such as lighting conditions, sediment deposition, and marine organism attachment can interfere with the detection of the pipeline surface. Therefore, the robustness of deep learning models in these complex environments has not been fully verified.

[0005] Currently, the You Only Look Once (YOLO) model using a convolutional neural network is used for pipeline image detection to comprehensively evaluate pipeline abnormalities. Although this system has shown high recognition accuracy in testing, it has not been studied in deep-sea environments. In deep-sea environments, light is extremely weak, and scattering and attenuation are severe, so conventional optical vision technology may not be able to function in such environments.

[0006] The existing submarine pipeline defect detection method has made certain progress in accuracy and real-time performance, but still has many limitations. According to the principle of deep learning model, the detection method based on deep learning is easy to be affected by the amount and distribution of data, especially when facing complex and variable submarine environment, its generalization ability is still insufficient. In addition, the existing method mostly depends on specific sensor data, lacks the ability of multi-source data fusion, and is difficult to cope with sudden environmental changes and noise interference.

[0007] Therefore, the present application is proposed. SUMMARY

[0008] To this end, the present application provides an underwater pipeline defect detection method based on an improved UNet model to solve different problems existing in underwater pipeline image enhancement and defect detection, improve the feature expression ability of the network model, and finally achieve the advantages of good generalization ability, good robustness, high enhancement and defect detection efficiency.

[0009] To achieve the above-mentioned purpose, the present application provides an underwater pipeline defect detection method based on an improved UNet model, comprising:

[0010] In the first branch, a prior distribution network for feature extraction is constructed based on the improved UNet network framework, the original underwater pipeline image is feature extracted and reconstructed through the prior distribution network, and a pipeline defect feature extraction image is generated;

[0011] In the second branch, a posterior distribution network for underwater image enhancement is constructed based on the improved UNet network framework, the original underwater pipeline image and the reference image are feature enhanced through the posterior distribution network, and a pipeline defect feature enhancement image is generated;

[0012] Through the Pr module, the Po module and the AdaIN module, the pipeline defect feature extraction image and the pipeline defect feature enhancement image are enhanced statistical transformation, and a pipeline defect recognition image is generated.

[0013] Further, the following steps are included in the prior distribution network / posterior distribution network:

[0014] Step S1, the original underwater pipeline image, or the original underwater pipeline image and the reference image, is converted into a first feature map through a series of convolution blocks and LeakyReLU activation functions;

[0015] Step S2, the first feature map is feature extracted through a dynamic convolution module to generate a second feature map;

[0016] Step S3, consolidating features of the second feature map through a max pooling layer to generate a third feature map;

[0017] Step S4, repeating the steps S1 to S3 at least twice to generate a fourth feature map, wherein the number of convolution kernels of the dynamic convolution module is increased successively;

[0018] Step S5, enhancing regional recognition of the fourth feature map through a convolution attention module to generate a fifth feature map;

[0019] Step S6, generating the pipeline defect feature extraction image or the pipeline defect feature enhancement image through an up-sampling layer, feature splicing and a convolution layer.

[0020] Further, in the step S2, the convolution operation is weighted and summed to generate the second feature map.

[0021] Further, the convolution attention module includes a channel attention submodule and a spatial attention submodule, and the step S5 includes steps S51 to S53 belonging to the channel attention submodule and steps S54 to S56 belonging to the spatial attention submodule, specifically:

[0022] Step S51, performing two-dimensional global average pooling and global maximum pooling on the fourth feature map element by element to generate a first descriptor and a second descriptor, respectively;

[0023] Step S52, after the first descriptor and the second descriptor pass through a fully connected network, sequentially passing through a ReLU activation function, channel attention calculation and a Sigmoid activation function to generate channel attention weights;

[0024] Step S53, multiplying the channel weight vector and the fourth feature map element by element to generate a first enhanced feature map;

[0025] Step S54, performing three-dimensional global average pooling and global maximum pooling on the first enhanced feature map to generate a third descriptor and a fourth descriptor, respectively;

[0026] Step S55, performing feature splicing, convolution layer and activation function operations on the third descriptor and the fourth descriptor to generate spatial attention weights;

[0027] Step S56, calculating the spatial attention weights and the first enhanced feature map to generate the fifth feature map.

[0028] Further, in the prior distribution network / posterior distribution network, before the step S1, the three primary colors of the original degraded underwater pipeline image are respectively color corrected by adjusting the color saturation to generate the original underwater pipeline image.

[0029] Further, the step S6 includes the following steps:

[0030] Step S61, generating a sixth feature map through an up-sampling layer from the fifth feature map;

[0031] Step S62, defining the input features of the step S3 as first spliced features to Nth spliced features in turn according to a repeating order, and sequentially performing feature splicing, a convolution layer, an up-sampling layer, feature splicing with the N-1th spliced feature, a convolution layer, an up-sampling layer, and feature splicing with the first spliced feature on the sixth feature map, to generate the pipeline defect feature extraction image or the pipeline defect feature enhancement image.

[0032] Further, the pipeline defect feature extraction image and the pipeline defect feature enhancement image are enhanced and statistically transformed by the Pr module, the Po module and the AdaIN module to generate the pipeline defect recognition image, including:

[0033] The pipeline defect feature extraction image is input into the AdaIN module through the Pr module;

[0034] The pipeline defect feature enhancement image is input into the AdaIN module through the Po module;

[0035] The AdaIN module generates the pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map.

[0036] Further, the AdaIN module generates the pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map, including:

[0037] The Po module receives the posterior distribution of the pipeline defect feature enhancement image generated from the original underwater pipeline image and the reference image, and extracts random samples from the posterior distribution to input the AdaIN module, so as to make the AdaIN module learn the enhancement image in the training stage;

[0038] The Pr module generates random samples of the pipeline defect feature extraction image, and inputs the random samples and the pipeline defect feature extraction image into the feature representation of the AdaIN module, so as to make the AdaIN module learn the enhancement image in the training stage and make the AdaIN module make predictions in the testing stage.

[0039] Further, the pipeline defect identification image sequentially passes through a series of convolution blocks and a LeakyReLU activation function, the convolution attention module, and a convolution layer to generate a final feature map, wherein the LeakyReLU activation function and the convolution attention module are used to increase the nonlinearity of the features, and the convolution layer is used for detail reconstruction.

[0040] Further, the underwater pipeline defect detection method is applied to a pipeline robot working underwater and equipped with a camera, and the pipeline defects include sediment, gas floatation, penetration and notches.

[0041] The collected images of the pipeline robot are generated into the underwater pipeline original degradation images through hybrid enhancement technology and multiple data enhancement technologies.

[0042] Compared with the prior art, the present application has the beneficial effects that,

[0043] 1. Compared with the traditional supervised learning method which needs a large amount of label data difficult to obtain in the underwater environment, the present application can learn the underwater image enhancement task without paired clear images through similar adversarial learning and self-supervised learning strategies, so as to solve the different problems of underwater pipeline image enhancement and defect detection, improve the feature expression ability of the network model, and finally achieve the advantages of good generalization ability, good robustness, high enhancement and defect detection efficiency.

[0044] 2. The present application generates degradation images using an imaging model, and optimizes the underwater restoration and degradation process through an adversarial learning method, so as to finally realize the improvement of image quality. This self-supervised feature not only reduces the demand for a large amount of labeled data, but also improves the generalization ability in complex scenes, so that it can adapt to different underwater environments and lighting conditions.

[0045] 3. The network of the present application sets a skip connection between each convolution module, which can directly transmit the detailed features of the low layer to the decoding part, avoiding the information loss caused by layer-by-layer encoding. This structure can ensure the complete transmission of multi-scale information, so that the final output image has better detail performance, especially when dealing with complex underwater environments, the key color and edge information is preserved.

[0046] 4. The network of the present application adds dynamic convolution and CBAM convolution attention modules, which help the network pay more attention to the key areas of the image. CBAM combines channel attention and spatial attention to give the network stronger selectivity in feature extraction, which can preferentially process the degradation area and target object, further improving the restoration effect. Dynamic convolution can adapt to different feature patterns and better cope with the complex lighting changes and noise of underwater images.

[0047] 5、The multi-functional characteristics of the network of the present application make it suitable for complex underwater image enhancement and defect detection applications. Through the double-path structure and feature fusion, it not only can enhance the definition of the image, but also can improve the recognition rate when detecting underwater pipeline defects, corrosion or other structural damage. This feature is of great help to underwater detection, engineering maintenance and other applications, and improves the overall work efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 A schematic diagram of the overall structure of the underwater pipeline defect detection method based on the improved UNet model of the embodiment of the present application;

[0049] Figure 2 A schematic diagram of the network framework structure process of the underwater pipeline defect detection method based on the improved UNet model of the embodiment of the present application;

[0050] Figure 3 A schematic diagram of the network construction process of the underwater pipeline defect detection method based on the improved UNet model of the embodiment of the present application;

[0051] Figure 4 A schematic diagram of the network training process of the underwater pipeline defect detection method based on the improved UNet model of the embodiment of the present application;

[0052] Figure 5 A schematic diagram of the channel attention submodule structure of the convolutional attention module of the underwater pipeline defect detection method based on the improved UNet model of the embodiment of the present application;

[0053] Figure 6 A schematic diagram of the spatial attention submodule structure of the convolutional attention module of the underwater pipeline defect detection method based on the improved UNet model of the embodiment of the present application;

[0054] Figure 7 A schematic diagram of the Pr module / Po module structure of the underwater pipeline defect detection method based on the improved UNet model of the embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose and advantages of the present application more clear and obvious, the present application will be further described below in combination with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the protection scope of the present application.

[0056] The preferred embodiments of the present application will be described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application, and are not used to limit the protection scope of the present application.

[0057] It should be noted that in the description of the present application, the terms indicating the direction or positional relationship of "upper", "lower", "left", "right", "inner", "outer" and the like are based on the direction or positional relationship shown in the drawings, which is only for the convenience of description, and does not indicate or imply that the device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0058] In addition, it should be noted that in the description of the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0059] As shown in Figures 1 to 7 The present application provides an underwater pipeline defect detection method based on an improved UNet model, which can solve different problems existing in the enhancement and defect detection of underwater pipeline images, improve the feature expression ability of the network model, and finally achieve the advantages of good generalization ability, good robustness, high enhancement and defect detection efficiency.

[0060] As shown in Figures 1 to 7 The present embodiment provides an underwater pipeline defect detection method based on an improved UNet model, which includes: in the first branch, constructing a prior distribution network for feature extraction based on the improved UNet network framework, performing feature extraction and reconstruction on the original underwater pipeline image through the prior distribution network, and generating a pipeline defect feature extraction image; in the second branch, constructing a posterior distribution network for underwater image enhancement based on the improved UNet network framework, performing feature enhancement on the original underwater pipeline image and the reference image through the posterior distribution network, and generating a pipeline defect feature enhancement image; through the Pr module, the Po module and the AdaIN module, the pipeline defect feature extraction image and the pipeline defect feature enhancement image are subjected to enhancement statistical transformation to generate a pipeline defect recognition image.

[0061] It should be noted that although the first branch and the second branch are the same improved UNet network framework, their task purposes, training targets, usage modes and input quantities are different. Specifically, the prior distribution can be understood as a hypothesis or expectation of image features before any data is observed, which helps the model understand what the image features may be without specific instances. The posterior distribution refers to the probability distribution of model parameters given prior information and observed data. The goal of this branch is to learn how to change the features of the input image to match the quality of the reference image based on the input image and the reference image. Both branches involve processing of image features, but their purposes and processing methods are not the same. The first branch focuses more on the characteristics of the original image itself, while the second branch enhances the image based on external standards (reference image). The same for the first branch and the second branch is that they both extract multi-scale features of the image through the encoder part of the UNet network. The difference is that in the second branch, the decoder part of the UNet network is used to generate the enhanced image in combination with the features extracted by the encoder.

[0062] The purpose of the prior distribution network of the first branch is to estimate the prior distribution of a single original underwater pipeline image. The implementation process can refer to the usage of UNet network framework convolutional neural network in medical image segmentation, which captures context information through a shrinking path and achieves precise positioning through a symmetric expanding path. In this branch, the prior distribution network is used to extract useful features from the original underwater pipeline image, which is crucial for subsequent image enhancement processing.

[0063] The improved UNet network framework of the second branch takes the original underwater pipeline image and the corresponding reference image as input to construct the posterior distribution network of UIE (underwater image enhancement). The posterior distribution network not only needs to consider the information of the original image, but also needs to consider the additional guidance information provided by the reference image, so that the posterior distribution network can learn how to convert the original underwater pipeline image into a clearer and higher quality pipeline defect feature enhancement image.

[0064] The task difference between the prior distribution network of the first branch and the posterior distribution network of the second branch: the prior distribution network focuses on extracting useful features from the original image, while the posterior distribution network uses the useful features extracted from the original image and the reference image to generate the enhanced image. This division of labor enables each branch to perform best on its specific task.

[0065] The input difference between the prior distribution network of the first branch and the posterior distribution network of the second branch: the first branch only takes the original underwater pipeline image as input, while the second branch combines the reference image in addition to the original underwater pipeline image. This differentiated input allows each branch to be optimized for its specific task.

[0066] The training target of the prior distribution network of the first branch and the posterior distribution network of the second branch: the training target of the first branch is to learn the feature representation of the original underwater pipeline image, while the training target of the second branch is to learn how to generate high-quality enhanced images given the original underwater pipeline image and the reference image. The difference between the two targets results in different results from the two branches even if the same network structure is used.

[0067] Further, the prior distribution network / posterior distribution network comprises the following steps: in the improved UNet network framework, the following steps are included: step S1, converting the channel number of the original underwater pipeline image or the original underwater pipeline image and the reference image through a series of convolution blocks and LeakyReLU activation functions to generate a first feature map; step S2, extracting features from the first feature map through a dynamic convolution module to generate a second feature map; step S3, consolidating features of the second feature map through a max-pooling layer to generate a third feature map; step S4, repeating steps S1 to S3 at least twice to generate a fourth feature map, wherein the number of convolution kernels of the dynamic convolution module increases gradually; step S5, enhancing regional recognition of the fourth feature map through a convolution attention module to generate a fifth feature map; step S6, generating the pipeline defect feature extraction image or the pipeline defect feature enhancement image through an upsampling layer, feature concatenation, and a convolution layer.

[0068] Further, in step S2, the convolution operation is weighted summed to generate the second feature map.

[0069] Specifically, in step S2, the convolution operation and the weighted sum are performed by the following formula to generate the second feature map:

[0070]

[0071] In the formula, * represents convolution operation, ω i is a weight vector, K is the number of convolution kernels, x represents input information, and y represents output information.

[0072] Specifically, the processing procedure of the dynamic convolution module in the step S2 is as follows: (1) assuming that the given output is CxHxW, wherein C is the number of input channels, and HxW is the input image resolution size; (2) the input image passes through a weight generator to obtain a weight vector ω, and if the convolution kernel is K, the size of ω is K; (3) each convolution kernel is weighted using the weight vector ω, and the weighted convolution kernel is used for convolution operation to obtain an output feature map, wherein the formula of the convolution operation is: y i =x*W i , W is a vector, x represents a training parameter, and y represents an output parameter; (3.4) the convolution results of multiple convolution kernels are weighted and summed to obtain the second feature map: wherein x*W i represents convolution operation, ω is a weight vector, and K is the number of convolution kernels.

[0073] The dynamic convolution allows the network to flexibly select the convolution kernel according to the characteristics of the input data by dynamically generating the weight and weighting multiple convolution kernels, thereby greatly enhancing the expression ability of the network. This mechanism enables the dynamic convolution to better extract effective features when processing input data with different characteristics, and improves the adaptability and generalization ability of the network to complex scenes.

[0074] Referring to Figure 2 , in the step S4, preferably, the steps S1 to S3 are repeated three times, and each time the number of convolution kernels of the dynamic convolution module is increased to improve the model performance, obtain stronger feature expression ability, and make the details and boundary information more obvious. The multi-channel high-dimensional features provide more abundant information, which can better support the processing of subsequent steps.

[0075] Further, a convolution attention module is used to cooperatively fuse spatial multi-scale features, process global features, and balance channel features, and the method comprises the following steps:

[0076] The CBAM is an effective attention mechanism, which improves the representation ability of the network by modeling the attention of space and channels. The CBAM is composed of two sub-modules: a channel attention module as shown in Figure 5 and a spatial attention module as shown in Figure 6The spatial attention module (S) shown, the connection structure of the channel attention module and the spatial attention module can refer to the structure of the conventional convolution attention module CBAM, and the improvement of the convolution attention module CBAM lies in the specific process of the channel attention module and the spatial attention module. The role of the channel attention module is to adaptively assign weights to each channel of the input feature map, so that the network can automatically focus on important channel features. The role of the spatial attention module is to weight each position of the feature map, emphasizing important regions in the image.

[0077] The step S5 includes steps S51 to S53 belonging to the channel attention submodule, steps S54 to S56 belonging to the spatial attention submodule, wherein steps S51 and S52 are specifically as shown in Figure 5 As shown, steps S54 and S55 are specifically as shown in Figure 6 As shown, specifically:

[0078] Step S51, performing two-dimensional global average pooling and global maximum pooling on the fourth feature map element by element to generate a first descriptor and a second descriptor, respectively;

[0079] Specifically, as shown in Figure 5 Step S51, performing global average pooling and global maximum pooling on the fourth feature map by the following formula:

[0080]

[0081] F max = max i,j F(i,j)

[0082] In the formula, F avg is the first descriptor generated by global average pooling, H and W are two dimensions of the fourth feature map, and F max is the second descriptor generated by global maximum pooling;

[0083] The results of the two pooling operations are both Cx1x1 tensors, representing global information of each channel.

[0084] Step S52, after passing the first descriptor and the second descriptor through a fully connected network, sequentially passing through a ReLU activation function, channel attention calculation and a Sigmoid activation function, to generate channel attention weights;

[0085] Specifically, as shown in Figure 5 Step S52, after passing the first descriptor and the second descriptor through a fully connected network, input the following formula to generate channel attention weights:

[0086] M c (F) = σ(W1(RELU(W0(F avg )))+W1(RELU(W0(F max ))))

[0087] In the formula, M c (F) is the channel attention weight, σ represents the Sigmoid activation function, ReLU represents the ReLU activation function, W0 and W1 are two learnable weights, F avg is the first descriptor, F max is the second descriptor.

[0088] The generated result is mapped to [0, 1] through the Sigmoid function to obtain a channel weight vector. This vector represents the importance of each channel.

[0089] Step S53, multiplying the channel weight vector and the fourth feature map element by element to generate a first enhanced feature map;

[0090] Specifically, the step S53 refers to the structure of the conventional convolution attention module CBAM, which is not shown in Figure 5 and 6 The step S53 performs channel weighting on the channel weight vector through the following formula to obtain the first enhanced feature map:

[0091]

[0092] In the formula, M c (F) is the channel attention weight, F represents the fourth feature map, represents element-wise multiplication, and F' is the first enhanced feature map.

[0093] Step S54, performing three-dimensional global average pooling and global maximum pooling on the first enhanced feature map to generate a third descriptor and a fourth descriptor, respectively;

[0094] Specifically, as shown in the step S54, the first enhanced feature map is subjected to global average pooling and global maximum pooling through the following formula: Figure 6

[0095]

[0096]

[0097] In the formula, is the third descriptor generated by global average pooling, and C is the first enhanced feature Figure 1 dimension, ​a fourth descriptor generated for the global max pooling, F' is the first enhanced feature map;

[0098] Both of the two pooling operation results are 1xHxW tensors.

[0099] Step S55, the third descriptor and the fourth descriptor are operated by feature splicing, convolution layer and activation function to generate spatial attention weight;

[0100] Specifically, as shown in step S55, the third descriptor and the fourth descriptor are operated by feature splicing, convolution layer and activation function through the following formula to generate spatial attention weight: Figure 6

[0101]

[0102] In the formula, M s (F') is the spatial attention weight, F' is the first enhanced feature map, σ represents the Sigmoid activation function, Conv represents the convolution layer, is the third descriptor, is the fourth descriptor, and [·] represents feature splicing in the channel dimension;

[0103] That is, the two pooling results are connected in the channel dimension to obtain a 2xHxW, and then a 7x7 convolution layer is performed to generate the spatial attention weight.

[0104] Step S56, the spatial attention weight and the first enhanced feature map are calculated to generate the fifth feature map;

[0105] Specifically, step S56 refers to the structure of the conventional convolution attention module CBAM, which is not shown in Figure 5 and 6 Step S56, the spatial attention weight and the first enhanced feature map are calculated to generate the fifth feature map through the following formula:

[0106]

[0107] In the formula, F'' represents the fifth feature map, M s (F') is the channel spatial attention weight, represents element-wise multiplication.

[0108] CBAM is a lightweight and effective attention module that weights features in two dimensions of channel and space, improving the attention ability of the network to important information. CBAM can be seamlessly embedded in existing convolutional neural networks to improve the feature representation ability.

[0109] ​The above modules are integrated and put into the improved GAN network framework to realize an effective defect detection method for underwater pipeline images as follows: the dynamic convolution module is put into the encoding stage of the GAN network framework, and the attention mechanism module of step (4) is put into the encoding stage of the GAN network. The key component of the framework GAN network framework is the AdaIN module, which encodes ambiguity after the feature extractor. The blocks before and after aim to build enhanced distributions. After the feature extractor is completed, the AdaIN module performs enhanced statistical transformation on the received deterministic features. In the training stage, random samples in the posterior distribution are injected into the AdaIN GAN module to transform the enhanced representation. In the verification stage, random samples in the AdaIN module are used to generate predictions. After the AdaIN module, two concatenated 3x3 size convolution blocks and LeakyReLU activation functions obtain complete output images, and finally a 1x1 size convolution block connects the features to obtain the final defect detection effect image.

[0110] Further, in the prior distribution network / posterior distribution network, before performing the step S1, the three primary colors of the original degraded underwater pipeline image are respectively color corrected by adjusting the color saturation to generate the original underwater pipeline image.

[0111] Specifically, before performing the step S1, the original degraded underwater pipeline image is color corrected by the following formula to generate the original underwater pipeline image:

[0112]

[0113] where c represents one of the RGB primary colors, UC represents the corrected original underwater pipeline image, S represents the original degraded underwater pipeline image, M and V represent the mean and variance of the corresponding primary color c in the original degraded underwater pipeline image S respectively, and μ represents the underwater enhanced result saturation.

[0114] It can be understood that due to the particularity of the underwater environment, such as changes in lighting conditions, absorption and scattering of water bodies, etc., the images captured by the camera often have color deviation and low contrast problems. Therefore, these images need to be color corrected by a formula to restore their natural colors and contrasts, thereby significantly improving the quality of the input images and thus improving the performance of the entire network.

[0115] The overall steps of the underwater pipeline defect detection method are as follows: 1. Image acquisition: the original degraded underwater image is collected by underwater camera or robot and the like. 2. Color correction: the original degraded underwater image is processed using the color correction formula to generate a corrected image. 3. Input UNet: the corrected image is input into the UNet network with different branches for feature extraction and image enhancement. 4. Feature extraction: the UNet network extracts multi-scale features of the image through its encoder part. 5. Image enhancement: the enhanced image is generated through the decoder part of the UNet combined with the features extracted by the encoder. 6. Post-processing: the final enhanced image can be used for further defect detection or other tasks.

[0116] It can be understood that the application of the color correction formula in the UNet network framework is mainly to improve the quality of the input image and ensure that the network can learn more effective features from high-quality input. The original degraded underwater image of the water pipe is obtained by actual collection, and then preprocessed by the color correction formula to generate a corrected image, which is then input into the UNet network for feature extraction and image enhancement. This preprocessing step is crucial to improve the performance of the entire image enhancement system.

[0117] Further, the step S6 includes the following steps: step S61, generating a sixth feature map by upsampling the fifth feature map; step S62, defining the input features of step S3 as first splicing features to Nth splicing features in turn according to the repetition order, and sequentially performing feature splicing, convolution layer, upsampling layer, feature splicing with the N-1th splicing feature, convolution layer, upsampling layer, and feature splicing with the first splicing feature on the sixth feature map to generate the pipeline defect feature extraction image or the pipeline defect feature enhancement image.

[0118] Further, the pipeline defect feature extraction image and the pipeline defect feature enhancement image are enhanced by the Pr module, the Po module and the AdaIN module to generate the pipeline defect recognition image, including: the pipeline defect feature extraction image is input into the AdaIN module through the Pr module; the pipeline defect feature enhancement image is input into the AdaIN module through the Po module; the AdaIN module generates the pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map.

[0119] Further, the AdaIN module generates the pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map, including:

[0120] The Po module receives the posterior distribution of the pipeline defect feature enhanced image generated by the original underwater pipeline image and the reference image, and extracts random samples therefrom as input to the AdaIN module, so as to make the AdaIN module learn enhanced images in the training stage;

[0121] The Pr module generates random samples of the pipeline defect feature extraction image, and inputs the random samples and the pipeline defect feature extraction image to the feature representation of the AdaIN module, so as to make the AdaIN module learn enhanced images in the training stage and make the AdaIN module make predictions in the testing stage.

[0122] Further, the pipeline defect recognition image sequentially passes through a serial convolution block and a LeakyReLU activation function, the convolution attention module, and a convolution layer to generate a final feature map, wherein the LeakyReLU activation function and the convolution attention module are used to increase the nonlinearity of the features, and the convolution layer is used for detail reconstruction.

[0123] Further, the underwater pipeline defect detection method is applied to a pipeline robot working underwater and equipped with a camera, and the pipeline defects include sediment, gas floatation, penetration, and notches.

[0124] Specifically, the collected images are preprocessed by the hybrid enhancement technology and the multiple data enhancement technologies to generate the original underwater pipeline image.

[0125] Preferably, the pipeline robot is equipped with a high-resolution camera, and various videos and spectral images of underwater pipeline defects are obtained by the robot.

[0126] Preferably, key defect images are extracted from each video by using a video frame extraction technology, and 1020 original defect images are obtained.

[0127] Since the training quality of the detection model is positively correlated with the size of the dataset, it is difficult to obtain good training quality when the size of the dataset is too small. Therefore, it is preferred to use data augmentation techniques including rotation, cropping, color adjustment, mosaic and mixing to increase the dataset formed by the original underwater pipeline images to 5650 images. The mosaic enhancement technique is to randomly crop four images and combine them into a new image. The mixing enhancement technique is to mix two images of different categories together to form a coherent image. The constructed dataset is cleaned to remove data that does not meet the requirements, and a certain proportion of samples in the total samples are extracted as the training set, and the remaining samples are the validation set. The number of training set and validation set is preferably 9 to 1.

[0128] Specifically, referring to Figure 2 The improved UNet network framework also improves the GAN network framework based on deep learning technology, and the input original underwater pipeline image / reference image is a RGB image with a size of CxHxW, where C is the number of channels, H is the image height, and W is the image width. The structure of the UNet network framework is similar to the GAN composed of two branches.

[0129] In the improved UNet network framework, the feature map of the input convolution attention module CBAM is generated through three levels of convolution layers, wherein the first level of convolution layer is the starting part of the network, mainly used to extract primary features from the preprocessed original underwater pipeline image / reference image data. The primary features usually include edges, textures and other basic visual elements in the image. In order to effectively capture the primary features, this layer adopts convolution operation with LeakyReLU activation function and dynamic convolution operation. The LeakyReLU activation function does not completely suppress the signal when processing negative input, but gives it a small positive slope. This means that even negative input will pass on some information, rather than being directly zeroed like standard ReLU. This feature helps to alleviate the problem of "dead neurons", that is, some neurons are in an inactive state for a long time during training, thereby improving the learning ability of the model.

[0130] In the first convolutional layer, dynamic convolution technology is also introduced. Dynamic convolution allows kernel weights to be adjusted in real-time based on the characteristics of input data, enabling better adaptation to the diversity of input data. An important feature of dynamic convolution is that its kernel size and stride can be dynamically adjusted based on the characteristics of input data. This means that for different input samples, the model can choose the filter size and moving step that best suits the current situation. For example, when dealing with small-scale features containing rich details, a smaller kernel can be used to capture these details; while for larger-scale objects, the kernel size can be increased to obtain a broader view. The implementation of dynamic convolution usually relies on additional network structures such as MLP (Multi-Layer Perceptron) to generate convolution kernel weights for specific inputs. This method allows the model to learn how to generate the optimal set of kernel weights based on the statistical characteristics of input data during training. In addition, dynamic convolution can also be implemented through conditional calculation, that is, dynamically selecting a set of predefined convolution kernels based on input conditions. By enabling convolution operations to respond to changes in input data, dynamic convolution improves the flexibility and adaptability of the model. This adaptability is particularly important for handling diverse input data, as fixed convolution kernels may not be able to fully capture all important patterns. Therefore, when faced with complex visual tasks, dynamic convolution layers can provide stronger expressive power and higher performance.

[0131] Since dynamic convolution can adaptively adjust its kernel size and stride based on input data, adding max pooling after dynamic convolution can further consolidate the extracted features and ensure that more abstract information can be processed in the subsequent part of the network. At the same time, max pooling can also help maintain the invariance of the features learned by dynamic convolution, thereby improving the robustness and generalization ability of the entire network.

[0132] First layer convolution of the first convolutional layer: use a 3x3 convolution kernel with a stride of 1 and set the output channel number to 64. Activation function: use LeakyReLU to increase non-linear characteristics and allow certain negative values to pass through. Second layer convolution of the first convolutional layer: perform another convolution immediately after, set the output channel number to 128, and use the same 3x3 convolution kernel and LeakyReLU activation.

[0133] Through two convolutions, the bottom-level features of the image are extracted layer by layer, including edges, textures, and shapes, etc., ensuring rich feature information. The LeakyReLU activation function allows the network to capture more complex feature patterns, avoiding the problem of neuron death and improving the learning ability of the network.

[0134] Next, DynamicConv generates dynamic convolution kernels based on the input feature map, usually using a 1x1 convolution to generate multiple convolution kernels. According to the generated convolution kernel, the input feature map is convolved, which adapts to the different input features, thereby enhancing the network's adaptability to features. The max pooling operation uses a 2x2 pooling window and a step size of 2, reducing the spatial resolution of the feature map. By selecting the maximum value in the pooling window, the most significant feature information is retained, enhancing the representativeness of the features.

[0135] The second-level convolutional layer is similar in structure and function to the first level, but has an increase in the number of convolution kernels, which aims to capture more complex feature relationships. This allows the second-level convolutional layer to learn higher-level features than the first-level convolutional layer, further abstracting the primary features extracted by the first level.

[0136] The first layer of convolution in the second-level convolutional layer: again using a 3x3 convolution kernel with a step size of 1 and an output channel number of 128. Activation function: LeakyReLU is used to enhance the non-linear features of the network. On the basis of the first level, deeper features are extracted to capture more complex structural information. With the increase in the number of channels, the expression ability of the network is further improved, which can better adapt to complex underwater scenes.

[0137] Next, DynamicConv generates corresponding convolution kernels based on the current input feature map, and performs convolution operations on the new feature map to adapt to the current layer's features. This provides the network with stronger feature extraction capabilities, enabling it to more effectively capture subtle changes in underwater images. Dynamic convolution allows the network to maintain good performance on different feature maps.

[0138] Max pooling is performed to further reduce the spatial size of the feature map, providing more streamlined input for subsequent layers. Through max pooling, the network ensures that the most significant features are retained, which helps to fuse and enhance subsequent features.

[0139] The second layer of convolution in the second-level convolutional layer: continues to perform convolution with an output channel number of 256.

[0140] The third-level convolutional layer further increases the number of parameters, aiming to further enhance the learning ability of the model. With the increase in the number of parameters, the third-level convolutional layer can learn more abstract and complex features. These features may involve larger visual patterns, or even be understood as high-level descriptions of entire objects or scenes.

[0141] The first layer of convolution in the third-level convolutional layer uses a 3x3 convolution kernel with a stride of 1 and an output channel number of 256. The activation function uses LeakyReLU to increase nonlinearity. The second layer of convolution performs convolution with an output channel number of 512. It extracts more abstract features that can capture global information and important contextual relationships. A higher number of channels further enhances the network's feature expression ability, adapting to more complex underwater image features. Dynamic convolution kernel generation generates new convolution kernels based on the current input. Max pooling is performed again to further reduce the spatial dimensions of the feature map, preserving the most significant feature information and laying the foundation for the processing of the feature enhancement path.

[0142] After three levels of convolution, the final feature map is sent to the Convolutional Block Attention Module (CBAM) for further processing. CBAM enhances important information in the feature map, thereby improving the network's focus on key features. Through the combination of channel attention and spatial attention, the network can better understand the key information in the input image, which is crucial for improving the precision and quality of underwater image restoration. The design of CBAM not only enhances the performance of the feature map but also lays a solid foundation for subsequent processing steps.

[0143] The next step is usually upsampling to restore or increase the spatial resolution of the feature map, so that better feature fusion and target detection can be performed. During downsampling, the spatial size of the feature map is reduced, and some detailed information is lost. Upsampling can help restore these details, increasing the spatial resolution of the feature map.

[0144] The Pr module and the Po module are two key modules in the network, playing a crucial role in underwater image enhancement and target detection processes. The Pr module is mainly used after feature extraction. The Pr module generates random samples that are used to generate prediction values during the testing phase. Specifically, after feature extraction, the Pr module converts deterministic features into a form that matches the enhancement statistics through the PAdaIN (Parametric Adaptive and Instance Normalization) module. This means that samples drawn from the prior distribution in the Pr module will be used to guide how to adjust the features of the input image to make it more suitable for subsequent detection tasks.

[0145] The Po module mainly plays a role in the training phase. It receives random samples drawn from the posterior distribution and injects them into the AdaIN (Adaptive Instance Normalization) layer to convert the enhanced representation. In this way, the Po module helps the network learn how to effectively perform image enhancement, so that it can produce high-quality enhancement results when faced with images under different conditions.

[0146] The design of the Pr and Po modules demonstrates fine-grained control over the feature flow within the network. The Pr module simulates images by generating random samples, thereby accounting for potential variations in the enhancement process, while the Po module ensures that the network learns effective enhancement strategies by adjusting feature representations. Furthermore, the introduction of the Pr and Po modules enhances the network's generalization capabilities by allowing the network to see diverse inputs during training, enabling it to handle unseen situations during testing.

[0147] The AdaIN module is an important component following the Pr module and the Po module. It plays a central role in the image enhancement process, especially when it comes to transforming features to match the enhancement statistics.

[0148] The main function of the AdaIN module is to achieve feature stylization by adaptively adjusting the mean and variance of the feature map, thereby changing the statistical properties of the input feature map to conform to the target style. In the network, the AdaIN module is used to inject random samples generated by the Pr module into the feature representation to achieve image enhancement. During the training phase, this process enables the model to learn how to effectively enhance images; during the testing phase, the random samples generated by the Pr module are used for prediction.

[0149] After the AdaIN module, the image passes through two convolutional layers with corresponding LeakyReLU activation functions to further extract features and increase nonlinearity. It then enters the CBAM module to further enhance the feature map. After CBAM, the final Conv layer fuses the features output by the CBAM with the previously extracted features to generate the final feature map. This layer also converts the feature map into an output image with the same resolution as the input image to reconstruct details. This image enhancement allows for more accurate detection of pipeline defects.

[0150] See also Figure 3 and 4 , set the training parameters to train the model as follows: set the learning rate to Lr, the loss function to L, the Batch-Size to n, the training set and the validation set to 9:1, the optimizer to D, the total number of training rounds to E, and then train the deep learning network model as follows: (1) The learning rate Lr is initially 0.0001, the loss function is selected as Hunber_Loss, the optimizer is selected as Adam, and the hyperparameters such as the training round, batch size Batch-Size and input image size InputSize are set; (2) The final improved network model is obtained through iterative training, and then the trained network model is used to enhance and detect the underwater pipeline image of the validation set to obtain the final detection result graph.

[0151] It can be understood that one of the major challenges in underwater image enhancement is the lack of high-quality paired data, and it is often difficult to obtain clear "ground truth" images. Traditional supervised learning methods require a large amount of labeled data, which is difficult to obtain in underwater environments. This network uses similar strategies to adversarial learning and self-supervised learning, enabling it to learn underwater image enhancement tasks without paired clear images. This method uses imaging models to generate degraded images, and the underwater restoration and degradation process is optimized through adversarial learning, ultimately improving image quality. This self-supervised feature not only reduces the need for a large amount of labeled data, but also improves the generalization ability in complex scenarios, making it suitable for different underwater environments and lighting conditions. The network sets a skip connection between each convolutional module, which can directly transmit the detailed features of the bottom layer to the decoding part, avoiding information loss caused by layer-by-layer encoding. This structure ensures the complete transmission of multi-scale information, making the final output image have better detail performance, especially when dealing with complex underwater environments, it preserves key color and edge information. Dynamic convolution and CBAM attention modules are added to the network, which help the network pay more attention to key areas in the image. CBAM combines channel attention and spatial attention, giving the network more selectivity during feature extraction, which can prioritize processing of degraded areas and target objects, further improving restoration results. Dynamic convolution can adapt to different feature patterns and better cope with complex underwater image lighting changes and noise.

[0152] In addition, the multifunctional nature of the network makes it suitable for complex underwater image enhancement and defect detection applications. Through the dual-path structure and feature fusion, it not only enhances image clarity, but also improves recognition rates when detecting underwater pipeline defects, corrosion, or other structural damage. This feature is very helpful for underwater exploration, engineering maintenance, and other applications, improving overall work efficiency.

[0153] So far, the technical solutions of the present application have been described in conjunction with the preferred embodiments shown in the drawings, but those skilled in the art will readily understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to related technical features without departing from the principles of the present application, and the technical solutions after these changes or replacements will fall within the protection scope of the present application.

[0154] The above description is only the preferred embodiments of the present application and is not intended to limit the present application; for those skilled in the art, the present application can have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included within the protection scope of the present application.

Claims

1. A method for underwater pipeline defect detection based on an improved UNet model, characterized in that: include: In the first branch, a prior distribution network for feature extraction is constructed based on the improved UNet network framework. The prior distribution network is used to extract and reconstruct features of the original underwater pipeline image to generate a pipeline defect feature extraction image. In the second branch, a posterior distribution network for underwater image enhancement is constructed based on the improved UNet network framework, and features of the original underwater pipeline image and the reference image are enhanced by the posterior distribution network to generate a pipeline defect feature-enhanced image; Performing enhanced statistical transformation on the pipeline defect feature extraction image and the pipeline defect feature enhanced image through the Pr module, the Po module, and the AdaIN module to generate a pipeline defect recognition image; The prior distribution network / the posterior distribution network includes the following steps: Step S1: convert the number of channels of the original underwater pipeline image, or the original underwater pipeline image and the reference image, by serially connecting a convolution block and a LeakyReLU activation function to generate a first feature map; Step S2: extracting features from the first feature map through a dynamic convolution module to generate a second feature map; Step S3: consolidating the features of the second feature map through a maximum pooling layer to generate a third feature map; Step S4, repeating steps S1 to S3 at least twice to generate a fourth feature map, wherein the number of convolution kernels of the dynamic convolution module increases successively; Step S5: applying a convolutional attention module to the fourth feature map to enhance region recognition and generate a fifth feature map; Step S6: Passing the fifth feature map through an upsampling layer, a feature splicing layer, and a convolution layer to generate the pipeline defect feature extraction image or the pipeline defect feature enhanced image; The pipeline defect feature extraction image and the pipeline defect feature enhanced image are subjected to enhanced statistical transformation by the Pr module, the Po module and the AdaIN module to generate the pipeline defect recognition image, including: The pipeline defect feature extraction image is input into the AdaIN module through the Pr module; The pipeline defect feature enhanced image is input into the AdaIN module through the Po module; The AdaIN module generates the pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map; The AdaIN module generates a pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map, including: The Po module receives the posterior distribution of the pipeline defect feature enhanced image generated by the original underwater pipeline image and the reference image, and extracts random samples from them and inputs them into the AdaIN module to enable the AdaIN module to learn the enhanced image during the training phase; A random sample of the pipeline defect feature extraction image is generated by the Pr module, and the random sample and the pipeline defect feature extraction image are input into the feature representation of the AdaIN module to enable the AdaIN module to learn the enhanced image in the training phase and to enable the AdaIN module to make predictions in the testing phase.

2. The underwater pipeline defect detection method based on the improved UNet model according to claim 1 is characterized in that: In step S2, weighted summation is performed on the convolution operation to generate the second feature map.

3. The underwater pipeline defect detection method based on the improved UNet model according to claim 1 is characterized in that: The convolutional attention module includes a channel attention submodule and a spatial attention submodule. Step S5 includes steps S51 to S53 belonging to the channel attention submodule and steps S54 to S56 belonging to the spatial attention submodule, specifically: Step S51: performing two-dimensional global average pooling and global maximum pooling on the fourth feature map element by element to generate a first descriptor and a second descriptor respectively; Step S52: After passing the first descriptor and the second descriptor through a fully connected network, they are sequentially subjected to a ReLU activation function, a channel attention calculation, and a Sigmoid activation function to generate a channel attention weight; Step S53: multiplying the channel weight vector and the fourth feature map element-wise to generate a first enhanced feature map; Step S54: performing three-dimensional global average pooling and global maximum pooling on the first enhanced feature map to generate a third descriptor and a fourth descriptor respectively; Step S55: performing feature concatenation, convolution layer, and activation function operations on the third descriptor and the fourth descriptor to generate a spatial attention weight; Step S56: Calculate the spatial attention weight and the first enhanced feature map to generate the fifth feature map.

4. The underwater pipeline defect detection method based on the improved UNet model according to claim 1 is characterized in that: In the prior distribution network / the posterior distribution network, before executing step S1, the original underwater pipeline image is generated by performing color correction on the three primary colors of the original degraded image of the underwater pipeline by adjusting the color saturation.

5. The underwater pipeline defect detection method based on the improved UNet model according to claim 1 is characterized in that: The step S6 comprises the following steps: Step S61: Pass the fifth feature map through an upsampling layer to generate a sixth feature map; Step S62: The input features of step S3 are defined as the first splicing feature to the Nth splicing feature in a repetitive order, and the sixth feature map is subjected to feature splicing with the Nth splicing feature, convolution layer, upsampling layer, feature splicing with the N-1th splicing feature, convolution layer, upsampling layer, until feature splicing with the first splicing feature is performed, to generate the pipeline defect feature extraction image or the pipeline defect feature enhanced image.

6. The underwater pipeline defect detection method based on the improved UNet model according to any one of claims 1 to 5, characterized in that: The pipeline defect recognition image is sequentially passed through a series convolution block and a LeakyReLU activation function, a convolution attention module, and a convolution layer to generate a final feature map, wherein the LeakyReLU activation function and the convolution attention module are used to increase the nonlinearity of the features, and the convolution layer is used to reconstruct the details.

7. The underwater pipeline defect detection method based on the improved UNet model according to claim 4 is characterized in that: The underwater pipeline defect detection method is applied to a pipeline robot equipped with a camera and working underwater. The pipeline defects include sediment, air flotation, penetration and gaps. The collected image of the pipeline robot is subjected to a hybrid enhancement technology and multiple data enhancement technologies to generate the original degraded image of the underwater pipeline.

Images