Underwater pipeline defect detection method based on improved UNet model

Abstract

The invention relates to the technical field of deep learning, and provides an underwater pipeline defect detection method based on an improved UNet model, and the method comprises the steps: constructing a prior distribution network for feature extraction based on an improved UNet network frame in a first branch, carrying out the feature extraction and reconstruction of an original underwater pipeline image, and generating a pipeline defect feature extraction image; in the second branch, constructing a posterior distribution network for underwater image enhancement based on an improved UNet network framework, and performing feature enhancement on the original underwater pipeline image and the reference image to generate a pipeline defect feature enhanced image; and performing enhanced statistical transformation on the pipeline defect feature extraction image and the pipeline defect feature enhanced image through a Pr module, a Po module and an AdaIN module to generate a pipeline defect recognition image. According to the method, different problems existing in enhancement and defect detection of the underwater pipeline image are solved, the feature expression ability of the network model is improved, and finally the advantages of being good in generalization ability, good in robustness and high in enhancement and defect detection efficiency are achieved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to an underwater pipeline defect detection method based on an improved UNet model. Background Art

[0002] Underwater pipelines are an important part of marine oil and gas exploitation and marine engineering, and they undertake the function of transporting energy. With the continuous deepening of marine development, the scale of submarine pipeline systems has gradually expanded. These pipelines face various complex environmental challenges when running on the seabed for a long time. Due to long-term exposure to harsh marine environments, underwater pipelines will be affected by a series of factors such as corrosion, wear, mechanical damage, and marine biological attachment. The cumulative effect of these factors will cause cracks, dents, perforations and other defects in the pipelines, which may lead to serious safety problems, such as energy leakage or pipeline rupture. The above problems will not only bring economic losses, but also cause serious pollution to the marine ecological environment. Therefore, how to effectively detect, maintain and manage submarine pipelines has become a research topic that has attracted much attention in the field of marine engineering and energy transportation today.

[0003] Traditional submarine pipeline inspection methods, such as sonar and laser scanning, can identify pipeline defects to a certain extent, but these methods have relatively low resolution and are susceptible to noise interference. At the same time, these methods usually require complex post-processing and manual intervention, are inefficient, and are difficult to adapt to the requirements of real-time inspection in modern marine engineering. In order to ensure the normal operation of the pipeline, it is necessary to upgrade the inspection methods with innovative technologies to ensure the long-term safe operation of the pipeline.

[0004] Therefore, research on deep learning-based methods has received widespread attention. For example, although the accuracy of neural networks in quantitatively identifying submarine pipeline defects is high, their performance may vary in different environments and conditions. The submarine pipeline environment is complex and changeable. Factors such as lighting conditions, sediment deposition, and marine biological attachment can interfere with the detection of pipeline surfaces. Therefore, the robustness of deep learning models in these complex environments has not been fully verified.

[0005] Currently, there is a You Only Look Once (YOLO) model that uses a convolutional neural network to detect pipeline images and comprehensively evaluate pipeline anomalies. Although the system showed a high recognition accuracy in the test, it has not yet been studied in the deep sea environment. The light in the deep sea environment is extremely weak, with severe scattering and attenuation. Conventional optical vision technology is likely to be difficult to work in this environment.

[0006] Existing submarine pipeline defect detection methods have made some progress in accuracy and real-time performance, but there are still many limitations. According to the principle of deep learning models, deep learning-based detection methods are easily affected by data volume and data distribution, especially when facing complex and changeable submarine environments, their generalization ability is still insufficient. Although the detection methods based on traditional visual technology and robotic systems perform well under laboratory conditions, they still have obvious deficiencies in real-time and robustness in complex deep-sea environments. In addition, most existing methods rely on specific sensor data, lack the ability to fuse multi-source data, and are difficult to cope with sudden environmental changes and noise interference.

[0007] In view of this, the present invention is proposed. Summary of the invention

[0008] To this end, the present invention provides an underwater pipeline defect detection method based on an improved UNet model, which is used to specifically solve the different problems existing in underwater pipeline image enhancement and defect detection, while improving the feature expression ability of the network model, and ultimately achieving the advantages of good generalization ability, good robustness, and high enhancement and defect detection efficiency.

[0009] To achieve the above object, the present invention proposes an underwater pipeline defect detection method based on an improved UNet model, comprising:

[0010] In the first branch, a prior distribution network for feature extraction is constructed based on the improved UNet network framework, and features of the original underwater pipeline image are extracted and reconstructed through the prior distribution network to generate a pipeline defect feature extraction image;

[0011] In the second branch, a posterior distribution network for underwater image enhancement is constructed based on the improved UNet network framework, and features of the original underwater pipeline image and the reference image are enhanced by the posterior distribution network to generate a pipeline defect feature enhanced image;

[0012] Through the Pr module, the Po module and the AdaIN module, the pipeline defect feature extraction image and the pipeline defect feature enhanced image are enhanced and statistically transformed to generate a pipeline defect recognition image.

[0013] Furthermore, the prior distribution network / the posterior distribution network includes the following steps:

[0014] Step S1, converting the number of channels of the original underwater pipeline image, or the original underwater pipeline image and the reference image, by serially connecting a convolution block and a LeakyReLU activation function to generate a first feature map;

[0015] Step S2: extracting features from the first feature map through a dynamic convolution module to generate a second feature map;

[0016] Step S3, consolidating the features of the second feature map through a maximum pooling layer to generate a third feature map;

[0017] Step S4, repeating the steps S1 to S3 at least twice to generate a fourth feature map, wherein the number of convolution kernels of the dynamic convolution module increases successively;

[0018] Step S5, enhancing region recognition by a convolutional attention module on the fourth feature map to generate a fifth feature map;

[0019] Step S6: Pass the fifth feature map through an upsampling layer, a feature concatenation layer, and a convolution layer to generate the pipeline defect feature extraction image or the pipeline defect feature enhanced image.

[0020] Furthermore, in step S2, weighted summation is performed on the convolution operation to generate the second feature map.

[0021] Further, the convolutional attention module includes a channel attention submodule and a spatial attention submodule, and the step S5 includes steps S51 to S53 belonging to the channel attention submodule and steps S54 to S56 belonging to the spatial attention submodule, specifically:

[0022] Step S51, performing two-dimensional global average pooling and global maximum pooling on the fourth feature map element by element to generate a first descriptor and a second descriptor respectively;

[0023] Step S52: After the first descriptor and the second descriptor pass through the fully connected network, they are sequentially subjected to the ReLU activation function, channel attention calculation and Sigmoid activation function to generate channel attention weights;

[0024] Step S53, multiplying the channel weight vector and the fourth feature map element by element to generate a first enhanced feature map;

[0025] Step S54: performing three-dimensional global average pooling and global maximum pooling on the first enhanced feature map to generate a third descriptor and a fourth descriptor respectively;

[0026] Step S55, performing feature concatenation, convolution layer and activation function operations on the third descriptor and the fourth descriptor to generate a spatial attention weight;

[0027] Step S56: Calculate the spatial attention weight and the first enhanced feature map to generate the fifth feature map.

[0028] Furthermore, in the prior distribution network / the posterior distribution network, before executing step S1, the three primary colors of the original degraded image of the underwater pipeline are color corrected by adjusting the color saturation respectively to generate the original underwater pipeline image.

[0029] Furthermore, step S6 includes the following steps:

[0030] Step S61, passing the fifth feature map through an upsampling layer to generate a sixth feature map;

[0031] Step S62, defining the input features of step S3 as the first stitching feature to the Nth stitching feature in a repetitive order, and subjecting the sixth feature map to feature stitching with the Nth stitching feature, a convolution layer, an upsampling layer, feature stitching with the N-1th stitching feature, a convolution layer, and an upsampling layer in sequence, until feature stitching with the first stitching feature is performed, to generate the pipeline defect feature extraction image or the pipeline defect feature enhanced image.

[0032] Furthermore, the pipeline defect feature extraction image and the pipeline defect feature enhancement image are subjected to enhanced statistical transformation through the Pr module, the Po module and the AdaIN module to generate the pipeline defect recognition image, including:

[0033] The pipeline defect feature extraction image is input into the AdaIN module through the Pr module;

[0034] The pipeline defect feature enhanced image is input into the AdaIN module through the Po module;

[0035] The AdaIN module generates the pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map.

[0036] Furthermore, the AdaIN module generates a pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map, including:

[0037] The Po module receives the posterior distribution of the pipeline defect feature enhanced image generated by the original underwater pipeline image and the reference image, and extracts random samples therefrom and inputs them into the AdaIN module, so as to enable the AdaIN module to learn the enhanced image in the training stage;

[0038] A random sample of the pipeline defect feature extraction image is generated by the Pr module, and the random sample and the pipeline defect feature extraction image are input into the feature representation of the AdaIN module to enable the AdaIN module to learn enhanced images in the training phase and to enable the AdaIN module to make predictions in the testing phase.

[0039] Furthermore, the pipeline defect recognition image is sequentially connected through a series of convolution blocks and a LeakyReLU activation function, a convolutional attention module, and a convolutional layer to generate a final feature map, wherein the LeakyReLU activation function and the convolutional attention module are used to increase the nonlinearity of the feature, and the convolutional layer is used to reconstruct the details.

[0040] Furthermore, the underwater pipeline defect detection method is applied to a pipeline robot equipped with a camera working underwater, and the pipeline defects include sediment, air flotation, penetration and gap;

[0041] The collected image of the pipeline robot is subjected to hybrid enhancement technology and multiple data enhancement technologies to generate the original degraded image of the underwater pipeline.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. Compared with the traditional supervised learning method which requires a large amount of labeled data that is difficult to obtain in underwater environments, the present invention uses similar adversarial learning and self-supervised learning strategies to enable it to learn the task of underwater image enhancement without paired clear images, so as to solve the different problems existing in underwater pipeline image enhancement and defect detection in a targeted manner, while improving the feature expression ability of the network model, and finally achieving the advantages of good generalization ability, good robustness, and high efficiency of enhancement and defect detection.

[0044] 2. The present invention uses an imaging model to generate degraded images, and adaptively optimizes the underwater restoration and degradation process through adversarial learning, ultimately achieving improved image quality. This self-supervision feature not only reduces the need for a large amount of labeled data, but also improves the generalization ability in complex scenes, enabling it to adapt to different underwater environments and lighting conditions.

[0045] 3. The network of the present invention sets a jump connection between each convolution module, which can directly transfer the low-level detail features to the decoding part, avoiding the information loss caused by layer-by-layer encoding. This structure can ensure the complete transmission of multi-scale information, so that the final output image has better detail performance, especially when dealing with complex underwater environments, retaining key color and edge information.

[0046] 4. Dynamic convolution and CBAM convolution attention modules are added to the network of the present invention, which help the network to focus on key areas in the image more effectively. CBAM gives the network stronger selectivity in the feature extraction process through the combination of channel attention and spatial attention, and can give priority to degraded areas and target objects, further improving the restoration effect. Dynamic convolution can adapt to different feature patterns and better cope with the complex illumination changes and noise of underwater images.

[0047] 5. The multifunctional nature of the network of the present invention makes it suitable for complex underwater image enhancement and defect detection applications. Through the dual-path structure and feature fusion, it can not only enhance the clarity of the image, but also improve the recognition rate when detecting underwater pipeline defects, corrosion or other structural damage. This feature is of great help to applications such as underwater detection and engineering maintenance, and improves the overall work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of the general structure of an underwater pipeline defect detection method based on an improved UNet model according to an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of a network framework structure flow chart of an underwater pipeline defect detection method based on an improved UNet model according to an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of a network construction process of an underwater pipeline defect detection method based on an improved UNet model according to an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of a network training process of an underwater pipeline defect detection method based on an improved UNet model according to an embodiment of the present invention;

[0052] Figure 5 It is a schematic diagram of the structure of a channel attention submodule of a convolutional attention module of an underwater pipeline defect detection method based on an improved UNet model according to an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of the structure of a spatial attention submodule of a convolutional attention module of an underwater pipeline defect detection method based on an improved UNet model according to an embodiment of the present invention;

[0054] Figure 7 Schematic diagram of the Pr module / Po module structure of the underwater pipeline defect detection method based on the improved UNet model in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0057] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0058] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0059] like Figures 1 to 7 As shown, the present invention provides an underwater pipeline defect detection method based on an improved UNet model, which can solve the different problems existing in underwater pipeline image enhancement and defect detection in a targeted manner, while improving the feature expression ability of the network model, and finally achieve the advantages of good generalization ability, good robustness, and high enhancement and defect detection efficiency.

[0060] like Figures 1 to 7 As shown, this embodiment proposes an underwater pipeline defect detection method based on an improved UNet model, including: in a first branch, a prior distribution network for feature extraction is constructed based on the improved UNet network framework, and features are extracted and reconstructed on the original underwater pipeline image through the prior distribution network to generate a pipeline defect feature extraction image; in a second branch, a posterior distribution network for underwater image enhancement is constructed based on the improved UNet network framework, and features are enhanced on the original underwater pipeline image and the reference image through the posterior distribution network to generate a pipeline defect feature enhanced image; and enhanced statistical transformation is performed on the pipeline defect feature extraction image and the pipeline defect feature enhanced image through the Pr module, the Po module and the AdaIN module to generate a pipeline defect recognition image.

[0061] It should be noted that although the first branch and the second branch are the same improved UNet network framework, their task objectives, training goals, usage methods, and input amounts are different. Specifically, the prior distribution can be understood as an assumption or expectation of image features before any data is observed, which helps the model understand what features the image may have in the absence of specific instances. The posterior distribution refers to the probability distribution of model parameters given known prior information and observed data. The goal of this branch is to learn how to change the features of the input image to match the quality of the reference image based on the input image and the reference image. Both branches involve the processing of image features, but their purposes and processing methods are different. The first branch focuses more on the characteristics of the original image itself, while the second branch enhances the image based on the consideration of external standards (reference images). The first branch and the second branch are the same in that they both extract multi-scale features of the image through the encoder part of the UNet network. The difference is that in the second branch, the decoder part of the UNet network is combined with the features extracted by the encoder to generate an enhanced image.

[0062] The purpose of the prior distribution network of the first branch is to estimate the prior distribution of a single original underwater pipeline image. The specific implementation process can refer to the use of the convolutional neural network of the UNet network framework in medical image segmentation, which captures contextual information through a contraction path and achieves precise positioning through a symmetrical expansion path. In this branch, the prior distribution network is used to extract useful features from the original underwater pipeline image, which are crucial for subsequent image enhancement processing.

[0063] The improved UNet network framework of the second branch takes the original underwater pipeline image and the corresponding reference image as input to construct the posterior distribution network of UIE (underwater image enhancement). The posterior distribution network needs to consider not only the information of the original image, but also the additional guidance information provided by the reference image, so that the posterior distribution network can learn how to convert the original underwater pipeline image into a clearer and higher quality pipeline defect feature enhanced image.

[0064] The task difference between the prior distribution network of the first branch and the posterior distribution network of the second branch: the prior distribution network focuses on extracting useful features from the original image, while the posterior distribution network uses the useful features extracted from the original image and the reference image to generate an enhanced image. This division of labor enables each branch to perform at its best on its specific task.

[0065] The input difference between the prior distribution network of the first branch and the posterior distribution network of the second branch: the first branch only takes the original underwater pipeline image as input, while the second branch combines the reference image in addition to the original underwater pipeline image. This differentiated input allows each branch to optimize for its specific task.

[0066] The training objectives of the prior distribution network of the first branch and the posterior distribution network of the second branch are: the training objective of the first branch is to learn the feature representation of the original underwater pipeline image, while the training objective of the second branch is to learn how to generate high-quality enhanced images given the original underwater pipeline image and the reference image. The difference between these two objectives leads to different results for the two branches even with the same network structure.

[0067] Furthermore, the prior distribution network / the posterior distribution network includes the following steps: the improved UNet network framework includes the following steps: step S1, converting the number of channels of the original underwater pipeline image, or the original underwater pipeline image and the reference image through a series convolution block and a LeakyReLU activation function to generate a first feature map; step S2, extracting features from the first feature map through a dynamic convolution module to generate a second feature map; step S3, consolidating features of the second feature map through a maximum pooling layer to generate a third feature map; step S4, repeating steps S1 to S3 at least twice to generate a fourth feature map, wherein the number of convolution kernels of the dynamic convolution module increases successively; step S5, enhancing region recognition of the fourth feature map through a convolution attention module to generate a fifth feature map; step S6, passing the fifth feature map through an upsampling layer, feature splicing and a convolution layer to generate the pipeline defect feature extraction image or the pipeline defect feature enhanced image.

[0068] Furthermore, in step S2, weighted summation is performed on the convolution operation to generate the second feature map.

[0069] Specifically, in step S2, the second feature map is generated by performing convolution operation and weighted summation according to the following formula:

[0070]

[0071] In the formula, * represents the convolution operation, ω i is the weight vector, K is the number of convolution kernels, x represents input information, and y represents output information.

[0072] Specifically, the processing process of the dynamic convolution module in step S2 is as follows: (1) Assume that the given output is C×H×W, where C is the number of input channels and H×W is the input image resolution; (2) The input image is passed through a weight generator to obtain a weight vector ω, and if the convolution kernel is K, the size of ω is K; (3) Use the weight vector ω to weight each convolution kernel, and use the weighted convolution kernel for convolution operation to obtain an output feature map, where the formula for the convolution operation is: i =x*W i , W is a vector, x represents the training parameter, and y represents the output parameter; (3.4) The convolution results of multiple convolution kernels are weighted and summed to obtain the second feature map: In the formula, x*W i Represents the convolution operation, ω is the weight vector, and K is the number of convolution kernels.

[0073] Dynamic convolution dynamically generates weights and weights multiple convolution kernels, allowing the network to flexibly select convolution kernels according to the characteristics of the input data, thereby greatly enhancing the network's expressiveness. This mechanism enables dynamic convolution to better extract effective features when processing input data with different characteristics, while improving the network's adaptability and generalization ability to complex scenarios.

[0074] See also Figure 2 In step S4, preferably, the steps S1 to S3 are repeated three times, and the number of convolution kernels of the dynamic convolution module is increased each time, so as to improve the model performance, obtain stronger feature expression ability, and make the details and boundary information more obvious. The high-dimensional features of multiple channels provide richer information and can better support the processing of subsequent steps.

[0075] Furthermore, the convolutional attention module is used to collaboratively fuse spatial multi-scale features, process global features, and balance channel features. The method includes:

[0076] CBAM is an effective attention mechanism that improves the representation ability of the network by modeling the attention of space and channels. CBAM consists of two sub-modules: Figure 5 The channel attention module (Channel AttentionModule) shown in Figure 6The spatial attention module (Spatial Attention Module) shown in the figure, the connection structure of the channel attention module and the spatial attention module can refer to the structure of the conventional convolutional attention module CBAM, and the improvement of the convolutional attention module CBAM lies in the specific process of the channel attention module and the spatial attention module. The role of the channel attention module is to adaptively assign weights to each channel of the input feature map, so that the network can automatically focus on important channel features. The role of the spatial attention module is to weight each position of the feature map and emphasize the important areas in the image.

[0077] The step S5 includes: steps S51 to S53 belonging to the channel attention submodule, and steps S54 to S56 belonging to the spatial attention submodule, wherein steps S51 and S52 are specifically as follows: Figure 5 As shown, step S54 and step S55 are specifically as follows Figure 6 As shown, specifically:

[0078] Step S51, performing two-dimensional global average pooling and global maximum pooling on the fourth feature map element by element to generate a first descriptor and a second descriptor respectively;

[0079] Specifically, Figure 5 In step S51, the fourth feature map is subjected to global average pooling and global maximum pooling by the following formula:

[0080]

[0081] F max =max i,j F(i,j)

[0082] In the formula, F avg is the first descriptor generated by global average pooling, H and W are the two dimensions of the fourth feature map, and F max The second descriptor generated by global maximum pooling;

[0083] The results of these two pooling operations are both C×1×1 tensors, representing the global information of each channel.

[0084] Step S52: After the first descriptor and the second descriptor pass through the fully connected network, they are sequentially subjected to the ReLU activation function, channel attention calculation and Sigmoid activation function to generate channel attention weights;

[0085] Specifically, Figure 5 In step S52, after the first descriptor and the second descriptor are passed through a fully connected network, the following formula is input to generate a channel attention weight:

[0086] M c (F) = σ(W 1 (RELU(W 0 (F avg )))+W 1 (RELU(W 0 (F max ))))

[0087] Where M c (F) is the channel attention weight, σ represents the Sigmoid activation function, ReLU represents the ReLU activation function, W 0 and W 1 are two learnable weights, F avg is the first descriptor, F max is the second descriptor;

[0088] The generated result is mapped to [0,1] through the Sigmoid function to obtain a channel weight vector, which represents the importance of each channel.

[0089] Step S53, multiplying the channel weight vector and the fourth feature map element by element to generate a first enhanced feature map;

[0090] Specifically, the step S53 refers to the structure of the conventional convolutional attention module CBAM, which is not Figure 5 and 6 As shown in FIG. 1 , the step S53 performs channel weighting on the channel weight vector by the following formula to obtain a first enhanced feature map:

[0091]

[0092] Where M c (F) is the channel attention weight, F represents the fourth feature map, represents element-wise multiplication, F′ is the first enhanced feature map;

[0093] Step S54: performing three-dimensional global average pooling and global maximum pooling on the first enhanced feature map to generate a third descriptor and a fourth descriptor respectively;

[0094] Specifically, Figure 6 In step S54, the first enhanced feature map is subjected to global average pooling and global maximum pooling by the following formula:

[0095]

[0096]

[0097] In the formula, is the third descriptor generated by global average pooling, and C is the first enhanced feature Figure 1 dimensions, is the fourth descriptor generated by global maximum pooling, and F′ is the first enhanced feature map;

[0098] The results of these two pooling operations are both 1×H×W tensors.

[0099] Step S55, performing feature concatenation, convolution layer and activation function operations on the third descriptor and the fourth descriptor to generate a spatial attention weight;

[0100] Specifically, Figure 6 In step S55, the third descriptor and the fourth descriptor are subjected to feature concatenation, convolution layer and activation function operations through the following formula to generate a spatial attention weight:

[0101]

[0102] Where M s (F′) is the spatial attention weight, F′ is the first enhanced feature map, σ represents the Sigmoid activation function, Conv represents the convolutional layer, is the third descriptor, is the fourth descriptor, [·] represents feature concatenation in the channel dimension;

[0103] That is, the two pooling results are connected in the channel dimension to obtain a 2×H×W, and then passed through a 7×7 convolution layer to generate the spatial attention weight.

[0104] Step S56: Calculate the spatial attention weight and the first enhanced feature map to generate the fifth feature map;

[0105] Specifically, step S56 refers to the structure of the conventional convolutional attention module CBAM, which is not Figure 5 and 6 As shown in FIG. 1 , in step S56, the spatial attention weight and the first enhanced feature map are calculated by the following formula to generate the fifth feature map:

[0106]

[0107] Wherein, F″ represents the fifth characteristic graph, M s (F′) is the channel space attention weight, Represents element-wise multiplication.

[0108] CBAM is a lightweight and effective attention module that weights features in two dimensions, channel and space, improving the network's ability to focus on important information. CBAM can be seamlessly embedded in existing convolutional neural networks to improve their feature representation capabilities.

[0109] The above modules are integrated and placed into the improved GAN network framework to achieve an effective defect detection method for underwater pipeline images as follows: the dynamic convolution module is placed in the encoding stage of the GAN network framework, and the attention mechanism module of step (4) is placed in the encoding stage of the GAN network. The key component of the framework GAN network framework is the AdaIN module, which encodes ambiguity after the feature extractor. The previous and subsequent blocks are designed to build an enhanced distribution. After the feature extractor is completed, the AdaIN module performs an enhanced statistical transformation on the received deterministic features. In the training phase, random samples from the posterior distribution are injected into the AdaINGAN module to transform the enhanced representation. In the verification phase, random samples in the AdaIN module are used to generate predictions. After the AdaIN module, two concatenated 3×3 convolution blocks and LeakyReLU activation functions are obtained to obtain the complete output image, and finally a 1×1 convolution block connects the features to obtain the final defect detection effect image.

[0110] Furthermore, in the prior distribution network / the posterior distribution network, before executing step S1, the three primary colors of the original degraded image of the underwater pipeline are color corrected by adjusting the color saturation respectively to generate the original underwater pipeline image.

[0111] Specifically, before executing step S1, the original underwater pipeline image is color corrected by the following formula to generate the original underwater pipeline image:

[0112]

[0113] Wherein c represents one of the three primary colors of RGB, UC represents the corrected original underwater pipeline image, S represents the original degraded image of the underwater pipeline, M and V represent the mean and variance of the corresponding primary color c in the original degraded image S of the underwater pipeline, respectively, and μ represents the saturation of the underwater enhancement result.

[0114] It is understandable that due to the particularity of the underwater environment, such as changes in lighting conditions, absorption and scattering of water bodies, the images collected by the camera often have color deviation and low contrast. Therefore, these images need to be corrected through color correction formulas to restore their natural color and contrast, thereby significantly improving the quality of the input image and thus improving the performance of the entire network.

[0115] The overall steps of the underwater pipeline defect detection method are as follows: 1. Image acquisition: The original degraded underwater image is collected through underwater cameras or robots and other equipment. 2. Color correction: The original degraded underwater image is processed using the above color correction formula to generate a corrected image. 3. Input UNet: The corrected image is input into the UNet network of different branches for feature extraction and image enhancement respectively. 4. Feature extraction: The UNet network extracts multi-scale features of the image through its encoder part. 5. Image enhancement: Through the decoder part of UNet, combined with the features extracted by the encoder, an enhanced image is generated. 6. Post-processing: The enhanced image finally generated can be used for further defect detection or other tasks.

[0116] It is understandable that the application of the color correction formula in the UNet network framework is mainly to improve the quality of the input image and ensure that the network can learn more effective features from high-quality input. The original degraded underwater image of the water pipe is obtained through actual acquisition, and then preprocessed by the color correction formula to generate the corrected image, which is then input into the UNet network for feature extraction and image enhancement. This preprocessing step is crucial to improving the performance of the entire image enhancement system.

[0117] Furthermore, the step S6 includes the following steps: step S61, passing the fifth feature map through an upsampling layer to generate a sixth feature map; step S62, defining the input features of the step S3 as the first splicing feature to the Nth splicing feature in a repetitive order, and passing the sixth feature map through feature splicing with the Nth splicing feature, a convolution layer, an upsampling layer, feature splicing with the N-1th splicing feature, a convolution layer, and an upsampling layer in sequence, until feature splicing with the first splicing feature, to generate the pipeline defect feature extraction image or the pipeline defect feature enhanced image.

[0118] Furthermore, the pipeline defect feature extraction image and the pipeline defect feature enhanced image are subjected to enhanced statistical transformation through the Pr module, the Po module and the AdaIN module to generate the pipeline defect recognition image, comprising: the pipeline defect feature extraction image is input into the AdaIN module through the Pr module; the pipeline defect feature enhanced image is input into the AdaIN module through the Po module; the AdaIN module generates the pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map.

[0119] Furthermore, the AdaIN module generates a pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map, including:

[0120] The Po module receives the posterior distribution of the pipeline defect feature enhanced image generated by the original underwater pipeline image and the reference image, and extracts random samples therefrom and inputs them into the AdaIN module, so as to enable the AdaIN module to learn the enhanced image in the training stage;

[0121] A random sample of the pipeline defect feature extraction image is generated by the Pr module, and the random sample and the pipeline defect feature extraction image are input into the feature representation of the AdaIN module to enable the AdaIN module to learn enhanced images in the training phase and to enable the AdaIN module to make predictions in the testing phase.

[0122] Furthermore, the pipeline defect recognition image is sequentially connected through a series of convolution blocks and a LeakyReLU activation function, a convolutional attention module, and a convolutional layer to generate a final feature map, wherein the LeakyReLU activation function and the convolutional attention module are used to increase the nonlinearity of the feature, and the convolutional layer is used to reconstruct the details.

[0123] Furthermore, the underwater pipeline defect detection method is applied to a pipeline robot working underwater and equipped with a camera, and the pipeline defects include sediments, air flotation, penetration and gaps; the collected image of the pipeline robot is subjected to hybrid enhancement technology and multiple data enhancement technologies to generate the original underwater pipeline image.

[0124] Specifically, the collected images are preprocessed by hybrid enhancement technology and multiple data enhancement technologies to generate original underwater pipeline images.

[0125] Preferably, the pipeline robot is equipped with a high-resolution camera, and the robot is used to obtain various videos and spectrograms of underwater pipeline defects, with a video resolution of 1080×720 pixels and a duration ranging from 1 minute to 3 minutes, depending on the specific situation.

[0126] Preferably, the key defect images are extracted from each video using video frame extraction technology to obtain 1020 original defect images.

[0127] Since the training quality of the detection model is positively correlated with the size of the data set, it is difficult to obtain good training quality when the data set size is too small. Therefore, it is preferred to use data enhancement techniques including rotation, cropping, color adjustment, mosaic and mixing to increase the data set formed by the original underwater pipeline image to 5650 images. Mosaic enhancement technology is to randomly crop four images and combine them into a new image. Hybrid enhancement technology is to mix two images of different categories together to form a coherent image. The constructed data set is cleaned to remove data that does not meet the requirements. At the same time, a certain proportion of samples are extracted from the total samples as training sets, and the remaining samples are validation sets. The number of training sets and validation sets is preferably divided into 9 to 1.

[0128] Specifically, see Figure 2 The improved UNet network framework is also an improved GAN network framework based on deep learning technology. The input original underwater pipeline image / reference image is an RGB image of size C×H×W, where C is the number of channels, H is the image height, and W is the image width. The structure of the UNet network framework is similar to that of GAN and consists of two branches.

[0129] In the improved UNet network framework, the feature map of the input convolutional attention module CBAM is generated through three levels of convolutional layers, wherein the first level of convolutional layer is the starting part of the network, which is mainly used to extract primary features from the preprocessed original underwater pipeline image / reference image data. Primary features usually include edges, textures and other basic visual elements in the image. In order to effectively capture the primary features, this layer uses convolution operations and dynamic convolution operations with LeakyReLU activation functions. The LeakyReLU activation function does not completely suppress the signal when processing negative inputs, but gives it a small positive slope. This means that even negative inputs will be passed certain information, rather than being directly set to zero like standard ReLU. This feature helps alleviate the problem of "dead neurons", that is, some neurons are inactive for a long time during training, thereby improving the learning ability of the model.

[0130] In the first level convolution layer, dynamic convolution technology is also introduced. Dynamic convolution allows kernel weights to be adjusted in real time according to the characteristics of the input data, so that it can better adapt to the diversity of input data. An important feature of dynamic convolution is that its kernel size and step size can be dynamically adjusted according to the characteristics of the input data. This means that for different input samples, the model can choose the filter size and moving step that best suits the current situation. For example, when processing small-scale features containing rich details, a smaller kernel can be used to capture these details; while for larger-scale objects, the kernel size can be increased to obtain a wider field of view. The implementation of dynamic convolution usually relies on additional network structures, such as MLP (multi-layer perceptron), to generate convolution kernel weights for specific inputs. This approach allows the model to learn how to generate the optimal set of kernel weights according to the statistical characteristics of the input data during training. In addition, dynamic convolution can also be implemented in a conditional calculation manner, that is, a set of predefined convolution kernels are dynamically selected according to the input conditions. By enabling the convolution operation to respond to changes in the input data, dynamic convolution improves the flexibility and adaptability of the model. This adaptability is particularly important for processing diverse input data, because fixed convolution kernels may not fully capture all important patterns. Therefore, when faced with complex visual tasks, dynamic convolutional layers can provide stronger expressive power and higher performance.

[0131] Since dynamic convolution can adaptively adjust its kernel size and stride according to the input data, adding maximum pooling after dynamic convolution can further consolidate the extracted features and ensure that more abstract information can be processed in subsequent parts of the network. At the same time, maximum pooling can also help maintain the invariance of the features learned by dynamic convolution, thereby improving the robustness and generalization ability of the entire network.

[0132] The first convolution of the first convolution layer: use a 3×3 convolution kernel, a step size of 1, and the number of output channels is set to 64. Activation function: use LeakyReLU, which adds nonlinear characteristics and allows a certain amount of negative value transmission. The second convolution of the first convolution layer: follow with another convolution, the number of output channels is set to 128, and the same 3×3 convolution kernel and LeakyReLU activation are used.

[0133] Through two convolutions, the underlying features of the image are extracted layer by layer, including edges, textures, and shapes, ensuring rich feature information. The LeakyReLU activation function enables the network to capture more complex feature patterns, avoids the problem of neuron death, and improves the network's learning ability.

[0134] Then DynamicConv generates a dynamic convolution kernel based on the input feature map, usually using 1×1 convolution to generate multiple convolution kernels. The input feature map is convolved according to the generated convolution kernel to adapt to the different input features, thereby enhancing the adaptability of the network to the features. The maximum pooling operation uses a 2×2 pooling window and a step size of 2 to reduce the spatial resolution of the feature map. By selecting the maximum value in the pooling window, the most significant feature information is retained, enhancing the representativeness of the feature.

[0135] The second-level convolutional layer is similar to the first-level in structure and function, but the number of convolution kernels is increased to capture more complex feature relationships. This enables the second-level convolutional layer to learn more advanced features than the first-level convolutional layer, further abstracting the primary features extracted by the first level.

[0136] The first convolution of the second convolution layer: Use the 3×3 convolution kernel again, with a step size of 1 and the number of output channels set to 128. Activation function: Use LeakyReLU to enhance the nonlinear characteristics of the network. On the basis of the first level, extract deeper features and capture more complex structural information. With the increase in the number of channels, the network's expressive ability is further improved, which can better adapt to complex underwater scenes.

[0137] Then DynamicConv generates the corresponding convolution kernel based on the current input feature map, and performs convolution operations on the new feature map to adapt to the features of the current layer. This provides the network with stronger feature extraction capabilities and can more effectively capture subtle changes in underwater images. Dynamic convolution enables the network to maintain good performance on different feature maps.

[0138] Perform maximum pooling to further reduce the spatial size of the feature map and provide more streamlined input for subsequent layers. Through maximum pooling, the network is ensured to retain the most significant features, which helps to fuse and enhance subsequent features.

[0139] The second convolution of the second convolution layer: Continue convolution and set the number of output channels to 256.

[0140] The third-level convolutional layer further increases the number of parameters, aiming to further enhance the learning ability of the model. With the increase in the number of parameters, the third-level convolutional layer can learn more abstract and complex features. These features may involve a wider range of visual patterns, or even be understood as a high-level description of the entire object or scene.

[0141] The first convolution of the third convolution layer: use 3×3 convolution kernel, stride 1, and set the number of output channels to 256. Activation function: use LeakyReLU to increase nonlinearity. The second convolution layer: perform convolution, and set the number of output channels to 512. Extract more abstract features to capture global information and important contextual relationships. A higher number of channels further improves the feature expression ability of the network and adapts to more complex underwater image features. Dynamic convolution kernel generation: Generate new convolution kernels based on the current input. Perform maximum pooling again to further reduce the spatial dimension of the feature map, retain the most significant feature information, and lay the foundation for the processing of the feature enhancement path.

[0142] After three levels of convolution, the final feature map is sent to the convolutional attention module CBAM for further processing. The role of CBAM is to enhance the important information in the feature map, thereby improving the network's attention to key features. Through the combination of channel attention and spatial attention, the network can better understand the key information in the input image, which is crucial to improving the accuracy and quality of underwater image restoration. The design of CBAM not only improves the performance of the feature map, but also lays a solid foundation for subsequent processing steps.

[0143] The next upsampling step is usually to restore or increase the spatial resolution of the feature map so that feature fusion and object detection can be better performed. During the downsampling process, the spatial size of the feature map is reduced and some detail information is lost. Upsampling can help restore these details and improve the spatial resolution of the feature map.

[0144] The Pr module and the Po module are two key modules in the network, which play a vital role in the underwater image enhancement and target detection process. The Pr module is mainly after the feature extraction stage. The role of the Pr module is to generate random samples, which are used to generate prediction values ​​in the test phase. Specifically, after the feature extraction is completed, the Pr module converts the deterministic features into a form that matches the enhanced statistics through the PAdaIN (parametric adaptation and instance normalization) module. This means that in the Pr module, the samples drawn from the prior distribution will be used to guide how to adjust the features of the input image to make it more suitable for subsequent detection tasks.

[0145] The Po module mainly plays a role in the training phase. It receives random samples drawn from the posterior distribution and injects them into the AdaIN (Adaptive Instance Normalization) layer to transform the enhanced representation. In this way, the Po module helps the network learn how to effectively perform image enhancement, so that it can produce high-quality enhancement results when facing images under different conditions.

[0146] The design of the Pr and Po modules reflects the fine control of the feature flow within the network. The Pr module simulates the image by generating random samples, thereby enhancing the possible changes in the process, while the Po module ensures that the network can learn effective enhancement strategies by adjusting the feature representation. In addition, the introduction of the Pr and Po modules also enhances the generalization ability of the network because they allow the network to see diverse inputs during training, so that it can also handle unseen situations during testing.

[0147] The AdaIN module is an important component following the Pr module and the Po module. It plays a central role in the image enhancement process, especially when it comes to transforming features to match the enhancement statistics.

[0148] The main function of the AdaIN module is to achieve feature stylization by adaptively adjusting the mean and variance of the feature map, thereby changing the statistical properties of the input feature map to make it consistent with the target style. In the network, the AdaIN module is used to inject random samples generated by the Pr module into the feature representation to achieve image enhancement. During the training phase, this process enables the model to learn how to effectively enhance images; while in the test phase, predictions are made through random samples generated by the Pr module.

[0149] After passing through the AdaIN module, it passes through two convolutional layers and the corresponding LeakyReLU activation function, which is used to further extract features and increase nonlinearity. Then it enters the CBAM module to further enhance the feature map. After passing through CBAM, the last Conv layer fuses the features output by CBAM with the previously extracted features to generate the final feature map. This layer is also responsible for converting the feature map into an output image with the same resolution as the input image for detail reconstruction. After this image enhancement operation, pipeline defects can be detected more accurately through the enhanced image.

[0150] See also Figure 3 and 4 , set the training parameters to train the model: set the learning rate to Lr, the loss function to L, the Batch-Size to n, the training set and the validation set to 9:1, the optimizer to D, the total number of training rounds to E, and then train the deep learning network model as follows: (1) The learning rate Lr is initially 0.0001, the loss function is selected as Hunber_Loss, the optimizer is selected as Adam, and the hyperparameters such as the training round, batch size Batch-Size and input image size InputSize are set; (2) The final improved network model is obtained through iterative training, and then the trained network model is used to enhance and detect the underwater pipeline images of the validation set to obtain the final detection result figure.

[0151] Understandably, a major challenge of underwater image enhancement is the lack of high-quality paired data, and it is usually difficult to obtain clear "ground truth" images. Traditional supervised learning methods require a large amount of labeled data, which is difficult to obtain in underwater environments. This network uses similar adversarial learning and self-supervised learning strategies to enable it to learn the task of underwater image enhancement without paired clear images. This method uses an imaging model to generate degraded images, and adaptively optimizes the underwater restoration and degradation processes through adversarial learning, ultimately achieving image quality improvement. This self-supervision feature not only reduces the need for a large amount of labeled data, but also improves the generalization ability in complex scenes, making it adaptable to different underwater environments and lighting conditions. The network sets up jump connections between each convolution module, which can directly pass the underlying detail features to the decoding part, avoiding the information loss caused by layer-by-layer encoding. This structure can ensure the complete transmission of multi-scale information, so that the final output image has better detail representation, especially when dealing with complex underwater environments, retaining key color and edge information. Dynamic convolution and CBAM attention modules are added to the network, which help the network focus on key areas in the image more effectively. CBAM, through the combination of channel attention and spatial attention, gives the network stronger selectivity in the feature extraction process, can give priority to degraded areas and target objects, and further improve the restoration effect. Dynamic convolution can adapt to different feature patterns and better cope with the complex lighting changes and noise of underwater images.

[0152] In addition, the multifunctional nature of the network makes it suitable for complex underwater image enhancement and defect detection applications. Through dual-path structure and feature fusion, it can not only enhance the clarity of the image, but also improve the recognition rate when detecting underwater pipeline defects, corrosion or other structural damage. This feature is of great help to applications such as underwater detection and engineering maintenance, and improves overall work efficiency.

[0153] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0154] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for underwater pipeline defect detection based on an improved UNet model, characterized in that: include: In the first branch, a prior distribution network for feature extraction is constructed based on the improved UNet network framework, and features of the original underwater pipeline image are extracted and reconstructed through the prior distribution network to generate a pipeline defect feature extraction image; In the second branch, a posterior distribution network for underwater image enhancement is constructed based on the improved UNet network framework, and features of the original underwater pipeline image and the reference image are enhanced by the posterior distribution network to generate a pipeline defect feature enhanced image; Through the Pr module, the Po module and the AdaIN module, the pipeline defect feature extraction image and the pipeline defect feature enhanced image are enhanced and statistically transformed to generate a pipeline defect recognition image.

2. The underwater pipeline defect detection method based on the improved UNet model according to claim 1 is characterized in that: The prior distribution network / the posterior distribution network includes the following steps: Step S1, converting the number of channels of the original underwater pipeline image, or the original underwater pipeline image and the reference image, by serially connecting a convolution block and a LeakyReLU activation function to generate a first feature map; Step S2: extracting features from the first feature map through a dynamic convolution module to generate a second feature map; Step S3, consolidating the features of the second feature map through a maximum pooling layer to generate a third feature map; Step S4, repeating the steps S1 to S3 at least twice to generate a fourth feature map, wherein the number of convolution kernels of the dynamic convolution module increases successively; Step S5, enhancing region recognition by a convolutional attention module on the fourth feature map to generate a fifth feature map; Step S6: Pass the fifth feature map through an upsampling layer, a feature concatenation layer, and a convolution layer to generate the pipeline defect feature extraction image or the pipeline defect feature enhanced image.

3. The underwater pipeline defect detection method based on the improved UNet model according to claim 2 is characterized in that: In the step S2, weighted summation is performed on the convolution operation to generate the second feature map.

4. The underwater pipeline defect detection method based on the improved UNet model according to claim 2 is characterized in that: The convolutional attention module includes a channel attention submodule and a spatial attention submodule, and step S5 includes steps S51 to S53 belonging to the channel attention submodule and steps S54 to S56 belonging to the spatial attention submodule, specifically: Step S51, performing two-dimensional global average pooling and global maximum pooling on the fourth feature map element by element to generate a first descriptor and a second descriptor respectively; Step S52: After the first descriptor and the second descriptor pass through the fully connected network, they are sequentially subjected to the ReLU activation function, channel attention calculation and Sigmoid activation function to generate channel attention weights; Step S53, multiplying the channel weight vector and the fourth feature map element by element to generate a first enhanced feature map; Step S54: performing three-dimensional global average pooling and global maximum pooling on the first enhanced feature map to generate a third descriptor and a fourth descriptor respectively; Step S55, performing feature concatenation, convolution layer and activation function operations on the third descriptor and the fourth descriptor to generate a spatial attention weight; Step S56: Calculate the spatial attention weight and the first enhanced feature map to generate the fifth feature map.

5. The underwater pipeline defect detection method based on the improved UNet model according to claim 2 is characterized in that: In the prior distribution network / the posterior distribution network, before executing step S1, the three primary colors of the original degraded image of the underwater pipeline are color corrected by adjusting the color saturation respectively to generate the original underwater pipeline image.

6. The underwater pipeline defect detection method based on the improved UNet model according to claim 2 is characterized in that: The step S6 comprises the following steps: Step S61, passing the fifth feature map through an upsampling layer to generate a sixth feature map; Step S62, defining the input features of step S3 as the first stitching feature to the Nth stitching feature in a repetitive order, and subjecting the sixth feature map to feature stitching with the Nth stitching feature, a convolution layer, an upsampling layer, feature stitching with the N-1th stitching feature, a convolution layer, and an upsampling layer in sequence, until feature stitching with the first stitching feature is performed, to generate the pipeline defect feature extraction image or the pipeline defect feature enhanced image.

7. The underwater pipeline defect detection method based on the improved UNet model according to claim 6 is characterized in that: The pipeline defect feature extraction image and the pipeline defect feature enhancement image are subjected to enhanced statistical transformation by the Pr module, the Po module and the AdaIN module to generate the pipeline defect recognition image, including: The pipeline defect feature extraction image is input into the AdaIN module through the Pr module; The pipeline defect feature enhanced image is input into the AdaIN module through the Po module; The AdaIN module generates the pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map.

8. The underwater pipeline defect detection method based on the improved UNet model according to claim 7 is characterized in that: The AdaIN module generates a pipeline defect recognition image by adaptively adjusting the mean and variance of its input feature map, including: The Po module receives the posterior distribution of the pipeline defect feature enhanced image generated by the original underwater pipeline image and the reference image, and extracts random samples therefrom and inputs them into the AdaIN module, so as to enable the AdaIN module to learn the enhanced image in the training stage; A random sample of the pipeline defect feature extraction image is generated by the Pr module, and the random sample and the pipeline defect feature extraction image are input into the feature representation of the AdaIN module to enable the AdaIN module to learn enhanced images in the training phase and to enable the AdaIN module to make predictions in the testing phase.

9. The underwater pipeline defect detection method based on the improved UNet model according to any one of claims 1 to 8, characterized in that: The pipeline defect recognition image is sequentially connected through a series of convolution blocks and a LeakyReLU activation function, a convolutional attention module, and a convolutional layer to generate a final feature map, wherein the LeakyReLU activation function and the convolutional attention module are used to increase the nonlinearity of the feature, and the convolutional layer is used to reconstruct the details.

10. The underwater pipeline defect detection method based on the improved UNet model according to claim 5 is characterized in that: The underwater pipeline defect detection method is applied to a pipeline robot equipped with a camera working underwater, and the pipeline defects include sediment, air flotation, penetration and gap; The collected image of the pipeline robot is subjected to hybrid enhancement technology and multiple data enhancement technologies to generate the original degraded image of the underwater pipeline.

Images