A Pipe Weld Image Segmentation Method Based on Improved UNet

Abstract

This invention proposes a pipeline weld image segmentation method based on an improved UNet. The method first preprocesses the acquired pipeline weld image data and establishes a dataset. Then, it constructs an improved UNet network pipeline weld image segmentation model. Specifically, it first utilizes densely connected blocks combined with a parallel attention mechanism to build a network downsampling module, enhancing the model's feature extraction capability. Next, it replaces ordinary convolution with depthwise separable convolution, reducing the computational cost. Finally, it employs a novel loss function combining weighted Dice and cross-entropy to mitigate the imbalance between weld and non-weld regions in the pipeline weld image. Finally, it trains the model's parameters using the training dataset and inputs the test set images into the trained network model to obtain the segmented images. The segmentation model of this invention achieves high accuracy and good segmentation results for pipeline weld images.

Description

Technical Field

[0001] This invention relates to the field of pipeline weld image segmentation and deep learning, specifically to a pipeline weld image segmentation method based on an improved UNet. Background Technology

[0002] The quality of pipeline welds plays a crucial role in the safety of oil and gas pipeline transportation. Non-destructive testing (NDT) can detect the type and nature of internal defects in pipeline welds without damaging the weld structure, and it has become the main method for pipeline weld inspection. X-ray digital imaging inspection is one of the important methods for NDT of oil and gas pipeline welds. This method first uses an X-ray machine to irradiate the pipeline weld, and then uses X-ray film to examine the weld defects. Traditional manual image interpretation is inefficient and greatly affected by subjective factors, making it difficult to meet the needs of industrial applications. Therefore, it is urgent to realize the automatic detection and recognition of pipeline weld image defects using computers. X-ray weld images have a lot of background interference information, making it difficult to directly detect and identify weld defects. Therefore, scholars at home and abroad use image segmentation methods in computer vision to extract accurate weld locations to more effectively assist in the identification and detection of weld defects.

[0003] In recent years, breakthroughs have been made in weld seam image segmentation algorithms based on deep learning. Among them, "Wang Rui, Hu Yunlei, Li Haitao, Gao Shaoze, Wang Gang. Defect Instance Segmentation Algorithm for X-ray Weld Seam Images [J]. Journal of Harbin Institute of Technology, 2022, Vol. 54(5):140-145, 116" utilizes a lightweight residual structure combined with a multi-layer visual fusion mechanism FPN to achieve accurate segmentation of weld seam regions in low-cost hardware; "Xia Junfei. Image Segmentation of Circular Weld Seam on Inner Wall of Cylindrical Pipe Based on Deep Learning Autoencoder Network [D]. Yunnan Normal University, 2021" proposes an image autoencoder network based on circular convolution operation to achieve effective segmentation of weld seam images; "Bin Wang, Fengshun Li, Rongjian Lu, Xiaoyu, Wenhan Zhu. Weld Feature Extraction Based on Semantic Segmentation" Network[J].Sensors,2022,Vol.22(11):4130》Designed a lightweight ERF-Net network that combines channel attention mechanism, which improves segmentation efficiency and achieves a segmentation accuracy of 96%; however, its original deep learning is insufficient for feature extraction of weld seams, and is greatly affected by background factors of weld seam images, so the segmentation accuracy still needs to be improved. Summary of the Invention

[0004] To overcome the shortcomings of existing methods, a pipe weld image segmentation method based on an improved UNet network is proposed. First, a densely connected network is used as the main downsampling module of the encoder to extract global features of the weld image. Combined with a parallel attention mechanism, the network encoder's perception and feature extraction capabilities for the weld region are improved, thus solving the problems of unclear gray-level distinction and large background interference in the weld region. Then, depthwise separable convolutions are used to replace the original convolution blocks. The presence of different convolution kernels reduces the number of model parameters and improves computational efficiency while extracting more weld features. Finally, the DSen-UNet network is trained by combining the cross-entropy loss function and Dice Loss to solve the problem of the imbalance between the weld region and the background region.

[0005] To achieve the aforementioned objective, the provided technical solution is a pipe weld image segmentation method based on an improved UNet, characterized by the following steps:

[0006] Step 1: Acquire pipeline weld image data and preprocess the image data using an image viewer. The specific method is as follows:

[0007] (1) Enhance and denoise the obtained weld image, and adjust the image to an n×n pixel size, where n∈Z. + ;

[0008] (2) The processed images are labeled with weld segmentation according to the standards provided by the weld defect detection personnel, and the labeled images are divided into training set and test set;

[0009] Step 2: Construct an improved UNet weld seam segmentation network. The specific processing method is as follows:

[0010] (1) Construct an improved dense connection block, which consists of depthwise separable convolutions ConvN×N and a designed parallel attention mechanism SENet. The computational structure of the convolutional structure is as follows:

[0011] x output =S(Conv N×N (x input ))

[0012] In the formula, x input The input image representing the model, x output The output image of the representative model, Conv N×N For depthwise separable convolution, N∈Z + S(·) represents the parallel attention mechanism SENet;

[0013] (2) Constructing the parallel attention mechanism SENet, which consists of channel attention, spatial attention, and a fusion mechanism. The structure of the channel attention is as follows:

[0014]

[0015] In the formula, Let j be the weight parameter of the i-th channel weight, where j∈[1,k], i∈(0,j). Let |A| be the set of global feature maps of the k neighboring channels of the i-th channel, C be the number of channels, and Y and b be hyperparameters. odd Let ω, k, C, Υ, b ∈ Z be the nearest odd number adjacent to A. + ;

[0016] The spatial attention structure is as follows:

[0017]

[0018] In the formula, σ is the ReLU activation function. is the dot product of corresponding data points in the two feature maps, where F is the input feature map;

[0019] The fusion mechanism consists of Concat feature fusion and the SeLU activation function. The Concat operation formula is as follows:

[0020]

[0021] In the formula, X and Y are both feature maps input to the channel and X = Y, k represents the number of channels of the input feature map X, d is the number of channels of the input feature map Y, i∈[1,k], j∈[1,d];

[0022] The SeLU activation function is:

[0023]

[0024] In the formula, z is the model output feature map, λ∈Z, α∈Z + ;

[0025] (3) The UNet segmentation network includes upsampling, downsampling and skip connection structures. Upsampling is composed of improved dense connection blocks and depthwise separable convolutions. Downsampling is composed of Conv1×1 convolutions and depthwise separable convolutions. Upsampling and downsampling are skipped by Concat.

[0026] (4) A mixed loss function is constructed by weighting the cross-entropy function and the Dice Loss used to calculate the F1 score of the model training. The mixed loss function is as follows:

[0027]

[0028] Where X represents the pixel prediction category; Y represents the pixel label category; X∩Y is the intersection of the two; |X|+|Y| is the number of elements in the union of the two; H and C are the width and height of the weld image; y represents the pixel label category; y′ represents the pixel prediction category; λ is the proportion of the weld region in the entire weld image; 1-λ is the proportion of the non-weld region in the entire weld image, λ∈(0,1);

[0029] Step 3: Input the training set into the segmentation network model built in Step 2 for training. Select the model with the smallest loss function during the training process.

[0030] Step 4: Input the test set into the trained parameter model to perform weld seam segmentation and obtain the weld seam segmentation results.

[0031] The innovation of this invention is reflected in:

[0032] (1) The weld seam segmentation method of the present invention adopts a dense connection structure combined with a parallel attention mechanism and uses depth-separable convolution as the main convolution module of the network, which enhances the network feature extraction capability while slightly increasing or not increasing the number of parameters.

[0033] (2) The weld seam segmentation method of the present invention adopts a parallel attention mechanism that simultaneously considers the dual factors of spatial attention and channel attention of the model;

[0034] (3) The weld seam segmentation method of the present invention uses Dice Loss and cross-entropy function to construct a hybrid loss function to train the recognition model;

[0035] Beneficial effects:

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] (1) This invention studies the performance of deep learning weld segmentation model when facing weld images with large differences between background and foreground. It uses dense connection structure combined with parallel attention mechanism and uses depth separable convolution as the main convolution module of the network to extract weld features of the network, so that the weld image method proposed in this invention has a good weld segmentation effect when facing weld images with large differences between background and foreground.

[0038] (2) The hybrid loss function constructed by Dice Loss and cross-entropy function proposed in this invention can effectively solve the problem of imbalance between the proportion of weld area and background area; Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the specific implementation of the present invention;

[0040] Figure 2 This is a diagram of the improved UNet network structure for the pipeline weld seam image segmentation method.

[0041] Figure 3 This is a training and testing loss function curve of the weld seam segmentation method of the present invention;

[0042] Figure 4 This is a comparison of the segmentation effects of the weld segmentation method of the present invention and the weld segmentation method. In the figure, (a) original image, (b) manually annotated, (c) SegNet, (d) DeepLab, (e) UNet, and (f) model of the present invention.

[0043] Figure 5 This is a comparison of the segmentation confusion matrix between the weld segmentation method of the present invention and the weld segmentation method. In the figure, (a) SegNet, (b) DeepLab, (c) UNet, and (d) the model of the present invention. Detailed Implementation

[0044] This invention is a method for segmenting pipe weld images based on an improved UNet, the specific process of which is as follows: Figure 1 As shown, the feature includes the following steps:

[0045] Step 1: Acquire pipeline weld image data and preprocess the image data using an image viewer. The specific method is as follows:

[0046] (1) Enhance and denoise the obtained weld image, and adjust the image to an n×n pixel size, where n∈Z. + ;

[0047] (2) The processed images are labeled with weld segmentation according to the standards provided by the weld defect detection personnel, and the labeled images are divided into training set and test set;

[0048] Step 2: Construct an improved UNet weld seam segmentation network. The specific processing method is as follows:

[0049] (1) Construct an improved dense connection block, which consists of depthwise separable convolutions ConvN×N and a designed parallel attention mechanism SENet. The computational structure of the convolutional structure is as follows:

[0050] x output =S(Conv N×N (x input ))

[0051] In the formula, x input The input image representing the model, x output The output image of the representative model, Conv N×N For depthwise separable convolution, N∈Z +S(·) represents the parallel attention mechanism SENet;

[0052] (2) Constructing the parallel attention mechanism SENet, which consists of channel attention, spatial attention, and a fusion mechanism. The structure of the channel attention is as follows:

[0053]

[0054] In the formula, Let j be the weight parameter of the i-th channel weight, where j∈[1,k], i∈(0,j). Let |A| be the set of global feature maps of the k neighboring channels of the i-th channel, C be the number of channels, and Y and b be hyperparameters. odd Let ω, k, C, Υ, b ∈ Z be the nearest odd number adjacent to A. + ;

[0055] The spatial attention structure is as follows:

[0056]

[0057] In the formula, σ is the ReLU activation function. is the dot product of corresponding data points in the two feature maps, where F is the input feature map;

[0058] The fusion mechanism consists of Concat feature fusion and the SeLU activation function. The Concat operation formula is as follows:

[0059]

[0060] In the formula, X and Y are both feature maps input to the channel and X = Y, k represents the number of channels of the input feature map X, d is the number of channels of the input feature map Y, i∈[1,k], j∈[1,d];

[0061] The SeLU activation function is:

[0062]

[0063] In the formula, z is the model output feature map, λ∈Z, α∈Z + ;

[0064] (3) The UNet segmentation network includes downsampling, upsampling and skip connection structures. Downsampling is composed of improved dense connection blocks and depthwise separable convolutions. Upsampling is composed of Conv1×1 convolutions and depthwise separable convolutions. Upsampling and downsampling are skipped by Concat.

[0065] (4) A mixed loss function is constructed by weighting the cross-entropy function and the Dice Loss used to calculate the F1 score of the model training. The mixed loss function is as follows:

[0066]

[0067] Where X represents the pixel prediction category; Y represents the pixel label category; X∩Y is the intersection of the two; |X|+|Y| is the number of elements in the union of the two; H and C are the width and height of the weld image; y represents the pixel label category; y′ represents the pixel prediction category; λ is the proportion of the weld region in the entire weld image; 1-λ is the proportion of the non-weld region in the entire weld image, λ∈(0,1);

[0068] Step 3: Input the training set into the segmentation network model built in Step 2 for training. Select the model with the smallest loss function during the training process.

[0069] Step 4: Input the test set into the trained parameter model to perform weld seam segmentation and obtain the weld seam segmentation results.

[0070] In summary, this is merely a preferred embodiment of the recommended method for pipeline weld image segmentation and is not intended to limit the invention. All related modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the protection scope of this invention.

Claims

1. A pipeline weld seam image segmentation method based on improved UNet, characterized by The method comprises the following steps: Step 1: Obtain pipeline weld image data, and perform data preprocessing on the image by using an image browser, in particular the following manner: (1) The obtained weld image is enhanced and denoised, and the image is adjusted to the size of n x n pixels, n∈Z + ; (2) The processed image is segmented and labeled according to the standard provided by the weld defect detection staff, and the labeled image is divided into a training set and a test set; Step 2: Construct an improved UNet weld segmentation network, in particular the following manner: (1) Construct an improved dense connection block, which comprises a deep separable convolution ConvN×N and a designed parallel attention mechanism SENet, and the convolution structure is calculated as follows: x output = S(Conv N×N (x input )) In the formula, x input Input image of the model, x output Output image of the model, Conv N×N Depth separable convolution, N∈Z + S() is a parallel attention mechanism SENet; (2) Construct the parallel attention mechanism SENet, which comprises channel attention, spatial attention and a fusion mechanism, wherein the channel attention structure is as follows: In the formula, is the jth weight parameter of the ith channel weight, j ∈ [1, k], i ∈ (0, j), is the kth adjacent channel global feature map set of the ith channel, C is the channel number, Y and b are hyperparameters, |A| odd is the nearest odd number adjacent to A, ω, k, C, Y, b ∈ Z + ; The spatial attention structure is as follows: where σ is a ReLu activation function, is the dot product of the two feature map corresponding position data, F is the input feature map; The fusion mechanism comprises a Concat feature fusion and a SeLU activation function, and the operation formula of the Concat is as follows: In the formula, X and Y are feature maps input in the channel, X=Y, k represents the number of channels of the input feature map X, d is the number of channels of the input feature map Y, i∈[1,k], j∈[1,d]; The SeLU activation function is as follows: In the formula, z is a model output feature map, λ∈Z, α∈Z + ; (3) The UNet segmentation network comprises up-sampling, down-sampling and a skip connection structure, the down-sampling comprises an improved dense connection block and a deep separable convolution, the up-sampling comprises a Conv1×1 convolution and a deep separable convolution connection, and the up-sampling and the down-sampling are connected by a Concat for skip connection; (4) A hybrid loss function is constructed by respectively weighting a cross-entropy function and a Dice Loss for calculating the F1 score of the model, and the hybrid loss function is as follows: Wherein, X represents the pixel prediction category; Y represents the pixel label category; X∩Y represents the intersection of the two; |X|+|Y| represents the number of elements of the union of the two; H and C represent the width and height of the weld image; y represents the pixel label category; y' represents the pixel prediction category; λ represents the proportion of the weld area in the entire weld image; 1-λ represents the proportion of the non-weld area in the entire weld image, and λ∈(0,1); Step 3: Input the training set into the segmentation network model constructed in step 2 for training, and select the model with the minimum loss function in the training process as the trained model; Step 4: Input the test set into the trained parameter model for weld segmentation to obtain the weld segmentation result.

Images