An improved pipeline defect detection method based on RefineDet

Abstract

This invention discloses an improved pipeline defect detection method based on RefineDet. First, the backbone network of RefineDet is modified to a Swin-transformer. Second, the features output by the multi-scale feature fusion of the Neck module are used to classify and fine-tune the anchors of the anchor refinement module. This makes the Refinedanchors output by the anchor refinement module more accurate, thereby enabling the target detection module to better locate the target. Finally, a learning rate adjustment strategy of warmup and cosine is used, and the improved RefineDet network is trained using the Adamw optimizer. This improved method based on RefineDet can effectively improve the average accuracy of pipeline defect detection.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision target detection, and particularly relates to a pipeline defect detection method based on an improved RefineDet. Background Art

[0002] Pipelines are commonly used in the industrial sector to transport liquids and gases. Due to the complexity of the working environment and the materials being transported, they are prone to corrosion, blockage, and even rupture during use. Therefore, regular pipeline inspections are essential to ensure their service life and safety. Traditional manual methods for detecting pipeline defects are time-consuming and prone to false detections and missed detections. Therefore, intelligent defect detection methods are a better choice. Pipeline defect detection refers to the detection of surface defects on pipelines. Surface defect detection generally utilizes advanced artificial intelligence visual inspection technology to detect defects such as spots, pits, scratches, color differences, and defects on the workpiece surface.

[0003] The present invention uses an improved RefineDet target detection model to detect defects in pipelines. The RefineDet model can be regarded as a combination of SSD, RPN and FPN algorithms. Its main idea is: two-stage algorithms such as Faster-RCNN perform two regressions on the box, so the accuracy is high but the speed is slow; one-stage algorithms such as YOLO only perform one regression on the box, which is fast but the accuracy is low; RefineDet combines the two and performs two regressions on the box, but it is still a one-stage algorithm, which not only improves the accuracy but also is faster; the framework used by RefineDet is SSD, and the feature fusion operation of FPN is introduced to improve the detection effect of small targets. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide an improved method based on RefineDet that overcomes the problem of low average accuracy in target defect detection. This invention improves the average accuracy of pipeline defect datasets by modifying RefineDet's network architecture and training strategy, resulting in a higher accuracy rate for the trained deep convolutional neural network model, thereby enhancing the network model's ability to classify and locate pipeline defects. To achieve this objective, the present invention employs the following technical solutions:

[0005] An improved pipeline defect detection method based on RefineDet is characterized by comprising the following steps:

[0006] Step 1: Obtain pipeline defect dataset X s (Including N s samples) and the corresponding labels Y S ; Xs Divide into training set X in a ratio of 7:3 train (Including N train samples) and the test set X test (Including N test samples);

[0007] Step 2: Build an improved RefineDet model, which consists of four branches: backbone network, Neck module, anchor refinement module, and target detection module:

[0008] Step 2.1: Construct the backbone network; the backbone network is the Swin Transformer network, which constructs four feature extraction stages in a hierarchical manner, denoted as S1, S2, S3 and S4, and the output feature map of each stage is denoted as O s1 , O s2 , O s3 and O s4 Each stage consists of two parts: the patch merging layer and the SwinTransformer block. The number of patch merging layers in the four feature extraction stages is 1, and the number of SwinTransformer blocks is 2, 2, 6, and 2 respectively. Two SwinTransformer blocks form a group. The patch merging operation is to divide the feature map with a height, width, and number of channels of H×W×C into four The feature maps of size are then concatenated in the channel dimension to obtain The feature map of size is normalized and linearly transformed into The patch merging operation in the first stage is a linear transformation, and the size of the feature map remains unchanged. Therefore, the H×W×C of the patch merging layer input in each stage are 80×80×48, 80×80×96, 40×40×192, and 20×20×384 respectively; in a group of Swin Transformer blocks, the structure of the first Swin Transformer block is a normalization layer, a window multi-head attention module, a normalization layer, and a multi-layer perceptron module, and the structure of the second Swin Transformer block is a normalization layer, an offset window multi-head attention module, a normalization layer, and a multi-layer perceptron module;

[0009] Step 2.2: Construct the Neck module. The function of the Neck module is similar to that of the feature pyramid module. It enriches semantic information by adding high-level features to low-level features to improve detection accuracy. The Neck module contains four transmission connection blocks, which are denoted as T1, T2, T3 and T4, corresponding to S1, S2, S3 and S4 of the backbone network. T4 contains only one branch, O s4 As the input of this branch, after 3 convolution operations, the output feature map size remains unchanged, and the remaining transmission connection block T i (i=1,2,3) contains two branches, the first branch is O si As the input of this branch, the second branch takes T i+1 The output feature map of is taken as the input of this branch. Since the sizes of the input feature maps of the first branch and the second branch do not match, in order to match their dimensions, the first branch uses a deconvolution operation to enlarge the input feature map so that it is the same size as the input feature map of the second branch. Then the second branch performs two convolution operations on the input feature map. At this time, the size of the feature map output by this branch remains unchanged. Then, the outputs of the two branches are summed in an element-by-element addition manner. Finally, a convolution layer is added after the summation to ensure the distinguishability of the detection features. The outputs of the four transmission connection blocks T1, T2, T3 and T4 of the Neck module are respectively denoted as O t1 , O t2 , O t3 and O t4 ;

[0010] Step 2.3: Construct the anchor refinement module; the anchor refinement module provides better initialization for the bounding box regression in the target detection module by adjusting the position and size of the anchors; the anchor refinement module contains 4 convolution blocks, each of which is composed of two convolution layers; the 4 convolution blocks are used to input 4 different scale feature maps O t1 , O t2 , O t3 and O t4 Predict the position of anchors for fine-tuning and predict the confidence that the anchors are positive samples;

[0011] Step 2.4: Construct the target detection module; the target detection module is similar to the anchor refinement module in structure and also contains 4 convolution blocks, each of which consists of two convolution layers; the 4 convolution blocks are used to perform four different scale feature maps of the input. t1 , O t2 , O t3 and O t4 Predict the offset of the target border and the defect category of the target;

[0012] Step 3: The loss function includes the loss function of the anchor refinement module and the loss function of the target detection module, specifically:

[0013] Step 3.1: The anchor refinement module divides each anchor into two categories based on whether there is a target, and performs binary classification and regression loss calculation on the anchors; the loss functions used are cross entropy loss function and smooth L1 loss function respectively;

[0014] The cross entropy loss function is:

[0015]

[0016] Among them, y i is the label value, y' i is the predicted value, n is the training set X train The number of samples N train ;

[0017] The formula for the smooth L1 loss function is:

[0018]

[0019] in v=(v x , v y , v w , v h ) represents the center coordinate of the real bounding box (v x ,v y ) and the width v of the box w and height v h , Represents the center coordinates of the prediction box and the width of the box and height k∈{x,y,h,w}, where x, y, w, and h represent the coordinates of the center point of the box (x, y) and the width w and height h of the box;

[0020] Step 3.2: The loss functions of the target detection module are cross entropy loss function and smooth L1 loss function respectively;

[0021] The cross entropy loss function is:

[0022]

[0023] Among them, y i is the label value, y' i is the predicted value, n is the training set X train The number of samples N train ;

[0024] The formula for the smooth L1 loss function is:

[0025]

[0026] in v=(v x , v y , v w , v h ) represents the center coordinate of the real bounding box (v x ,v y ) and the width v of the box w and height v h , Represents the center coordinates of the prediction box and the width of the box and height k∈{x,y,h,w}, where x, y, w, and h represent the coordinates of the center point of the box (x, y) and the width w and height h of the box;

[0027] Step 4: Use the learning rate adjustment strategy of warmup and cosine and train the improved RefineDet network using the Adamw optimizer;

[0028] Step 5: Test the trained improved RefineDet network on the test set and calculate the average accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the basic method flow of the present invention;

[0030] Figure 2 Design of the overall network architecture for this article;

[0031] Figure 3 As the backbone network structure diagram;

[0032] Figure 4 This is the structure diagram of the transmission connection block included in the Neck module;

[0033] Figure 5 Refine the module structure diagram for anchor;

[0034] Figure 6 This is the structure diagram of the target detection module;

[0035] Table 1 shows the average accuracy results of the RefineDet model on the test set before and after improvement; DETAILED DESCRIPTION

[0036] The present invention provides an improved method based on RefineDet, which is explained and illustrated below with reference to the accompanying drawings:

[0037] The pipeline defect dataset used in the present invention contains 20 categories, a total of 6010 samples and corresponding labels, of which the training set samples are 4258 and the test set samples are 1752.

[0038] The embodiment of the present invention is as follows:

[0039] Step 1: Obtain pipeline defect dataset X s (Contains 6010 samples) and the corresponding label Y S (Contains 6010 samples); X s Divide into training set X in a ratio of 7:3 train (Contains 4258 samples and corresponding labels) and the test set X test (Contains 1752 samples and corresponding labels);

[0040] Step 2: Build an improved RefineDet model, which consists of four branches: backbone network, Neck module, anchor refinement module, and target detection module:

[0041] Step 2.1: Construct the backbone network; the backbone network is the Swin Transformer network, which constructs four feature extraction stages in a hierarchical manner, denoted as S1, S2, S3 and S4, and the output feature map of each stage is denoted as O s1 , O s2 , O s3 and O s4 Each stage consists of two parts: the patch merging layer and the SwinTransformer block. The number of patch merging layers in the four feature extraction stages is 1, and the number of SwinTransformer blocks is 2, 2, 6, and 2 respectively. Two SwinTransformer blocks form a group. The patch merging operation is to divide the feature map with the size of H×W×C in terms of the number of samples, height, width, and number of channels into four The feature maps of size are then concatenated in the channel dimension to obtain The feature map of size is normalized and linearly transformed into The patch merging operation in the first stage is a linear transformation, and the size of the feature map remains unchanged. Therefore, the H×W×C of the patch merging layer input in each stage are 80×80×48, 80×80×96, 40×40×192, and 20×20×384 respectively. In a group of Swin Transformer blocks, the structure of the first Swin Transformer block is a normalization layer, a window multi-head attention module, a normalization layer, and a multi-layer perceptron module. The structure of the second Swin Transformer block is a normalization layer, an offset window multi-head attention module, a normalization layer, and a multi-layer perceptron module.

[0042] Step 2.2: Construct the Neck module. The function of the Neck module is similar to that of the feature pyramid module. It enriches semantic information by adding high-level features to low-level features to improve detection accuracy. The Neck module contains four transmission connection blocks, which are denoted as T1, T2, T3 and T4, corresponding to S1, S2, S3 and S4 of the backbone network. T4 contains only one branch, O s4 As the input of this branch, after 3 convolution operations, the output feature map size remains unchanged, and the remaining transmission connection block T i (i=1,2,3) contains two branches, the first branch is O si As the input of this branch, the second branch takes T i+1 The output feature map of is taken as the input of this branch. Since the sizes of the input feature maps of the first branch and the second branch do not match, in order to match their dimensions, the first branch uses a deconvolution operation to enlarge the input feature map so that it is the same size as the input feature map of the second branch. Then the second branch performs two convolution operations on the input feature map. At this time, the size of the feature map output by this branch remains unchanged. Then, the outputs of the two branches are summed in an element-by-element addition manner. Finally, a convolution layer is added after the summation to ensure the distinguishability of the detection features. The outputs of the four transmission connection blocks T1, T2, T3 and T4 of the Neck module are respectively denoted as O t1 , O t2 , O t3 and O t4 ;

[0043] Step 2.3: Construct the anchor refinement module; the anchor refinement module provides better initialization for the bounding box regression in the target detection module by adjusting the position and size of the anchors; the anchor refinement module contains 4 convolution blocks, each of which is composed of two convolution layers; the 4 convolution blocks are used to input 4 different scale feature maps O t1 , O t2 , O t3 and Ot4 Predict the position of anchors for fine-tuning and predict the confidence that the anchors are positive samples;

[0044] Step 2.4: Construct the target detection module; the target detection module is similar to the anchor refinement module in structure and also contains 4 convolution blocks, each of which is composed of two convolution layers; the 4 convolution blocks are respectively used to input the feature maps of 4 different scales. t1 , O t2 , O t3 and O t4 Predict the offset of the target border and the defect category of the target;

[0045] Step 3: The loss function includes the loss function of the anchor refinement module and the loss function of the target detection module, specifically:

[0046] Step 3.1: The anchor refinement module divides each anchor into two categories based on whether there is a target, and performs binary classification and regression loss calculation on the anchors; the loss functions used are cross entropy loss function and smooth L1 loss function respectively;

[0047] The cross entropy loss function is:

[0048]

[0049] Among them, y i is the label value, y′ i is the predicted value, n is the training set X train Number of samples, a total of 4258 samples;

[0050] The formula for the smooth L1 loss function is:

[0051]

[0052] in v=(v x , v y , v w , v h ) represents the center coordinate of the real bounding box (v x ,v y ) and the width v of the box w and height v h , Represents the center coordinates of the prediction box and the width of the box and height k∈{x,y,h,w}, where x, y, w, and h represent the coordinates of the center point of the box (x, y) and the width w and height h of the box;

[0053] Step 3.2: The loss functions of the target detection module are cross entropy loss function and smooth L1 loss function respectively;

[0054] The cross entropy loss function is:

[0055]

[0056] Among them, y i is the label value, y′ i is the predicted value, n is the training set X train Number of samples, a total of 4258 samples;

[0057] The formula for the smooth L1 loss function is:

[0058]

[0059] in v=(v x , v y , v w , v h ) represents the center coordinate of the real bounding box (v x ,v y ) and the width v of the box w and height v h , Represents the center coordinates of the prediction box and the width of the box and height k∈{x,y,h,w}, where x, y, w, and h represent the coordinates of the center point of the box (x, y) and the width w and height h of the box;

[0060] Step 4: Use the learning rate adjustment strategy of warmup and cosine and train the improved RefineDet network using the Adamw optimizer;

[0061] Step 5: Test the trained improved RefineDet network on the test set, and the calculated average accuracy is 91.8%.

[0062] Table 1 Test results of different models

[0063]

[0064] The above examples are only used to illustrate the present invention, not to limit the technical solutions described in the present invention. Therefore, all technical solutions and improvements that do not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A pipeline defect detection method based on an improved RefineDet, characterized in that: The following steps are involved: Step 1: Obtain pipeline defect dataset X s And the corresponding label Y S ; X s Divide the training set into X in a ratio of 7:3 train and the test set X test ;X s Contains N s samples, training set X train Contains N train samples, test set X test Contains N test samples; Step 2: Build an improved RefineDet model. The improved RefineDet model consists of four branches: backbone network, Neck module, anchor refinement module, and target detection module: Step 3: The loss function includes the loss function of the anchor refinement module and the loss function of the target detection module, specifically: Step 3.1: The anchor refinement module divides each anchor into two categories based on whether there is a target, and performs binary classification and regression loss calculation on the anchors; the loss functions used are cross entropy loss function and smooth L1 loss function respectively; The cross entropy loss function is: Among them, y i is the label value, y′ i is the predicted value, n is the training set X train The number of samples N train ; The formula for the smooth L1 loss function is: in v=(v x , v y , v w , v h ) represents the center coordinate of the real bounding box (v x , v y ) and the width v of the box w and height v h , Represents the center coordinates of the prediction box and the width of the box and height x, y, w, and h represent the coordinates of the center point of the box (x, y) and the width w and height h of the box; Step 3.2: The loss functions of the target detection module are cross entropy loss function and smooth L1 loss function respectively; The cross entropy loss function is: Among them, y i is the label value, y′ i is the predicted value, n is the training set X train The number of samples N train ; The formula for the smooth L1 loss function is: in v=(v x , v y , v w , v h ) represents the center coordinate of the real bounding box (v x , v y ) and the width v of the box w and height v h , Represents the center coordinates of the prediction box and the width of the box and height x, y, w, and h represent the coordinates of the center point of the box (x, y) and the width w and height h of the box; Step 4: Use the learning rate adjustment strategy of warmup and cosine and train the improved RefineDet network using the Adamw optimizer; Step 5: Test the trained improved RefineDet network on the test set and calculate the average accuracy.

2. The pipeline defect detection method based on the improved RefineDet according to claim 1, characterized in that: Step 2.1: Construct the backbone network; the backbone network is the Swin Transformer network, which constructs four feature extraction stages in a hierarchical manner, denoted as S1, S2, S3 and S4, and the output feature map of each stage is denoted as O s1 , O s2 , O s3 and O s4 ; Each stage consists of two parts, namely the Patch Merging layer and the SwinTransformer block; The number of patch merging layers in the four feature extraction stages is 1, and the number of Swin Transformer blocks is 2, 2, 6, and 2 respectively, with two Swin Transformer blocks forming a group. The patch merging operation is to divide the feature map with a height, width, and number of channels of C×H×W into four The feature maps of size are then concatenated in the channel dimension to obtain The feature maps of different sizes are normalized and linearly transformed into In a set of Swin Transformer blocks, the structure of the first Swin Transformer block is a normalization layer, a window multi-head attention module, a normalization layer and a multi-layer perceptron module, and the structure of the second Swin Transformer block is a normalization layer, an offset window multi-head attention module, a normalization layer and a multi-layer perceptron module; Step 2.2: Construct the Neck module. The Neck module enriches semantic information by adding high-level features to low-level features. The Neck module contains four transmission connection blocks, which are denoted as T1, T2, T3 and T4, corresponding to S1, S2, S3 and S4 of the backbone network. T4 contains only one branch, O s4 As the input of this branch, after 3 convolution operations, the output feature map size remains unchanged, and the remaining transmission connection block T i Contains two branches, i = 1, 2, 3, the first branch is O si As the input of this branch, the second branch takes T i+1 The output feature map of is taken as the input of this branch. Since the sizes of the input feature maps of the first branch and the second branch do not match, in order to match their dimensions, the first branch uses a deconvolution operation to enlarge the input feature map so that it is the same size as the input feature map of the second branch. Then the second branch performs two convolution operations on the input feature map. At this time, the size of the feature map output by this branch remains unchanged. Then, the outputs of the two branches are summed in an element-by-element addition manner. Finally, a convolution layer is added after the summation to ensure the distinguishability of the detection features. The outputs of the four transmission connection blocks T1, T2, T3 and T4 of the Neck module are respectively denoted as O t1 , O t2 , O t3 and O t4 ; Step 2.3: Construct the anchor refinement module; the anchor refinement module provides better initialization for the bounding box regression in the target detection module by adjusting the position and size of the anchors; the anchor refinement module contains 4 convolution blocks, each of which is composed of two convolution layers; the 4 convolution blocks are used to input 4 different scale feature maps O t1 , O t2 , O t3 and O t4 Predict the position of anchors for fine-tuning and predict the confidence that the anchors are positive samples; Step 2.4: Construct the target detection module; the target detection module is similar to the anchor refinement module in structure and also contains 4 convolution blocks, each of which consists of two convolution layers; the 4 convolution blocks are used to perform four different scale feature maps of the input. t1 , O t2 , O t3 and O t4 Perform target bounding box offset prediction and target defect category prediction.

Images