A fine-grained segmentation method for internal tiny defects of spiral steel pipe weld

Abstract

The application provides a fine-grained segmentation method for internal tiny defects of spiral steel pipe welds, and relates to the technical field of nondestructive testing based on machine vision. In the case that internal tiny defects of spiral steel pipe welds in a static scene are difficult to distinguish, a fine-grained tiny defect segmentation network model is constructed, including a feature extraction network, a semantic discrimination encoder, a contrast conversion mechanism, a progressive iterative decoding strategy and a plurality of convolution modules. A semantic discrimination architecture is designed to adaptively initialize features guided by semantics and alleviate defect size differences. The contrast conversion mechanism is introduced to dynamically optimize defect features. Finally, the progressive iterative decoding strategy is used to perform fine-grained pixel-level segmentation on the fused feature map. The method can still achieve good segmentation effect under adverse scenes such as unbalanced sample category distribution, dense distribution of tiny defects, pixel redundancy and background interference, and effectively improves the segmentation performance of the network.

Description

Technical Field

[0001] The present invention relates to the technical field of non-destructive testing based on machine vision, and in particular to a fine-grained segmentation method for extremely small defects inside a spiral steel pipe weld. Background Art

[0002] With continuous technological advancements, spiral steel pipes have become widely used in various modern fields, including energy transportation and urban construction. However, during the steel pipe manufacturing process, dynamic changes in welding technology and production environments lead to a wide variety of complex and variable weld defects. These defects directly affect pipeline quality and can cause serious safety accidents. Therefore, timely and professional comprehensive inspection and evaluation of spiral welds is crucial.

[0003] Among various inspection technologies, X-ray-based nondestructive testing (NDT) dominates the quality control process for spiral steel pipes, providing accurate information for repair decisions. First, an X-ray detector receives X-rays emitted by an emitter inside the spiral steel pipe, which are then converted into image information by an image acquisition card. When encountering weld defects, human experts roughly estimate the size of the defect to guide secondary repair decisions. However, in actual manufacturing environments, the probability of minor defects greatly exceeds that of other types of defects. This seriously affects the judgment of professionals, resulting in subjectivity, inefficiency, false positives, false negatives, and inaccurate positioning results. Therefore, automated visual inspection is of great value for the fine-grained positioning of minor spiral weld defects.

[0004] Compared with other defect detection tasks, weld defect detection is more challenging due to the influence of factors such as welding process, X-ray detection, and dynamic digitization. To date, X-ray weld inspection technology has shown considerable advantages and promoted substantial progress in this particular field. However, there are significant differences in appearance and distribution between spiral steel pipe weld images and other weld images. When these models are directly applied for pixel-level segmentation, the detection performance will drop rapidly. In addition, the current spiral steel pipe weld defect detection methods rely on unsupervised and target detection frameworks. This rough positioning method is not sufficient to support accurate secondary repair decisions. Therefore, despite significant progress, the development of fine-grained tiny spiral steel pipe weld defect detection is still full of the following complex challenges and requires comprehensive exploration: (1) Dense distribution of small defects: Affected by the complexity of the welding process, insufficient exhaust time will cause a large number of small porosity defects. These porosity defects often only occupy a few pixels of the entire image, which can easily lead to missed detection and failure of weld quality assessment. (2) Strong coupling relationship. Spiral steel pipe weld X-ray images usually show low-quality contrast. In addition, the image blur caused by imaging brings obstacles to intelligent detection. These two characteristics lead to a tight coupling between the characteristics of small defects and low-quality welds, which often hinders the distinction between small defects and surrounding welds. (3) Small defects under interference. Unlike other types of weld images, due to the characteristics of X-ray imaging and the spiral pattern, X-ray images of spiral steel pipe welds usually show a large amount of pixel redundancy. These redundant areas introduce additional interference such as text, slag inclusions, and lighting changes.

[0005] Previous research has used end-to-end deep learning methods to identify defects in weld images, which is considered a promising approach for weld defect detection. Deep learning methods for weld images can be roughly categorized into two frameworks: region-level detection and pixel-level detection. Region-level detection frameworks only provide approximate bounding boxes and struggle to accurately describe defect size and boundaries, especially for small and irregular defects. Therefore, to obtain more accurate detection results to guide secondary repair decisions, researchers have increasingly focused on pixel-level detection frameworks. In the prior art, an end-to-end pixel-level localization architecture was designed for weld defect segmentation. This architecture detects weld defects through local feature mapping and improved skip connections, aiming to eliminate the limitations of manual interpretation. This network utilizes a sparse attention strategy to reduce the strong coupling between welds and defects, achieving good segmentation performance in the detection of in-service annular welds. However, this network does not account for the large intra-class variability of spiral steel pipe weld defect images, the large number of extremely small defects, and the interference of invalid regions. Therefore, when encountering small and scattered defects, the network struggles to effectively establish connections between discrete defects. Therefore, although the above methods have shown impressive performance in specific areas, they have limitations when directly applied to spiral steel pipe weld defect detection. Decoupling low-quality weld features from defect features to ensure accurate fine-grained defect segmentation is a major challenge facing spiral steel pipe welding systems. However, existing spiral steel pipe weld inspection methods generally rely on region-level inspection frameworks to roughly locate defects, which is insufficient to provide precise guidance for professionals to make secondary repair decisions. Therefore, there is an urgent need to research and develop a fine-grained defect detection framework specifically tailored for spiral steel pipe welds. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned existing technologies and provide a fine-grained segmentation method for extremely small defects inside spiral steel pipe welds, so as to solve the problem that in the actual setting of spiral welding defect detection, the segmentation performance of the segmentation network drops sharply under unfavorable scenarios such as unbalanced distribution of defect sample types and dense distribution of extremely small defects.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A fine-grained segmentation method for extremely small defects inside spiral steel pipe welds includes the following steps:

[0009] Step 1: Construct a dataset of internal defects of granular spiral steel pipe welds, including a training set and a test set. Both the training set and the test set consist of original weld defect images and pixel-level segmentation labels, and use the saliency detection pixel-level segmentation task as the basic data unit.

[0010] The pixel-level segmentation labels in the training set are the precise positions and sizes of the surface defects represented by the original weld defect images in the training set, and the pixel-level segmentation labels in the test set are the precise positions and sizes of the surface defects represented by the original weld defect images in the test set. In the saliency detection segmentation task, the defects represented by the original weld defect images in the training set and the original weld defect images in the test set are pixel-level binary classification tasks;

[0011] Step 2: Construct a fine-grained minimal defect segmentation network model, which consists of a feature extraction network, a semantic clue generator (SCG), a semantic discriminant encoder (SDE), a contrastive transformation mechanism (CTM), a progressive iterative decoding strategy (PID), and several convolutional modules (RBC).

[0012] The feature extraction network extracts the first-level, second-level, third-level and fourth-level feature maps of the original weld defect image respectively, and feeds the extracted feature maps of the original weld defect image into the semantic clue generator SCG to obtain the global semantic clue T scg ;

[0013] The semantic clue generator SCG includes a first dense receptive field DRF unit, a first dense attention guided DAG, a second dense receptive field DRF unit, a second dense attention guided DAG, a third dense receptive field DRF unit, a third dense attention guided DAG, a global average pooling layer and a 1*1 convolution layer; the first dense receptive field DRF unit, the first dense attention guided DAG, the second dense receptive field DRF unit, the second dense attention guided DAG, the third dense receptive field DRF unit and the third dense attention guided DAG are connected in sequence, the global average pooling layer and the 1*1 convolution layer are connected in sequence, and the two are then connected together in parallel, and the feature map of the extracted original weld defect image is fed into the semantic clue generator SCG to obtain a global semantic clue T scg ;

[0014] The semantic discriminant encoder SDE combines the second-level, third-level and fourth-level feature maps extracted by the feature extraction network with the global semantic clues T scg Connect in the channel dimension and apply 3*3 convolutional layers to obtain the three-level output of the semantic discriminant encoder SDE. The three-level output is connected with three contrast transformation mechanisms CTM respectively to obtain the output of three contrast transformation mechanisms CTM, i.e. multi-scale output. The multi-scale output and the first-level output of the feature extraction network are integrated and fed into the progressive iterative decoding strategy PID in sequence. The multi-scale output is fused in sequence to obtain the final segmentation map.

[0015] The contrast conversion mechanism CTM includes a group of 3*3 maximum pooling layers, a first decoupling mechanism and a first 2-fold upsampling bilinear interpolation layer connected in sequence, and a group of 3*3 average pooling layers, a second decoupling mechanism and a second 2-fold upsampling bilinear interpolation layer connected in sequence. The two groups are connected together in parallel and fused through a convolution module RBC; the multi-scale output is sequentially passed through the 3*3 maximum pooling layer, the first decoupling mechanism and the first 2-fold upsampling bilinear interpolation layer to obtain the maximum coupled smooth defect feature, and the multi-scale output is sequentially passed through the 3*3 average pooling layer, the second decoupling mechanism and the second 2-fold upsampling bilinear interpolation layer to obtain the average coupled smooth defect feature, and then the maximum coupled smooth defect feature and the average coupled smooth defect feature are fused through the convolution module RBC to obtain the three output features of the contrast conversion mechanism CTM;

[0016] The progressive iterative decoding strategy PID includes three sequentially connected cross-scale weighted modules CSWM; the cross-scale weighted module CSWM includes two sets of parallel spatial attention residual architectures, and two sets of parallel sequentially connected convolutional layers and element-level multiplications; the spatial attention residual architecture is implemented by spatial attention SA, element-level multiplication and element-level addition, and the spatial attention mechanism SA is used to calculate the low-level feature F low and high-level features F high , with low-level features F low and high-level features F high As input, spatial attention is applied to the two features to obtain two different attention maps, which are calibrated and residually connected with the original low-level features and high-level features to obtain interactive low-level verification features. and advanced calibration features The two different sets of spatial attention architectures are sequentially passed through a convolution module RBC, a 3*3 convolution layer and a Sigmoid activation function to generate weighted information W mid , the interactive low-level validation features and advanced calibration features Respectively with weighted information W mid and 1-W mid Multiply to obtain two sets of weighted feature maps, which are concatenated according to the channel dimension and then element-wise multiplied and added with another set of spatial attention residual architectures to obtain the output of the cross-scale weighted module CSWM.

[0017] The convolution module RBC consists of a 3*3 convolution layer, a BN normalization layer and a ReLu activation function connected in sequence;

[0018] Step 3: Take the original weld defect images and corresponding pixel-level segmentation labels in the training set as input to train the fine-grained minimal defect segmentation network model and obtain a converged trained fine-grained minimal defect segmentation network model;

[0019] Step 3.1: The original weld defect images and corresponding pixel-level segmentation labels in the training set are input into the fine-grained minimal defect segmentation network model as fine-grained minimal defect samples. The feature extraction network is used to extract the first-level, second-level, third-level, and fourth-level feature maps of the original weld defect images respectively.

[0020] Step 3.2: Feed the second-level, third-level, and fourth-level feature maps of the original weld defect images in the training set into the semantic clue generator SCG to obtain the global semantic clue T scg ;

[0021] Step 3.3: The first-level, second-level, third-level and fourth-level feature maps of the original weld defect images in the training set are respectively compared with the global semantic clues T scg Connect according to the channel dimension and apply 3*3 convolutional layers to obtain the three-layer output of the semantic discriminant encoder SDE. The three-layer outputs are denoted as F2~F4, and these three-layer outputs are collectively referred to as multi-scale output F;

[0022] Step 3.4: Inject the obtained multi-scale output F = {F2, F3, F4} into the contrast transformation mechanism CTM respectively to obtain the three output features of the contrast transformation mechanism CTM;

[0023] Step 3.4.1: Inject the obtained multi-scale outputs F = {F2, F3, F4} into the contrast transformation mechanism CTM respectively, and perform decoupling transformation on each multi-scale output F;

[0024] Step 3.4.2: Apply RBC blocks to the three multi-scale outputs after decoupling transformation to fuse them, perform 2*2 average pooling operation and 2*2 maximum pooling operation in parallel, and apply RBC blocks to the two pooled features to fuse them again to obtain two different pooled features {F max ,F avg}, the formula is as follows:

[0025]

[0026] Among them, F max represents the maximum pooling feature, F avg Represents the average pooling feature, Max(·) represents the 2*2 maximum pooling operation, and Avg(·) represents the 2*2 average pooling operation;

[0027] Step 3.4.3: Use the decoupling mechanism to decouple the maximum pooling feature and the average pooling feature, further determine the coupling relationship between the defect and the weld, and obtain the maximum coupling defect feature and the average coupling defect feature. The formula is as follows:

[0028]

[0029] in, represents the maximum coupling defect characteristic, represents the average coupling defect feature, σ represents the sigmoid activation function, Γ is a 3*3 convolutional layer with a batch normalization layer, ζ represents a 3*3 average pooling operation, and ⊙ represents element-wise multiplication;

[0030] Step 3.4.4: Maximum coupling defect characteristics and average coupling defect characteristics The maximum coupled smooth defect feature and the average coupled smooth defect feature are obtained by upsampling through the 2-fold upsampling bilinear interpolation layer, and the maximum coupled smooth defect feature and the average coupled smooth defect feature are fused through the convolution module RBC to generate the output feature of the contrast conversion mechanism CTM. Therefore, the three hierarchical features F = {F2, F3, F4} are input into the three contrast conversion mechanisms CTM respectively, that is, the output features F of the three contrast conversion mechanisms CTM are obtained. ctm ={F ctm2 ,F ctm3 ,F ctm4};

[0031] Step 3.5: Use the cross-scale weighted modules CSWM connected in sequence in the progressive iterative decoding strategy PID to calculate the first-level features of the feature extraction network and the output features of the three contrast transformation mechanisms CTM, gradually highlight the defect features, and obtain the output result F of each cross-scale weighted module CSWM cswm ={F cswm1 ,F cswm2 ,F cswm3}, and for feature F cswm1 Perform upsampling operation to obtain the final fusion feature map of the defect;

[0032] Step 3.5.1: Compute low-level features F using the spatial attention mechanism SA low and high-level features F high ; The input features are the first-level feature map of the extraction network and the output features F of the three contrast transformation mechanisms CTM ctm ={F ctm2 ,F ctm3 ,F ctm4}, the hierarchical features in the cross-scale weighted modules CSWM connected in sequence are {F ctm4 ,F ctm3}, {F cswm3 ,F ctm2}, {F cswm2 , first-level feature map}, among the above hierarchical features, the former are all from high-level features, namely high-level features F high ; The latter are all from low-level features, that is, low-level features F low ;

[0033] Step 3.5.2: Take low-level features F low and high-level features F high As input, spatial attention is applied to the two features to obtain two different attention maps, which are calibrated and residually connected with the original features to obtain interactive low-level verification features. and advanced calibration features The formula is as follows:

[0034]

[0035] Where SA(·) represents spatial attention;

[0036] Step 3.5.3: Check low-level features along the channel dimension and advanced calibration features Splicing is performed, and the splicing features are fused using the RBC module. The 3*3 convolution layer, batch normalization, and Sigmoid activation function are applied in sequence to activate the feature vector and generate the activation weight map W. mid , the formula is as follows:

[0037]

[0038] Step 3.5.4: Use 3*3 convolutional layers to fuse low-level features F low With high-level features F high , and use the activation weight map W mid Perform element-wise multiplication with the fused high-level features to obtain a high-level weighted feature map Using the weighted activation feature vector 1-W mid Multiply with the fused low-level features to obtain a low-level weighted feature map High-level weighted feature maps according to channel dimension and low-level weighted feature maps Perform splicing to obtain the intermediate fusion features. The formula is as follows:

[0039]

[0040] Among them, F mid Represents the generated intermediate fusion features;

[0041] Step 3.5.5: Using the intermediate fusion feature Fmid Advanced check features Calibration is performed by multiplying the advanced check features with the intermediate fusion features F mid to obtain the final fusion feature output F CSWM , as follows:

[0042]

[0043] where F CSWM represents the defective final fusion feature map;

[0044] Step 3.6: Feature decoding is performed on the defective final fusion feature map using a cross-scale weighting module CSWM in a progressive iterative decoding strategy PID, and a segmentation result of the training set image is calculated to obtain a supervision feature map;

[0045] Step 3.7: A loss between the supervision feature map and a pixel-level segmentation label of the training set is calculated through a cross-entropy loss function BCE and an intersection over union loss function IOU, and network model parameters in the fine-grained micro-defect segmentation network model are iteratively updated using a stochastic gradient descent algorithm, the training is stopped when the cross-entropy loss function converges, all trainable parameters of the network are frozen, and a converged trained fine-grained micro-defect segmentation network model is obtained.

[0046] Step 4: A test image in the test set is input into the converged trained fine-grained micro-defect segmentation network model as a defect sample, a test set segmentation map is obtained after calculation, and the accurate position and size of the defect corresponding to each test image in the test set are obtained.

[0047] The beneficial effects produced by the above technical solutions are as follows:

[0048] (1) The fine-grained micro-defect segmentation method of the present application adopts a semantic discrimination and contrast transformation strategy, uses defect data labeled at the pixel level of the weld as a training set to train the segmentation network, and realizes fine-grained segmentation of internal defects of the spiral steel pipe weld through the network. The method of the present application can still achieve good segmentation effect under adverse scenes such as unbalanced sample category distribution, dense micro-defect distribution, pixel redundancy and background interference.

[0049] (2) The method of the present application suppresses the expression of invalid area background features and irrelevant noise in the feature map through a semantic discriminative encoder (SDE) and a contrast transformation mechanism (CTM), thereby improving the model's attention to defects, activating effective defect information response, and providing more robust feature input for subsequent iterative decoding, thereby effectively improving the segmentation performance of the network.

[0050] (3) The method of the present application introduces a progressive iterative decoding strategy that refines the initial supervision clues through successive iterations using a cross-scale weighting module (CSWM). It is beneficial for dynamic adaptive optimization, multi-scale information decoding of defect features after feature matching, aggregation of defect information with different receptive fields, and processing of features combined with residual connection and hybrid attention mechanism to better decode defect information and ultimately generate fine-grained detection results. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 Flowchart of a fine-grained segmentation method for internal tiny defects of spiral steel pipe welds in the present embodiment;

[0052] Figure 2 Principle diagram of a fine-grained segmentation method for internal tiny defects of spiral steel pipe welds in the present embodiment;

[0053] Figure 3 Principle diagram of the semantic discriminative encoder architecture in the present embodiment;

[0054] Figure 4 Principle diagram of the progressive iterative decoding strategy PID in the present embodiment;

[0055] Figure 5 Principle diagram of the cross-scale weighting module in the present embodiment. DETAILED DESCRIPTION

[0056] The specific embodiments of the present application will be further described in detail below in conjunction with the drawings and examples. The following examples are used to illustrate the present application, but are not intended to limit the scope of the present application.

[0057] The core idea of ​​the present invention is: in the face of the imbalance of defect categories and dense distribution of extremely small defects in the weld of spiral steel pipes, a semantic discriminant coding strategy is adopted to fully activate the defect information in the defect image, enhance the model's semantic information association with defects, and reduce invalid background interference; then, the strong coupling association between the weld and the defect is used by the contrast conversion mechanism to obtain differentiated high-quality defect features; finally, a progressive iterative decoder and a cross-scale weighted module are used to decode the fused feature map and segment it to obtain the final output result.

[0058] like Figure 1 As shown, the method of this embodiment is as follows.

[0059] Step 1: Construct a granular spiral steel pipe weld internal defect dataset, including a training set and a test set. Both the training set and the test set consist of original weld defect images and pixel-level segmentation labels, and use the saliency detection pixel-level segmentation task as the basic data unit.

[0060] The pixel-level segmentation labels of the training set are the precise positions and sizes of the surface defects represented by the original weld defect images of the training set, and the pixel-level segmentation labels of the test set are the precise positions and sizes of the surface defects represented by the original weld defect images of the test set. In the saliency detection segmentation task, the defects represented by the original weld defect images of the training set and the original weld defect images of the test set are pixel-level binary classification tasks;

[0061] Since there is currently no segmentation dataset for extremely small defects inside spiral steel pipe welds, and there are relatively few samples of internal defects in spiral steel pipe welds, in order to solve this problem, this embodiment constructs a fine-grained spiral steel pipe weld internal defect dataset, namely the Tiny Spiral Weld X-ray dataset (T-SWX-ray), which fills the gap in the dataset of extremely small defects inside spiral steel pipe welds. The dataset consists of a training set and a test set. The types of severe defects contained in the above training set and test set are unbalanced, mainly extremely small porosity defects, which is consistent with the defect distribution in the actual working conditions of spiral steel pipes.

[0062] Step 2: Construct a fine-grained minimal defect segmentation network model; the fine-grained minimal defect segmentation network model consists of a feature extraction network, a semantic cues generator (SCG), a semantic discriminative encoder (SDE), a contrast transformation mechanism (CTM), a progressive iterative decoding strategy (PID), and several convolutional modules (RBC); the progressive iterative decoding strategy PID includes a cross-scale weighting module (CSWM);

[0063] The feature extraction network extracts the first-level, second-level, third-level and fourth-level feature maps of the original weld defect image respectively, and feeds the extracted feature maps of the original weld defect image into the semantic clue generator SCG to obtain the global semantic clue T scg ; The semantic clue generator SCG includes a first dense receptive field DRF unit, a first dense attention guided DAG, a second dense receptive field DRF unit, a second dense attention guided DAG, a third dense receptive field DRF unit, a third dense attention guided DAG, a global average pooling layer and a 1*1 convolution layer; the first dense receptive field DRF unit, the first dense attention guided DAG, the second dense receptive field DRF unit, the second dense attention guided DAG, the third dense receptive field DRF unit and the third dense attention guided DAG are connected in sequence, the global average pooling layer and the 1*1 convolution layer are connected in sequence, and the two are then connected together in parallel, and the feature map of the extracted original weld defect image is fed into the semantic clue generator SCG to obtain the global semantic clue T scg ;

[0064] The semantic discriminant encoder SDE combines the second-level, third-level and fourth-level feature maps extracted by the feature extraction network with the global semantic clues T scg Connect in the channel dimension and apply 3*3 convolutional layers to obtain the three-level output of the semantic discriminant encoder SDE. The three-level outputs are denoted as F2~F4. The three-level outputs are connected with three contrast transformation mechanisms CTM respectively to obtain the outputs of three contrast transformation mechanisms CTM, i.e. multi-scale outputs. The multi-scale outputs and the first-level output of the feature extraction network are integrated and fed into the progressive iterative decoding strategy PID in sequence. The multi-scale outputs are fused in sequence to obtain the final segmentation map.

[0065] The contrast conversion mechanism CTM includes a group of 3*3 maximum pooling layers, a first decoupling mechanism (DM) and a first 2-fold upsampling bilinear interpolation layer connected in sequence, and a group of 3*3 average pooling layers, a second decoupling mechanism and a second 2-fold upsampling bilinear interpolation layer connected in sequence. The two groups are connected together in parallel and fused through a convolution module RBC; the multi-scale output is sequentially passed through the 3*3 maximum pooling layer, the first decoupling mechanism (DM) and the first 2-fold upsampling bilinear interpolation layer to obtain the maximum coupling smooth defect feature, and the multi-scale output is sequentially passed through the 3*3 average pooling layer, the second decoupling mechanism and the second 2-fold upsampling bilinear interpolation layer to obtain the average coupling smooth defect feature, and then the maximum coupling smooth defect feature and the average coupling smooth defect feature are fused through the convolution module RBC to obtain the output feature of the contrast conversion mechanism CTM;

[0066] The progressive iterative decoding strategy PID includes three sequentially connected cross-scale weighted modules CSWM; the cross-scale weighted module CSWM includes two sets of parallel spatial attention residual architectures, and two sets of parallel sequentially connected convolutional layers and element-level multiplications; the spatial attention residual architecture is implemented by spatial attention SA, element-level multiplication and element-level addition, and the spatial attention mechanism SA is used to calculate the low-level feature F low and high-level features F high , with low-level features F low and high-level features F high As input, spatial attention is applied to the two features to obtain two different attention maps, which are calibrated and residually connected with the original low-level features and high-level features to obtain interactive low-level verification features. and advanced calibration features The two different sets of spatial attention architectures are sequentially passed through a convolution module RBC, a 3*3 convolution layer and a Sigmoid activation function to generate weighted information W mid , the interactive low-level validation features and advanced calibration features Respectively with weighted information W mid and 1-W mid Multiply to obtain two sets of weighted feature maps, which are concatenated according to the channel dimension and then element-wise multiplied and added with another set of spatial attention residual architectures to obtain the output of the cross-scale weighted module CSWM.

[0067] The convolution module RBC consists of a 3*3 convolution layer, a BN normalization layer and a ReLu activation function connected in sequence;

[0068] Step 3: The original weld defect images and corresponding pixel-level segmentation labels in the training set are input into the fine-grained minimal defect segmentation network model as fine-grained minimal defect samples, and the pixel-level segmentation map is calculated. The more accurate position and size of the corresponding defect in each original weld defect image are predicted step by step through iteration. By training the fine-grained minimal defect segmentation network model, a converged and trained fine-grained minimal defect segmentation network model is obtained.

[0069] Step 3.1: The original weld defect images and corresponding pixel-level segmentation labels in the training set are input into the fine-grained minimal defect segmentation network model as fine-grained minimal defect samples. The feature extraction network is used to extract the first-level, second-level, third-level, and fourth-level feature maps of the original weld defect images respectively.

[0070] In this embodiment, if Figure 2 As shown in the figure, the fine-grained minimal defect segmentation network model can use Convolutional Neural Networks (CNN), Transformer or Multi-Layer Perceptron (MLP) as the backbone network of the feature extraction network. The original weld defect images in the training set are input into the backbone network, and the feature extraction network is used to extract the first, second, third and fourth level feature maps of the original weld defect images respectively. The extracted training set image feature maps are denoted as T1 to T4.

[0071] Step 3.2: Feed the second-level, third-level, and fourth-level feature maps of the original weld defect images in the training set into the semantic clue generator SCG to obtain global semantic clues;

[0072] In this embodiment, if Figure 3 As shown in the figure, the second-level, third-level and fourth-level feature maps of the original weld defect images in the training set are spliced ​​along the channel dimension, and the global context information is extracted by cascading three dense receptive field DRF units and dense attention guided DAG in sequence. Each dense receptive field DRF unit consists of three parallel 3*3 expanded convolutional layers with an expansion rate r of (1, 2, 4). Formally, it can be expressed as:

[0073]

[0074] in, represents the output of the i+1th dense receptive field DRF unit, Concat[·] represents the concatenation operation along the channel dimension, represents dilated convolution with different dilation rates r = (1, 2, 4), (to denote the output of the i-th dense attention guided DAG, the process can be expressed as:

[0075]

[0076] Afterwards, the initial features Apply global average pooling and 1*1 convolution layer operations, and use bilinear upsampling function to obtain global semantic features in the spatial dimension. Finally, through multiple dense receptive field DRF units and dense attention guidance Adaptive aggregation of global semantic clues T scg , the whole process is defined as:

[0077]

[0078] Where RBC(·) consists of a convolutional layer, a batch normalization layer, and a ReLU activation function, C 1×1 represents a 1×1 convolution operation, and Gap represents global average pooling;

[0079] Step 3.3: The first-level, second-level, third-level and fourth-level feature maps of the original weld defect images in the training set are respectively compared with the global semantic clues T scg In the semantic discriminant encoder SDE, connections are made according to the channel dimension, and 3*3 convolutional layers are applied to obtain the three-layer outputs F2~F4 of the semantic discriminant encoder SDE, and these three-layer outputs are collectively referred to as multi-scale output F={F2,F3,F4};

[0080] Step 3.4: Inject the obtained multi-scale output F = {F2, F3, F4} into the three contrast transformation mechanisms CTM respectively to obtain the output features of the three contrast transformation mechanisms CTM;

[0081] Step 3.4.1: Inject the obtained multi-scale output F = {F2, F3, F4} into the three contrast transformation mechanisms CTM respectively, and perform decoupling transformation on each multi-scale output;

[0082] Step 3.4.2: Apply RBC blocks to each multi-scale output after decoupling transformation for fusion, perform 2*2 average pooling operation and 2*2 maximum pooling operation in parallel, and apply RBC blocks to the two pooled features to fuse them again to obtain two different pooled features {F max ,F avg}, the process can be expressed as:

[0083]

[0084] Among them, F max represents the maximum pooling feature, F avg Represents the average pooling feature, Max(·) represents the 2*2 maximum pooling operation, and Avg(·) represents the 2*2 average pooling operation;

[0085] Step 3.4.3: Use the decoupling mechanism to decouple the maximum pooling feature and the average pooling feature, further determine the coupling relationship between the defect and the weld, and obtain more distinct maximum coupling defect features and average coupling defect features. The formula is as follows:

[0086]

[0087] in, represents the maximum coupling defect characteristic, represents the average coupling defect feature, σ represents the sigmoid activation function, Γ is a 3*3 convolutional layer with a batch normalization layer, ζ represents a 3*3 average pooling operation, and ⊙ represents element-wise multiplication;

[0088] Step 3.4.4: Maximum coupling defect characteristics and average coupling defect characteristics The maximum coupled smooth defect feature and the average coupled smooth defect feature are obtained by upsampling through the 2-fold upsampling bilinear interpolation layer, and the maximum coupled smooth defect feature and the average coupled smooth defect feature are fused through the convolution module RBC to generate the output feature of the contrast conversion mechanism CTM. Therefore, the three hierarchical features F = {F2, F3, F4} are input into the three contrast conversion mechanisms CTM respectively, that is, the output features F of the three contrast conversion mechanisms CTM are obtained. ctm ={F ctm2 ,F ctm3 ,F ctm4};

[0089] Step 3.5: The key component of the progressive iterative decoding strategy PID, namely the sequentially connected cross-scale weighted modules CSWM, gradually highlights the defect features, and compares the first-level features F1 extracted by the feature extraction network and the output features F of the three contrast transformation mechanisms CTM. ctm ={F ctm2 ,F ctm3 ,F ctm4} to calculate and gradually highlight the defect characteristics, and obtain the output result F of each cross-scale weighted module CSWM cswm ={Fx swm1 ,F cswm2 ,F cswm3}, and for feature F cswm1 Perform upsampling operation to obtain the final fusion feature map of the defect;

[0090] In this embodiment, if Figure 4 and Figure 5As shown, the progressive iterative decoding strategy PID is composed of three cross-scale weighting modules CSWM connected in sequence, and four stages are called stage 1, stage 2, stage 3 and stage 4 respectively; the first level feature F1 of the feature extraction network is taken as the input feature of stage 1, and the three output features F ctm ctm2 ctm3 ctm4 of the contrast conversion mechanism CTM are taken as the input features of stage 2, stage 3 and stage 4 respectively, and the multi-scale defect features of the four stages are fused in sequence by the cross-scale weighting module CSWM to obtain the final segmentation result; the cross-scale weighting module CSWM takes the low-level feature F low and the high-level feature F high as input, applies spatial attention to the two features, and obtains two different attention maps; the two attention maps are respectively calibrated and residual connected with the original low-level feature and the high-level feature to obtain the interactive low-level verification feature F and the high-level verification feature F The process can be expressed as:

[0091]

[0092] Wherein SA(·) represents spatial attention;

[0093] The low-level feature F low and the high-level feature F high are calculated by using the spatial attention mechanism SA; the input features are the first level feature map of the extraction network and the output features F ctm ctm2 ctm3 ctm4 of the three contrast conversion mechanisms CTM, and the hierarchical features in the cross-scale weighting module CSWM connected in sequence are {F ctm4 ctm3}, {F cswm3 ctm2} and {F cswm2 , the first level feature map}; among the above hierarchical features, the former are all features from high level, and the latter are all features from low level, so as to facilitate description, the hierarchical input features of any cross-scale weighting module CSWM can be uniformly represented as low-level feature F low and high-level feature F high ;

[0094] In order to further enhance the interaction between scales, the low-level verification feature F and the high-level verification feature F ​​​​​​​​Splicing is performed, and the splicing features are fused using the RBC module. The 3*3 convolution layer, batch normalization, and Sigmoid activation function are applied in sequence to activate the feature vector and generate the activation weight map W. mid , the process can be expressed as:

[0095]

[0096] Use 3*3 convolutional layers to fuse low-level verification features and advanced calibration features And use the activation weight map W mid Perform element-wise multiplication with the fused advanced verification features to obtain an advanced weighted feature map Using the weighted activation feature vector 1-W mid Multiply with the fused low-level verification features to obtain a low-level weighted feature map High-level weighted feature maps according to channel dimension and low-level weighted feature maps Splicing is performed to obtain the intermediate fusion features. The process can be expressed as:

[0097]

[0098] Among them, F mid Represents the generated intermediate fusion features;

[0099] Finally, using the intermediate fusion feature F mid Advanced calibration features Calibrate by using the Advanced Calibration feature and intermediate fusion feature F mid Perform multiplication and residual connection to obtain the final fusion feature, which is mathematically:

[0100]

[0101] Among them, F CSWM Represents the final fused feature map of the defect;

[0102] Step 3.6: Use the progressive iterative decoding strategy PID to finally fusion feature map F of the defect cswm ={F cswm1 ,F cswm2 ,F cswm3 ,F cswm4} Perform feature decoding and calculate the segmentation results of the training set image to obtain the supervised feature map S sup ;

[0103] In this embodiment, if Figure 4 As shown, the final fusion feature map F of the defect cswm ={F cswm1,F cswm2 ,F cswm3 ,F cswm4}Use bilinear interpolation to upsample to the same resolution as the original weld defect image, and use 3*3 convolution layers to fused feature maps F cswm Perform dimensionality reduction fusion to obtain four supervised feature maps S sup ={S sup1 ,S sup2 ,S sup3 ,S sup4};

[0104] Step 3.7: Calculate the supervised feature map S through the cross entropy loss function BCE and the intersection-over-union loss function IOU sup ={S sup1 ,S sup2 ,S sup3 ,S sup4} and training set labels S gt The network model parameters in the fine-grained minimal defect segmentation network model are iteratively updated using the stochastic gradient descent algorithm. When the cross entropy loss function converges, the training is stopped and all the trainable parameters of the network are frozen to obtain a converged and trained fine-grained minimal defect segmentation network model.

[0105] In this embodiment, the supervised feature map S is calculated by the cross entropy loss function BCE and the intersection-over-union loss function IOU. sup ={S sup1 ,S sup2 ,S sup3 ,S sup4} and pixel-level segmentation labels S of the training set gt The loss between them is calculated and the stochastic gradient descent (SGD) optimizer with an exponential decay coefficient of 0.9 is adopted. The initial learning rate is set to 0.05. The network model parameters in the fine-grained minimal defect segmentation network model are iteratively updated using the stochastic gradient descent algorithm. When the number of training iterations is 150, the intersection over union loss function (IOU) converges, the model training is completed, and all trainable parameters of the network are frozen to obtain a converged and trained fine-grained minimal defect segmentation network model. This model can be directly applied to the minimal defect segmentation task of internal defects in welds.

[0106] Step 4: Use the test images in the test set as defect samples to input the converged trained fine-grained minimal defect segmentation network model. After calculation, the test set segmentation map is obtained, and the accurate position and size of the defect corresponding to each test image in the test set are obtained.

[0107] Compared with the existing technology, the fine-grained segmentation method for extremely small defects inside spiral steel pipe welds proposed in this embodiment can achieve good segmentation effects in the task of segmenting internal defects of spiral steel pipe welds where there is an imbalance in defect categories and extremely small defects are densely distributed. In order to verify the performance of this method in segmenting internal defects in welds, the method proposed in this embodiment and the existing segmentation methods were tested on the fine-grained spiral steel pipe weld internal defect dataset T-SWX-ray. All test experimental methods were performed on the same equipment, and the hyperparameters of each existing method were set according to the hyperparameters in the original paper proposing the method. To visually demonstrate the quantitative performance comparison results between our method and existing detection methods, we employed a common setup: training model parameters on a training set and evaluating model segmentation performance on a test set. We used VGG16, ResNet50, and PVT-V2 as the backbone networks for the feature extraction networks, respectively. We evaluated and validated our method using common metrics in the field of saliency detection and segmentation: structural similarity measure (S-measure), mean absolute error (MAE), maximum E-measure (Em), maximum F-measure (Fm), and weighted F-measure (WF). Higher values ​​for these metrics indicate better segmentation performance. The quantitative performance comparison results, shown in Table 1, demonstrate that our method significantly outperforms existing methods in segmentation accuracy, achieving superior segmentation performance. When using VGG16 as the backbone network for the feature extraction network, the WF and S-measure achieved segmentation accuracies of 75.57%, 85.52%, MAE of 0.0029, Em of 93.39%, and F-measure of 77.38%.

[0108] Table 1 Quantitative index results of the proposed method and other saliency segmentation methods on the T-SWX-ray dataset

[0109]

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A fine-grained segmentation method for extremely small defects inside spiral steel pipe welds, characterized by: The following steps are involved: Step 1: Construct a dataset of internal defects of granular spiral steel pipe welds, including a training set and a test set. Both the training set and the test set consist of original weld defect images and pixel-level segmentation labels, and use the saliency detection pixel-level segmentation task as the basic data unit. Step 2: Construct a fine-grained minimal defect segmentation network model, which consists of a feature extraction network, a semantic clue generator (SCG), a semantic discriminant encoder (SDE), a contrastive transformation mechanism (CTM), a progressive iterative decoding strategy (PID), and several convolutional modules (RBC). The feature extraction network extracts the first-level, second-level, third-level and fourth-level feature maps of the original weld defect image respectively, and feeds the extracted feature maps of the original weld defect image into the semantic clue generator SCG to obtain the global semantic clue T scg ; The semantic clue generator SCG includes a first dense receptive field DRF unit, a first dense attention guided DAG, a second dense receptive field DRF unit, a second dense attention guided DAG, a third dense receptive field DRF unit, a third dense attention guided DAG, a global average pooling layer and a 1*1 convolution layer; the first dense receptive field DRF unit, the first dense attention guided DAG, the second dense receptive field DRF unit, the second dense attention guided DAG, the third dense receptive field DRF unit and the third dense attention guided DAG are connected in sequence, the global average pooling layer and the 1*1 convolution layer are connected in sequence, and the two are then connected together in parallel, and the feature map of the extracted original weld defect image is fed into the semantic clue generator SCG to obtain a global semantic clue T scg ; The contrast conversion mechanism CTM includes a group of 3*3 maximum pooling layers, a first decoupling mechanism and a first 2-fold upsampling bilinear interpolation layer connected in sequence, and a group of 3*3 average pooling layers, a second decoupling mechanism and a second 2-fold upsampling bilinear interpolation layer connected in sequence. The two groups are connected together in parallel and fused through a convolution module RBC; the multi-scale output is sequentially passed through the 3*3 maximum pooling layer, the first decoupling mechanism and the first 2-fold upsampling bilinear interpolation layer to obtain the maximum coupled smooth defect feature, and the multi-scale output is sequentially passed through the 3*3 average pooling layer, the second decoupling mechanism and the second 2-fold upsampling bilinear interpolation layer to obtain the average coupled smooth defect feature, and then the maximum coupled smooth defect feature and the average coupled smooth defect feature are fused through the convolution module RBC to obtain the three output features of the contrast conversion mechanism CTM; The progressive iterative decoding strategy PID includes three sequentially connected cross-scale weighted modules CSWM; the cross-scale weighted module CSWM includes two sets of parallel spatial attention residual architectures, and two sets of parallel sequentially connected convolutional layers and element-level multiplications; the spatial attention residual architecture is implemented by spatial attention SA, element-level multiplication and element-level addition, and the spatial attention mechanism SA is used to calculate the low-level feature F low and high-level features F high , with low-level features F low and high-level features F high As input, spatial attention is applied to the two features to obtain two different attention maps, which are calibrated and residually connected with the original low-level features and high-level features to obtain interactive low-level verification features. and advanced calibration features The two different sets of spatial attention architectures are sequentially passed through a convolution module RBC, a 3*3 convolution layer and a Sigmoid activation function to generate weighted information W mid , the interactive low-level validation features and advanced calibration features Respectively with weighted information W mid and 1-W mid Multiply to obtain two sets of weighted feature maps, which are concatenated according to the channel dimension and then element-wise multiplied and added with another set of spatial attention residual architectures to obtain the output of the cross-scale weighted module CSWM. The semantic discriminant encoder SDE combines the second-level, third-level and fourth-level feature maps extracted by the feature extraction network with the global semantic clues T scg Connect in the channel dimension and apply 3*3 convolutional layers to obtain the three-level output of the semantic discriminant encoder SDE. The three-level output is connected with three contrast transformation mechanisms CTM respectively to obtain the output of three contrast transformation mechanisms CTM, i.e. multi-scale output. The multi-scale output and the first-level output of the feature extraction network are integrated and fed into the progressive iterative decoding strategy PID in sequence. The multi-scale output is fused in sequence to obtain the final segmentation map. Step 3: Take the original weld defect images and corresponding pixel-level segmentation labels in the training set as input to train the fine-grained minimal defect segmentation network model and obtain a converged trained fine-grained minimal defect segmentation network model; Step 4: Use the test images in the test set as defect samples to input the converged trained fine-grained minimal defect segmentation network model. After calculation, the test set segmentation map is obtained, and the accurate position and size of the defect corresponding to each test image in the test set are obtained.

2. The fine-grained segmentation method for extremely small defects inside spiral steel pipe welds according to claim 1 is characterized by: The pixel-level segmentation labels in the training set of step 1 are the precise positions and sizes of the surface defects represented by the original weld defect images in the training set, and the pixel-level segmentation labels in the test set are the precise positions and sizes of the surface defects represented by the original weld defect images in the test set. In the saliency detection segmentation task, the defects represented by the original weld defect images in the training set and the original weld defect images in the test set are pixel-level binary classification tasks.

3. The fine-grained segmentation method for extremely small defects inside spiral steel pipe welds according to claim 1 is characterized by: The convolution module RBC consists of a 3*3 convolution layer, a BN normalization layer and a ReLu activation function connected in sequence.

4. The fine-grained segmentation method for extremely small defects inside spiral steel pipe welds according to claim 3 is characterized by: The step 3 specifically includes the following steps: Step 3.1: The original weld defect images and corresponding pixel-level segmentation labels in the training set are input into the fine-grained minimal defect segmentation network model as fine-grained minimal defect samples. The feature extraction network is used to extract the first-level, second-level, third-level, and fourth-level feature maps of the original weld defect images respectively. Step 3.2: Feed the second-level, third-level, and fourth-level feature maps of the original weld defect images in the training set into the semantic clue generator SCG to obtain the global semantic clue T scg ; Step 3.3: The first-level, second-level, third-level and fourth-level feature maps of the original weld defect images in the training set are respectively compared with the global semantic clues T scg Connect according to the channel dimension and apply 3*3 convolutional layers to obtain the three-layer output of the semantic discriminant encoder SDE. The three-layer outputs are denoted as F2~F4, and these three-layer outputs are collectively referred to as multi-scale output F; Step 3.4: Inject the obtained multi-scale output F = {F2, F3, F4} into the contrast transformation mechanism CTM respectively to obtain the three output features of the contrast transformation mechanism CTM; Step 3.5: Use the cross-scale weighted modules CSWM connected in sequence in the progressive iterative decoding strategy PID to calculate the first-level features of the feature extraction network and the output features of the three contrast transformation mechanisms CTM, gradually highlight the defect features, and obtain the output result F of each cross-scale weighted module CSWM cswm ={F cswm1 ,F cswm2 ,F cswm3 }, and for feature F cswm1 Perform upsampling operation to obtain the final fusion feature map of the defect; Step 3.6: Use the cross-scale weighted module CSWM in the progressive iterative decoding strategy PID to decode the final fusion feature map of the defect and calculate the segmentation results of the training set image to obtain the supervised feature map; Step 3.7: Calculate the loss between the supervised feature map and the pixel-level segmentation label of the training set through the cross-entropy loss function BCE and the intersection-over-union loss function IOU, and use the stochastic gradient descent algorithm to iteratively update the network model parameters in the fine-grained minimal defect segmentation network model. When the cross-entropy loss function converges, stop training, freeze all trainable parameters of the network, and obtain a converged and trained fine-grained minimal defect segmentation network model.

5. The fine-grained segmentation method for extremely small defects inside spiral steel pipe welds according to claim 4 is characterized in that: The step 3.4 specifically includes the following steps: Step 3.4.1: Inject the obtained multi-scale outputs F = {F2, F3, F4} into the contrast transformation mechanism CTM respectively, and perform decoupling transformation on each multi-scale output F; Step 3.4.2: Apply RBC blocks to the three multi-scale outputs after decoupling transformation to fuse them, perform 2*2 average pooling operation and 2*2 maximum pooling operation in parallel, and apply RBC blocks to the two pooled features to fuse them again to obtain two different pooled features {F max ,F avg }, the formula is as follows: Among them, F max represents the maximum pooling feature, F avg Represents the average pooling feature, Max(·) represents the 2*2 maximum pooling operation, and Avg(·) represents the 2*2 average pooling operation; Step 3.4.3: Use the decoupling mechanism to decouple the maximum pooling feature and the average pooling feature, further determine the coupling relationship between the defect and the weld, and obtain the maximum coupling defect feature and the average coupling defect feature. The formula is as follows: in, represents the maximum coupling defect characteristic, represents the average coupling defect feature, σ represents the sigmoid activation function, Γ is a 3*3 convolutional layer with a batch normalization layer, ζ represents a 3*3 average pooling operation, and ⊙ represents element-wise multiplication; Step 3.4.4: Maximum coupling defect characteristics and average coupling defect characteristics The maximum coupled smooth defect feature and the average coupled smooth defect feature are obtained by upsampling through the 2-fold upsampling bilinear interpolation layer, and the maximum coupled smooth defect feature and the average coupled smooth defect feature are fused through the convolution module RBC to generate the output feature of the contrast conversion mechanism CTM. Therefore, the three hierarchical features F = {F2, F3, F4} are input into the three contrast conversion mechanisms CTM respectively, that is, the output features F of the three contrast conversion mechanisms CTM are obtained. ctm ={F ctm2 ,F ctm3 ,F ctm4 }.

6. The fine-grained segmentation method for extremely small defects inside spiral steel pipe welds according to claim 5, characterized in that: The step 3.5 specifically includes the following steps: Step 3.5.1: Compute low-level features F using the spatial attention mechanism SA low and high-level features F high ; The input features are the first-level feature map of the extraction network and the output features F of the three contrast transformation mechanisms CTM ctm ={F ctm2 ,F ctm3 ,F ctm4 }, the hierarchical features in the cross-scale weighted modules CSWM connected in sequence are {F ctm4 ,F ctm3 }, {F cswm3 ,F ctm2 }, {F cswm2 , first-level feature map}, among the above hierarchical features, the former are all from high-level features, namely high-level features F high ; The latter are all from low-level features, that is, low-level features F low ; Step 3.5.2: Take low-level features F low and high-level features F high As input, spatial attention is applied to the two features to obtain two different attention maps, which are calibrated and residually connected with the original features to obtain interactive low-level verification features. and advanced calibration features The formula is as follows: Where SA(·) represents spatial attention; Step 3.5.3: Check low-level features along the channel dimension and advanced calibration features Splicing is performed, and the splicing features are fused using the RBC module. The 3*3 convolution layer, batch normalization, and Sigmoid activation function are applied in sequence to activate the feature vector and generate the activation weight map W. mid , the formula is as follows: Step 3.5.4: Use 3*3 convolutional layers to fuse low-level features F low With high-level features F high , and use the activation weight map W mid Perform element-wise multiplication with the fused high-level features to obtain a high-level weighted feature map Using the weighted activation feature vector 1-W mid Multiply with the fused low-level features to obtain a low-level weighted feature map High-level weighted feature maps according to channel dimension and low-level weighted feature maps Perform splicing to obtain the intermediate fusion features. The formula is as follows: Among them, F mid Represents the generated intermediate fusion features; Step 3.5.5: Using the intermediate fusion feature F mid Advanced calibration features Calibrate by using the Advanced Calibration feature and intermediate fusion feature F mid Perform multiplication and residual connection to obtain the final fusion feature output F CSWM , the formula is as follows: Among them, F CSWM Represents the final fused feature map of the defect.

Images