Fine-grained segmentation method for tiny defects in welding seam of spiral steel pipe

Abstract

The invention provides a fine-grained segmentation method for extremely small defects in a welding seam of a spiral steel pipe, and relates to the technical field of nondestructive testing based on machine vision. According to the method, a fine-grained minimum defect segmentation network model is constructed under the condition that minimum defects in a spiral steel pipe welding seam are difficult to distinguish in a static scene, the model comprises a feature extraction network, a semantic discrimination encoder, a comparison conversion mechanism, a progressive iterative decoding strategy and a plurality of convolution modules, and a semantic discrimination system structure is designed. Features are self-adaptively initialized with semantics as guidance, and defect scale differences are relieved; introducing a comparison conversion mechanism to dynamically optimize defect features; and finally, performing fine-grained pixel-level segmentation on the fused feature map by adopting a progressive iterative decoding strategy. According to the method, a good segmentation effect can still be achieved in adverse scenes such as unbalanced sample category distribution, dense minimum defect distribution, pixel redundancy and background interference, and the segmentation performance of the network is effectively improved.

Description

Technical Field

[0001] The invention relates to the technical field of nondestructive testing based on machine vision, and in particular to a fine-grained segmentation method for extremely small defects inside a spiral steel pipe weld. Background Art

[0002] With the continuous advancement of technology, spiral steel pipes have been widely used in various modern fields such as energy transportation and urban construction. However, in the process of steel pipe manufacturing, due to the dynamic changes in welding technology and production environment, weld defects are various and complex. These defects directly affect the quality of pipelines and cause major safety accidents. Therefore, it is crucial to conduct a comprehensive inspection and evaluation of spiral welds in a timely and professional manner.

[0003] Among various inspection technologies, X-ray-based nondestructive testing dominates the quality control process of spiral steel pipes, providing accurate information for repair decisions. First, the X-ray detector receives the X-rays emitted by the emitter inside the spiral steel pipe, which are then converted into image information by the image acquisition card. When encountering welding defects, human experts guide secondary repair decisions by roughly estimating the size of the defects. However, in actual manufacturing environments, the probability of occurrence of tiny defects greatly exceeds the probability of occurrence of other types of defects. This seriously affects the judgment of professionals, resulting in subjectivity, inefficiency, false positives, false negatives, and inaccurate positioning results. Therefore, automatic visual inspection is of great value for fine-grained positioning of tiny spiral weld defects.

[0004] Compared with other defect detection tasks, weld defect detection is more challenging due to the influence of welding process, X-ray detection, dynamic digitization and other factors. So far, X-ray weld flaw detection technology has shown considerable advantages and promoted substantial progress in this specific field. However, there are significant differences in the appearance and distribution of spiral steel pipe weld images from other weld images. When these models are directly applied for pixel-level segmentation, the detection performance will drop rapidly. In addition, the current spiral steel pipe weld defect detection methods rely on unsupervised and target detection frameworks. This rough positioning method is not sufficient to support accurate secondary repair decisions. Therefore, despite significant progress, the development of fine-grained tiny spiral steel pipe weld defect detection is still full of the following complex challenges and requires comprehensive exploration: (1) Dense distribution of small defects: Affected by the complexity of the welding process, insufficient exhaust time will cause a large number of small pore defects. These pore defects often occupy only a few pixels of the entire image, which is easy to cause missed detection and weld quality assessment failure. (2) Strong coupling relationship. Spiral steel pipe weld X-ray images usually show low-quality contrast. In addition, the image blur caused by imaging brings obstacles to intelligent detection. These two features lead to a tight coupling between the characteristics of tiny defects and low-quality welds, which often hinders the distinction between tiny defects and surrounding welds. (3) Tiny defects under interference. Unlike other types of weld images, due to the X-ray imaging characteristics and spiral patterns, spiral steel pipe weld X-ray images usually present a large amount of pixel redundancy. These redundant areas introduce additional interference, such as text, slag inclusions, and lighting changes.

[0005] Previous studies have used end-to-end deep learning methods to identify defects in weld images, which is considered to be a promising method for weld defect detection. Deep learning methods for welding images can be roughly divided into two frameworks: region-level detection and pixel-level detection. The region-level detection framework only provides an approximate bounding box, which is difficult to provide an accurate description of the defect size and boundary, especially for small and irregular defects. Therefore, in order to obtain more accurate detection results to guide secondary repair decisions, researchers have increasingly focused on pixel-level detection frameworks. In the prior art, an end-to-end pixel-level localization architecture is designed for weld defect segmentation, which detects weld defects through local feature mapping and improved skip connections, aiming to eliminate the limitations of manual interpretation. The network uses a sparse attention strategy to reduce the strong coupling relationship between welds and defects, and achieves good segmentation performance in the detection of annular welds in service. However, the network does not consider the problems of large intra-class differences in spiral steel pipe weld defect images, numerous extremely small defects, and interference from invalid regions. When encountering scenes with small and scattered defect areas, the network is difficult to effectively establish connections between discrete defects. Therefore, although the above methods show impressive performance in specific areas, they have limitations when directly applied to spiral steel pipe weld defect detection. Decoupling low-quality weld features from defect features to ensure accurate fine-grained defect segmentation is the main challenge facing spiral steel pipe welding systems. However, existing spiral steel pipe weld inspection methods usually rely on a regional-level inspection framework to roughly locate defects, which is insufficient to provide precise guidance for professionals to make secondary repair decisions. Therefore, there is an urgent need to research and develop a fine-grained defect detection framework tailored specifically for spiral steel pipe welds. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a fine-grained segmentation method for extremely small defects inside the weld of a spiral steel pipe in response to the above-mentioned deficiencies in the prior art, so as to solve the problem that in the actual setting of spiral welding defect detection, the segmentation performance of the segmentation network drops sharply under unfavorable scenarios such as unbalanced distribution of defect sample types and dense distribution of extremely small defects.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A fine-grained segmentation method for extremely small defects inside a spiral steel pipe weld comprises the following steps:

[0009] Step 1: Construct a granular spiral steel pipe weld internal defect dataset, including a training set and a test set; both the training set and the test set are composed of original weld defect images and pixel-level segmentation labels, and the saliency detection pixel-level segmentation task is used as the basic data unit;

[0010] The pixel-level segmentation labels in the training set are the precise positions and sizes of the surface defects represented by the original weld defect images in the training set, and the pixel-level segmentation labels in the test set are the precise positions and sizes of the surface defects represented by the original weld defect images in the test set, and in the saliency detection segmentation task, the defects represented by the original weld defect images in the training set and the original weld defect images in the test set are pixel-level binary classification tasks;

[0011] Step 2: construct a fine-grained extremely small defect segmentation network model, which consists of a feature extraction network, a semantic clue generator SCG, a semantic discriminant encoder SDE, a contrast transformation mechanism CTM, a progressive iterative decoding strategy PID and several convolution modules RBC;

[0012] The feature extraction network extracts the first-level, second-level, third-level and fourth-level feature maps of the original weld defect image respectively, and feeds the extracted feature maps of the original weld defect image into the semantic clue generator SCG to obtain the global semantic clue T scg ;

[0013] The semantic clue generator SCG includes a first dense receptive field DRF unit, a first dense attention guided DAG, a second dense receptive field DRF unit, a second dense attention guided DAG, a third dense receptive field DRF unit, a third dense attention guided DAG, a global average pooling layer and a 1*1 convolution layer; the first dense receptive field DRF unit, the first dense attention guided DAG, the second dense receptive field DRF unit, the second dense attention guided DAG, the third dense receptive field DRF unit and the third dense attention guided DAG are connected in sequence, the global average pooling layer and the 1*1 convolution layer are connected in sequence, and the two are then connected together in parallel, and the feature map of the extracted original weld defect image is fed into the semantic clue generator SCG to obtain the global semantic clue T scg ;

[0014] The semantic discriminative encoder SDE combines the second-level, third-level, and fourth-level feature maps extracted by the feature extraction network with the global semantic clues T scg Connect in the channel dimension and apply 3*3 convolutional layers to obtain the three-level output of the semantic discriminative encoder SDE. The three-level output is connected with three contrast transformation mechanisms CTM respectively to obtain the output of three contrast transformation mechanisms CTM, that is, multi-scale output. The multi-scale output and the first-level output of the feature extraction network are integrated and fed into the progressive iterative decoding strategy PID in turn, and the multi-scale output is fused in turn to obtain the final segmentation map.

[0015] The contrast conversion mechanism CTM includes a group of 3*3 maximum pooling layers, a first decoupling mechanism and a first 2-fold upsampling bilinear interpolation layer connected in sequence, and a group of 3*3 average pooling layers, a second decoupling mechanism and a second 2-fold upsampling bilinear interpolation layer connected in sequence. The two groups are connected together in parallel and fused through a convolution module RBC; the multi-scale output is sequentially passed through the 3*3 maximum pooling layer, the first decoupling mechanism and the first 2-fold upsampling bilinear interpolation layer to obtain the maximum coupling smooth defect feature, and the multi-scale output is sequentially passed through the 3*3 average pooling layer, the second decoupling mechanism and the second 2-fold upsampling bilinear interpolation layer to obtain the average coupling smooth defect feature, and then the maximum coupling smooth defect feature and the average coupling smooth defect feature are fused through the convolution module RBC to obtain three output features of the contrast conversion mechanism CTM;

[0016] The progressive iterative decoding strategy PID includes three sequentially connected cross-scale weighted modules CSWM; the cross-scale weighted module CSWM includes two sets of parallel spatial attention residual architectures, and two sets of parallel sequentially connected convolutional layers and element-level multiplications; the spatial attention residual architecture is implemented by spatial attention SA, element-level multiplication and element-level addition, and the spatial attention mechanism SA is used to calculate the low-level feature F low and high-level features F high , with low-level features F low and high-level features F high As input, spatial attention is applied to the two features to obtain two different attention maps. The two attention maps are calibrated and residually connected with the original low-level features and high-level features to obtain interactive low-level verification features. and advanced validation features The two different sets of spatial attention architectures are sequentially passed through a convolution module RBC, a 3*3 convolution layer and a Sigmoid activation function to generate weighted information W. mid , the interactive low-level validation features and advanced validation features Respectively with weighted information W mid and 1-W mid Multiply to obtain two sets of weighted feature maps, which are concatenated according to the channel dimension and element-wise multiplied and added with another set of spatial attention residual architecture to obtain the output of the cross-scale weighted module CSWM;

[0017] The convolution module RBC consists of a 3*3 convolution layer, a BN normalization layer and a ReLu activation function connected in sequence;

[0018] Step 3: Take the original weld defect images and the corresponding pixel-level segmentation labels in the training set as input, train the fine-grained minimal defect segmentation network model, and obtain a converged trained fine-grained minimal defect segmentation network model;

[0019] Step 3.1: The original weld defect image in the training set and the corresponding pixel-level segmentation label are input into the fine-grained minimal defect segmentation network model as fine-grained minimal defect samples, and the first-level, second-level, third-level and fourth-level feature maps of the original weld defect image are extracted by using the feature extraction network;

[0020] Step 3.2: Feed the second-level, third-level, and fourth-level feature maps of the original weld defect images in the training set into the semantic clue generator SCG to obtain the global semantic clue T scg ;

[0021] Step 3.3: The first, second, third and fourth level feature maps of the original weld defect images in the training set are respectively compared with the global semantic clues T scg Connect according to the channel dimension, and apply 3*3 convolutional layers respectively to obtain the three-layer output of the semantic discriminative encoder SDE, and record the three-layer output as F2~F4, and these three-layer outputs are collectively referred to as multi-scale output F;

[0022] Step 3.4: Inject the obtained multi-scale output F = {F2, F3, F4} into the contrast transformation mechanism CTM respectively to obtain the three output features of the contrast transformation mechanism CTM;

[0023] Step 3.4.1: Inject the obtained multi-scale outputs F = {F2, F3, F4} into the contrast transformation mechanism CTM respectively, and perform decoupling transformation on each multi-scale output F;

[0024] Step 3.4.2: Apply RBC blocks to the three multi-scale outputs after the decoupling transformation to fuse them, perform 2*2 average pooling operations and 2*2 maximum pooling operations in parallel, and apply RBC blocks to the two pooled features to fuse them again to obtain two different pooled features {F max ,F avg}, the formula is as follows:

[0025]

[0026] Among them, F max represents the maximum pooling feature, F avg represents the average pooling feature, Max(·) represents the 2*2 maximum pooling operation, and Avg(·) represents the 2*2 average pooling operation;

[0027] Step 3.4.3: Use the decoupling mechanism to decouple the maximum pooling feature and the average pooling feature, further determine the coupling relationship between the defect and the weld, and obtain the maximum coupling defect feature and the average coupling defect feature. The formula is as follows:

[0028]

[0029] in, represents the maximum coupling defect characteristic, represents the average coupling defect feature, σ represents the sigmoid activation function, Γ is a 3*3 convolutional layer with a batch normalization layer, ζ represents a 3*3 average pooling operation, and ⊙ represents element-wise multiplication;

[0030] Step 3.4.4: Maximum coupling defect characteristics and average coupling defect characteristics The maximum coupled smooth defect feature and the average coupled smooth defect feature are obtained by upsampling through a 2-fold upsampling bilinear interpolation layer, and the maximum coupled smooth defect feature and the average coupled smooth defect feature are fused through a convolution module RBC to generate the output feature of the contrast conversion mechanism CTM. Therefore, the three hierarchical features F = {F2, F3, F4} are respectively input into the three contrast conversion mechanisms CTM, that is, the output features F of the three contrast conversion mechanisms CTM are obtained. ctm ={F ctm2 ,F ctm3 ,F ctm4};

[0031] Step 3.5: Use the cross-scale weighted modules CSWM connected in sequence in the progressive iterative decoding strategy PID to calculate the first-level features of the feature extraction network and the output features of the three contrast transformation mechanisms CTM, gradually highlight the defect features, and obtain the output result F of each cross-scale weighted module CSWM cswm ={F cswm1 ,F cswm2 ,F cswm3}, and for feature F cswm1 Perform upsampling operation to obtain the final fused feature map of defects;

[0032] Step 3.5.1: Compute low-level features F using spatial attention mechanism SA low and high-level features F high ; The input features are the first-level feature map of the extraction network and the output features F of the three contrast transformation mechanisms CTM ctm ={F ctm2 ,F ctm3 ,F ctm4}, the hierarchical features in the sequentially connected cross-scale weighted modules CSWM are {F ctm4 ,F ctm3}, {F cswm3 ,F ctm2}, {F cswm2 , the first level feature map}, among the above hierarchical features, the former are all from high-level features, that is, high-level features F high ; The latter are all from low-level features, that is, low-level features F low ;

[0033] Step 3.5.2: Take low-level features F low and high-level features F high As input, spatial attention is applied to the two features to obtain two different attention maps, which are calibrated and residually connected with the original features to obtain interactive low-level verification features. and advanced validation features The formula is as follows:

[0034]

[0035] Where SA(·) represents spatial attention;

[0036] Step 3.5.3: Low-level validation features along the channel dimension and advanced validation features Splicing is performed, and the splicing features are fused using the RBC module. The 3*3 convolution layer, batch normalization, and Sigmoid activation function are applied in sequence to activate the feature vector to generate the activation weight map W mid , the formula is as follows:

[0037]

[0038] Step 3.5.4: Use 3*3 convolutional layers to fuse low-level features F low With advanced features F high , and use the activation weight map W mid Perform element-wise multiplication with the fused high-level features to obtain a high-level weighted feature map Using the weighted activation feature vector 1-W mid Multiply with the fused low-level features to obtain a low-level weighted feature map High-level weighted feature maps according to channel dimension and low-level weighted feature maps Splicing is performed to obtain the intermediate fusion features. The formula is as follows:

[0039]

[0040] Among them, F mid Represents the generated intermediate fusion features;

[0041] Step 3.5.5: Using the intermediate fusion feature Fmid Advanced Verification Features Calibrate by using the Advanced Calibration feature and the intermediate fusion feature F mid Perform multiplication and residual connection to obtain the final fusion feature output F CSWM , the formula is as follows:

[0042]

[0043] Among them, F CSWM Represents the final fused feature map of the defect;

[0044] Step 3.6: Use the cross-scale weighted module CSWM in the progressive iterative decoding strategy PID to decode the final fusion feature map of the defect and calculate the segmentation result of the training set image to obtain the supervised feature map;

[0045] Step 3.7: The loss between the supervised feature map and the pixel-level segmentation label of the training set is calculated by the cross entropy loss function BCE and the intersection over union loss function IOU, and the network model parameters in the fine-grained minimal defect segmentation network model are iteratively updated using the stochastic gradient descent algorithm. When the cross entropy loss function converges, the training is stopped, and all trainable parameters of the network are frozen to obtain a converged and trained fine-grained minimal defect segmentation network model.

[0046] Step 4: Use the test images in the test set as defect samples to input the converged trained fine-grained minimal defect segmentation network model, obtain the test set segmentation map after calculation, and get the accurate position and size of the defect corresponding to each test image in the test set.

[0047] The beneficial effects of adopting the above technical solution are:

[0048] (1) The method of the present invention adopts a fine-grained small defect segmentation method based on semantic discrimination and contrast transformation strategy, uses defect data annotated at the weld pixel level as a training set to train the segmentation network, and uses the network to achieve fine-grained segmentation of internal defects of spiral steel pipe welds. The method of the present invention can still achieve good segmentation results in unfavorable scenarios such as unbalanced sample category distribution, dense distribution of small defects, pixel redundancy and background interference;

[0049] (2) The method of the present invention suppresses the expression of background features and irrelevant noise in invalid areas of the feature map through a semantic discriminative encoder (SDE) and a contrast transformation mechanism (CTM), thereby increasing the model's attention to defects, activating effective defect information responses, and providing more robust feature inputs for subsequent iterative decoding, thereby effectively improving the segmentation performance of the network.

[0050] (3) The method of the present invention introduces a progressive iterative decoding strategy, which refines the initial supervision clues by using continuous iterations of the cross-scale weighting module (CSWM). It is conducive to dynamic adaptive optimization, multi-scale information decoding of defect features after feature matching, aggregation of defect information with different receptive fields, and processing features by combining residual connections and hybrid attention mechanisms to better decode defect information and finally generate fine-grained detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of a fine-grained segmentation method for extremely small defects inside a spiral steel pipe weld in this embodiment;

[0052] Figure 2 This is a schematic diagram of a fine-grained segmentation method for extremely small defects inside a spiral steel pipe weld in this embodiment;

[0053] Figure 3 is a schematic diagram of the semantic discriminative encoder architecture in this embodiment;

[0054] Figure 4 It is a schematic diagram of the progressive iterative decoding strategy PID in this implementation mode;

[0055] Figure 5 Schematic diagram of the cross-scale weighting module in this implementation. DETAILED DESCRIPTION

[0056] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0057] The core idea of ​​the present invention is: in the face of unbalanced defect categories and dense distribution of extremely small defects in the weld of spiral steel pipes, a semantic discriminant coding strategy is adopted to fully activate the defect information in the defect image, enhance the semantic information association of the model with defects, and reduce invalid background interference; then the strong coupling association between the weld and the defect of the contrast conversion mechanism is used to obtain differentiated high-quality defect features; finally, a progressive iterative decoder and a cross-scale weighted module are used to decode the fused feature map and segment it to obtain the final output result.

[0058] like Figure 1 As shown, the method of this embodiment is described as follows.

[0059] Step 1: Construct a granular spiral steel pipe weld internal defect dataset, including a training set and a test set, wherein both the training set and the test set are composed of original weld defect images and pixel-level segmentation labels, and the saliency detection pixel-level segmentation task is used as the basic data unit;

[0060] The pixel-level segmentation label of the training set is the precise position and size of the surface defect represented by the original weld defect image of the training set, and the pixel-level segmentation label of the test set is the precise position and size of the surface defect represented by the original weld defect image of the test set, and in the saliency detection segmentation task, the defects represented by the original weld defect image of the training set and the original weld defect image of the test set are pixel-level binary classification tasks;

[0061] Since there is currently no segmentation dataset of extremely small defects inside spiral steel pipe welds, and there are few samples of internal defects in spiral steel pipe welds, in order to solve this problem, this embodiment constructs a fine-grained spiral steel pipe weld internal defect dataset, namely a tiny spiral weld X-ray dataset (T-SWX-ray), which fills the gap in the dataset of extremely small defects inside spiral steel pipe welds. The dataset consists of a training set and a test set. The types of severe defects contained in the above training set and test set are unbalanced, mainly extremely small pore defects, which is consistent with the defect distribution in the actual working conditions of spiral steel pipes;

[0062] Step 2: construct a fine-grained minimal defect segmentation network model; the fine-grained minimal defect segmentation network model consists of a feature extraction network, a semantic cues generator (SCG), a semantic discriminative encoder (SDE), a contrast transformation mechanism (CTM), a progressive iterative decoding strategy (PID) and several convolutional modules (RBC); the progressive iterative decoding strategy PID includes a cross-scale weighting module (CSWM);

[0063] The feature extraction network extracts the first-level, second-level, third-level and fourth-level feature maps of the original weld defect image respectively, and feeds the extracted feature maps of the original weld defect image into the semantic clue generator SCG to obtain the global semantic clue T scg ; The semantic clue generator SCG includes a first dense receptive field DRF unit, a first dense attention guided DAG, a second dense receptive field DRF unit, a second dense attention guided DAG, a third dense receptive field DRF unit, a third dense attention guided DAG, a global average pooling layer and a 1*1 convolution layer; the first dense receptive field DRF unit, the first dense attention guided DAG, the second dense receptive field DRF unit, the second dense attention guided DAG, the third dense receptive field DRF unit and the third dense attention guided DAG are connected in sequence, the global average pooling layer and the 1*1 convolution layer are connected in sequence, and the two are then connected together in parallel, and the feature map of the extracted original weld defect image is fed into the semantic clue generator SCG to obtain the global semantic clue T scg ;

[0064] The semantic discriminative encoder SDE combines the second-level, third-level, and fourth-level feature maps extracted by the feature extraction network with the global semantic clues T scg Connect in the channel dimension, and apply 3*3 convolutional layers respectively to obtain the three-level output of the semantic discriminant encoder SDE, and record the three-level output as F2~F4. The three-level output is connected with three contrast transformation mechanisms CTM respectively to obtain the output of three contrast transformation mechanisms CTM, that is, multi-scale output. The multi-scale output and the first-level output of the feature extraction network are integrated and fed into the progressive iterative decoding strategy PID in sequence, and the multi-scale output is fused in sequence to obtain the final segmentation map;

[0065] The contrast conversion mechanism CTM includes a group of 3*3 maximum pooling layers, a first decoupling mechanism (Decoupling Mechanism, DM) and a first 2-fold upsampling bilinear interpolation layer connected in sequence, and a group of 3*3 average pooling layers, a second decoupling mechanism and a second 2-fold upsampling bilinear interpolation layer connected in sequence, the two groups are connected together in parallel and fused through a convolution module RBC; the multi-scale output is sequentially passed through the 3*3 maximum pooling layer, the first decoupling mechanism (Decoupling Mechanism, DM) and the first 2-fold upsampling bilinear interpolation layer to obtain the maximum coupling smooth defect feature, and the multi-scale output is sequentially passed through the 3*3 average pooling layer, the second decoupling mechanism and the second 2-fold upsampling bilinear interpolation layer to obtain the average coupling smooth defect feature, and then the maximum coupling smooth defect feature and the average coupling smooth defect feature are fused through the convolution module RBC to obtain the output feature of the contrast conversion mechanism CTM;

[0066] The progressive iterative decoding strategy PID includes three sequentially connected cross-scale weighted modules CSWM; the cross-scale weighted module CSWM includes two sets of parallel spatial attention residual architectures, and two sets of parallel sequentially connected convolutional layers and element-level multiplications; the spatial attention residual architecture is implemented by spatial attention SA, element-level multiplication and element-level addition, and the spatial attention mechanism SA is used to calculate the low-level feature F low and high-level features F high , with low-level features F low and high-level features F high As input, spatial attention is applied to the two features to obtain two different attention maps. The two attention maps are calibrated and residually connected with the original low-level features and high-level features to obtain interactive low-level verification features. and advanced validation features The two different sets of spatial attention architectures are sequentially passed through a convolution module RBC, a 3*3 convolution layer and a Sigmoid activation function to generate weighted information W. mid , the interactive low-level validation features and advanced validation features Respectively with weighted information W mid and 1-W mid Multiply to obtain two sets of weighted feature maps, which are concatenated according to the channel dimension and element-wise multiplied and added with another set of spatial attention residual architecture to obtain the output of the cross-scale weighted module CSWM;

[0067] The convolution module RBC consists of a 3*3 convolution layer, a BN normalization layer and a ReLu activation function connected in sequence;

[0068] Step 3: Input the original weld defect images and corresponding pixel-level segmentation labels in the training set as fine-grained minimal defect samples into the fine-grained minimal defect segmentation network model, calculate the pixel-level segmentation map, and gradually iterate and predict to obtain a more accurate position and size of the corresponding defect in each original weld defect image. By training the fine-grained minimal defect segmentation network model, a converged trained fine-grained minimal defect segmentation network model is obtained;

[0069] Step 3.1: The original weld defect image in the training set and the corresponding pixel-level segmentation label are input into the fine-grained minimal defect segmentation network model as fine-grained minimal defect samples, and the first-level, second-level, third-level and fourth-level feature maps of the original weld defect image are extracted by using the feature extraction network;

[0070] In this embodiment, if Figure 2 As shown in the figure, the fine-grained minimal defect segmentation network model can use Convolutional Neural Networks (CNN), Transformer or Multi-Layer Perceptron (MLP) as the backbone network of the feature extraction network, input the original weld defect image in the training set into the backbone network, and use the feature extraction network to extract the first, second, third and fourth level feature maps of the original weld defect image respectively, and the extracted training set image feature maps are recorded as T1~T4;

[0071] Step 3.2: Feed the second-level, third-level, and fourth-level feature maps of the original weld defect images in the training set into the semantic clue generator SCG to obtain global semantic clues;

[0072] In this embodiment, if Figure 3 As shown in the figure, the second, third and fourth level feature maps of the original weld defect images in the training set are spliced ​​along the channel dimension, and the global context information is extracted by cascading three dense receptive field DRF units and dense attention guided DAG in sequence. Each dense receptive field DRF unit consists of three parallel 3*3 expanded convolutional layers, and the expansion rate r is (1, 2, 4). Formally, it can be expressed as:

[0073]

[0074] in, represents the output of the i+1th dense receptive field DRF unit, Concat[·] represents the concatenation operation along the channel dimension, represents dilated convolution with different dilation rates r = (1, 2, 4), (to denote the output of the i-th dense attention guided DAG, the process can be expressed as:

[0075]

[0076] Afterwards, the initial features Apply global average pooling and 1*1 convolution layer operations, and use bilinear upsampling function to obtain global semantic features in the spatial dimension. Finally, multiple dense receptive field DRF units and dense attention guidance are used to The global semantic clues T are obtained by adaptive aggregation scg , the whole process is defined as:

[0077]

[0078] RBC(·) consists of a convolutional layer, a batch normalization layer, and a ReLU activation function. 1×1 represents a 1×1 convolution operation, and Gap represents global average pooling;

[0079] Step 3.3: The first, second, third and fourth level feature maps of the original weld defect images in the training set are respectively compared with the global semantic clues T scg In the semantic discriminant encoder SDE, connections are made according to the channel dimension, and 3*3 convolutional layers are applied respectively to obtain the three-layer output F2~F4 of the semantic discriminant encoder SDE, and these three-layer outputs are collectively referred to as multi-scale output F={F2,F3,F4};

[0080] Step 3.4: Inject the obtained multi-scale output F = {F2, F3, F4} into the three contrast transformation mechanisms CTM respectively to obtain the output features of the three contrast transformation mechanisms CTM;

[0081] Step 3.4.1: Inject the obtained multi-scale output F = {F2, F3, F4} into the three contrast transformation mechanisms CTM respectively, and perform decoupling transformation on each multi-scale output;

[0082] Step 3.4.2: Apply RBC blocks to each multi-scale output after the decoupling transformation for fusion, perform 2*2 average pooling operation and 2*2 maximum pooling operation in parallel, and apply RBC blocks to the two pooled features to fuse them again to obtain two different pooled features {F max ,F avg}, the process can be expressed as:

[0083]

[0084] Among them, F max represents the maximum pooling feature, F avg represents the average pooling feature, Max(·) represents the 2*2 maximum pooling operation, and Avg(·) represents the 2*2 average pooling operation;

[0085] Step 3.4.3: Use the decoupling mechanism to decouple the maximum pooling feature and the average pooling feature, further determine the coupling relationship between the defect and the weld, and obtain more distinct maximum coupling defect features and average coupling defect features. The formula is as follows:

[0086]

[0087] in, represents the maximum coupling defect characteristic, represents the average coupling defect feature, σ represents the sigmoid activation function, Γ is a 3*3 convolutional layer with a batch normalization layer, ζ represents a 3*3 average pooling operation, and ⊙ represents element-wise multiplication;

[0088] Step 3.4.4: Maximum coupling defect characteristics and average coupling defect characteristics The maximum coupled smooth defect feature and the average coupled smooth defect feature are obtained by upsampling through a 2-fold upsampling bilinear interpolation layer, and the maximum coupled smooth defect feature and the average coupled smooth defect feature are fused through a convolution module RBC to generate the output feature of the contrast conversion mechanism CTM. Therefore, the three hierarchical features F = {F2, F3, F4} are respectively input into the three contrast conversion mechanisms CTM, that is, the output features F of the three contrast conversion mechanisms CTM are obtained. ctm ={F ctm2 ,F ctm3 ,F ctm4};

[0089] Step 3.5: The key component of the progressive iterative decoding strategy PID, namely the sequentially connected cross-scale weighted modules CSWM, gradually highlights the defect features, and the first-level features F1 extracted by the feature extraction network and the output features F of the three contrast transformation mechanisms CTM are compared. ctm ={F ctm2 ,F ctm3 ,F ctm4} to calculate and gradually highlight the defect characteristics, and obtain the output result F of each cross-scale weighted module CSWM cswm ={Fx swm1 ,F cswm2 ,F cswm3}, and for feature F cswm1 Perform upsampling operation to obtain the final fused feature map of defects;

[0090] In this embodiment, if Figure 4 and Figure 5As shown in the figure, the progressive iterative decoding strategy PID consists of three sequentially connected cross-scale weighted modules CSWM, which are called stage 1, stage 2, stage 3 and stage 4 respectively; the first-level feature F1 of the feature extraction network is used as the input feature of stage 1, and the three output features F ctm ={F ctm2 ,F ctm3 ,F ctm4} are used as the input features of stage 2, stage 3 and stage 4 respectively, and the multi-scale defect features of these four stages are fused in turn through the cross-scale weighted module CSWM to obtain the final segmentation result; the cross-scale weighted module CSWM uses the low-level feature F low and high-level features F high As input, spatial attention is applied to the two features to obtain two different attention maps; the two attention maps are calibrated and residually connected with the original low-level features and high-level features to obtain interactive low-level verification features and advanced validation features The process can be described as:

[0091]

[0092] Where SA(·) represents spatial attention;

[0093] The low-level features F low and high-level features F high It is calculated using the spatial attention mechanism SA; the input features are the first-level feature map of the extraction network and the output features F of the three contrast transformation mechanisms CTM ctm ={F ctm2 ,F ctm3 ,F ctm4}, the hierarchical features in the sequentially connected cross-scale weighted modules CSWM are {F ctm4 ,F ctm3}, {F cswm3 ,F ctm2}, {F cswm2 , the first-level feature map}; Among the above hierarchical features, the former are all from high-level features, and the latter are all from low-level features. Therefore, for the convenience of expression, the hierarchical input features of any cross-scale weighted module CSWM can be uniformly expressed as low-level features F low and high-level features F high ;

[0094] To further enhance the interaction between scales, low-level validation features are and advanced validation features Splicing is performed, and the splicing features are fused using the RBC module. The 3*3 convolution layer, batch normalization, and Sigmoid activation function are applied in sequence to activate the feature vector to generate the activation weight map W mid , the process can be expressed as:

[0095]

[0096] Use 3*3 convolutional layers to fuse low-level verification features and advanced validation features And use the activation weight map W mid Perform element-wise multiplication with the fused advanced verification features to obtain an advanced weighted feature map Using the weighted activation feature vector 1-W mid Multiply with the fused low-level check feature to obtain a low-level weighted feature map High-level weighted feature maps according to channel dimension and low-level weighted feature maps Splicing is performed to obtain the intermediate fusion features. The process can be expressed as:

[0097]

[0098] Among them, F mid Represents the generated intermediate fusion features;

[0099] Finally, using the intermediate fusion feature F mid Advanced Verification Features Calibrate by using the Advanced Calibration feature and the intermediate fusion feature F mid Perform multiplication and residual connection to obtain the final fusion feature. In mathematical form, we have:

[0100]

[0101] Among them, F CSWM Represents the final fused feature map of the defect;

[0102] Step 3.6: Use the progressive iterative decoding strategy PID to finally fusion feature map F of the defect cswm ={F cswm1 ,F cswm2 ,F cswm3 ,F cswm4} Perform feature decoding and calculate the segmentation results of the training set image to obtain the supervised feature map S sup ;

[0103] In this embodiment, if Figure 4 As shown, the final fusion feature map F of the defect cswm ={F cswm1,F cswm2 ,F cswm3 ,F cswm4} respectively use bilinear interpolation to upsample to the same resolution as the original weld defect image, and use a 3*3 convolution layer to fused the final feature map F of the defect. cswm Perform dimensionality reduction fusion to obtain four supervised feature maps S sup ={S sup1 ,S sup2 ,S sup3 ,S sup4};

[0104] Step 3.7: Calculate the supervised feature map S through the cross entropy loss function BCE and the intersection-over-union loss function IOU sup ={S sup1 ,S sup2 ,S sup3 ,S sup4} and training set labels S gt The network model parameters in the fine-grained minimal defect segmentation network model are iteratively updated using the stochastic gradient descent algorithm. When the cross entropy loss function converges, the training is stopped and all trainable parameters of the network are frozen to obtain a converged and trained fine-grained minimal defect segmentation network model.

[0105] In this embodiment, the supervised feature map S is calculated by the cross entropy loss function BCE and the intersection-over-union loss function IOU. sup ={S sup1 ,S sup2 ,S sup3 ,S sup4} and the pixel-level segmentation labels S of the training set gt The loss between , the stochastic gradient descent (SGD) optimizer with an exponential decay coefficient of 0.9 is adopted, the initial learning rate is set to 0.05, and the stochastic gradient descent algorithm is used to iteratively update the network model parameters in the fine-grained extremely small defect segmentation network model. When the number of training iterations is 150, the intersection over union loss function IOU converges, the model training is completed, and all trainable parameters of the network are frozen to obtain a converged and trained fine-grained extremely small defect segmentation network model, which can be directly applied to the extremely small defect segmentation task of internal defects of welds.

[0106] Step 4: Use the test images in the test set as defect samples to input the converged trained fine-grained minimal defect segmentation network model, obtain the test set segmentation map after calculation, and get the accurate position and size of the defect corresponding to each test image in the test set.

[0107] Compared with the prior art, the fine-grained segmentation method for extremely small defects inside spiral steel pipe welds proposed in this embodiment can achieve good segmentation effects in the task of segmenting internal defects of spiral steel pipe welds with imbalanced defect categories and dense distribution of extremely small defects. In order to verify the performance of this method in the segmentation of internal defects in welds, the method proposed in this embodiment and the existing segmentation methods were tested on the fine-grained spiral steel pipe weld internal defect dataset T-SWX-ray. All test experimental methods were carried out on the same equipment, and the hyperparameters of each existing method were set according to the hyperparameters in the original paper proposing the method. In order to intuitively show the comparison results of quantitative performance indicators between the present method and the existing detection methods, the general setting of training set training model parameters, testing set testing model segmentation performance and evaluation is adopted, and VGG16, ResNet50 and PVT-V2 are used as the backbone network of the feature extraction network respectively. The performance of the present method is evaluated and verified by the general indicators in the field of saliency detection segmentation: structural similarity measure (S-measure), mean absolute error (MAE), maximum E-measure (Em), maximum F-measure (Fm) and weighted F-measure (WF). The higher the value of the above evaluation index, the better the segmentation performance of the corresponding method. The comparison results of quantitative indicators shown in Table 1 show that compared with other existing methods, the present method has great advantages in segmentation accuracy and can obtain better segmentation performance. Among them, when using VGG16 as the backbone network of the feature extraction network, the evaluation index WF reaches 75.57%, S-measure reaches 85.52%, MAE reaches 0.0029, Em reaches 93.39%, and F reaches 77.38% segmentation accuracy.

[0108] Table 1 Quantitative index results of the proposed method and other saliency segmentation methods on the T-SWX-ray dataset

[0109]

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A fine-grained segmentation method for extremely small defects inside spiral steel pipe welds, characterized by: The following steps are involved: Step 1: Construct a granular spiral steel pipe weld internal defect dataset, including a training set and a test set; both the training set and the test set are composed of original weld defect images and pixel-level segmentation labels, and the saliency detection pixel-level segmentation task is used as the basic data unit; Step 2: construct a fine-grained extremely small defect segmentation network model, which consists of a feature extraction network, a semantic clue generator SCG, a semantic discriminant encoder SDE, a contrast transformation mechanism CTM, a progressive iterative decoding strategy PID and several convolution modules RBC; The feature extraction network extracts the first-level, second-level, third-level and fourth-level feature maps of the original weld defect image respectively, and feeds the extracted feature maps of the original weld defect image into the semantic clue generator SCG to obtain the global semantic clue T scg ; The semantic discriminative encoder SDE combines the second-level, third-level, and fourth-level feature maps extracted by the feature extraction network with the global semantic clues T scg Connect in the channel dimension and apply 3*3 convolutional layers to obtain the three-level output of the semantic discriminative encoder SDE. The three-level output is connected with three contrast transformation mechanisms CTM respectively to obtain the output of three contrast transformation mechanisms CTM, that is, multi-scale output. The multi-scale output and the first-level output of the feature extraction network are integrated and fed into the progressive iterative decoding strategy PID in turn, and the multi-scale output is fused in turn to obtain the final segmentation map. Step 3: Take the original weld defect images and the corresponding pixel-level segmentation labels in the training set as input, train the fine-grained minimal defect segmentation network model, and obtain a converged trained fine-grained minimal defect segmentation network model; Step 4: Use the test images in the test set as defect samples to input the converged trained fine-grained minimal defect segmentation network model, obtain the test set segmentation map after calculation, and get the accurate position and size of the defect corresponding to each test image in the test set.

2. According to claim 1, a fine-grained segmentation method for extremely small defects inside the weld of a spiral steel pipe is characterized by: The pixel-level segmentation labels in the training set of step 1 are the precise positions and sizes of the surface defects represented by the original weld defect images in the training set, and the pixel-level segmentation labels in the test set are the precise positions and sizes of the surface defects represented by the original weld defect images in the test set. In the saliency detection segmentation task, the defects represented by the original weld defect images in the training set and the original weld defect images in the test set are pixel-level binary classification tasks.

3. According to claim 1, a fine-grained segmentation method for extremely small defects inside the weld of a spiral steel pipe is characterized by: The semantic clue generator SCG includes a first dense receptive field DRF unit, a first dense attention guided DAG, a second dense receptive field DRF unit, a second dense attention guided DAG, a third dense receptive field DRF unit, a third dense attention guided DAG, a global average pooling layer and a 1*1 convolution layer; the first dense receptive field DRF unit, the first dense attention guided DAG, the second dense receptive field DRF unit, the second dense attention guided DAG, the third dense receptive field DRF unit and the third dense attention guided DAG are connected in sequence, the global average pooling layer and the 1*1 convolution layer are connected in sequence, and the two are then connected together in parallel, and the feature map of the extracted original weld defect image is fed into the semantic clue generator SCG to obtain the global semantic clue T scg .

4. The fine-grained segmentation method for extremely small defects inside the spiral steel pipe weld according to claim 3 is characterized by: The contrast conversion mechanism CTM includes a group of 3*3 maximum pooling layers, a first decoupling mechanism and a first 2-fold upsampling bilinear interpolation layer connected in sequence, and a group of 3*3 average pooling layers, a second decoupling mechanism and a second 2-fold upsampling bilinear interpolation layer connected in sequence. The two groups are connected together in parallel and fused through a convolution module RBC; the multi-scale output is sequentially passed through the 3*3 maximum pooling layer, the first decoupling mechanism and the first 2-fold upsampling bilinear interpolation layer to obtain the maximum coupled smooth defect feature, and the multi-scale output is sequentially passed through the 3*3 average pooling layer, the second decoupling mechanism and the second 2-fold upsampling bilinear interpolation layer to obtain the average coupled smooth defect feature, and then the maximum coupled smooth defect feature and the average coupled smooth defect feature are fused through the convolution module RBC to obtain the three output features of the contrast conversion mechanism CTM.

5. The fine-grained segmentation method for extremely small defects inside the weld of a spiral steel pipe according to claim 3 is characterized in that: The progressive iterative decoding strategy PID includes three sequentially connected cross-scale weighted modules CSWM; the cross-scale weighted module CSWM includes two sets of parallel spatial attention residual architectures, and two sets of parallel sequentially connected convolutional layers and element-level multiplications; the spatial attention residual architecture is implemented by spatial attention SA, element-level multiplication and element-level addition, and the spatial attention mechanism SA is used to calculate the low-level feature F low and high-level features F high , with low-level features F low and high-level features F high As input, spatial attention is applied to the two features to obtain two different attention maps. The two attention maps are calibrated and residually connected with the original low-level features and high-level features to obtain interactive low-level verification features. and advanced validation features The two different sets of spatial attention architectures are sequentially passed through a convolution module RBC, a 3*3 convolution layer and a Sigmoid activation function to generate weighted information W. mid , the interactive low-level validation features and advanced validation features Respectively with weighted information W mid and 1-W mid Two sets of weighted feature maps are obtained by multiplication. The two sets of weighted feature maps are concatenated according to the channel dimension and element-wise multiplication and element-wise addition operations are performed with another set of spatial attention residual architectures to obtain the output of the cross-scale weighted module CSWM.

6. The fine-grained segmentation method for extremely small defects inside the weld of a spiral steel pipe according to claim 5 is characterized in that: The convolution module RBC consists of a 3*3 convolution layer, a BN normalization layer and a ReLu activation function connected in sequence.

7. The fine-grained segmentation method for extremely small defects inside the weld of a spiral steel pipe according to claim 6 is characterized in that: The step 3 specifically comprises the following steps: Step 3.1: The original weld defect image in the training set and the corresponding pixel-level segmentation label are input into the fine-grained minimal defect segmentation network model as fine-grained minimal defect samples, and the first-level, second-level, third-level and fourth-level feature maps of the original weld defect image are extracted by using the feature extraction network; Step 3.2: Feed the second-level, third-level, and fourth-level feature maps of the original weld defect images in the training set into the semantic clue generator SCG to obtain the global semantic clue T scg ; Step 3.3: The first, second, third and fourth level feature maps of the original weld defect images in the training set are respectively compared with the global semantic clues T scg Connect according to the channel dimension, and apply 3*3 convolutional layers respectively to obtain the three-layer output of the semantic discriminative encoder SDE, and record the three-layer output as F2~F4, and these three-layer outputs are collectively referred to as multi-scale output F; Step 3.4: Inject the obtained multi-scale output F = {F2, F3, F4} into the contrast transformation mechanism CTM respectively to obtain the three output features of the contrast transformation mechanism CTM; Step 3.5: Use the cross-scale weighted modules CSWM connected in sequence in the progressive iterative decoding strategy PID to calculate the first-level features of the feature extraction network and the output features of the three contrast transformation mechanisms CTM, gradually highlight the defect features, and obtain the output result F of each cross-scale weighted module CSWM cswm ={F cswm1 ,F cswm2 ,F cswm3 }, and for feature F cswm1 Perform upsampling operation to obtain the final fused feature map of defects; Step 3.6: Use the cross-scale weighted module CSWM in the progressive iterative decoding strategy PID to decode the final fusion feature map of the defect and calculate the segmentation result of the training set image to obtain the supervised feature map; Step 3.7: The loss between the supervised feature map and the pixel-level segmentation label of the training set is calculated by the cross entropy loss function BCE and the intersection over union loss function IOU, and the network model parameters in the fine-grained minimal defect segmentation network model are iteratively updated using the stochastic gradient descent algorithm. When the cross entropy loss function converges, the training is stopped and all trainable parameters of the network are frozen to obtain a converged and trained fine-grained minimal defect segmentation network model.

8. The fine-grained segmentation method for extremely small defects inside the weld of a spiral steel pipe according to claim 7 is characterized in that: The step 3.4 specifically includes the following steps: Step 3.4.1: Inject the obtained multi-scale outputs F = {F2, F3, F4} into the contrast transformation mechanism CTM respectively, and perform decoupling transformation on each multi-scale output F; Step 3.4.2: Apply RBC blocks to the three multi-scale outputs after the decoupling transformation to fuse them, perform 2*2 average pooling operations and 2*2 maximum pooling operations in parallel, and apply RBC blocks to the two pooled features to fuse them again to obtain two different pooled features {F max ,F avg }, the formula is as follows: Among them, F max represents the maximum pooling feature, F avg represents the average pooling feature, Max(·) represents the 2*2 maximum pooling operation, and Avg(·) represents the 2*2 average pooling operation; Step 3.4.3: Use the decoupling mechanism to decouple the maximum pooling feature and the average pooling feature, further determine the coupling relationship between the defect and the weld, and obtain the maximum coupling defect feature and the average coupling defect feature. The formula is as follows: in, represents the maximum coupling defect characteristic, represents the average coupling defect feature, σ represents the sigmoid activation function, Γ is a 3*3 convolutional layer with a batch normalization layer, ζ represents a 3*3 average pooling operation, and ⊙ represents element-wise multiplication; Step 3.4.4: Maximum coupling defect characteristics and average coupling defect characteristics The maximum coupled smooth defect feature and the average coupled smooth defect feature are obtained by upsampling through a 2-fold upsampling bilinear interpolation layer, and the maximum coupled smooth defect feature and the average coupled smooth defect feature are fused through a convolution module RBC to generate the output feature of the contrast conversion mechanism CTM. Therefore, the three hierarchical features F = {F2, F3, F4} are respectively input into the three contrast conversion mechanisms CTM, that is, the output features F of the three contrast conversion mechanisms CTM are obtained. ctm ={F ctm2 ,F ctm3 ,F ctm4 }.

9. The fine-grained segmentation method for extremely small defects inside the weld of a spiral steel pipe according to claim 8, characterized in that: The step 3.5 specifically includes the following steps: Step 3.5.1: Compute low-level features F using spatial attention mechanism SA low and high-level features F high ; The input features are the first-level feature map of the extraction network and the output features F of the three contrast transformation mechanisms CTM ctm ={F ctm2 ,F ctm3 ,F ctm4 }, the hierarchical features in the sequentially connected cross-scale weighted modules CSWM are {F ctm4 ,F ctm3 }, {F cswm3 ,F ctm2 }, {F cswm2 , the first level feature map}, among the above hierarchical features, the former are all from high-level features, that is, high-level features F high ; The latter are all from low-level features, that is, low-level features F low ; Step 3.5.2: Take low-level features F low and high-level features F high As input, spatial attention is applied to the two features to obtain two different attention maps, which are calibrated and residually connected with the original features to obtain interactive low-level verification features. and advanced validation features The formula is as follows: Where SA(·) represents spatial attention; Step 3.5.3: Low-level validation features along the channel dimension and advanced validation features Splicing is performed, and the splicing features are fused using the RBC module. The 3*3 convolution layer, batch normalization, and Sigmoid activation function are applied in sequence to activate the feature vector to generate the activation weight map W mid , the formula is as follows: Step 3.5.4: Use 3*3 convolutional layers to fuse low-level features F low With advanced features F high , and use the activation weight map W mid Perform element-wise multiplication with the fused high-level features to obtain a high-level weighted feature map Using the weighted activation feature vector 1-W mid Multiply with the fused low-level features to obtain a low-level weighted feature map High-level weighted feature maps according to channel dimension and low-level weighted feature maps Splicing is performed to obtain the intermediate fusion features. The formula is as follows: Among them, F mid Represents the generated intermediate fusion features; Step 3.5.5: Using the intermediate fusion feature F mid Advanced Verification Features Calibrate by using the Advanced Calibration feature and the intermediate fusion feature F mid Perform multiplication and residual connection to obtain the final fusion feature output F CSWM , the formula is as follows: Among them, F CSWM Represents the final fused feature map of defects.

Images