Small-diameter pipe welding defect detection method based on space-frequency characteristic cooperative enhancement

Abstract

The invention belongs to the technical field of industrial defect detection, and particularly relates to a small-diameter tube welding defect detection method based on space-frequency characteristic collaborative enhancement. The defect detection model comprises a backbone network based on a defect feature multi-dimensional sensing unit, a feature enhancement module based on space-frequency feature joint enhancement, a feature fusion pyramid network and a final detection head; the backbone network based on the defect feature multi-dimensional sensing unit comprises three defect feature multi-dimensional sensing units which are sequentially connected, and the feature enhancement module based on space-frequency feature joint enhancement comprises a frequency domain detail attention layer, a space-frequency fusion buffer layer and a spatial pyramid pooling layer. Output feature maps of the three defect feature multi-dimensional sensing units are input feature maps of a frequency domain detail attention layer, a space-frequency fusion buffer layer and a spatial pyramid pooling layer respectively, and three enhanced feature maps output by a feature enhancement module based on space-frequency feature joint enhancement are input into a feature fusion pyramid network; three output feature maps of the feature fusion pyramid network respectively pass through a final detection head to obtain a defect detection result. According to the method, the capability of sensing the edge and detail information of the welding defects is enhanced, and the method has better detection precision for the welding defects with different scales and shapes.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial defect detection, and specifically is a small-diameter pipe welding defect detection method based on space-frequency feature collaborative enhancement, which can be used for defect detection of various welding structures in the industrial field. Background Art

[0002] Small-diameter pipes are widely used in the power, petrochemical and other industries. Small-diameter pipes generally refer to pipes with an outer diameter of less than or equal to 100 mm, which are mainly used for the transportation and heat exchange of steam, combustible gas, oil, and reaction materials. A large part of them operate under high temperature, high pressure or corrosive medium conditions. Therefore, the welding quality of small-diameter pipes fundamentally determines whether the entire heating surface can work normally. The welding quality problems of small-diameter pipes will lead to potential safety hazards, which seriously threaten the safety of equipment and staff. Due to the small radius of curvature and small wall thickness of small-diameter pipes, it is difficult to perform ultrasonic testing. At present, the detection of small-diameter pipe welding defects is generally achieved through X-ray non-destructive testing technology.

[0003] Traditional methods mainly rely on manual work to determine whether there are defects in X-ray weld images. This is not only highly subjective and overly dependent on the experience and technical level of the assessors, but also inefficient, time-consuming and labor-intensive, affecting the quality and speed of factory production. With the continuous development of computer vision technology, computer image-assisted detection has become a trend. Traditional machine learning methods identify defects by classifying the features of each pixel and its surrounding local neighborhood, but they require manual design and selection of features and cannot achieve true automatic recognition. The target detection technology based on deep learning can automatically extract target features in the image without the need for manual design and selection of features, reducing the cost of manual intervention and the impact of subjective factors.

[0004] However, the deep learning-based defect detection method directly inputs the preprocessed images into the model for training. The convolutional network will learn the texture, color and other features of the defects, and finally locate and classify the defects. Since the small-diameter pipe welding defect images usually show low contrast, the defect details are not easy to perceive, and the grayscale difference between the defect edge and the background is not obvious, there is a problem that welding defects are difficult to detect effectively. Summary of the invention

[0005] In view of the deficiencies in the prior art, the technical problem that the present invention intends to solve is to propose a small-diameter pipe welding defect detection method based on collaborative enhancement of space-frequency features.

[0006] The present invention solves the technical problem by adopting the following technical solution:

[0007] A method for detecting small-diameter pipe welding defects based on space-frequency feature collaborative enhancement, characterized in that the method comprises the following steps:

[0008] The first step is to obtain several X-ray small-diameter tube welding images and normalize their sizes to obtain a data set;

[0009] The second step is to build a welding defect detection model, including a backbone network based on a multi-dimensional perception unit of defect features, a feature enhancement module based on joint enhancement of space-frequency features, a feature fusion pyramid network and a final detection head.

[0010] The backbone network based on the defect feature multidimensional perception unit includes three defect feature multidimensional perception units connected in sequence, and the defect feature multidimensional perception unit includes a parallel frequency domain feature perception branch and a spatial domain feature perception branch; the input feature map of the frequency domain feature perception branch is decomposed into a low frequency subband, a horizontal high frequency subband, a vertical high frequency subband and a diagonal high frequency subband through a two-dimensional discrete wavelet transform; after the four subbands are subjected to wavelet domain differential convolution, they are subjected to inverse wavelet transform and 1×1 convolution in sequence to obtain the output feature map of the frequency domain feature perception branch; wherein the wavelet domain differential convolution is expressed as:

[0011] WDDC(X)=CDC-F(Concat(X LL ,HDC(X HL ),VDC(X LH ),X HH )) (1)

[0012] Where WDDC(X) is the output feature map of wavelet domain differential convolution, CDC-F(·) is the frequency domain center differential convolution, Concat(·) is the concatenation operation, HDC(·) is the horizontal differential convolution, VDC(·) is the vertical differential convolution, X LL , X HL , X LH and X HH They are low-frequency sub-band, horizontal high-frequency sub-band, vertical high-frequency sub-band and diagonal high-frequency sub-band;

[0013] The spatial domain feature perception branch includes five convolutional layers, each of which is connected to a normalization layer and an activation layer; the output feature maps of the first convolutional layer, the third convolutional layer, and the fifth convolutional layer are connected to the output feature map of the frequency domain feature perception branch, and then a 1×1 convolution is performed to obtain the output feature map of the defect feature multidimensional perception unit;

[0014] The feature enhancement module based on joint enhancement of space-frequency features comprises a frequency domain detail attention layer, a space-frequency fusion buffer layer, and a space pyramid pooling layer. The output feature map C3 of the first defect feature multidimensional perception unit is enhanced by the frequency domain detail attention layer to obtain a feature map C3', the output feature map C4 of the second defect feature multidimensional perception unit is enhanced by the space-frequency fusion buffer layer to obtain a feature map C4', and the output feature map C5 of the third defect feature multidimensional perception unit is enhanced by the space pyramid pooling layer to obtain a feature map C5';

[0015] In the frequency domain detail attention layer, the feature map C3 is decomposed into four sub-bands: low frequency, horizontal high frequency, vertical high frequency, and diagonal high frequency through two-dimensional discrete wavelet transform; the four sub-bands are stacked in the frequency domain dimension to obtain the feature map X 11 ; Concatenate the four sub-bands in order on the channel dimension to obtain the feature map X2; 11 The channels and frequency domain dimensions of are swapped to obtain the feature map X 12 ; Feature map X 11 and X 12 After global average pooling, 3D convolution and Sigmoid activation function respectively, the channel attention weight w about the frequency band dimension is obtained. 11 The sum of the band’s attention weights w about the channel dimension 12 ; The feature map X2 is sequentially subjected to global average pooling, 2D convolution and Sigmoid activation function to obtain the channel-band mixed attention weight w2; the four sub-bands are divided according to the feature map X 11 After feature alignment with the shape of the channel, the attention weight w about the frequency band dimension 11 , the attention weight w of the frequency band on the channel dimension 12 Element-by-element multiplication is performed, and after sub-band decoupling, it is spliced ​​in the channel dimension according to the shape of the feature map X2. The spliced ​​feature map undergoes 2D convolution, element-by-element multiplication with the channel-band mixed attention weight w2, and then undergoes inverse wavelet transform. It is residually connected with the input feature map of the frequency domain detail attention layer to obtain the enhanced feature map C3';

[0016] In the space-frequency fusion buffer layer, the feature map C4 is sequentially subjected to fast Fourier transform, 2D convolution and inverse Fourier transform, and then concatenated with the feature map C4 after 2D convolution in the channel dimension to obtain the enhanced feature map C4';

[0017] The enhanced feature maps C5', C4' and C3' are input into the feature fusion pyramid network, and the feature fusion pyramid network outputs feature maps P5, P4 and P3; the feature maps P5, P4 and P3 are respectively input into the final detection head for welding defect detection;

[0018] The third step is to construct a loss function, train the welding defect detection model, and use the trained welding defect detection model for small-diameter pipe welding defect detection.

[0019] Furthermore, the frequency domain central difference convolution is sampled by three-dimensional convolution, first sampling the input feature map to obtain a local receptive field cube , and then get the sampling value through weighted summation aggregation; the three-dimensional convolution is expressed as:

[0020]

[0021] In the formula, Vanilla(p i ) represents the current position p in the feature map i The sampling value, p n is the local receptive field cube The relative position offset in n ) is the relative position offset p n The corresponding weight, x(p i +p n ) is the position p i +p n The characteristic value of

[0022] Relative position offset p in the current frequency step n The weight is calculated by the following formula:

[0023]

[0024] In the formula, is the relative position offset p in the current frequency step n The weight of are the relative position offset p in the previous frequency step and the next frequency step respectively. n The weight of

[0025] The frequency domain central difference convolution is expressed as:

[0026]

[0027] Where CDC-F(·) is the frequency domain central difference convolution, θ is a hyperparameter, and R -1 Indicates the previous frequency step, R 1 Indicates the next frequency step, R 0 Indicates the current frequency step.

[0028] Furthermore, three aspect ratio-weighted detection heads are introduced in the model training process for auxiliary training. The three feature maps on the top-down feature aggregation path of the feature fusion pyramid network are the inputs of the three aspect ratio-weighted detection heads. The total loss of model training is calculated by the following formula;

[0029] L total =L Last +εL ARDS (12)

[0030] Where, L total is the total loss of model training, L Last is the total loss of the final detection head, L ARDS is the total loss of the detection head weighted based on the aspect ratio, ε is the weight coefficient;

[0031] The total loss of the final detection head is L Last Calculated by the following formula:

[0032]

[0033] In the formula, and are the bounding box loss, confidence loss, and classification loss of the final detection head, All are weight coefficients;

[0034] Bounding box loss of the final detection head Calculated by the following formula:

[0035]

[0036] Where, CIoU k is the CIoU value between the kth positive sample in the current batch and the true bounding box, and K is the number of positive samples in the current batch;

[0037] The total loss L of the detection head based on the aspect ratio weighting ARDS Calculated by the following formula:

[0038]

[0039] In the formula, and is the bounding box loss, confidence loss, and classification loss of the detection head weighted by aspect ratio, are weight coefficients;

[0040] Bounding box loss for the detection head based on aspect ratio weighting Calculated by the following formula:

[0041]

[0042] In the formula, Q1 is the central positive sample set, n1 represents the number of central positive samples, Q2 is the additional selected positive sample set, and h k 、w k are the length and width of the kth positive sample, h j 、w jare the length and width of the jth positive sample respectively.

[0043] Furthermore, in the final detection head, the input feature map is gridded, the true bounding box is mapped to the feature map, the grid cell where the center point of the true bounding box is located is determined, and the grid cell and the two grid cells above or below, left or right of the center point of the true bounding box are marked as positive sample grid cells; an anchor box is generated on each positive sample grid cell, and the IoU value of the anchor box and the true bounding box is calculated. If the IoU value exceeds the set threshold, the anchor box is regarded as a positive sample, otherwise it is regarded as a negative sample; in the detection head based on aspect ratio weighting, on the basis of the positive sample grid cells of the final detection head, the upper or lower grid cells adjacent to and farther away from the center point of the true bounding box are also marked as positive sample grid cells.

[0044] Furthermore, the convolution kernel size of the first convolution layer of the spatial domain feature perception branch is 1×1, the padding is 0, and the stride is 1, and the convolution kernel size of the remaining convolution layers is 3×3, the padding is 1, and the stride is 1.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The backbone network based on the defect feature multi-dimensional perception unit extracts defect features from multiple dimensions (frequency domain and spatial domain) and multiple scales (different frequency sub-bands, different receptive fields). After fusing the spatial domain features with the frequency domain features, a richer defect feature representation can be obtained. In the defect feature multi-dimensional perception unit, wavelet domain differential convolution is used to enhance the expression of defect edge information and perceive the detailed differences of defect features in different frequency sub-bands, thereby enhancing the discriminability between defects and backgrounds.

[0047] Since deep feature maps contain rich semantic and position information, and shallow feature maps contain more defect details and texture information, the feature enhancement module based on joint enhancement of space-frequency features designs different enhancement methods for different characteristics of feature maps at different levels: deep feature maps perceive the semantic information of targets of different scales through spatial pyramid pooling operations; shallow feature maps use frequency domain detail attention mechanisms to enhance the expression of defect detail texture information and enhance the model's perception of tiny defects; since deep and shallow feature maps are enhanced in different domains, in order to avoid significant feature deviations in the subsequent feature fusion process, the intermediate feature map is used as the input of the space-frequency fusion buffer layer. The feature enhancement module based on joint enhancement of space-frequency features makes full use of the complementarity of deep and shallow feature maps in semantic information and detail information, thereby improving the model's perception of targets of different scales and tiny defects.

[0048] The feature fusion pyramid network adopts the PAFPN structure, uses top-down and bottom-up path aggregation methods and jump connections, effectively fuses feature maps from different levels, and improves the expression ability of multi-scale features and the robustness of the model.

[0049] On the basis of the original three final detection heads, three detection heads based on aspect ratio weighting are introduced to supervise the feature fusion effect of the intermediate layer, accelerate model convergence, and improve model stability. At the same time, the detection head based on aspect ratio weighting is more inclined to learn defects with larger aspect ratios and slender shapes in the positive sample selection strategy, which can balance the loss contribution of imbalanced categories. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a structural schematic diagram of the welding defect detection model of the present invention;

[0051] Figure 2 It is a principle diagram of the wavelet domain differential convolution and the frequency domain central differential convolution of the present invention;

[0052] Figure 3 It is a schematic diagram of the structure of the frequency domain detail attention layer and the space-frequency fusion buffer layer of the present invention;

[0053] Figure 4 Schematic diagram of positive sample grid unit sampling of the final detection head and the detection head based on aspect ratio weighting of the present invention. DETAILED DESCRIPTION

[0054] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods, but the protection scope of the present application is not limited thereto.

[0055] The present invention is a small diameter pipe welding defect detection method based on space-frequency feature collaborative enhancement (hereinafter referred to as method, see Figure 1-4 ), comprising the following steps:

[0056] The first step is to obtain a certain number of X-ray small-diameter tube welding images with defect oriented bounding boxes (OBB) and defect category labels, and perform size normalization on them. The image size is 300*300 pixels to obtain a data set for training the model. As an embodiment, the data set for training the model contains 2902 images, and the defect category labels include: circular defect, incomplete penetration and incomplete fusion defect, weld nodule defect, circular foreign body defect, strip defect, and strip foreign body defect.

[0057] The second step is to build a welding defect detection model based on the deep learning network; the welding defect detection model consists of four parts: a backbone network based on the multi-dimensional perception unit (MDPU) of defect features, a feature enhancement module based on spatial-frequency joint enhancement layer (SFJE), a feature fusion pyramid network (PAFPN) and three final detection heads (Last head).

[0058] like Figure 1 As shown in the figure, the small-diameter pipe defect image is extracted through the backbone network, and the results of each layer of MDPU are input into SFJE. SFJE performs targeted enhancement according to the different characteristics of each layer of feature map. The feature map after feature enhancement is input into the PAFPN feature fusion network to improve the expression ability of multi-scale features. The result after feature fusion is input into the Last head. Since the deep supervision strategy is used, the intermediate features are input into the ARDS head for auxiliary training.

[0059] The backbone network based on defect feature multidimensional perception unit includes three defect feature multidimensional perception units connected in sequence. The three MDPUs output feature maps of different levels. The feature map C3 output by the first MDPU serves as the input of the second MDPU, and the feature map C4 output by the second MDPU serves as the input of the third MDPU. The third MDPU outputs feature map C5. The feature map C5 contains rich semantic information, and the feature map C3 contains more texture detail information. The feature maps of three different levels serve as the input of the feature enhancement layer based on the joint enhancement of space-frequency features.

[0060] The MDPU includes two feature perception branches, namely, a frequency domain feature perception branch and a space domain feature perception branch; the frequency domain feature perception branch includes a two-dimensional discrete wavelet transform (Discrete Wavelet Transform, DWT), a wavelet-domain differential convolution (Wavelet-domain Differential Convolution, WDDC), an inverse wavelet transform (InverseDiscrete Wavelet Transform, IWT) and a convolution for adjusting the channel dimension;

[0061] The input feature map of MDPU is decomposed into low-frequency subbands X by DWT. LL , horizontal high frequency subband X HL , vertical high frequency subband X LH and diagonal high frequency subband X HH ; Low frequency subband X LLRepresents the low-frequency information of the feature map, mostly background information; horizontal high-frequency subband X HL and vertical high frequency subband X LH Respectively represent the high-frequency information of the feature map in the horizontal and vertical directions, describing the horizontal edge features and vertical edge features of the feature map; the diagonal high-frequency subband X HH Describes the diagonal high-frequency information of the feature map.

[0062] The four sub-bands are input into the wavelet domain differential convolution WDDC, which includes three differential convolutions: horizontal differential convolution (HDC), vertical differential convolution (VDC) and frequency domain center differential convolution (CDC-F); the high frequency sub-band X in the horizontal direction HL The horizontal edge features are further enhanced through horizontal differential convolution to obtain the enhanced horizontal high-frequency sub-band; the vertical high-frequency sub-band X LH The vertical edge features are further enhanced through vertical differential convolution to obtain the enhanced vertical high-frequency sub-band; the low-frequency sub-band X LL , diagonal high frequency subband X HH , the enhanced horizontal high-frequency sub-band and the vertical high-frequency sub-band are spliced ​​to obtain the input feature map of the frequency domain center difference convolution, and the wavelet domain difference convolution WDDC is expressed by the following formula:

[0063] WDDC(X)=CDC-F(Concat(X LL ,HDC(X HL ),VDC(X LH ),X HH )) (1)

[0064] Where WDDC(X) is the output feature map of wavelet domain differential convolution WDDC, CDC-F(·) is the frequency domain center differential convolution, Concat(·) is the concatenation operation, HDC(·) is the horizontal differential convolution, and VDC(·) is the vertical differential convolution.

[0065] The frequency domain center difference convolution CDC-F captures the frequency domain difference information between adjacent frequency steps to perceive the subtle difference between defects and background, thereby enhancing the discriminability between defects and background. The frequency domain center difference convolution CDC-F uses three-dimensional convolution for sampling. First, the local receptive field cube is sampled on the input feature map. , and then the sampling value is obtained by weighted summation aggregation, so the three-dimensional convolution can be expressed as:

[0066]

[0067] In the formula, Vanilla(p i ) represents the current position p in the feature map iThe sampling value, p n is the local receptive field cube The relative position offset in n ) is the relative position offset p n The corresponding weight, x(p i +p n ) is the position p i +p n The characteristic value of

[0068] In order to highlight the difference in frequency domain dimension, the frequency domain center difference convolution CDC-F adjusts the weighting method, keeps the local receptive field cube unchanged, and assigns the weight of the current frequency step to the sum of the weights of the adjacent frequency steps. Then the relative position offset p in the current frequency step n The weight can be calculated by the following formula:

[0069]

[0070] In the formula, is the relative position offset p in the current frequency step n The weight of are the relative position offset p in the previous frequency step and the next frequency step respectively. n The weight of

[0071] The operation of frequency domain center difference convolution CDC-F can be expressed as:

[0072]

[0073] Where CDC-F(·) is the frequency domain central difference convolution, θ is a hyperparameter, and R -1 Indicates the previous frequency step, R 1 Indicates the next frequency step, R 0 Indicates the current frequency step.

[0074] The output feature map of WDDC is restored to the spatial domain through inverse wavelet transform, and then the dimension is adjusted through a 1×1 convolution to obtain the frequency domain feature map, that is, the output feature map of the frequency domain feature perception branch.

[0075] The spatial domain feature perception branch includes five convolutional layers for extracting spatial domain feature maps at different levels. Each convolutional layer is connected to a normalization layer and an activation layer, and the activation function adopts the SiLU function. The parameter settings of each convolutional layer are shown in Table 1.

[0076] Table 1 Parameters of each convolutional layer of the spatial domain feature perception branch

[0077]

[0078] The output feature maps of the first convolutional layer Conv 1, the third convolutional layer Conv 3, and the fifth convolutional layer Conv 5 are connected to the output feature map of the frequency domain feature perception branch, and then the channel dimension is adjusted through 1×1 convolution to obtain the output feature map of the defect feature multidimensional perception unit.

[0079] The feature enhancement module based on joint enhancement of space-frequency features consists of three layers: frequency detail attention (FDA) layer, spatial-frequency fusion buffer (SFB) layer, and spatial pyramid pooling (SPPCSPC) layer. The input feature map of the FDA layer is C3, the input feature map of the SFB layer is C4, and the input feature map of the SPPCSPC layer is C5. The output feature map of the feature enhancement module based on joint enhancement of space-frequency features is the enhanced feature map C5. ’ , C4' and C3'.

[0080] The FDA layer aims to enhance the texture detail information of shallow features. The feature map C3 is decomposed into four sub-bands: low frequency, horizontal high frequency, vertical high frequency, and diagonal high frequency. The four sub-bands are stacked in the frequency domain to obtain the feature map X 11 ; Concatenate the four sub-bands in order on the channel dimension to obtain the feature map X2; 11 The channels and frequency domain dimensions of are swapped to obtain the feature map X 12 ; Feature map X 11 and X 12 After the first frequency domain attention branch, the channel attention weight w about the frequency band dimension is obtained. 11 The sum of the band’s attention weights w about the channel dimension 12 , realizing bidirectional attention between channels and frequency bands; the first frequency domain attention branch aggregates the spatial dimensions through the global average pooling operation, and then fine-tunes and activates the feature map through 3D convolution and Sigmoid function, which can be described by the following formula:

[0081] w 11 =σ(W(GAP(X 11 ))) (5)

[0082] w 12 =σ(W(GAP(X 12 ))) (6)

[0083] Where GAP represents the global average pooling operation, W is the learnable weight matrix, and σ represents the Sigmoid activation function.

[0084] The feature map X2 is input into the second frequency domain attention branch, and the spatial dimension is squeezed through the global average pooling operation. Then, the feature response of the channel-frequency domain mixed dimension is adaptively calibrated through the 2D convolution operation. After the Sigmoid activation function, the channel-frequency mixed attention weight w2 is obtained, which can be described by the following formula:

[0085] w2=σ(W(GAP(X2))) (7)

[0086] The four subbands are divided according to the feature map X 11 After feature alignment with the shape of the channel, the attention weight w about the frequency band dimension 11 , the attention weight w of the frequency band on the channel dimension 12 Element-by-element multiplication is performed, and after sub-band decoupling, it is spliced ​​in the channel dimension according to the shape of the feature map X2. The spliced ​​feature map is 2D convolved, element-by-element multiplied with the channel-band mixed attention weight w2, and then converted to the spatial domain through inverse wavelet transform. It is residually connected with the input feature map of the FDA layer to obtain the output feature map of the FDA layer, that is, the enhanced feature map C3'.

[0087] In order to avoid significant feature deviations during feature fusion, the SFB layer is used to smooth the feature fusion process. The SFB layer contains two branches. The first branch calibrates the feature map C4 in the channel dimension through a 2D convolution; the second branch first performs a fast Fourier transform on the feature map C4 to extract spectral features, and then after 2D convolution adjustment, the features are converted to the spatial domain through an inverse Fourier transform; the output feature maps of the two branches are spliced ​​in the channel dimension to obtain the output feature map of the SFB layer, that is, the enhanced feature map C4'.

[0088] The enhanced feature maps C5', C4' and C3' are input into the feature fusion pyramid network. The feature fusion pyramid network PAFPN adopts top-down and bottom-up feature aggregation methods. On the one hand, the information of each feature map is fused layer by layer through upsampling, so that the low-level feature map can better integrate the information of the high-level feature map; on the other hand, the detail information of the low-level feature map is propagated layer by layer to the high-level feature map through downsampling, while maintaining the semantic information, the detail expression ability of the high-resolution feature map is enhanced; on the feature map obtained by fusion at each layer, the multi-level feature map is further fused through jump connection, and three feature maps of different resolutions P5, P4 and P3 are obtained.

[0089] The feature maps P5, P4, and P3 are input into the three final detection heads respectively for welding defect detection. The input image is divided into S×S grid cells according to the feature maps, and each grid cell generates a set of anchor boxes to predict whether the grid cell contains the target. First, the ground truth (GT) box is mapped to the feature map, and the grid cell where the center point of the ground truth box is located is determined. The grid cell and the two grid cells above, below, left, or right of the center point of the GT box are marked as positive sample grid cells. On each positive sample grid cell, an anchor box is generated according to the anchor box size preset in the input feature map, and the IoU value between the anchor box and the GT box is calculated. If the IoU value exceeds the set threshold, the anchor box is regarded as a positive sample, which is used to calculate the loss with the GT box and is used to predict the category and bounding box of the ground truth box. If the IoU value is lower than the set threshold, the anchor box is regarded as a negative sample and does not participate in the prediction of the ground truth box.

[0090] The third step is to construct a loss function, train the welding defect detection model, and use the trained welding defect detection model for small-diameter pipe welding defect detection;

[0091] The total loss of the final detection head Last head consists of three parts: bounding box loss Confidence loss and classification loss The bounding box loss is used to optimize the positional relationship between the predicted box and the true bounding box, the confidence loss is used to measure whether there is an object in the predicted box, and the classification loss is used to measure whether the target category in the predicted box is correctly predicted; the total loss L Last The calculation formula is:

[0092]

[0093] In the formula, All are weight coefficients;

[0094] Among them, the bounding box loss Calculated by the following formula:

[0095]

[0096] Where, CIoU k is the CIoU value between the kth positive sample in the current batch and the true bounding box, and K is the number of positive samples in the current batch;

[0097] In the small-diameter pipe welding defect detection task, taking into account the large aspect ratio, slender shape, small number of defect instances, high defect hazard and high detection difficulty of defects such as unfused and incomplete welds, a deep supervision strategy is adopted to introduce three aspect ratio weighted detection heads (Aspect Ratio-aware Depth Supervision head, ARDS head). The three feature maps on the top-down feature aggregation path of the feature fusion pyramid network are input into the three ARDS heads respectively. When the ARDS head selects the positive sample grid cell, the upper or lower grid cells adjacent to and farther from the center point of the GT box will also be marked as positive sample grid cells based on the positive sample grid cells of the Last head. Similarly, the total loss of the ARDS head also includes the bounding box loss. Confidence loss and classification loss The calculation formula is:

[0098]

[0099] Where, L ARDS is the total loss of the ARDS head, are weight coefficients;

[0100] In ARDS head, the positive samples with the same position as the last head are regarded as the central positive samples. For the additionally selected positive samples, the aspect ratio weighting strategy is used to weight them according to the aspect ratio and area of ​​the predicted box. The bounding box loss is Calculated by the following formula:

[0101]

[0102] In the formula, Q1 is the central positive sample set, n1 represents the number of central positive samples, Q2 is the additional selected positive sample set, and h k 、w k are the length and width of the kth positive sample, h j 、w j are the length and width of the jth positive sample respectively.

[0103] Through deep supervision and aspect ratio weighting strategy, ARDS head can increase the number of positive samples, balance the loss contribution of imbalanced category defects, and improve the detection rate of long strip defects.

[0104] Since ARDS head supervises the intermediate feature map, it is only used to balance the loss contribution of imbalanced categories and the auxiliary training of the model. Compared with Lasthead, the prediction accuracy is usually slightly lower. Therefore, a smaller weight coefficient ε is introduced to reduce the impact of ARDShead on the total loss of model training. The calculation formula of the total loss of model training is:

[0105] L total =L Last +εL ARDS (12).

[0107] Example

[0108] Set the maximum training rounds, learning rate and batch size, and use the random assignment method to initialize the training parameters of the defect detection model; change the data set in the first step to a size of 640*640 through grayscale filling operation to suit model training, and then divide it into several equal parts according to the quantity ratio, and divide no less than 60% of the shares into training sets, no less than 10% of the shares into validation sets, and the remaining shares into test sets to obtain a series of data sets; keep the ratio of the number of pictures in the training set, validation set, and test set unchanged, and exchange the pictures contained in them in equal quantities according to the shares, and perform multiple cross transformations to obtain multiple series of data sets;

[0109] For a batch of X-ray small-diameter tube welding images in a training set in a series of data sets, the defect detection model outputs the location box and classification information of the detected small-diameter tube welding defects. The detection results and their annotation information are input into the final loss formula of the defect detection model in the third step to obtain the detection loss of this batch of X-ray small-diameter tube welding images. According to the detection loss, the training parameters of the defect detection model are updated once by back propagation to complete the iterative training of a batch of images; then the next batch of X-ray small-diameter tube welding images in the training set are input into the defect detection model with the updated training parameters, and then the training parameters of the defect detection model are updated again according to the detection loss, and the cycle continues until the iterative training of the last batch of images in a round of the training set is completed. After a round of training is completed, the validation set images in the same series of data sets are input into the model that has completed a round of training, so as to observe the dynamic changes in the model effect of each round in the training; proceed to the next In the training of each round, the training parameters of the previous round of training are used as the initial parameters of the next round of training, and the cycle continues until the training of the last round of images in the training set is completed. When the training reaches the maximum training round, the training is stopped, and the training parameters of the last round of training are saved to obtain the defect detection model trained by the series of data sets. The average recall rate of the model prediction value and the true value of the validation set images is calculated. When the average recall rate is not less than 70%, it is the effective model of the defect detection model trained this time; if the average recall rate is too small, adjust the hyperparameters such as the learning rate and batch, and restart the training from the first round;

[0110] Using the same method, the defect detection model is trained using multiple series of data sets, and the test set in the corresponding series of data sets is input into the defect detection model trained by the corresponding series of data sets. The model prediction value and the true value of the image defect category are evaluated using mAP (average accuracy of all categories, also known as average precision of all categories) and average recall rate AR (Average Recall). The defect detection model with the highest average recall rate is preferred. When the average recall rates are the same, the defect detection model with the highest average precision of all categories is selected; the selected defect detection model is the optimal defect detection model. As an embodiment, during initialization, the model parameters are set to the maximum training round Epoch = 300, the learning rate lr = 0.01, the image input size Image_scale = 640*640, the input batch size of each batch = 16, and the graphics card uses Tesla V100.

[0111] The X-ray small-diameter tube welding image to be inspected is subjected to a linear transformation operation to enhance the light and dark contrast between the defect and the background. The input image is then gray-filled to a size of 640×640 and input into the trained welding defect detection model to obtain the category and location information of the defect.

[0112] In order to verify the effectiveness of the method of the present invention, the same small-diameter pipe welding defect data set was used for testing. The baseline model uses YOLOv7, which includes a backbone network, a PAFPN feature fusion network and three final detection heads. The backbone network does not introduce frequency domain information. The test results show that the baseline model performs poorly in defect detection. Due to insufficient image information mining, the detection rate is low for defects with unclear differences between the edge and the background. The present invention introduces frequency domain information on the baseline model, extracts defect features through the backbone network composed of MDPU, and then enhances the defect features in a targeted manner by the SFJE module, which can effectively improve the detection accuracy of defects. In addition, defects such as unfused and incomplete penetration are also difficult to be effectively detected by the baseline model due to the small number of instances and the fact that most of them have pathological aspect ratios. After adding ARDS head auxiliary training, the detection rate of unfused and incomplete penetration defects increased significantly.

[0113] Table 2 Ablation experimental results of the present invention (%)

[0114]

[0115] Among them, R (Recall) represents the recall rate of a single-class detection, and AP (Average Precision) comprehensively considers the recall rate and precision rate of the model for a single-class defect, calculates the precision rate under each recall rate through different recall rate thresholds, and then averages these precision rates. AR (Average Recall) represents the average recall rate of multiple classes of defects, and mAP represents the average AP rate of multiple classes of defects.

[0116] From the above test results, it can be found that the baseline model has serious missed detection phenomena for strip defects, strip foreign bodies, and unfused and incomplete weld defects. The defect detection model of the present invention uses MDPU to introduce frequency domain information in the feature extraction stage, and then uses the SFJE module to synergistically enhance the frequency domain and spatial domain features according to the characteristics of different feature maps. Compared with the baseline model, the detection rate of defects such as circular defects, circular foreign bodies, and strip defects, which are easily confused with the background and have small differences, has been improved, and the recall rate of strip defects has been improved the most significantly, reaching 84.9%. Unfused and incomplete weld defects are slender in shape, have a large longitudinal span, and have a small number of defect instances. It is difficult for the baseline model to learn effective feature information. After adding the ADRS detection head, the loss contribution of this defect category is balanced through the aspect ratio weighted deep supervision strategy, and the selection range of positive samples is relaxed, so that more discriminative defect features can be learned. The defect detection model of the present invention significantly improves the detection rate of unfused and incomplete weld defects, and the recall rate is increased by 6.5% compared with the baseline model.

[0117] In summary, the defect detection model of the present invention can effectively improve the detection rate of defects by introducing frequency domain features in the feature extraction and feature enhancement stages and combining the aspect ratio weighted deep supervision method.

[0118] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A small diameter pipe welding defect detection method based on space-frequency feature collaborative enhancement, characterized in that: The method comprises the following steps: The first step is to obtain several X-ray small-diameter tube welding images and normalize their sizes to obtain a data set; The second step is to build a welding defect detection model, including a backbone network based on a multi-dimensional perception unit of defect features, a feature enhancement module based on joint enhancement of space-frequency features, a feature fusion pyramid network and a final detection head. The backbone network based on the defect feature multidimensional perception unit includes three defect feature multidimensional perception units connected in sequence, and the defect feature multidimensional perception unit includes a parallel frequency domain feature perception branch and a space domain feature perception branch; The input feature map of the frequency domain feature perception branch is decomposed into low-frequency sub-band, horizontal high-frequency sub-band, vertical high-frequency sub-band and diagonal high-frequency sub-band through two-dimensional discrete wavelet transform; the four sub-bands are subjected to wavelet domain differential convolution, and then sequentially subjected to inverse wavelet transform and 1×1 convolution to obtain the output feature map of the frequency domain feature perception branch; among which, the wavelet domain differential convolution is expressed as: WDDC(X)=CDC-F(Concat(X LL ,HDC(X HL ),VDC(X LH ),X HH )) (1) Where WDDC(X) is the output feature map of wavelet domain differential convolution, CDC-F(·) is the frequency domain center differential convolution, Concat(·) is the concatenation operation, HDC(·) is the horizontal differential convolution, VDC(·) is the vertical differential convolution, X LL , X HL , X LH and X HH They are low-frequency sub-band, horizontal high-frequency sub-band, vertical high-frequency sub-band and diagonal high-frequency sub-band; The spatial domain feature perception branch includes five convolutional layers, each of which is connected to a normalization layer and an activation layer; the output feature maps of the first convolutional layer, the third convolutional layer, and the fifth convolutional layer are connected to the output feature map of the frequency domain feature perception branch, and then a 1×1 convolution is performed to obtain the output feature map of the defect feature multidimensional perception unit; The feature enhancement module based on joint enhancement of space-frequency features comprises a frequency domain detail attention layer, a space-frequency fusion buffer layer, and a space pyramid pooling layer. The output feature map C3 of the first defect feature multidimensional perception unit is enhanced by the frequency domain detail attention layer to obtain a feature map C3', the output feature map C4 of the second defect feature multidimensional perception unit is enhanced by the space-frequency fusion buffer layer to obtain a feature map C4', and the output feature map C5 of the third defect feature multidimensional perception unit is enhanced by the space pyramid pooling layer to obtain a feature map C5'; In the frequency domain detail attention layer, the feature map C3 is decomposed into four sub-bands: low frequency, horizontal high frequency, vertical high frequency, and diagonal high frequency through two-dimensional discrete wavelet transform; the four sub-bands are stacked in the frequency domain dimension to obtain the feature map X 11 ; Concatenate the four sub-bands in order on the channel dimension to obtain the feature map X2; 11 The channels and frequency domain dimensions of are swapped to obtain the feature map X 12 ; Feature map X 11 and X 12 After global average pooling, 3D convolution and Sigmoid activation function respectively, the channel attention weight w about the frequency band dimension is obtained. 11 The sum of the band’s attention weights w about the channel dimension 12 ; The feature map X2 is sequentially subjected to global average pooling, 2D convolution and Sigmoid activation function to obtain the channel-band mixed attention weight w2; the four sub-bands are divided according to the feature map X 11 After feature alignment with the shape of the channel, the attention weight w about the frequency band dimension 11 , the attention weight w of the frequency band on the channel dimension 12 Element-by-element multiplication is performed, and after sub-band decoupling, it is spliced ​​in the channel dimension according to the shape of the feature map X2. The spliced ​​feature map undergoes 2D convolution, element-by-element multiplication with the channel-band mixed attention weight w2, and then undergoes inverse wavelet transform. It is residually connected with the input feature map of the frequency domain detail attention layer to obtain the enhanced feature map C3'; In the space-frequency fusion buffer layer, the feature map C4 is sequentially subjected to fast Fourier transform, 2D convolution and inverse Fourier transform, and then concatenated with the feature map C4 after 2D convolution in the channel dimension to obtain the enhanced feature map C4'; The enhanced feature maps C5', C4' and C3' are input into the feature fusion pyramid network, and the feature fusion pyramid network outputs feature maps P5, P4 and P3; the feature maps P5, P4 and P3 are respectively input into the final detection head for welding defect detection; The third step is to construct a loss function, train the welding defect detection model, and use the trained welding defect detection model for small-diameter pipe welding defect detection.

2. The small-diameter pipe welding defect detection method based on space-frequency feature collaborative enhancement according to claim 1 is characterized in that: The frequency domain center difference convolution is sampled by three-dimensional convolution. First, the local receptive field cube is sampled on the input feature map Then the sample value is obtained by weighted summation aggregation; the three-dimensional convolution is expressed as: In the formula, Vanilla(p i ) represents the current position p in the feature map i The sampling value, p n is the relative position offset in the local receptive field cube c, w(p n ) is the relative position offset p n The corresponding weight, x(p i +p n ) is the position p i +p n The characteristic value of Relative position offset p in the current frequency step n The weight is calculated by the following formula: In the formula, is the relative position offset p in the current frequency step n The weight of are the relative position offset p in the previous frequency step and the next frequency step respectively. n The weight of The frequency domain central difference convolution is expressed as: Where CDC-F(·) is the frequency domain central difference convolution, θ is a hyperparameter, and R -1 Indicates the previous frequency step, R 1 Indicates the next frequency step, R 0 Indicates the current frequency step.

3. The small diameter pipe welding defect detection method based on space-frequency feature collaborative enhancement according to claim 1 or 2 is characterized in that: During the model training process, three aspect ratio-weighted detection heads are introduced for auxiliary training. The three feature maps on the top-down feature aggregation path of the feature fusion pyramid network are the inputs of the three aspect ratio-weighted detection heads. The total loss of model training is calculated by the following formula; L total =L Last +εL ARDS (12) Where, L total is the total loss of model training, L Last is the total loss of the final detection head, L ARDS is the total loss of the detection head weighted based on the aspect ratio, ε is the weight coefficient; The total loss of the final detection head is L Last Calculated by the following formula: In the formula, and are the bounding box loss, confidence loss, and classification loss of the final detection head, All are weight coefficients; Bounding box loss of the final detection head Calculated by the following formula: Where, CIoU k is the CIoU value between the kth positive sample in the current batch and the true bounding box, and K is the number of positive samples in the current batch; The total loss L of the detection head based on the aspect ratio weighting ARDS Calculated by the following formula: In the formula, and is the bounding box loss, confidence loss, and classification loss of the detection head weighted by aspect ratio, are weight coefficients; Bounding box loss for the detection head based on aspect ratio weighting Calculated by the following formula: In the formula, Q1 is the central positive sample set, n1 represents the number of central positive samples, Q2 is the additional selected positive sample set, and h k 、w k are the length and width of the kth positive sample, h j 、w j are the length and width of the jth positive sample respectively.

4. The small-diameter pipe welding defect detection method based on space-frequency feature collaborative enhancement according to claim 3 is characterized in that: In the final detection head, the input feature map is gridded, the true bounding box is mapped to the feature map, the grid cell where the center point of the true bounding box is located is determined, and the grid cell and the two grid cells above or below, left or right of the center point of the true bounding box are marked as positive sample grid cells; an anchor box is generated on each positive sample grid cell, and the IoU value of the anchor box and the true bounding box is calculated. If the IoU value exceeds the set threshold, the anchor box is regarded as a positive sample, otherwise it is regarded as a negative sample; in the detection head based on aspect ratio weighting, on the basis of the positive sample grid cells of the final detection head, the upper or lower grid cells adjacent to and farther away from the center point of the true bounding box are also marked as positive sample grid cells.

5. The small diameter pipe welding defect detection method based on space-frequency feature collaborative enhancement according to claim 1 is characterized in that: The convolution kernel size of the first convolution layer of the spatial domain feature perception branch is 1×1, the padding is 0, and the stride is 1. The convolution kernel size of the remaining convolution layers is 3×3, the padding is 1, and the stride is 1.

Images