Welding defect detection method based on grayscale and defect feature perception

Abstract

The application discloses a welding defect detection method based on gray scale and defect feature perception, and a welding defect detection model used in the method comprises a Darknet53 network based on gray scale attention, a feature pyramid network based on shape perception and a center perception detection head; in the feature extraction process, the Darknet53 network based on gray scale attention utilizes a gray scale attention module to pay attention to the gray scale information of an image, and enhances the expression capability of defect features; feature maps C3-C5 extracted by the third-fifth convolution stages of the Darknet53 network based on gray scale attention are input into the feature pyramid network based on shape perception for feature fusion, and feature maps T2-T5 are obtained; the feature maps T2-T5 are input into the center perception detection head for prediction, and defects are classified and positioned. The method fully utilizes the gray scale information difference between defect and non-defect areas in an image to extract defect features, improves the defect detection capability of the model, and the model also has good detection effect on defects with a pathological length-width ratio.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of welding defect detection, and specifically relates to a welding defect detection method based on gray scale and defect feature perception, which can be used for defect detection of various welding structures in the industrial field. BACKGROUND

[0002] Welding process is widely used in pressure vessels, automobiles, robots, ships and other equipment with complex structure and a large amount of steel connection due to its good connectivity, large welding structure stiffness, good integrity and wide adaptability. For the equipment with complex structure and high quality required by high-end manufacturing industry, due to its high value, high strength and high parameter characteristics, higher requirements are also put forward for the quality of the welding structure. Due to the influence of welding environment and various factors, defects including strip, circle, line and the like will inevitably occur in the welding process, which will affect the welding quality and further affect the equipment quality. Therefore, in order to ensure the quality of the welding product, it is necessary to detect the welding defects.

[0003] At present, non-destructive testing technology is generally used for welding defect detection. X-ray detection, as the most commonly used and most intuitive non-destructive testing technology, can effectively find the internal defects of the welding structure and ensure the safety and reliability of the welding structure. X-ray non-destructive testing technology is mainly based on the different densities and thicknesses of X-ray penetrating welding products, and utilizes different gray scale image features to detect the internal defects of the welding structure. With the development of machine learning technology, the target detection technology based on deep learning opens up a new field of view for welding defect detection. Compared with the traditional welding defect detection technology, the welding defect detection based on deep learning can automatically extract the complex features of the defects, thereby reducing the influence of human subjective factors in the process of manually extracting features and improving the detection accuracy and objectivity of the welding defects. In addition, the welding defect detection technology based on deep learning can also effectively reduce the labor cost and time cost of enterprises in the quality inspection link, and realize the automatic detection of welding defects.

[0004] Different types of welding defects have great differences in shape. For example, the incomplete penetration defect and the incomplete fusion defect are very flat, which belong to the long and narrow ratio defect. The conventional detection model is easy to divide this long and narrow ratio defect into several parts, and it is difficult to accurately locate and detect these defects. In addition, in the same X-ray welding image, due to the different metal densities of the defect positions, the brightness of the defect and non-defect areas in the welding image will also be different. These brightness differences provide rich gray scale information for welding defect detection, which is beneficial to defect positioning and classification. Therefore, the present application proposes a welding defect detection method based on gray scale and defect feature perception, which fully utilizes the gray scale information to extract defect features and improves the detection ability of the long and narrow ratio defect. SUMMARY

[0005] In view of the deficiencies of the prior art, the technical problem to be solved by the present application is to provide a welding defect detection method based on gray scale and defect feature perception.

[0006] The technical solution adopted by the present application to solve the technical problem is as follows:

[0007] A welding defect detection method based on gray scale and defect feature perception, characterized in that the welding defect detection model used by the method comprises a Darknet53 network based on gray scale attention, a feature pyramid network based on shape perception, and a center perception detection head; the Darknet53 network based on gray scale attention is obtained by embedding a gray scale attention module between adjacent two convolution stages of the Darknet53 network, and the welding defect image is input into the Darknet53 network based on gray scale attention for feature extraction; in the feature extraction process, the gray scale attention module is used to focus on the gray scale information of the image, thereby enhancing the expression ability of the defect feature; the feature maps C3-C5 extracted by the third to fifth convolution stages of the Darknet53 network based on gray scale attention are input into the feature pyramid network based on shape perception for feature fusion; the feature pyramid network based on shape perception adopts an inflation convolution and texture enhancement manner to improve the detection ability and precision of the model for defects with a pathological length-width ratio, thereby obtaining feature maps T2-T5; the feature maps T2-T5 are input into the center perception detection head for prediction, and the defects are classified and located.

[0008] The input feature map X of the gray scale attention module is subjected to convolution, normalization and activation operations to obtain a feature map U; the feature map U is subjected to convolution, normalization and activation operations to obtain a feature map K; the feature map K is subjected to global average pooling to obtain channel descriptors, and the channel descriptors are subjected to feature extraction to obtain channel thresholds; the channel descriptors are multiplied by the channel thresholds to obtain thresholded channel descriptors, and the feature map K is binarized according to the thresholded channel descriptors; at the same time, the feature map K is subjected to global average pooling and feature extraction to obtain weights of each channel, the binarized feature map is multiplied by the weights of each channel, and then the binarized feature map is spliced with the input feature map X to obtain an output feature map of the gray scale attention module.

[0009] Further, the feature map C5 is input to a shape-aware feature pyramid network, i.e., a feature map H5, the feature map H5 is fused with C4 to obtain a feature map H4, the feature map H4 is fused with C3 to obtain a feature map H3, and the feature map H3 is the feature map P3; the feature map P3 is fused with C3 to obtain a feature map P4, and the feature map P4 is fused with C4 to obtain a feature map P5; meanwhile, the feature map H3 is input to a multi-scale dilated convolution module, and the feature map H3 is preprocessed and then respectively input to three parallel dilated convolution branches in the multi-scale dilated convolution module, and the output feature maps of the three dilated convolution branches are post-processed to obtain a feature map M3; each dilated convolution branch includes four consecutive hole residual units, and the feature map input to the hole residual unit is sequentially subjected to 1x1 convolution, dilated convolution, and 1x1 convolution, and then spliced with the input feature map of the hole residual unit to obtain the output feature map of the hole residual unit.

[0010] The feature maps P3-P5 and M3 are input to a texture feature enhancement module for texture enhancement, and the texture feature enhancement module adopts four sub-modules to process feature maps of different levels in parallel, each sub-module includes a CPre module, a Canny operator and a CPost module, the CPre module includes consecutive 1x1 convolution, 3x3 convolution and activation operation, and the feature map obtained by the CPre module is input to the Canny operator to generate a texture heat map; after the texture heat map is spliced with the input feature map of the sub-module, the texture enhancement is completed through the CPost module; and the CPost module includes convolution and ReLU activation operation.

[0011] Further, the dilated convolution sizes of the three dilated convolution branches of the multi-scale dilated convolution module are 1x3, 2x3 and 3x3 respectively.

[0012] Further, in the center-aware detection head, a prediction box is generated at each pixel point of the feature map, and the center degree of each prediction box is calculated by using formula (6);

[0013]

[0014] Wherein, θ represents the center degree of the prediction box, and l, t, r and b are the distances from the pixel point to the left, upper, right and lower boundaries of the prediction box respectively;

[0015] The prediction box is offset processed according to the center degree, and the offset prediction box is represented as:

[0016] priors * =cat(priors,1.5x priorsx(1-Sigmoid(θ))) (8)

[0017] Wherein, priors, priors *cat(Q, GT) and cat(Q, GT) represent the prediction boxes before and after offset respectively, cat(·) represents the concatenation operation, and Sigmoid(·) represents the Sigmoid activation function;

[0018] According to the defect category information, the label score lable_score is calculated by using formula (9); the label score is taken as the offset, and the center region of the real box is calculated by using formula (10);

[0019] lable_score = Sigmoid(gt_lables / gt_nums) + k (9)

[0020]

[0021] wherein gt_lables represents the numerical defect category, gt_nums represents the number of defect categories, k is a constant, and l * , t * , r * , b * respectively represent the distances from the center of the real box to the left, top, right and bottom boundaries;

[0022] The intersection of the real box after offset and the region in the center region of the real box is obtained to obtain the positive sample candidate region, and the intersection over union I gt of the positive sample candidate region and the corresponding real box is calculated by using formulas (12) and (13), respectively. gt

[0023] I gt = IoU(Q, GT) (12)

[0024] cost = cls_loss + l * IoU_loss + 100000 * (-Q) (13)

[0025] wherein Q represents the positive sample candidate region, GT represents the real box, IoU(·) represents the calculation of the intersection over union, cls_loss and IoU_loss respectively represent the classification loss and positioning loss of the prediction box, and l is the positioning loss balance coefficient.

[0026] A plurality of prediction boxes with large intersection over union are selected from the prediction box set corresponding to the real box, and a plurality of prediction boxes with small cost are selected from the selected plurality of prediction boxes as positive samples of the real box, and the rest are negative samples, to complete the positive and negative sample distribution; the positive and negative samples are input into the detection head for classification and positioning.

[0027] Further, the distances of the pixel points to the four boundaries of the prediction box are calculated according to formula (7);

[0028]

[0029] where (x, y) is the position coordinate of the pixel point, [x min ,y min ,x max ,y max ] is the real box corresponding to the pixel point, x min , x max are the minimum and maximum values of the horizontal coordinate of the real box, y min , y max are the minimum and maximum values of the vertical coordinate of the real box.

[0030] Further, in the gray attention module, the feature extraction includes sequentially connected full connection layer, normalization layer, ReLU activation layer, full connection layer and Sigmoid activation layer.

[0031] Compared with the prior art, the beneficial effects of the present application are:

[0032] 1. In the same welding image, due to the different metal densities of the defect locations, the lightness of the defect and non-defect areas in the welding image will also be different, and these lightness differences provide rich gray information, and the present application proposes a gray attention module in order to fully utilize the gray information in the image, the gray attention module makes the network focus on the gray information of the image, learns a threshold value for each channel, and uses a multi-threshold value method to obtain defect feature information under different threshold values and suppress invalid features. The gray attention module utilizes the different gray features of different welding defects, highlights the gray differences of the defect features, and thus more easily detects defects.

[0033] 2. In order to improve the detection ability of the model for pathological aspect ratio welding defects, a feature pyramid network based on shape perception is proposed, which adds a multi-scale dilated convolution module and a texture feature enhancement module on the basis of the traditional FPN+PAN module, which can effectively capture the features of pathological aspect ratio defects and avoid the pathological aspect ratio defects being segmented into several parts; the FPN+PAN module adds a bottom-up feature pyramid to the FPN to obtain strong positioning information at the bottom layer; the multi-scale dilated convolution module uses multiple scales of dilated convolution to capture the features of pathological aspect ratio defects at different scales; the texture feature enhancement module highlights the texture information of the defects through a CNN network and an edge detection operator, and the three modules form a cascade structure for enhancing the features of pathological aspect ratio defects and extracting the texture information of the defects.

[0034] 3. In the welding defects, small-scale defects are the majority, so there are more samples of small-scale defects and fewer samples of large-scale defects in the data set. In order to alleviate the problem of too large size difference between defect classes, the center perception detection head offsets the prediction box by introducing the centrality and the class information of the defect, and different allocation strategies are adopted for defects of different sizes to complete the matching of positive and negative samples. In addition, the center loss is introduced to suppress low-quality prediction boxes far from the center of the real box, so as to improve the precision of defect detection. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is the overall flowchart of the present application;

[0036] Figure 2 is the structural schematic diagram of the welding defect detection model of the present application;

[0037] Figure 3 is the structural schematic diagram of the gray attention module of the present application;

[0038] Figure 4 is the structural schematic diagram of the shape perception based feature pyramid network of the present application;

[0039] Figure 5 is the structural schematic diagram of the multi-scale dilated convolution module of the present application;

[0040] Figure 6 is the structural schematic diagram of the texture feature enhancement module of the present application;

[0041] Fig. 7(a) is a test result diagram of a baseline model detecting a circular defect;

[0042] Fig. 7(b) is a test result diagram of a detection model composed of a baseline model and a gray attention module of the present application detecting a circular defect;

[0043] Figure 8 is a test result diagram of a baseline model and a detection model composed of a baseline model and a multi-scale dilated convolution module detecting a strip-shaped defect;

[0044] Fig. 9(a) is a real box of a strip-shaped defect;

[0045] Fig. 9(b) is a visual heat map of a baseline model detecting a strip-shaped defect;

[0046] Fig. 9(c) is a visual heat map of a detection model composed of a baseline model and a multi-scale dilated convolution module detecting a strip-shaped defect;

[0047] Fig. 9(d) is a visual heat map of a detection model composed of a baseline model and a texture feature enhancement module detecting a strip-shaped defect. DETAILED DESCRIPTION

[0048] The technical solutions of the present application will be described in detail below in combination with the drawings and specific embodiments, which do not limit the protection scope of the present application.

[0049] The present application is a welding defect detection method based on gray scale and defect feature perception (referred to as method, see Figure 1 -9), comprising the following steps:

[0050] The first step is to obtain an X-ray welding defect image with OBB (Oriented Bounding Boxes) annotation, and to construct a data set.

[0051] The second step is to construct a welding defect detection model based on a deep learning network; as shown in Figure 2 The welding defect detection model includes three parts, namely a GAM-Darknet53 network based on grayscale attention, an S-FPN network based on shape perception, and a C-Head network based on center perception. The X-ray welding defect image is input into the GAM-Darknet53 network to extract a multi-scale feature map. The feature maps C3-C5 extracted from the third to fifth convolution stages of the GAM-Darknet53 network are input into the S-FPN network for feature fusion to obtain feature maps T2-T5. The feature maps T2-T5 are input into the C-Head network for prediction to classify and locate the defects.

[0052] The Darknet53 network is divided into five convolution stages, and a GAM is embedded between each two adjacent convolution stages of the Darknet53 network to obtain the GAM-Darknet53 network based on grayscale attention; as shown in Figure 3As shown, the input feature map X of the grayscale attention module is subjected to convolution, normalization and activation operations to obtain a feature map U; the feature map U is subjected to convolution, normalization and activation operations to obtain a feature map K; the feature map K enters a threshold learning module to learn the threshold of each channel; the threshold learning module includes a global average pooling (GAP) and a feature extraction operation, the feature map K is subjected to a global average pooling operation to generate descriptors of various channels, and the descriptors of various channels are respectively subjected to a feature extraction operation to capture the connection between channels to obtain the threshold of each channel; in the feature extraction operation, the channel descriptors are subjected to a fully connected layer (FC), a normalization layer (BN), a ReLU activation layer and a fully connected layer to obtain a feature map, which is then subjected to a Sigmoid activation layer to obtain the channel threshold, and the expression is:

[0053] s = F ex (z, W) = σ (W2δ (BN (W1z))) (1)

[0054] In the formula, s represents the channel threshold, F ex (·) represents a feature extraction operation, z represents a channel descriptor, W represents a weight matrix, W1 and W2 represent weight matrices of two fully connected layers, BN(·) represents a normalization operation, δ represents a ReLU activation function, and σ represents a Sigmoid activation function.

[0055] The channel descriptor is multiplied by the threshold value of the corresponding channel to obtain a thresholded channel descriptor, and the expression is:

[0056] S = ε (z, s) (2)

[0057] In the formula, ε(·) represents a channel descriptor multiplication operation with a corresponding channel threshold, and S represents a thresholded channel descriptor.

[0058] The thresholded channel descriptor is subjected to a binarization operation on the feature map K to obtain a binarized feature map B; the binarization operation is represented as:

[0059]

[0060] In the formula, υ(·) represents a binarization operation, and the specific meaning of the binarization operation is that if K>S, that is, the gray value of a pixel point of the feature map K is greater than the thresholded channel descriptor S, then the gray value of the pixel point takes the maximum gray value Max(K) of the feature map K, otherwise takes the minimum gray value Min(K);

[0061] The feature map K also enters another threshold learning module to learn the weight of each channel according to the importance of the gray information contained in the channel; the binarized feature map is multiplied by the weight of each channel to obtain a weighted binarized feature map U * , which is represented as:

[0062] U * = ω (F ex , B) = F ex (z, W) · B = s * B (4)

[0063] wherein ω represents the multiplication of the binarized feature map and the corresponding channel weight, s * represents the channel weight;

[0064] The weighted binarized feature map is spliced with the input feature map X of the gray attention module to complete the binarized feature fusion process, and the output feature map of the gray attention module is obtained.

[0065]

[0066] wherein, represents the splicing operation, X * represents the output feature map of the gray attention module.

[0067] As shown in Figure 4 , the feature maps C3-C5 extracted by the third-fifth convolution stages of the GAM-Darknet53 network are input into the S-FPN network for feature fusion, and the feature map C5 input into the S-FPN network is the feature map H5; the feature map H5 is fused with C4 to obtain the feature map H4; the feature map H4 is fused with C3 to obtain the feature map H3; the feature map H3 is the feature map P3, and the feature map P3 is fused with C3 to obtain the feature map P4; the feature map P4 is fused with C4 to obtain the feature map P5; the feature map H3 is simultaneously input into the multi-scale dilated convolution (MSLD) module for fusion to obtain the feature map M3; then the feature maps P3-P5 and M3 are input into the texture feature enhancement (TFEM) module for texture enhancement to obtain the feature maps T2-T5, and the feature maps T2-T5 are the output feature maps of the S-FPN network;

[0068] As shown in Figure 5As shown, the multi-scale dilated convolution module focuses on the shape features of defects by expanding the kernel size multiple times, thereby improving the detection capability of the network for defects with a pathological aspect ratio; the feature map H3 is preprocessed by 1x1 convolution to reduce the number of channels and 3x3 convolution to refine semantic information (Prejector), to obtain a preprocessed feature map; the preprocessed feature map is input into three parallel dilated convolution branches, each of which includes four consecutive dilated residual units (Dilated Encoder, DE); the dilated convolution kernels of the four dilated residual units have different dilation ratios, which can obtain features with a larger receptive field without changing the size of the feature map, avoid information loss caused by down-sampling of ordinary convolution, and better extract defects with a pathological aspect ratio such as incomplete penetration defects and incomplete fusion defects; the dilated residual unit first uses 1x1 convolution to reduce the dimension to reduce the parameter operation amount, then uses i x 3 dilated convolution, and finally uses 1x1 convolution to restore the dimension; the feature map with restored dimension is spliced with the input feature map of the dilated residual unit to obtain the output feature map of the dilated residual unit; in order to obtain features of defects with a pathological aspect ratio of multiple scales, the value of i in the three dilated convolution branches is 1, 2 and 3 respectively, so the dilated convolution size of the three dilated convolution branches is 1x3, 2x3 and 3x3 respectively; in this way, different scale convolution kernels are used to obtain features of defects with different scale pathological aspect ratios; the output feature maps of the three dilated convolution branches are post-processed (Postjector) to obtain the output feature map M3 of the multi-scale dilated convolution module; the post-processing includes three operations of splicing, convolution and activation;

[0069] The texture feature enhancement module enhances the texture features of the defect area, improves the perception ability of the network for defect texture information, and thereby helps to improve the accuracy of defect detection; the feature maps P3-P5 and M3 are input into the texture feature enhancement module for feature enhancement to obtain feature maps T2-T5, which are the output feature maps of the S-FPN network; as shown in FIG. 4, the texture feature enhancement module includes three parallel branches, each of which includes a 1x1 convolution layer, a 3x3 convolution layer and a ReLU activation function; the 1x1 convolution layer in each branch is used to reduce the dimension of the input feature map, the 3x3 convolution layer is used to refine the semantic information of the feature map, and the ReLU activation function is used to obtain the output feature map of the branch; the output feature maps of the three branches are spliced to obtain the output feature map of the texture feature enhancement module, which is the output feature map of the S-FPN network; Figure 6As shown, the texture feature enhancement module adopts four sub-modules to process different levels of feature maps in parallel; each sub-module includes a CPre module, a Canny operator and a CPost module, wherein the CPre module includes continuous 1x1 convolution, 3x3 convolution and activation operation, aiming to reduce the number of channels and refine semantic information; the feature map obtained by the CPre module is input into the Canny operator to obtain the texture information of the feature map, the Canny operator first smoothes the noise through Gaussian filtering, then calculates the gradient and gradient direction of the pixel points in the feature map and uses maximum suppression to eliminate the stray effect caused by edge detection, finally completes edge detection through double threshold detection and weak edge suppression to generate a texture heat map; the texture heat map is spliced with the input feature map of the sub-module, and the spliced feature map is processed by the CPost module to complete texture enhancement and obtain the output feature map of the sub-module; the CPost module includes convolution and ReLU activation operation.

[0070] Third step, obtaining training samples, training the welding defect detection model by using the training samples, and using the trained welding defect detection model for welding defect detection;

[0071] According to the characteristics of the size difference between defect categories, the sample allocation strategy is improved, different allocation strategies are adopted for defects of different sizes, so that the model is more suitable for welding defects of different sizes; at the same time, the center loss is introduced to suppress low-quality prediction boxes far away from the center of the real box, and the weight of the prediction box far away from the center of the real box is reduced. First, a prediction box is generated at each pixel point of the feature map, and the center degree of each prediction box is calculated by formula (6), then the prediction box is offset according to the center degree of the prediction box, the closer the center degree of the prediction box is to 1, the less it needs to be offset, and the closer the center degree is to 0, the more it needs to be offset, and the offset prediction box is shown in formula (8);

[0072]

[0073] Wherein, θ represents the center degree of the prediction box, l, t, r, b are the distances from the pixel point to the left, upper, right and lower boundaries of the prediction box respectively; if the real box corresponding to the pixel point with position coordinates (x, y) is GT=[x min ,y min ,x max ,y max ], x min , x max are the minimum and maximum values of the horizontal coordinates of the real box, y min , y max are the minimum and maximum values of the vertical coordinates of the real box, then the distances from the pixel point to the four boundaries of the prediction box can be calculated by formula (7);

[0074]

[0075] priors * = cat(priors, 1.5 x priors x (1 - Sigmoid(0))) (8)

[0076] where priors and priors * represent the bounding boxes before and after offset respectively, cat(·) represents the concatenation operation, and Sigmoid(·) represents the Sigmoid activation function;

[0077] Calculate the label score lable_score of each label according to formula (9):

[0078] lable_score = Sigmoid(gt_lables / gt_nums) + k (9)

[0079] where gt_lables represents the numerical defect category, taking values 0, 1, 2, 3, representing circular defects, tungsten clamping defects, strip-shaped defects, and linear defects, respectively, gt_nums represents the number of defect categories, and k is a constant, taking 2.0;

[0080] Large defects have a small probability of occurring in the welding process, and the number of large defect samples that can be collected is limited. These large defects often have a pathological aspect ratio, and the shape of the defect with a pathological aspect ratio may be very long and flat, which will cause the defect feature to be very weak in the image, making it difficult to accurately detect and identify. Therefore, large defects are regarded as difficult samples. In the positive and negative sample matching process, the label score is used as the offset, and the center area of the real box is calculated according to formula (10). Large defects use large offset, and small defects use small offset, which is more tolerant to large defects to ensure that they have enough positive samples. Using the label score, the center area of each real box is calculated according to formula (10);

[0081]

[0082] where l * , t * , r * , b * represent the distances from the center of the real box to the left, top, right, and bottom boundaries, respectively;

[0083] The intersection of the area of the offset prediction box inside the real box and the area inside the center area of the real box is obtained as the positive sample candidate area Q, which is represented as:

[0084] Q = GT & center (11)

[0085] where & represents the intersection, and center represents the center area of the real box.

[0086] The intersection over union I of each positive sample candidate region and the corresponding real box is calculated by formula (12) and (13) respectively gt , the cost of each real box;

[0087] I gt = IoU (Q, GT) (12)

[0088] cost = cls_loss + l·IoU_loss + 100000·(-Q) (13)

[0089] Wherein, IoU(·) represents calculating the intersection over union, cls_loss and IoU_loss represent the classification loss and positioning loss of the predicted box respectively, and l is the positioning loss balance coefficient, which is 3.0;

[0090] From the predicted box set corresponding to the real box, multiple predicted boxes with large intersection over union are selected, and multiple predicted boxes with small cost are selected from these predicted boxes as positive samples of the real box, and the rest are negative samples, to complete the positive and negative sample distribution; The positive and negative samples are input into the detection head for classification and positioning, and each detection head is responsible for different feature level prediction tasks; In the model training process, the center loss, classification loss, confidence loss and IoU loss are combined into a hybrid loss, and the center perception detection head outputs high-quality predicted boxes under the supervision of the hybrid loss; On the feature level i, the hybrid loss is defined as follows:

[0091]

[0092] Wherein, represents the classification loss, represents the IoU loss, represents the center loss, represents the confidence loss.

[0093] Embodiment 1

[0094] To verify the effectiveness of the method of the present application, the experimental data set is used for testing. Figures 7(a), (b) are respectively the test result comparison chart of the baseline model and the detection model composed of the baseline model and the gray attention module of the present application for detecting circular defects. The baseline model adopts YOLOX model, and YOLOX takes Darknet53 as the backbone network for feature extraction, and adds a spatial pyramid pooling structure (SPP). As can be seen from the figure, the baseline model has poor detection effect on defects, and there is missed detection, which cannot effectively detect and distinguish defects. After adding the gray attention module, all defects can be effectively detected and distinguished, verifying that the gray attention module of the present application can effectively improve the defect detection precision and avoid missed detection and false detection.

[0095] Figure 8 Figures 8(a), (b) are respectively the test result chart of the baseline model and the detection model composed of the baseline model and the multi-scale dilated convolution module (MSLD) of the present application for detecting linear defects. As can be seen from the figure, the original FPN network of the baseline model will detect a linear defect into multiple defects of different sizes in the detection process, that is, a linear defect is divided into two parts, and these detection boxes are too small to be identified as positive samples, which will lead to low recall rate and precision rate of defects. After introducing the MSLD module in the FPN, the linear defect can be completely detected, because the MSLD module introduces a flat multi-scale dilated convolution, which can adapt to the long and narrow ratio defect of the linear defect, so a detection box that can completely cover the linear defect is generated, which shows that by adding the MSLD module on the FPN, the problem of poor detection effect of linear defects can be effectively solved, and the detection precision of the model for linear defects is improved.

[0096] Figure 9(a) is the real box of the linear defect, and figures 9(b)-(d) respectively show the detection results of the baseline model, the detection model composed of the baseline model and the multi-scale dilated convolution module of the present application, and the detection model composed of the baseline model and the texture feature enhancement module of the present application for linear defects in the form of visual heat map. As can be seen from figure 9, the baseline model performs relatively weak in the aspect of linear defect feature acquisition, and it is difficult to accurately reflect the contour of the linear defect; in order to enhance the acquisition ability of defect features, the baseline model introduces a multi-scale dilated convolution module, and the dilated convolution can expand the receptive field and enhance the acquisition ability of features. After introducing the texture feature enhancement module on the basis of the baseline model, the feature information of the defect can be accurately captured, and the interference of irrelevant factors is effectively reduced. Table 1 is the comparison result of various indicators of the baseline model and each module of the present application.

[0097] Table 1 Comparison results of each module and baseline model (%)

[0098]

[0099] From the test results, it can be seen that the baseline model has a missed detection phenomenon, the welding defect detection model of the application improves the backbone network, FPN and detection head in YOLOX model, and proposes GAM, S-FPN and C-Head modules. As can be seen from Table 1, among the three sub-modules proposed in this paper, compared with the baseline model, the S-FPN module has the most significant improvement in the average recall rate, and the average recall rate is increased by 3.42%; GAM has the best effect on improving the average precision rate, and the average precision rate is increased by 4.91%; adding the C-Head module can also obviously improve the average precision rate, and the average precision rate is increased by 3.91%. When the three sub-modules are combined with each other, the combination of S-FPN and C-Head has the best effect on improving the model, and the average recall rate is increased by 5.94%, the average precision rate is increased by 4.05%, and the average accuracy is increased by 6.53%. The detection result of the combination of the three modules is the highest, compared with the baseline model, the average recall rate is increased by 6.19%, the average precision rate is increased by 6.45%, and the average accuracy is increased by 8.82%. Through the experiment of combining different modules, it is verified that the GAM, S-FPN and C-Head modules have complementary relationship, and the effectiveness of the overall model is verified.

[0100] The unmentioned part of the application is applicable to the prior art.

Claims

1. A welding defect detection method based on gray scale and defect feature perception, characterized in that, The welding defect detection model used in the method comprises a Darknet53 network based on grayscale attention, a shape perception based feature pyramid network and a center perception detection head; the Darknet53 network based on grayscale attention is obtained by embedding a grayscale attention module between adjacent two convolution stages of the Darknet53 network, and a welding defect image is input into the Darknet53 network based on grayscale attention for feature extraction; in the feature extraction process, the grayscale attention module is used to pay attention to the grayscale information of the image, and the expression ability of the defect features is enhanced; the feature maps C3-C5 extracted by the third-fifth convolution stages of the Darknet53 network based on grayscale attention are input into the shape perception based feature pyramid network for feature fusion; the shape perception based feature pyramid network adopts an inflation convolution and a texture enhancement mode to improve the detection ability and precision of the model on defects with a pathological length-width ratio, and obtains feature maps T2-T5; the feature maps T2-T5 are input into the center perception detection head for prediction, and the defects are classified and positioned; The input feature map X of the grayscale attention module is subjected to convolution, normalization and activation operations to obtain a feature map U; The feature map U is subjected to convolution, normalization and activation operations to obtain a feature map K; the feature map K is subjected to global average pooling to obtain channel descriptors, and the channel descriptors are subjected to feature extraction to obtain channel thresholds; The channel descriptors are multiplied by the channel thresholds to obtain thresholded channel descriptors, and the feature map K is binarized according to the thresholded channel descriptors; meanwhile, the feature map K is subjected to global average pooling and feature extraction to obtain channel weights, the binarized feature map is multiplied by the channel weights, and then is spliced with the input feature map X to obtain an output feature map of the grayscale attention module; The channel threshold extraction process is represented as: s = F ex (z, W) = σ(W2δ(BN(W1z))) (1) where s denotes a channel threshold, F ex (·) denotes a feature extraction operation, z denotes a channel descriptor, W denotes a weight matrix, W1, W2 denote weight matrices of two fully connected layers, BN(·) denotes a normalization operation, δ denotes a ReLU activation function, and σ denotes a Sigmoid activation function.

2. The method for detecting welding defects based on gray scale and defect feature perception according to claim 1, characterized in that, The feature map C5 is input into the shape perception based feature pyramid network to obtain a feature map H5, the feature map H5 is fused with C4 to obtain a feature map H4, the feature map H4 is fused with C3 to obtain a feature map H3, and the feature map H3 is a feature map P3; the feature map P3 is fused with C3 to obtain a feature map P4, and the feature map P4 is fused with C4 to obtain a feature map P5; meanwhile, the feature map H3 is input into a multi-scale inflation convolution module, the feature map H3 is preprocessed and then respectively input into three parallel inflation convolution branches in the multi-scale inflation convolution module, and the output feature maps of the three inflation convolution branches are post-processed to obtain a feature map M3; each inflation convolution branch comprises four consecutive hollow residual units, and the feature map input into the hollow residual unit is sequentially subjected to 1x1 convolution, inflation convolution and 1x1 convolution, and then is spliced with the input feature map of the hollow residual unit to obtain the output feature map of the hollow residual unit; The feature maps P3-P5 and M3 are input into a texture feature enhancement module for texture enhancement, the texture feature enhancement module adopts four sub-modules to process feature maps of different levels in parallel, each sub-module includes a CPre module, a Canny operator and a CPost module, the CPre module includes continuous 1x1 convolution, 3x3 convolution and activation operation, the feature map obtained by the CPre module is input into the Canny operator to generate a texture heat map; the texture heat map is spliced with the input feature map of the sub-module, and then passes through the CPost module to complete texture enhancement; the CPost module includes convolution and ReLU activation operation.

3. The method of claim 2, wherein the welding defect detection method based on gray scale and defect feature perception is characterized by, The sizes of the dilated convolutions of the three dilated convolution branches of the multi-scale dilated convolution module are 1x3, 2x3 and 3x3 respectively.

4. The method of claim 1 to 3, wherein In the center perception detection head, a prediction box is generated at each pixel point of the feature map, and the center degree of each prediction box is calculated by formula (6); Wherein, θ represents the center degree of the prediction box, l, t, r and b are the distances from the pixel point to the left, upper, right and lower boundaries of the prediction box respectively; The prediction box is offset according to the center degree, and the offset prediction box is represented as: priors * = cat(priors, 1.5 x priors x (1 - Sigmoid(0)) (8) where priors, priors * denote the pre- and post-offset bounding boxes, respectively, cat(·) denotes the concatenation operation, and Sigmoid(·) denotes the Sigmoid activation function. According to the defect category information, the label score lable_score is calculated by formula (9); the label score is used as an offset, and the center area of the real box is calculated by formula (10); lable_score=Sigmoid(gt_lables / gt_nums)+k (9) wherein gt_lables represents the numerical defect categories, gt_nums represents the number of defect categories, k is a constant, l * , t * , r * , b * respectively represent the distances from the center of the real frame to the left, top, right and bottom boundaries; The intersection of the offset prediction box being inside the real box and being inside the central region of the real box is obtained as a positive sample candidate region, and the intersection over union I of the positive sample candidate region and the corresponding real box is calculated using equations (12) and (13) gt and the real box cost I gt = IoU(Q, GT) (12) cost=cls_loss+λ·IoU_loss+100000·(-Q) (13) Wherein, Q represents a positive sample candidate region, GT represents a real box, IoU(·) represents calculating the intersection over union, cls_loss and IoU_loss represent the classification loss and positioning loss of the prediction box respectively, and λ is a positioning loss balance coefficient; A plurality of prediction boxes with large intersection over union are selected from the prediction box set corresponding to the real box, and a plurality of prediction boxes with small replacement value are selected from the selected plurality of prediction boxes as positive samples of the real box, and the rest are negative samples, to complete positive and negative sample distribution; the positive and negative samples are input into the detection head for classification and positioning.

5. The method of claim 4, wherein the welding defect detection method based on gray scale and defect feature perception is characterized by, The distances of the pixel point to the four boundaries of the prediction box are calculated according to formula (7); wherein (x, y) is the position coordinate of the pixel point, [x min ,y min ,x max ,y max ] is the real box corresponding to the pixel point, x min , x max are the minimum and maximum values of the horizontal coordinate of the real box, y min , y max are the minimum and maximum values of the vertical coordinate of the real box.

6. The method for detecting welding defects based on gray scale and defect feature perception according to claim 1, characterized in that, In the gray attention module, feature extraction includes sequentially connected full connection layer, normalization layer, ReLU activation layer, full connection layer and Sigmoid activation layer.

Images