Weld defect detection method and system based on small sample target detection network

Abstract

The application discloses a kind of based on small sample target detection network's weld defect detection method, system, detection method includes: using the two-stage target detection algorithm of Faster R-CNN and the small sample detection algorithm based on fine-tuning constructs small sample target detection network;Small diameter pipe ring weld area image is obtained;Small diameter pipe ring weld area image is input to the small sample target detection network after training, to obtain the defect data of small diameter pipe ring weld.The application uses the two-stage target detection algorithm of Faster R-CNN and the small sample detection algorithm based on fine-tuning constructs small sample target detection network small diameter pipe ring's weld defect detection, improves the accuracy rate of small sample target detection network to rare defect detection and the detection rate of small sample defect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of welding seams, in particular to a welding seam defect detection method and system based on a small sample target detection network. BACKGROUND

[0002] In the welding process, due to improper adjustment of welding parameters (welding power, protective gas flow, welding speed) and complex welding environment, etc. uncertain factors, internal defects such as pores, incomplete fusion, incomplete penetration, cracks, etc. are prone to occur inside the welding seam. These defects not only affect the appearance of the welding seam, but also cause the welding seam connection strength to decrease, thereby affecting the product quality of the welded component, and even causing the product to be scrapped. In order to ensure the product quality of the welded component, efficient and accurate defect detection needs to be performed on the welding seam area.

[0003] In order to ensure product quality, factories usually set up a detection link to check whether there are defects in the welding position to ensure that the product can be safely and stably used for a long time in an industrial environment. X-ray imaging is a common non-destructive testing method for welding seams, and the image has the characteristics of extremely small welding defect targets and sparse distribution.

[0004] The commonly used technical solution is a detection system based on a deep learning model, which is described in detail as follows:

[0005] (1) Take a number of pictures containing welding seams and defects, pre-process and label the defects;

[0006] (2) Design a neural network model based on deep learning;

[0007] (3) Use the labeled pictures and the neural network model for training, and the trained model can be used to detect defects in the image.

[0008] The existing defect detection method based on the deep learning model has the following problems:

[0009] (1) The training of the model requires a large number of images to achieve good results, and a large amount of manpower and machine resources are required for image labeling work;

[0010] (2) When the sample size is small, the defect detection model cannot achieve good detection performance. SUMMARY

[0011] The technical problem to be solved by the present application is to overcome the defects in the prior art that the defect detection method based on the deep learning model requires a large number of images for training to achieve good results, and a large amount of manpower and machine resources are required for image labeling work, and when the sample size is small, the defect detection model cannot achieve good detection performance. A welding seam defect detection method and system based on a small sample target detection network are provided.

[0012] The present application solves the above technical problems by the following technical solutions:

[0013] The first aspect of the present application provides a weld defect detection method based on a small sample target detection network, the detection method comprising:

[0014] A two-stage target detection algorithm of a Faster R-CNN (target detection network) and a small sample detection algorithm based on fine-tuning are used to construct a small sample target detection network;

[0015] An image of a small-diameter pipe girth weld area is obtained;

[0016] The small-diameter pipe girth weld area image is input into the trained small sample target detection network to obtain defect data of the small-diameter pipe girth weld.

[0017] Preferably, the detection head in the small sample target detection network comprises a CosHead (a kind of detection head).

[0018] Preferably, the Faster R-CNN comprises a convolutional neural network, a region generation network, a feature pyramid network, and a Guide Anchor RPN (Guide Anchor RPN) network.

[0019] Preferably, before the step of inputting the small-diameter pipe girth weld area image into the trained small sample target detection network to obtain defect data of the small-diameter pipe girth weld, the detection method further comprises:

[0020] Obtaining a weld defect annotation data set;

[0021] Performing data enhancement processing and non-linear grayscale transformation processing on the weld defect annotation data set to obtain a processed weld defect annotation data set, the processed weld defect annotation data set comprising a training set;

[0022] Training a small sample target detection network using the training set to obtain a trained small sample target detection network.

[0023] Preferably, the step of performing data enhancement processing and non-linear grayscale transformation processing on the weld defect annotation data set to obtain a processed weld defect annotation data set comprises:

[0024] Random flipping and / or random cropping algorithms are used to perform data enhancement processing on the weld defect annotation data set to obtain a data enhancement processed weld defect annotation data set;

[0025] The data enhanced weld defect annotation dataset is subjected to non-linear gray scale transformation processing by using a sin (sine) transformation to obtain a processed weld defect annotation dataset.

[0026] Preferably, the processed weld defect annotation dataset comprises a test set, and the detection method further comprises:

[0027] The trained small sample target detection network is tested by using the test set until the detection accuracy of the trained small sample target detection network reaches a target condition.

[0028] Preferably, the small sample target detection network comprises a base model and a fine-tuned model.

[0029] The step of training a small sample target detection network by using the training set to obtain a trained small sample target detection network comprises:

[0030] The processed weld defect annotation dataset is used to train a small sample target detection network to obtain the base model.

[0031] The weights of the base model are frozen to form the fine-tuned model.

[0032] A small sample weld defect annotation dataset is obtained from the processed weld defect annotation dataset.

[0033] The fine-tuned model is trained by using the small sample weld defect annotation dataset to obtain a trained fine-tuned model.

[0034] The second aspect of the present application provides a small sample target detection network based weld defect detection system, which comprises:

[0035] A construction module is configured to construct a small sample target detection network by using a two-stage target detection algorithm of a Faster R-CNN and a small sample detection algorithm based on fine-tuning.

[0036] A weld area image acquisition module is configured to acquire a small-diameter pipe girth weld area image.

[0037] A detection module is configured to input the small-diameter pipe girth weld area image into a trained small sample target detection network to obtain defect data of the small-diameter pipe girth weld.

[0038] Preferably, the detection head in the small sample target detection network comprises a CosHead.

[0039] Preferably, the Faster R-CNN comprises a convolutional neural network, a region generation network, a feature pyramid network, and a Guide Anchor RPN network.

[0040] Preferably, the detection system further comprises:

[0041] a data set acquisition module configured to acquire a weld defect annotation data set;

[0042] a processing module configured to perform data enhancement processing and non-linear grayscale transformation processing on the weld defect annotation data set to obtain a processed weld defect annotation data set, the processed weld defect annotation data set comprising a training set;

[0043] a training module configured to train a small sample target detection network using the training set to obtain a trained small sample target detection network.

[0044] Preferably, the processing module comprises:

[0045] a data enhancement processing unit configured to perform data enhancement processing on the weld defect annotation data set using a random flipping and / or random cropping algorithm to obtain a data enhancement processed weld defect annotation data set;

[0046] a transformation processing unit configured to perform non-linear grayscale transformation processing on the data enhancement processed weld defect annotation data set using a sin transformation to obtain the processed weld defect annotation data set.

[0047] Preferably, the processed weld defect annotation data set comprises a test set, and the detection system further comprises:

[0048] a testing module configured to test the trained small sample target detection network using the test set until a detection accuracy of the trained small sample target detection network reaches a target condition.

[0049] Preferably, the small sample target detection network comprises a base model and a fine-tuning model, and the training module comprises:

[0050] a first training unit configured to train a small sample target detection network using the processed weld defect annotation data set to obtain the base model;

[0051] a freezing unit configured to freeze weights of the base model to form the fine-tuning model;

[0052] an acquisition unit configured to acquire a small sample weld defect annotation data set from the processed weld defect annotation data set;

[0053] a second training unit configured to train the fine-tuning model using the small sample weld defect annotation data set to obtain a trained fine-tuning model.

[0054] The third aspect of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and configured to run on the processor, wherein the processor executes the computer program to implement the method for detecting weld defects based on a small sample target detection network according to the first aspect.

[0055] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the method for detecting weld defects based on a small sample target detection network according to the first aspect.

[0056] On the basis of common knowledge in the art, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred examples of the present application.

[0057] The positive progress effect of the present application is that:

[0058] The present application uses the two-stage target detection algorithm of Faster R-CNN and the small sample detection algorithm based on fine-tuning to construct a small sample target detection network to detect the weld defects of the small diameter pipe ring, thereby improving the accuracy of the small sample target detection network in detecting rare defects and the detection rate of small sample defects. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 The flowchart of the method for detecting weld defects based on a small sample target detection network according to the first aspect of the present application is shown.

[0060] Figure 2 The first module schematic diagram of the detection system for weld defects based on a small sample target detection network according to the third aspect of the present application is shown.

[0061] Figure 3 The second module schematic diagram of the detection system for weld defects based on a small sample target detection network according to the fourth aspect of the present application is shown.

[0062] Figure 4 The structure schematic diagram of the electronic device according to the fifth aspect of the present application is shown. DETAILED DESCRIPTION

[0063] The present application will be further described by way of examples below, but the present application is not limited in the scope of the examples.

[0064] Example 1

[0065] The present embodiment provides a method for detecting weld defects based on a small sample target detection network, as shown in the figure, the detection method comprises: Figure 1 The detection method comprises:

[0066] Step 101, a two-stage target detection algorithm of Faster R-CNN and a small sample detection algorithm based on fine-tuning are used to construct a small sample target detection network.

[0067] As an optional implementation, the detection head in the small sample target detection network comprises a CosHead.

[0068] As an optional implementation, the Faster R-CNN comprises a convolutional neural network, a region generation network, a feature pyramid network and a Guide Anchor RPN network.

[0069] In the process of constructing the small sample target detection network, due to the serious long-tail effect of the defect samples, in order to meet the detection requirements under the condition of small samples, the two-stage target detection algorithm of Faster R-CNN is combined with the small sample detection algorithm TFA (two-stage fine-tuning approach) based on fine-tuning to construct the small sample target detection network. The Faster R-CNN comprises two stages: the first stage is composed of a convolutional neural network (CNN) and a region generation network (RPN), wherein the CNN network is used to extract general features of an image, the general feature extraction network of the small sample target detection network used in the application is ResNet-101, and the RPN network extracts a candidate target bounding box according to the general features. The RPN network is usually composed of a simple convolutional network (for example, a deformable convolutional layer network) and a classification branch and a regression branch; the second stage extracts features of the candidate regions from the RPN network and performs class classification and bounding box regression.

[0070] Specifically, the construction process of the small sample target detection network comprises a-h.

[0071] a, normalization layer: a large input image resolution can ensure good detection effect, but there is a problem that the number of images trained by a single GPU (graphics processing unit) is small, resulting in poor Batch Normalization (batch normalization) effect, therefore, Group Normalization (a new deep learning normalization method) is used as the normalization layer of the small sample target detection network, which can have better effect than the default Batch Normalization layer;

[0072] b, modify the optimizer: an Adam (an optimizer) optimizer is used to accelerate the training process of the small sample target detection network and prevent the optimization point from falling into a saddle point;

[0073] c. Adding a Feature Pyramid Network (FPN): For example, at an image resolution of 1120x 1120, the total number of pixels of round defects is between 200 and 12544, and the total number of pixels of incomplete fusion or incomplete penetration is between 2000 and 12544, with a very large size difference, so an FPN network is added to enhance the network's processing capability for target scales;

[0074] d. The RPN network part is modified to a Guide Anchor RPN network: The original RPN network uses fixed anchors for classification and regression at each location, and the Guide Anchor can adaptively set the anchor according to the features of the current location, thereby improving the quality of the foreground part output by the RPN network;

[0075] e. The CNN network of the RPN network uses deformable convolution (DCN): The theoretical receptive field of a traditional convolutional neural network is usually a square with consistent length and width, and deformable convolution makes the theoretical receptive field of the neural network more flexible, which has better results for objects with varying shapes;

[0076] f. The loss function of the regression branch of the RPN network and the RCNN network uses GIoU loss, and the calculation method of average precision (AP) is based on Intersection-over-Union (IoU), which makes the target of model training and inference more consistent;

[0077] g. The detection head is modified to a CosHead;

[0078] h, The RCNN network part of the base model and the fine-tuning model in the small sample target detection network adopts CosHead. The difference between the detection head and the original detection head of Faster R-CNN is that the output of the classification branch adopts a cosine similarity function for normalization operation, which helps to reduce the variance of small sample categories, alleviate the problem of too large difference between different classification weights, prevent the model from tending to detect a larger number of one class of positive samples, and thus improve the detection performance of the smaller number of samples, especially when the number of training samples is very small. In the prior art, the base model uses a normal detection head, and the fine-tuning model uses CosHead. After the base model is trained, the initial stage of the fine-tuning model is greatly reduced due to the introduction of normalization, thereby greatly prolonging the time from training to ideal AP, and the risk of overfitting is also greatly improved. Therefore, the detection head of the base model is also used CosHead, so that the model structure of the base model and the fine-tuning model is completely consistent. After improvement, the performance of the initial stage of the fine-tuning model is completely consistent with the base model, and only a small amount of time is needed to intuitively perceive the improvement of the small sample performance.

[0079] For example, when constructing a small sample target detection network, the input image size is configured as 1120 and 1120, the pre-processing mean is [128, 128, 128], and the variance is [35, 35, 35]; the CNN network in the RPN network adopts ResNet-101, for example, the basic size of the anchor is 8, there are five height-width ratios: 0.25, 0.5, 1.0, 2.0, 4.0, the number of feature scales used for detection is 5, and the step lengths are 4, 8, 16, 32, and 64; the hole rate of the convolution in the last stage of the CNN network is 2; the output channel of the FPN network can be 256, and the default method is used for upsampling; the RPN network uses Guide Anchor RPN Head, that is, a 3x3 convolution is used first, and then the classification branch and the regression branch are used to output results, the maximum number before NMS is 2000, the output number is 1000, and the IoU threshold of NMS is 0.7; the ROI pooling part adopts Deform ROIPooling to improve the feature information granularity of the pooling; the RCNN network part adopts CosHead, and the IoU threshold of NMS is 0.3 (to reduce the phenomenon of multiple same class boxes overlapping).

[0080] Step 102, acquire a small-diameter pipe girth weld area image;

[0081] In this embodiment, the small-diameter pipe girth weld area image is acquired through the weld area detection model.

[0082] Step 103, input the small-diameter pipe girth weld area image into the small sample target detection network trained to obtain defect data of the small-diameter pipe girth weld.

[0083] In this embodiment, the defect data includes defect categories and defect positions, and the defect categories include, but are not limited to, circular defects, strip defects, clamps, cracks, incomplete fusion, and incomplete welding.

[0084] It should be noted that, since defect identification is also a very challenging task for humans, and there are often missing annotations in the annotated samples; on the other hand, there is less existing annotated data. Therefore, the present application adopts a human-in-the-loop technology to provide guidance for researchers during model learning, and continuously improves the recognition ability of the model and the size and accuracy of the annotated data set in human-computer interaction iteration. The specific method is: using the annotated weld defect data set to train a small sample target detection network, using the network to infer the unannotated images, then manually auditing the targets detected by the small sample target detection network, deleting the incorrect detected targets, and adding the correct detected targets to the weld defect annotation data set. Through multiple rounds of the above process, the ability of the small sample target detection network and the accuracy of the weld defect annotation data set can be improved.

[0085] The small sample target detection network trained in this embodiment realizes the detection of small-diameter pipe girth weld defects, improves the detection rate of types with less positive samples such as strip defects, clamps, cracks, and other defects, and effectively reduces the large amount of human and machine resources consumed in image annotation work by training and detecting the model under the condition of less image data.

[0086] In addition, the weld defect detection software provides auxiliary quality inspection functions: the present application aims to provide an auxiliary tool for welding software, and the application scenarios can be extended to scenarios other than small-diameter pipe girth welds to assist researchers to realize rapid weld quality grading.

[0087] Embodiment 2

[0088] The weld defect detection method based on the small sample target detection network of this embodiment is a further improvement of embodiment 1, specifically:

[0089] As an optional implementation, before step 103, the detection method further includes:

[0090] Step 102-1, obtaining a weld defect annotation data set;

[0091] In this embodiment, small-diameter pipe girth weld X-ray images are acquired, including DR (Digital Radiography) images and RT (Radiographic Testing) images, and the RT film-form weld X-ray images are converted into digital images using a film scanner. The small-diameter pipe girth weld X-ray images are labeled for defect targets using a labeling tool, and the labeling method is to frame and label specific defect types. The labeling labels include weld area, circular defect, strip defect, incomplete fusion, incomplete penetration, crack, etc. to form a full-image labeling dataset. The labeling steps are divided into steps (1)-(5):

[0092] (1) Framing: frame the weld area and welding defects in the small-diameter pipe girth weld X-ray image respectively, and only frame the complete target, which is tightly attached to the target edge and completely includes the target.

[0093] (2) Label: The target label type of the weld area framed in step (1) is formed into a weld area labeling dataset.

[0094] (3) Label: The target label category (for example, circular defect, strip defect, incomplete fusion, incomplete penetration, crack, etc.) framed in step (1) is labeled.

[0095] (4) According to the weld area labeling dataset in step (2), the weld area is cut and the defect labeling frame is mapped to form a weld defect labeling dataset containing only the weld area image.

[0096] (5) The weld defect labeling dataset in step (4) is screened, and n samples (n can be 10, 20, 30..) of each type of defect are selected to form a small-sample weld defect labeling dataset.

[0097] Step 102-2, the weld defect labeling dataset is processed by data enhancement and nonlinear gray scale transformation to obtain a processed weld defect labeling dataset, and the processed weld defect labeling dataset includes a training set.

[0098] In this embodiment, data enhancement is performed using cropping and contrast enhancement, and nonlinear gray scale transformation preprocessing of the weld defect labeling dataset is performed using sin transformation to obtain a preprocessed weld defect labeling dataset, and the preprocessed weld defect labeling dataset is divided into a training set and a test set.

[0099] It should be noted that, due to the small number of samples of strip defects, inclusion defects, cracks and other defects in the weld defects, the sample enhancement method based on cropping is added. Since the shape of the weld defect is an important discriminant in the quality inspection standard, the complete defect target needs to be retained, so the cropping strategy is modified as follows: after a cropped image is randomly generated, the ratio of the area of the intersection of the target box and the cropped image to the area of the target box is calculated, denoted as A, if A < a, the cropped image is discarded, if A = a, the cropped image is retained. The value of a is set according to the actual situation.

[0100] Step 102-3, training the small sample target detection network with the training set to obtain the trained small sample target detection network.

[0101] In this embodiment, the training process of the small sample target detection network generally includes two steps:

[0102] Step 1, training the small sample target detection network using the training set in the processed weld defect annotation data set, updating all weights in the small sample target detection network until the small sample target detection network converges;

[0103] Step 2, creating a new detection head, using the small sample weld defect annotation data set in the processed weld defect annotation data set as the training data set in the second stage, freezing the weights of the small sample target detection network before the detection head, and only updating the weights of the new detection head during the training process.

[0104] The embodiment performs data enhancement on the weld defect annotation data set, and automatically detects the weld defect in the case of limited training set, thereby improving the detection efficiency.

[0105] As an optional implementation, step 102-2 includes:

[0106] Step 102-21, performing data enhancement processing on the weld defect annotation data set by using a random flip and / or random cropping algorithm to obtain the weld defect annotation data set after data enhancement processing;

[0107] In this embodiment, the random flip and random cropping algorithm is used for data enhancement during training of the base model; and only the random flip algorithm is used for data enhancement during training of the fine-tuning model.

[0108] Step 102-22, performing non-linear grayscale transformation processing on the weld defect annotation data set after data enhancement processing by using sin transformation to obtain the processed weld defect annotation data set.

[0109] The embodiment uses data enhancement means of cropping and contrast enhancement to increase the number of small sample defects, and adjusts the cropping strategy to ensure the availability of the enhanced weld defect annotation data set. The weld defect annotation data set preprocessing is modified from normalization to nonlinear gray scale transformation based on sin transformation, which can make the defects more prominent.

[0110] As an optional implementation, the processed weld defect annotation data set includes a test set, and the detection method further includes:

[0111] The trained small sample target detection network is tested by using the test set until the detection accuracy of the trained small sample target detection network reaches the target condition.

[0112] In the embodiment, the weight of the trained small sample target detection network is used for auxiliary annotation, and the weight is substituted into the forward inference script of the small sample target detection network. The performance of the trained small sample target detection network is tested by using the test set in the processed weld defect annotation data set. If the detection accuracy of the trained small sample target detection network reaches the target condition, the trained small sample target detection network is effective, and the next step can be performed. If the detection accuracy of the trained small sample target detection network does not reach the target condition, the labeled defect data can be appropriately increased or the samples screened in the small sample weld defect annotation data set can be replaced, and the training of the small sample target detection network is performed again until the performance of the small sample target detection network meets the standard. The weight of the small sample target detection network whose detection accuracy meets the target condition is saved, and the forward inference script of the small sample target detection network is packaged as a microservice, which aims to interact with the weld defect detection software for image and detection result. The detection result is displayed as follows: if there is a weld defect in the image, the bounding box region of the above target, the attribute and state description, and the confidence are output. The microservice can be used as an auxiliary detection tool in the weld quality inspection task to provide convenient pre-annotation reference for quality inspection personnel. In the testing process, the IoU threshold is set to 0.3, that is, if the IoU loss of the predicted detection box and the labeled real target box is greater than 0.3, it is considered to be correct detection, otherwise it is considered to be incorrect detection. The AP@0.3 and Recall@0.3 of the fine-tuned model of the small sample target detection network on the six types of defects are shown in Table 1:

[0113] Table 1

[0114] Category Circular defect Unfused Bar defect Other defect Clamp Crack AP@0.3 0.848 0.809 0.204 0.397 0.598 0.500 Recall@0.3 0.648 0.945 0.625 0.700 0.916 1.000

[0115] It should be noted that the target condition is set according to the actual situation, which is not limited here.

[0116] As an optional implementation, the small sample target detection network includes a base model and a fine-tuned model; step 102-3 includes:

[0117] Step 102-31, training the small sample target detection network by using the processed weld defect annotation data set to obtain a basic model;

[0118] Step 102-32, freezing the weights of the basic model to form a fine-tuning model;

[0119] In the embodiment, the weights of other modules of the basic model except the last classification branch weight or the regression branch weight can be frozen.

[0120] It should be noted that the meaning of freezing is that the weights after freezing will not be updated in the subsequent fine-tuning process.

[0121] Step 102-33, obtaining a small sample weld defect annotation data set from the processed weld defect annotation data set;

[0122] Step 102-34, training the fine-tuning model by using the small sample weld defect annotation data set to obtain a trained fine-tuning model.

[0123] In the process of training the small sample target detection network, for example, the RPN network adopts MaxIoU matching, the positive sample threshold is 0.7, the negative sample threshold is 0.3, the minimum positive sample threshold is 0.3, and all samples are selected for training; the RCNN network adopts MaxIoU matching, the positive sample threshold is 0.5, the negative sample threshold is 0.3, the minimum positive sample threshold is 0.4, the samples are randomly selected during training, the positive sample to negative sample ratio is 1:3, and the maximum number is 256; the Group Normalization layer of the CNN network is updated, and a pre-trained model provided by Pytorch is used; the Adam optimizer is used for training the basic model, the learning rate is set to 0.0001, a total of 50 epochs are trained, and gradient clipping is used; the Adam optimizer is used for training the fine-tuning model, the learning rate is set to 1e-5, a total of 500 epochs are trained, and gradient clipping is not used.

[0124] The embodiment introduces a small sample detection algorithm based on fine-tuning and a data enhancement method, and can still obtain good detection effect for tail defect categories in the case of long-tail effect of defect data, improves the generalization ability and robustness of the small sample target detection network, improves the accuracy of the small sample target detection network for rare defects by increasing the small sample training link, and fully utilizes a large number of common defect samples to improve the detection rate of rare sample defects.

[0125] The embodiment utilizes a two-stage target detection algorithm of Faster R-CNN and a small sample detection algorithm based on fine-tuning to construct a small sample target detection network to detect weld defects of small diameter pipe rings, and improves the accuracy of the small sample target detection network in detecting rare defects and the detection rate of small sample defects.

[0126] Embodiment 3

[0127] The embodiment provides a weld defect detection system based on a small sample target detection network, as shown in the following figure, the detection system comprises: Figure 2

[0128] The construction module 21 is configured to utilize a two-stage target detection algorithm of Faster R-CNN and a small sample detection algorithm based on fine-tuning to construct a small sample target detection network.

[0129] As an optional implementation, the detection head in the small sample target detection network comprises a CosHead.

[0130] As an optional implementation, the Faster R-CNN comprises a convolutional neural network, a region generation network, a feature pyramid network and a Guide Anchor RPN network.

[0131] In the process of constructing the small sample target detection network, since the defect samples have a serious long-tail effect, in order to meet the detection requirements under the condition of small samples, the two-stage target detection algorithm of Faster R-CNN is combined with the small sample detection algorithm TFA (two-stage fine-tuning approach) based on fine-tuning to construct the small sample target detection network. The Faster R-CNN comprises the following two stages: the first stage is composed of a convolutional neural network (CNN) and a region generation network (RPN), wherein the CNN network is used to extract general features of an image, the general feature extraction network of the small sample target detection network used in the embodiment is ResNet-101, and the RPN network extracts a candidate target bounding box according to the general features. The RPN network is usually composed of a simple convolutional network (for example, a deformable convolutional layer network) and a classification branch and a regression branch; the second stage extracts features of the candidate regions from the RPN network and performs class classification and bounding box regression.

[0132] Specifically, the construction process of the small sample target detection network comprises a-h.

[0133] ​a、Normalization layer: The input image resolution is large enough to ensure good detection effect, but the number of single GPU training images is small, which leads to poor Batch Normalization effect. Therefore, Group Normalization is used as the normalization layer of the small sample target detection network, which can have better effect than the default Batch Normalization layer;

[0134] b、Modify the optimizer: Adam optimizer is used to accelerate the training process of the small sample target detection network and prevent the optimization point from falling into the saddle point;

[0135] c、Add Feature Pyramid Network (FPN): For example, under the image resolution of 1120x 1120, the total number of pixels of the round defects is 200-12544, and the total number of pixels of the incomplete fusion or incomplete penetration is 2000-12544, the size difference is very large, therefore, the FPN network is added to enhance the network's processing capacity for target scale;

[0136] d、Modify the RPN network to Guide Anchor RPN network: The original RPN network uses fixed Anchor for classification and regression at each location, Guide Anchor can adaptively set Anchor according to the current location features, thus improving the quality of the RPN network output foreground part;

[0137] e、The CNN network of the RPN network uses deformable convolution (DCN): The traditional convolutional neural network has a consistent square shape in the theoretical receptive field, deformable convolution makes the theoretical receptive field of the neural network more flexible, and has better effect on objects with variable shapes;

[0138] f、The loss function of the regression branch of the RPN network and the RCNN network uses GIoU loss, the average precision (AP) calculation method is based on the intersection over union (IoU), and the loss based on IoU can make the model training and inference more consistent;

[0139] g、Modify the detection head to CosHead;

[0140] h, The RCNN network part of the base model and the fine-tuning model in the small sample target detection network adopts CosHead. The difference between the detection head and the original detection head of Faster R-CNN is that the output of the classification branch adopts a cosine similarity function for normalization operation, which helps to reduce the variance of small sample categories, alleviate the problem of too large difference between different classification weights, prevent the model from tending to detect a large number of one class of positive samples, and thus improve the detection performance of the small number of samples, especially when the number of training samples is very small. In the prior art, the base model uses a normal detection head, and the fine-tuning model uses CosHead. After the base model is trained, the initial stage of the fine-tuning model is greatly reduced due to the introduction of normalization, thereby greatly prolonging the time from training to ideal AP, and the risk of overfitting is also greatly improved. Therefore, the detection head of the base model is also used CosHead, so that the model structure of the base model and the fine-tuning model is completely consistent. After improvement, the performance of the initial stage of the fine-tuning model is completely consistent with the base model, and only a small amount of time is needed to intuitively perceive the improvement of the small sample performance.

[0141] For example, when constructing a small sample target detection network, the input image size is configured as 1120 and 1120, the preprocessing mean is [128, 128, 128], and the variance is [35, 35, 35]; the CNN network in the RPN network adopts ResNet-101, for example, the basic size of the anchor is 8, there are five height-width ratios: 0.25, 0.5, 1.0, 2.0, 4.0, the number of feature scales used for detection is 5, and the step lengths are 4, 8, 16, 32 and 64; the hole rate of the convolution in the last stage of the CNN network is 2; the output channel of the FPN network can be 256, and the default method is used for upsampling; the RPN network uses Guide Anchor RPNHead, that is, a 3x3 convolution is used first, and then the classification branch and the regression branch are used to output results, the maximum number before NMS is 2000, the output number is 1000, and the IoU threshold of NMS is 0.7; the ROI pooling part adopts Deform ROIPooling to improve the feature information granularity of the pooling; the RCNN network part adopts CosHead, and the IoU threshold of NMS is 0.3 (to reduce the phenomenon of multiple same class boxes overlapping).

[0142] The weld area image acquisition module 22 is configured to acquire the small-diameter pipe girth weld area image.

[0143] In this embodiment, the small-diameter pipe girth weld area image is acquired by the weld area detection model.

[0144] The detection module 23 is configured to input the small-diameter pipe girth weld area image into the trained small sample target detection network to obtain the defect data of the small-diameter pipe girth weld.

[0145] In this embodiment, the defect data includes defect categories and defect positions, and the defect categories include, but are not limited to, circular defects, strip defects, clamps, cracks, un-fusion, and un-welding.

[0146] It should be noted that, since defect identification is also a very challenging task for humans, and there are often missing annotations in the annotated samples; on the other hand, there is less existing annotated data. Therefore, the present application adopts a human-in-the-loop technology to provide guidance for researchers during model learning, and continuously improves the recognition ability of the model and the size and accuracy of the annotated data set in human-computer interaction iteration. The specific method is: using the annotated weld defect data set to train a small sample target detection network, using the network to infer the unannotated images, then manually auditing the targets detected by the small sample target detection network, deleting the incorrect detected targets, and adding the correct detected targets to the weld defect annotation data set. Through multiple rounds of the above process, the ability of the small sample target detection network and the accuracy of the weld defect annotation data set can be improved.

[0147] The small sample target detection network trained in this embodiment realizes the detection of small-diameter pipe girth weld defects, improves the detection rate of types with less positive samples such as strip defects, clamps, cracks, and other defects, and effectively reduces the large amount of human and machine resources consumed in image annotation work by training and detecting the model under the condition of less image data.

[0148] In addition, the auxiliary quality inspection function is provided in combination with the weld defect detection software: the present application aims to provide an auxiliary tool for welding software, and the application scenarios can be extended to scenarios other than small-diameter pipe girth welds to assist researchers to realize rapid weld quality grading.

[0149] Embodiment 4

[0150] The weld defect detection system based on the small sample target detection network of this embodiment is a further improvement of embodiment 3, specifically:

[0151] As an optional implementation, as shown in Figure 3 , the detection system further includes:

[0152] The data set acquisition module 24 is configured to acquire a weld defect annotation data set.

[0153] In this embodiment, the small diameter pipe girth weld X-ray image is obtained, including DR (Digital Radiography) image and RT (Radiographic Testing) image, and the weld X-ray image in the form of RT film is converted into digital image using a film scanner. The small diameter pipe girth weld X-ray image is labeled using a labeling tool, and the labeling method is to frame and label the specific defect type. The labeling labels include weld area, circular defect, strip defect, incomplete fusion, incomplete penetration, crack, etc. to form a full image labeling data set. The labeling steps are divided into steps (1)-(5):

[0154] (1) Framing: frame the weld area and welding defects in the small diameter pipe girth weld X-ray image respectively, and only frame the complete target, which is close to the target edge and completely includes the target;

[0155] (2) Label: the target label type of the weld area framed in step (1) is formed to form a weld area labeling data set;

[0156] (3) Label: the target label category (for example, circular defect, strip defect, incomplete fusion, incomplete penetration, crack, etc.) framed in step (1) is labeled;

[0157] (4) According to the weld area labeling data set in step (2), the weld area is cut and the defect labeling frame is mapped to form a weld defect labeling data set containing only the weld area image;

[0158] (5) The weld defect labeling data set in step (4) is screened, and n samples (n can be 10, 20, 30..) of each type of defect are selected to form a small sample weld defect labeling data set.

[0159] The processing module 25 is configured to perform data enhancement processing and nonlinear gray scale transformation processing on the weld defect labeling data set to obtain a processed weld defect labeling data set, and the processed weld defect labeling data set includes a training set.

[0160] In this embodiment, the data enhancement is performed by cropping and contrast enhancement, and the nonlinear gray scale transformation preprocessing of the weld defect labeling data set is performed by using sin transformation to obtain a preprocessed weld defect labeling data set, and the preprocessed weld defect labeling data set is divided into a training set and a test set.

[0161] It should be noted that, due to the small number of samples of strip defects, clamping pits, cracks and other defects in the weld defects, the sample enhancement method based on cropping is added. Since the shape of the weld defect is an important discriminant in the quality inspection standard, the complete defect target needs to be retained, so the cropping strategy is modified as follows: after a cropped image is randomly generated, the ratio of the area of the intersection of the target box and the cropped image to the area of the target box is calculated, denoted as A, if A < alpha, the cropped image is discarded, if A = alpha, the cropped image is retained. The value of alpha is set according to the actual situation.

[0162] The training module 26 is configured to train the small sample target detection network using the training set to obtain the trained small sample target detection network.

[0163] In this embodiment, the training process of the small sample target detection network generally includes two steps:

[0164] Step 1: using the training set in the processed weld defect annotation data set to train the small sample target detection network, updating all weights in the small sample target detection network until the small sample target detection network converges;

[0165] Step 2: creating a new detection head, using the small sample weld defect annotation data set in the processed weld defect annotation data set as the training data set in the second stage, freezing the weights of the small sample target detection network before the detection head, and only updating the weights of the new detection head during the training process.

[0166] The embodiment performs data enhancement on the weld defect annotation data set, and automatically detects the weld defect in the case of limited training set, thereby improving the detection efficiency.

[0167] As an optional implementation, as shown in Figure 3 The processing module 25 includes:

[0168] The data enhancement processing unit 251 is configured to perform data enhancement processing on the weld defect annotation data set by using a random flipping algorithm and / or a random cropping algorithm to obtain the weld defect annotation data set after data enhancement processing.

[0169] In this embodiment, the random flipping algorithm and the random cropping algorithm are used for data enhancement during training of the base model; and only the random flipping algorithm is used for data enhancement during training of the fine-tuning model.

[0170] The transformation processing unit 252 is configured to perform non-linear grayscale transformation processing on the weld defect annotation data set after data enhancement processing by using a sin transformation to obtain the processed weld defect annotation data set.

[0171] The embodiment uses data enhancement means of cropping and contrast enhancement to increase the number of small sample defects, and adjusts the cropping strategy to ensure the availability of the enhanced weld defect annotation data set. The weld defect annotation data set preprocessing is modified from normalization to nonlinear gray scale transformation based on sin transformation, which can make the defects more prominent.

[0172] As an optional implementation, the processed weld defect annotation data set includes a test set, as shown in the figure, the detection system further includes: Figure 3

[0173] The test module 27 is configured to test the trained small sample target detection network by using the test set until the detection accuracy of the trained small sample target detection network reaches the target condition.

[0174] In the embodiment, the weight of the trained small sample target detection network is used for auxiliary annotation, the weight is substituted into the forward inference script of the small sample target detection network, the performance of the trained small sample target detection network is tested by using the test set in the processed weld defect annotation data set, if the detection accuracy of the trained small sample target detection network reaches the target condition, that is, the trained small sample target detection network is effective, the next step can be performed, if the detection accuracy of the trained small sample target detection network does not reach the target condition, the data with labeled defects can be appropriately increased or the samples screened in the small sample weld defect annotation data set are replaced, and the training of the small sample target detection network is performed again until the performance of the small sample target detection network meets the standard. The weight of the small sample target detection network whose detection accuracy meets the target condition is saved, and the forward inference script of the small sample target detection network is packaged as a micro service, which aims to interact with the weld defect detection software for image and detection result. The detection result is displayed as follows: if there is a weld defect in the image, the bounding box region of the above target, the attribute and state description and the confidence are output. The micro service can be used as an auxiliary detection tool in the weld quality inspection task to provide convenient pre-annotation reference for quality inspection personnel. In the test process, the IoU threshold is set to 0.3, that is, if the IoU loss of the predicted detection box and the labeled real target box is greater than 0.3, it is considered to be correct detection, otherwise it is considered to be incorrect detection. The AP@0.3 and Recall@0.3 of the fine-tuning model of the small sample target detection network are shown in Table 1 in the embodiment 2;

[0175] It should be noted that the target condition is set according to the actual situation, which is not limited here.

[0176] As an optional implementation, the small sample target detection network includes a basic model and a fine-tuning model; as shown in the figure, the training module 26 includes: Figure 3 ​​

[0177] The first training unit 261 is configured to train the small sample target detection network by using the processed weld defect annotation dataset, so as to obtain the base model.

[0178] The freezing unit 262 is configured to freeze the weights of the base model, so as to form a fine-tuning model.

[0179] In this embodiment, the weights of other modules of the base model except the last classification branch weight or the regression branch weight can be frozen.

[0180] It should be noted that the meaning of freezing is that the weights after freezing will not be updated in the subsequent fine-tuning process.

[0181] The acquisition unit 263 is configured to acquire a small sample weld defect annotation dataset from the processed weld defect annotation dataset.

[0182] The second training unit 264 is configured to train the fine-tuning model by using the small sample weld defect annotation dataset, so as to obtain a trained fine-tuning model.

[0183] In the process of training the small sample target detection network, for example, the RPN network adopts MaxIoU matching, the positive sample threshold is 0.7, the negative sample threshold is 0.3, the minimum positive sample threshold is 0.3, and all samples are selected for training; the RCNN network adopts MaxIoU matching, the positive sample threshold is 0.5, the negative sample threshold is 0.3, the minimum positive sample threshold is 0.4, the samples are randomly selected during training, the positive sample to negative sample ratio is 1:3, and the maximum number is 256; the Group Normalization layer of the CNN network is updated, and a pre-trained model provided by Pytorch is used; the Adam optimizer is used for training the base model, the learning rate is set to 0.0001, a total of 50 epochs are trained, and gradient clipping is used; the Adam optimizer is used for training the fine-tuning model, the learning rate is set to 1e-5, a total of 500 epochs are trained, and gradient clipping is not used.

[0184] By introducing the small sample detection algorithm based on fine-tuning and the data enhancement means, the embodiment can still obtain good detection effect for tail defect categories in the case of long tail effect of defect data, improves the generalization ability and robustness of the small sample target detection network, improves the accuracy of the small sample target detection network for rare defects by increasing the small sample training link, and fully utilizes a large number of common defect samples to improve the detection rate of rare sample defects.

[0185] The embodiment utilizes a two-stage target detection algorithm of Faster R-CNN and a small sample detection algorithm based on fine-tuning to construct a small sample target detection network to detect the weld defects of the small diameter pipe ring, and improve the accuracy of the small sample target detection network in detecting rare defects and the detection rate of small sample defects.

[0186] Embodiment 5

[0187] Figure 4 A structural schematic diagram of an electronic device provided in the embodiment 3 of the present application is provided. The electronic device includes a memory, a processor, and a computer program stored in the memory and used for running on the processor, and the processor implements the method for detecting the weld defects based on the small sample target detection network in the embodiment 1 when executing the program. Figure 4 The electronic device 30 shown is merely an example and should not bring any limitation to the function and use range of the embodiment of the present application.

[0188] As shown in Figure 4 The electronic device 30 can be in the form of a general computing device, for example, it can be a server device. The components of the electronic device 30 can include but are not limited to the above-mentioned at least one processor 31, the above-mentioned at least one memory 32, a bus 33 connecting different system components including the memory 32 and the processor 31.

[0189] The bus 33 includes a data bus, an address bus and a control bus.

[0190] The memory 32 can include a volatile memory, for example, a random access memory (RAM) 321 and / or a cache memory 322, and can further include a read-only memory (ROM) 323.

[0191] The memory 32 can further include a program / utility 325 having a set of the program modules 324, such as an operating system, one or more application programs, other program modules, and program data, and each of these examples, or some combination thereof, can include implementation of a network environment.

[0192] The processor 31 performs various function applications and data processing by running the computer program stored in the memory 32, for example, the method for detecting the weld defects based on the small sample target detection network in the embodiment 1 of the present application.

[0193] The electronic device 30 can also communicate with one or more external devices 34 such as a keyboard, a pointing device, etc. through an input / output (I / O) interface 35. Further, the model generation device 30 can communicate with one or more networks such as a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet, through a network adapter 36. As Figure 4 illustrated, the network adapter 36 communicates with the other modules of the model generation device 30 through the bus 33. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with the model generation device 30 such as, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0194] It should be noted that although several means / modules or sub-means / modules of an electronic device are mentioned in the foregoing detailed description, such division is merely exemplary and not mandatory. Indeed, according to an embodiment of the application, the features and functionalities of two or more means / modules described above can be embodied in one means / module. Conversely, the features and functionalities of one means / module described above can be further divided into several means / modules.

[0195] Embodiment 6

[0196] The embodiment provides a computer readable storage medium, and a computer program is stored on the computer readable storage medium. The program is executed by a processor to implement the weld defect detection method based on the small sample target detection network provided in embodiment 1.

[0197] More specifically, the readable storage medium can include, but is not limited to, a portable disc, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0198] In a possible implementation, the present application can also be implemented in the form of a program product, which includes program code for causing a terminal device to execute the weld defect detection method based on the small sample target detection network described in embodiment 1 when the program product is run on the terminal device.

[0199] The program code for executing the present application can be written in any combination of one or more programming languages, and can be executed entirely on the user device, partly on the user device and partly on a remote device, or entirely on a remote device.

[0200] Although the specific embodiments of the present application have been described above, it is understood by those skilled in the art that the present application is only illustrated by way of example, and the scope of protection of the present application is defined by the appended claims. Those skilled in the art can make various changes or modifications to the embodiments without departing from the principles and essence of the present application, and such changes and modifications fall within the scope of protection of the present application.

Claims

1. A method for detecting weld defects based on a small sample target detection network, characterized in that, The detection method comprises: a small sample target detection network is constructed by using a two-stage target detection algorithm of Faster R-CNN and a small sample detection algorithm based on fine tuning; an image of a small-diameter pipe girth weld area is acquired; the small sample target detection network is trained by using a training set to obtain a trained small sample target detection network; the image of the small-diameter pipe girth weld area is input into the trained small sample target detection network to obtain defect data of the small-diameter pipe girth weld; the small sample target detection network comprises a base model and a fine tuning model; detection heads of the base model and the fine tuning model are both CosHead; the step of training the small sample target detection network by using the training set to obtain the trained small sample target detection network comprises: the small sample target detection network is trained by using the processed girth weld defect annotation data set to obtain the base model; weights of the base model are frozen to form the fine tuning model. 2.The method of claim 1, wherein, The Faster R-CNN comprises a convolutional neural network, a region generation network, a feature pyramid network and a Guide AnchorRPN network. 3.The method of claim 1, wherein, Before the step of inputting the image of the small-diameter pipe girth weld area into the trained small sample target detection network to obtain the defect data of the small-diameter pipe girth weld, the detection method further comprises: a girth weld defect annotation data set is acquired; the girth weld defect annotation data set is subjected to data enhancement processing and non-linear gray scale transformation processing to obtain a processed girth weld defect annotation data set, wherein the processed girth weld defect annotation data set comprises a training set. 4.The method of claim 3, wherein, The step of subjecting the girth weld defect annotation data set to data enhancement processing and non-linear gray scale transformation processing to obtain the processed girth weld defect annotation data set comprises: a random flipping and / or random cropping algorithm is used to subject the girth weld defect annotation data set to data enhancement processing to obtain a girth weld defect annotation data set subjected to data enhancement processing; a sin transformation is used to subject the girth weld defect annotation data set subjected to data enhancement processing to non-linear gray scale transformation processing to obtain the processed girth weld defect annotation data set. 5.The method of claim 3, wherein, The processed girth weld defect annotation data set comprises a test set, and the detection method further comprises: the trained small sample target detection network is tested by using the test set until a detection accuracy of the trained small sample target detection network reaches a target condition. 6.The method of claim 3, wherein, The step of training the small sample target detection network by using the training set to obtain the trained small sample target detection network further comprises: a small sample girth weld defect annotation data set is acquired from the processed girth weld defect annotation data set; the fine tuning model is trained by using the small sample girth weld defect annotation data set to obtain a trained fine tuning model.

7. A system for detecting a weld defect based on a small sample target detection network, characterized by, The detection system comprises: a construction module configured to construct a small sample target detection network by using a two-stage target detection algorithm of Faster R-CNN and a small sample detection algorithm based on fine tuning; a girth weld area image acquisition module configured to acquire an image of a small-diameter pipe girth weld area; a training module configured to train the small sample target detection network by using a training set to obtain a trained small sample target detection network; The detection module is configured to input the small-diameter pipe girth weld area image into the trained small sample target detection network to obtain defect data of the small-diameter pipe girth weld. The small sample target detection network comprises a base model and a fine-tuning model; and detection heads of the base model and the fine-tuning model are both CosHead. The training module comprises a first training unit configured to train the small sample target detection network by using the processed weld defect annotation data set to obtain the base model. The freezing unit is configured to freeze the weight of the base model to form the fine-tuning model.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory for running on the processor, characterized in that, The processor executes the computer program to implement the weld defect detection method based on the small sample target detection network according to any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the weld defect detection method based on the small sample target detection network according to any one of claims 1-6.

Images