Weak supervision defect detection method for component image

Abstract

The invention provides a weak supervision defect detection method for a component image, and belongs to the field of digital image processing and industrial detection. According to the method, the significant characteristics of an attention mechanism are considered, the advantage that the U-Net + + can achieve high-precision pixel-level image segmentation under a small sample is utilized, a triple convolution attention module and a self-supervision convolution attention module are introduced into a U-Net + + network for structure optimization, and a weak supervision mode training model is provided. The method is suitable for component image defect detection tasks under the conditions that the number of normal samples is large and the number of defect samples is small, the method is reliable and good in stability, the problems existing in a supervised method and an unsupervised method can be effectively solved, the component image covers a surface image for detecting surface defects and a CT image for detecting internal defects, and the detection accuracy is high. The method can be applied to high-precision internal and external defect detection of components.

Description

Technical Field

[0001] The invention belongs to the field of digital image processing and industrial detection, and relates to a weakly supervised defect detection method for component images. Background Art

[0002] In the industrial production process, the quality of finished components is easily affected by the deficiencies and limitations of existing technologies, working conditions and other factors. Defects are an important manifestation of product quality, which can generally be divided into surface defects and internal defects. Surface defects of components are generally detected based on the surface images taken. There are many means of detecting internal defects of components, and industrial CT (Computed Tomography) that detects in the form of intuitive tomographic images is one of the important detection methods. Surface images and CT images have great similarities and can be understood as digital images obtained by "shooting". Defect detection of actual industrial components may only perform surface defect detection, or only perform internal defect detection, or both surface defects and internal defects need to be detected.

[0003] Manual defect detection used to be the mainstream method, but this method is inefficient, the detection results are easily affected by human subjective factors, and cannot meet the requirements of real-time detection. The comprehensive intelligent development of the manufacturing industry has put forward new requirements for the quality inspection of industrial products. Traditional feature-based machine vision defect detection methods generally include three major methods based on texture features, color features, and shape features. In addition to these three types of features, some other features, such as spatial relationship features, can also be used for industrial product defect detection. Since most industrial products contain a variety of information, it is usually not enough to use only a single feature or a single category of features. In practical applications, multiple features and multiple categories of features are often used in combination.

[0004] The rapid development of deep learning has made it more and more widely used in the field of defect detection, which can be roughly divided into three categories: supervised methods, unsupervised methods and weakly supervised methods. Among the three methods of deep learning, the supervised method is the most widely used because it has good accuracy, but the cost of this defect labeling method is undoubtedly huge and does not meet the actual production needs; the unsupervised method is in line with the progress of industrial development. Although it does not require defective samples, its detection accuracy is low and cannot meet the detection needs; the weakly supervised method is not widely used at present, but only a small number of defective samples and normal samples are needed to achieve higher detection performance, which is more in line with actual needs. In actual industrial production, quality control must be carried out, which makes normal samples abundant, but due to the existence of objective unstable factors, it is inevitable that products will have flaws or defects, but there are still few defective samples, which increases the difficulty of defect detection to a certain extent.

[0005] In recent years, many deep learning-based algorithms have been successfully applied to the field of defect detection, such as Convolutional Neural Networks (CNN), Autoencoder, Generative Adversarial Networks (GAN), U-Net, Faster R-CNN, etc. Among them, U-Net is favored by researchers because it can still achieve high-precision pixel-level image segmentation with fewer samples, and is widely used in surface and internal defect detection of industrial products. At present, defect detection of industrial products still needs further research and performance improvement in terms of real-time problems, small sample problems, small target problems, and unbalanced sample problems. Summary of the invention

[0006] According to the situation that normal samples are far more than defective samples in actual production, the present invention provides a weakly supervised defect detection method for component images. Taking into account the significant characteristics of the attention mechanism and utilizing the advantage of U-Net++ that it can achieve high-precision pixel-level image segmentation even with small samples, the triple convolutional attention module and the self-supervised convolutional attention module are introduced into the U-Net++ network for structural optimization, and a weakly supervised training model is provided, which can be applied to defect detection of surface images and CT images of components to achieve high-precision defect positioning.

[0007] The technical solution adopted by the present invention to solve the technical problem comprises the following steps:

[0008] Step 1: According to the component defect detection task, a defect detection dataset with the same image type and resolution, object characteristics and image quality is constructed, in which normal samples are the main body and defective samples account for only a small number;

[0009] Step 2: Build a U-Net++ network defect detection model including a triple convolutional attention module and a self-supervised convolutional attention module;

[0010] Step 3: Use a weakly supervised training method to introduce the defect samples in the defect detection dataset into the training process of the U-Net++ network defect detection model, and complete the training of the defect detection model together with the synthetic defect samples;

[0011] Step 4: Use the trained U-Net++ network defect detection model to perform component defect detection, input the image of the component to be detected, and output the defect detection result image of the image. If a defect exists, the location of the defect and its shape will be displayed in the result image.

[0012] In the above step 1, the image types in the component defect detection task include component surface images and component CT images. The image type of one detection task can only be one of the two.

[0013] In the above step 2, the U-Net++ network defect detection model including the triple convolutional attention module and the self-supervised convolutional attention module is constructed with the following features:

[0014] The U-Net++ network defect detection model includes 15 convolution blocks, 4 downsampling (MaxPool2d), 10 deconvolution operations (ConvTranspose2d), and 20 jump connections. Except for the convolution blocks at the output and input ends, the structure of each convolution block is a triple convolution attention module CTAM + 2 convolution layers; the convolution block at the input end is 2 convolution layers; the convolution blocks at the four output ends are CTAM + 2 convolution layers + self-supervised convolution attention module SSPCAB + 1 convolution layer, where the structure of each convolution layer is Conv2d + BatchNorm2d + ReLu, and the convolution kernel size is 3×3.

[0015] The above-mentioned CTAM has three branches, each of which is essentially a spatial attention module, which consists of a pooling layer + a convolutional layer. The pooling layer includes maximum pooling and average pooling. The convolutional layer is Conv2d+BatchNorm2d+Sigmoid, and the convolution kernel size is 7×7. CTAM is located after the skip connection and upsampling fusion features in the U-Net++ network.

[0016] The above SSPCAB includes a masked convolution and a channel attention module (ECAnet); the masked convolution consists of four sub-convolutions, which process the upper left, upper right, lower left and lower right areas of the target feature respectively, and their weighted results are the output values ​​of the target bit features; the feature map after the masked convolution is then weighted by the channel dimension by ECANet to obtain the final feature map. SSPCAB is located before the last convolution layer at the output end of the U-Net++ network.

[0017] In the above step 3, a weakly supervised training method is adopted to introduce the defect samples in the defect detection data set into the training process of the U-Net++ network defect detection model, and complete the training of the defect detection model together with the synthetic defect samples. The specific steps include:

[0018] (1) Extract the defect mask M from Perlin noise a , using the defect detection dataset as the defect data source E d ;

[0019] (2) M a With E dPerform element-wise multiplication ⊙, by M a ⊙E d Obtain intermediate product I1;

[0020] (3) M a Negate Compared with the normal sample N in the defect detection dataset d Perform element-wise multiplication ⊙, by Obtaining intermediate product I3;

[0021] (4) By M a ⊙N d The intermediate product I2 is obtained. I2 is present only in M a The pixel value is 1, which shows N d Texture data;

[0022] (5) Perform element-wise addition on I1, I2, and I3 Depend on Get the synthetic defect sample S d ;

[0023] (6) S d and the defect sample R in the defect detection dataset d Concatenate to form training data T d , the splicing method is to add S d Randomly add R d One or several defective samples;

[0024] (7) Select Focal Loss, Focal Tversky Loss and L1 Loss as loss functions for each predicted output of the U-Net++ network defect detection model. (Total 4), and label it with the defect Using deep supervision, M a With R d The set of true value graphs, then for each loss function:

[0025]

[0026] (8) Calculate the target loss function Loss = FLoss + (FTLoss / λ1) + (L1Loss / λ2), where λ1 and λ2 are weight coefficients, λ1 is set to the gradient cutoff of FTLoss / FLoss, and λ2 is set to the gradient cutoff of L1Loss / FLoss, so that λ1 and λ2 are constants;

[0027] (9) Set the relevant parameters of the U-Net++ network defect detection model based on the T dAs input, As output, the training of the defect detection model is completed.

[0028] The beneficial effects of the present invention are as follows: a weakly supervised defect detection method for component images provided by the present invention is suitable for component image defect detection tasks when the number of normal samples is abundant and the number of defect samples is small. The method is reliable and has good stability, and can effectively solve the problems existing in supervised methods and unsupervised methods. In addition, the component image covers surface images for detecting surface defects and CT images for detecting internal defects, and can be applied to high-precision internal and external defect detection of components. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the process of the present invention.

[0030] Figure 2 This is the overall schematic diagram of the U-Net++ network defect detection model.

[0031] Figure 3 Schematic diagram of the triple convolutional attention module.

[0032] Figure 4 Schematic diagram of the self-supervised convolutional attention module.

[0033] Figure 5 Schematic diagram of synthetic defects.

[0034] Figure 6 This is a CT image defect detection diagram of a composite material component. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0036] like Figure 1 As shown, it is a schematic diagram of the process of the present invention, for detecting defects in CT images of composite material components, a weakly supervised defect detection method of component images is performed, including the following steps:

[0037] Step 1: According to the component defect detection task, a defect detection dataset with the same image type and resolution, object characteristics and image quality is constructed. The image types in the component defect detection task include component surface images and component CT images. The image type of a detection task can only be one of the two. The defect detection dataset constructed in this embodiment contains CT images of various composite material components, of which the number of normal samples is 500, the number of defective samples is 10, and the image resolution is 256×256.

[0038] Step 2: Build a U-Net++ network defect detection model containing a triple convolutional attention module and a self-supervised convolutional attention module, such as Figure 2 shown.

[0039] The U-Net++ network defect detection model includes 15 convolution blocks, 4 downsampling (MaxPool2d), 10 deconvolution operations (ConvTranspose2d), and 20 jump connections. Except for the convolution blocks at the output and input ends, the structure of each convolution block is a triple convolution attention module CTAM + 2 convolution layers; the convolution block at the input end is 2 convolution layers; the convolution blocks at the four output ends are CTAM + 2 convolution layers + self-supervised convolution attention module SSPCAB + 1 convolution layer, where the structure of each convolution layer is Conv2d + BatchNorm2d + ReLu, and the convolution kernel size is 3×3.

[0040] The above CTAM has three branches, each of which is essentially a spatial attention module, which consists of a pooling layer + a convolutional layer. The pooling layer includes maximum pooling and average pooling, and the convolutional layer is Conv2d+BatchNorm2d+Sigmoid, with a convolution kernel size of 7×7. Figure 3 As shown in the figure, the two branches on the right are responsible for calculating the attention weights of the channel dimension C and the spatial dimensions H and W respectively, and the branch on the left is responsible for calculating the relationship between the spatial dimensions H and W. Finally, the simple average of the three weights is taken as the final output feature. CTAM is located after the skip connection and upsampling fusion features in the U-Net++ network.

[0041] The above SSPCAB (such as Figure 4 The masked convolution consists of four sub-convolutions K i (i: 1~4), which process the upper left, upper right, lower left and lower right areas of the focus area M respectively, and their weighted results are the values ​​of M. The subconvolution size is represented by K', and d is the expansion domain, which represents the distance from the subconvolution kernel to the receptive field center M. In this embodiment, M, K' and d are all set to 1, then the size of the mask convolution receptive field (K M Represents) as:

[0042] K M =2K'+2d+M.

[0043] ECAnet is a lightweight channel attention module, in which global average pooling performs feature aggregation on the input features. ECAnet generates the weights of each channel through a one-dimensional convolution of size K, where K is adaptively determined by the mapping relationship with the channel dimension C through the following function:

[0044]

[0045] Among them, |t oodTo represent the nearest odd integer to t, set γ and b to 2 and 1 respectively.

[0046] The feature map after mask convolution is then weighted by ECANet on the channel dimension to obtain the final feature map. SSPCAB is located before the last convolution layer at the output of the U-Net++ network.

[0047] Step 3: Use a weakly supervised training method to introduce the defect samples in the defect detection data set into the training process of the U-Net++ network defect detection model, and complete the training of the defect detection model together with the synthetic defect samples. The specific steps include:

[0048] (1) Extract the defect mask M from Perlin noise a , using the defect detection dataset as the defect data source E d ;

[0049] (2) M a With E d Perform element-wise multiplication ⊙, by M a ⊙E d Obtain intermediate product I1;

[0050] (3) M a Negate Compared with the normal sample N in the defect detection dataset d Perform element-wise multiplication ⊙, by Obtaining intermediate product I3;

[0051] (4) By M a ⊙N d The intermediate product I2 is obtained. I2 is present only in M a The pixel value is 1, which shows N d Texture data;

[0052] (5) Perform element-wise addition on I1, I2, and I3 Depend on Get the synthetic defect sample S d ,like Figure 5 As shown;

[0053] (6) S d and the defect sample R in the defect detection dataset d Concatenate to form training data T d , the splicing method is to add S d Randomly add R d One or several defective samples;

[0054] (7) Select Focal Loss, Focal Tversky Loss and L1 Loss as loss functions for each predicted output of the U-Net++ network defect detection model. (Total 4), and label it with the defect Using deep supervision, M a With R d The set of true value graphs, then for each loss function:

[0055]

[0056]

[0057] (8) Calculate the target loss function Loss = FLoss + (FTLoss / λ1) + (L1Loss / λ2), where λ1 and λ2 are weight coefficients, λ1 is set to the gradient cutoff of FTLoss / FLoss, and λ2 is set to the gradient cutoff of L1Loss / FLoss, so that λ1 and λ2 are constants;

[0058] (9) Set the learning rate of the U-Net++ network defect detection model to 0.001, the training epoch to 2000, the learning rate adjustment strategy parameter to an exponential decay of 0.992, and the optimizer to select ADAM. d As input, As output, the training of the defect detection model is completed.

[0059] Step 4: Use the trained U-Net++ network defect detection model to perform CT image defect detection on composite components. Input the image of the component to be detected and output the defect detection result image of the image. If there is a defect, the location of the defect and its shape will be displayed in the result image, such as Figure 6 shown.

[0060] The specific embodiments of the present invention are described in detail above, but they are only examples. The present invention is not equivalent to the specific embodiments described above, and the present invention can realize the surface defect detection and internal defect detection of components, covering its surface image and internal CT image. For those skilled in the art, any equivalent modification and substitution of the present invention is also within the scope of the present invention. Therefore, the equivalent transformations and modifications made without departing from the spirit and scope of the present invention should be included in the scope of the present invention.

Claims

1. A weakly supervised defect detection method for component images, characterized in that The steps include: Step 1: According to the component defect detection task, a defect detection dataset with the same image type and resolution, object characteristics and image quality is constructed, in which normal samples are the main body and defective samples account for only a small number; Step 2: Build a U-Net++ network defect detection model including a triple convolutional attention module and a self-supervised convolutional attention module; Step 3: Use a weakly supervised training method to introduce the defect samples in the defect detection dataset into the training process of the U-Net++ network defect detection model, and complete the training of the defect detection model together with the synthetic defect samples; Step 4: Use the trained U-Net++ network defect detection model to perform component defect detection, input the image of the component to be detected, and output the defect detection result image of the image. If a defect exists, the location of the defect and its shape will be displayed in the result image.

2. The weakly supervised defect detection method for component images according to claim 1 is characterized in that: In step 1, the image types in the component defect detection task include component surface images and component CT images. The image type of one detection task can only be one of the two.

3. The weakly supervised defect detection method for component images according to claim 1 is characterized in that: In step 2, the constructed U-Net++ network defect detection model including the triple convolutional attention module and the self-supervised convolutional attention module has the following characteristics: The U-Net++ network defect detection model includes 15 convolution blocks, 4 downsamplings (MaxPool2d), 10 deconvolution operations (ConvTranspose2d), and 20 jump connections; except for the convolution blocks at the output and input ends, the structure of each convolution block is a triple convolution attention module CTAM+2 convolution layers; the convolution block at the input end is 2 convolution layers; the convolution blocks at the four output ends are CTAM+2 convolution layers+self-supervised convolution attention module SSPCAB+1 convolution layer, where the structure of each convolution layer is Conv2d+BatchNorm2d+ReLu, and the convolution kernel size is 3×3.

4. The triple convolutional attention module (CTAM) according to claim 3, characterized in that: It has three branches, each of which is essentially a spatial attention module, which consists of a pooling layer + a convolutional layer. The pooling layer includes maximum pooling and average pooling. The convolutional layer is Conv2d+BatchNorm2d+Sigmoid, and the convolution kernel size is 7×7. CTAM is located after the skip connection and upsampling fusion features in the U-Net++ network.

5. The self-supervised convolutional attention module SSPCAB according to claim 3, characterized in that: It includes a masked convolution and a channel attention module (ECAnet); the masked convolution consists of four sub-convolutions, which process the upper left, upper right, lower left and lower right areas of the target feature respectively, and their weighted results are the output values ​​of the target bit features; the feature map after the masked convolution is then weighted by the ECANet on the channel dimension to obtain the final feature map. SSPCAB is located before the last convolution layer at the output end of the U-Net++ network.

6. The weakly supervised defect detection method for component images according to claim 1 is characterized in that: In step 3, a weakly supervised training method is adopted to introduce the defect samples in the defect detection data set into the training process of the U-Net++ network defect detection model, and complete the training of the defect detection model together with the synthetic defect samples. The specific steps include: (1) Extract the defect mask M from Perlin noise a , using the defect detection dataset as the defect data source E d ; (2) M a With E d Perform element-wise multiplication ⊙, by M a ⊙E d Obtain intermediate product I1; (3) M a Negate Compared with the normal sample N in the defect detection dataset d Perform element-wise multiplication ⊙, by Obtain intermediate product I3; (4) By M a ⊙N d The intermediate product I2 is obtained. I2 is present only in M a The pixel value is 1, which shows N d Texture data; (5) Perform element addition operation ⊕ on I1, I2, and I3, and obtain the synthetic defect sample S from I1⊕I2⊕I3 d ; (6) S d and the defect sample R in the defect detection dataset d Concatenate to form training data T d , the splicing method is to add S d Randomly add R d One or several defective samples; (7) Select Focal Loss, Focal Tversky Loss and L1 Loss as loss functions for each predicted output of the U-Net++ network defect detection model. (Total 4), and label it with the defect Using deep supervision, M a With R d The set of true value graphs, then for each loss function: (8) Calculate the target loss function Loss = FLoss + (FTLoss / λ1) + (L1Loss / λ2), where λ1 and λ2 are weight coefficients, λ1 is set to the gradient cutoff of FTLoss / FLoss, and λ2 is set to the gradient cutoff of L1Loss / FLoss, so that λ1 and λ2 are constants; (9) Set the relevant parameters of the U-Net++ network defect detection model based on the T d As input, As output, the training of the defect detection model is completed.

Images