A weakly supervised defect detection method for component images

Abstract

The present invention provides a weakly supervised defect detection method for component images, which belongs to the field of digital image processing and industrial inspection. This method takes into account the significant characteristics of the attention mechanism and utilizes the advantage of U-Net++ in being able to achieve high-precision pixel-level image segmentation even with small samples. The triple convolutional attention module and the self-supervised convolutional attention module are introduced into the U-Net++ network for structural optimization, and a weakly supervised training model is provided. This method is suitable for component image defect detection tasks in situations where the number of normal samples is abundant and the number of defective samples is small. The method is reliable and stable, and can effectively solve the problems existing in supervised and unsupervised methods. In addition, the component image covers surface images for detecting surface defects and CT images for detecting internal defects, and can be applied to high-precision internal and external defect detection of components.

Description

Technical Field

[0001] The invention belongs to the fields of digital image processing and industrial detection, and relates to a weakly supervised defect detection method for component images. Background Art

[0002] In industrial production, the quality of finished components is easily affected by the shortcomings and limitations of existing technologies, working conditions, and other factors. Defects are an important manifestation of product quality impairment and can generally be divided into surface defects and internal defects. Surface defects of components are generally detected based on captured surface images. There are various methods for detecting internal defects of components, and industrial CT (Computed Tomography), which performs detection in the form of intuitive tomographic images, is one of the most important detection methods. Surface images and CT images are very similar and can both be understood as digital images obtained by "photography." In actual industrial component defect detection, surface defect detection may only be performed, internal defect detection may only be performed, or both surface and internal defects may need to be detected.

[0003] Manual defect detection was once the mainstream method, but this method is inefficient, and detection results are easily affected by human subjective factors, which cannot meet the requirements of real-time detection. The comprehensive intelligent development of the manufacturing industry has put forward new requirements for the quality inspection of industrial products. Traditional feature-based machine vision defect detection methods generally fall into three categories: texture features, color features, and shape features. In addition to these three categories of features, other features, such as spatial relationship features, can also be used for industrial product defect detection. Because most industrial products contain multiple types of information, using only a single feature or a single category of features is usually not enough. In practical applications, multiple features and multiple categories of features are often combined.

[0004] The rapid development of deep learning has led to its increasingly widespread application in the field of defect detection. These methods can be broadly categorized into three categories: supervised, unsupervised, and weakly supervised. Of the three deep learning methods, supervised methods are the most widely used due to their high accuracy. However, the cost of this defect labeling method is undoubtedly enormous and does not meet actual production needs. Unsupervised methods are in line with industrial development and, while they do not require defective samples, their detection accuracy is low and does not meet detection needs. Weakly supervised methods are not currently widely used, but they can achieve high detection performance with only a small number of defective and normal samples, making them more in line with practical needs. In actual industrial production, quality control is essential, which results in an abundance of normal samples. However, due to objective instability, products inevitably have flaws or defects, yet defective samples remain scarce, which to some extent increases the difficulty of defect detection.

[0005] In recent years, many deep learning-based algorithms have been successfully applied to defect detection, such as convolutional neural networks (CNNs), autoencoders, generative adversarial networks (GANs), U-Net, and Faster R-CNN. U-Net, among others, has been favored by researchers for its ability to achieve high-precision pixel-level image segmentation even with relatively few samples, and is widely used for surface and internal defect detection in industrial products. Currently, industrial product defect detection still requires further research and performance improvement in areas such as real-time performance, small sample sizes, small targets, and unbalanced sample sizes. Summary of the Invention

[0006] In view of the fact that normal samples far outnumber defective samples in actual production, the present invention provides a weakly supervised defect detection method for component images. Taking into account the significant characteristics of the attention mechanism and utilizing the advantage of U-Net++ that it can achieve high-precision pixel-level image segmentation even with small samples, the triple convolutional attention module and the self-supervised convolutional attention module are introduced into the U-Net++ network for structural optimization, and a weakly supervised training model is provided, which can be applied to defect detection in surface images and CT images of components to achieve high-precision defect positioning.

[0007] The technical solution adopted by the present invention to solve the technical problem includes the following steps:

[0008] Step 1: Based on the component defect detection task, a defect detection dataset is constructed with the same image type and resolution, and similar object characteristics and image quality. Normal samples are the majority, and defective samples account for only a small number.

[0009] Step 2: Build a U-Net++ network defect detection model that includes a triple convolutional attention module and a self-supervised convolutional attention module;

[0010] Step 3: Using a weakly supervised training method, the defect samples in the defect detection dataset are introduced into the training process of the U-Net++ network defect detection model, and the training of the defect detection model is completed together with the synthetic defect samples.

[0011] Step 4: Use the trained U-Net++ network defect detection model to perform component defect detection. Input the component image to be inspected and output the defect detection result image of the image. If a defect exists, the location of the defect and its shape will be displayed in the result image.

[0012] In the above step 1, the image types in the component defect detection task include component surface images and component CT images. The image type of a detection task can only be one of the two.

[0013] In step 2 above, the U-Net++ network defect detection model built with the triple convolutional attention module and the self-supervised convolutional attention module has the following features:

[0014] The U-Net++ network defect detection model consists of 15 convolutional blocks, four downsampling operations (MaxPool2d), 10 deconvolution operations (ConvTranspose2d), and 20 skip connections. Except for the convolutional blocks at the output and input, each convolutional block is structured as a triple convolutional attention module (CTAM) followed by two convolutional layers. The convolutional block at the input consists of two convolutional layers. The convolutional blocks at the four outputs consist of a CTAM followed by two convolutional layers, a self-supervised convolutional attention module (SSPCAB), and one convolutional layer. Each convolutional layer uses a Conv2d, BatchNorm2d, and ReLu architecture, with a 3×3 kernel size.

[0015] The above-mentioned CTAM has three branches, each of which is essentially a spatial attention module, which consists of a pooling layer + a convolution layer. The pooling layer includes maximum pooling and average pooling. The convolution layer is Conv2d+BatchNorm2d+Sigmoid, and the convolution kernel size is 7×7; CTAM is located after the skip connection and upsampling fusion features in the U-Net++ network.

[0016] The SSPCAB algorithm consists of a masked convolution and a channel attention module (ECAnet). The masked convolution consists of four sub-convolutions, which process the upper left, upper right, lower left, and lower right regions of the target feature, respectively. The weighted result of these sub-convolutions forms the output value of the target bit feature. The feature map after the masked convolution is then weighted by the ECANet along the channel dimension to produce the final feature map. SSPCAB is placed before the last convolutional layer at the output of the U-Net++ network.

[0017] In step 3 above, a weakly supervised training method is adopted to introduce the defect samples in the defect detection dataset into the training process of the U-Net++ network defect detection model. The training of the defect detection model is completed together with the synthetic defect samples. The specific steps include:

[0018] (1) Extracting the defect mask M from Perlin noise a , using the defect detection dataset as the defect data source E d ;

[0019] (2) M a With E dPerform element-wise multiplication ⊙, by M a ⊙E d Obtain intermediate product I1;

[0020] (3) M a Negate Compared with the normal sample N in the defect detection dataset d Perform element-wise multiplication ⊙, by Obtain intermediate product I3;

[0021] (4) By M a ⊙N d The intermediate product I2 is obtained. I2 is only a The pixel value is 1, and N is displayed. d Texture data;

[0022] (5) Perform element-wise addition on I1, I2, and I3 Depend on Get the synthetic defect sample S d ;

[0023] (6) S d and the defect sample R in the defect detection dataset d Splicing to form training data T d , the splicing method is to add S d Randomly add R d One or more defective samples;

[0024] (7) Select Focal Loss, Focal Tversky Loss and L1 Loss as loss functions, and perform the weighted regression on each prediction output of the U-Net++ network defect detection model. (Total 4), and label it with the defect label Using deep supervision, M a With R d The set of true value maps, then for each loss function:

[0025]

[0026] (8) Calculate the target loss function Loss = FLoss + (FTLoss / λ1) + (L1Loss / λ2), where λ1 and λ2 are weight coefficients, λ1 is set to the gradient truncation of FTLoss / FLoss, and λ2 is set to the gradient truncation of L1Loss / FLoss, so that λ1 and λ2 are constants;

[0027] (9) Set the relevant parameters of the U-Net++ network defect detection model to the T dAs input, As output, the training of the defect detection model is completed.

[0028] The beneficial effects of the present invention are: a weakly supervised defect detection method for component images provided by the present invention is suitable for component image defect detection tasks when the number of normal samples is abundant and the number of defect samples is small. The method is reliable and stable, and can effectively solve the problems existing in supervised methods and unsupervised methods. In addition, the component image covers surface images for detecting surface defects and CT images for detecting internal defects, and can be applied to high-precision internal and external defect detection of components. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the process of the present invention.

[0030] Figure 2 This is the overall schematic diagram of the U-Net++ network defect detection model.

[0031] Figure 3 Schematic diagram of the triple convolutional attention module.

[0032] Figure 4 Schematic diagram of the self-supervised convolutional attention module.

[0033] Figure 5 Schematic diagram of synthetic defects.

[0034] Figure 6 This is a CT image defect detection diagram of a composite material component. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0036] like Figure 1 FIG. 1 is a flow chart of the present invention, which shows a method for weakly supervised defect detection of component images for CT image defect detection of composite material components, including the following steps:

[0037] Step 1: Based on the component defect detection task, construct a defect detection dataset with the same image type and resolution, and similar object characteristics and image quality. The image types in the component defect detection task include component surface images and component CT images. A single detection task can only have one of these two types of images. The defect detection dataset constructed in this example contains CT images of various composite components, with 500 normal samples, 10 defective samples, and an image resolution of 256×256.

[0038] Step 2: Build a U-Net++ network defect detection model that includes a triple convolutional attention module and a self-supervised convolutional attention module, as shown in Figure 2 shown.

[0039] The U-Net++ network defect detection model consists of 15 convolutional blocks, four downsampling operations (MaxPool2d), 10 deconvolution operations (ConvTranspose2d), and 20 skip connections. Except for the convolutional blocks at the output and input, each convolutional block is structured as a triple convolutional attention module (CTAM) followed by two convolutional layers. The convolutional block at the input consists of two convolutional layers. The convolutional blocks at the four outputs consist of a CTAM followed by two convolutional layers, a self-supervised convolutional attention module (SSPCAB), and one convolutional layer. Each convolutional layer uses a Conv2d, BatchNorm2d, and ReLu architecture, with a 3×3 kernel size.

[0040] The above CTAM has three branches, each of which is essentially a spatial attention module, which consists of a pooling layer + a convolution layer. The pooling layer includes maximum pooling and average pooling, and the convolution layer is Conv2d+BatchNorm2d+Sigmoid, with a convolution kernel size of 7×7. Figure 3 As shown in the figure, the two branches on the right are responsible for calculating the attention weights for the channel dimension C and the spatial dimensions H and W, respectively. The branch on the left is responsible for calculating the relationship between the spatial dimensions H and W. Finally, the simple average of the three weights is taken as the final output feature. CTAM is located after the skip connection and upsampling fusion feature in the U-Net++ network.

[0041] The above SSPCAB (such as Figure 4 As shown in Figure 2, the masked convolution consists of four sub-convolutions K. i (i: 1 to 4), which process the upper left, upper right, lower left and lower right areas of the focus area M respectively, and their weighted results are the value of M. The subconvolution size is represented by K', and d is the expansion domain, which represents the distance from the subconvolution kernel to the receptive field center M. In this embodiment, M, K' and d are all set to 1, then the size of the mask convolution receptive field (K M is represented by:

[0042] K M =2K'+2d+M.

[0043] ECAnet is a lightweight channel attention module, in which global average pooling aggregates input features. ECAnet generates the weights of each channel through a one-dimensional convolution of size K, where K is adaptively determined by the mapping relationship with the channel dimension C through the following function:

[0044]

[0045] Among them, |t oodTo represent the nearest odd integer to t, set γ and b to 2 and 1 respectively.

[0046] The feature map after mask convolution is then weighted by the channel dimension through ECANet to obtain the final feature map. SSPCAB is placed before the last convolution layer at the output of the U-Net++ network.

[0047] Step 3: Use a weakly supervised training method to introduce the defect samples in the defect detection dataset into the training process of the U-Net++ network defect detection model. The training of the defect detection model is completed together with the synthetic defect samples. The specific steps include:

[0048] (1) Extracting the defect mask M from Perlin noise a , using the defect detection dataset as the defect data source E d ;

[0049] (2) M a With E d Perform element-wise multiplication ⊙, by M a ⊙E d Obtain intermediate product I1;

[0050] (3) M a Negate Compared with the normal sample N in the defect detection dataset d Perform element-wise multiplication ⊙, by Obtain intermediate product I3;

[0051] (4) By M a ⊙N d The intermediate product I2 is obtained. I2 is only a The pixel value is 1, and N is displayed. d Texture data;

[0052] (5) Perform element-wise addition on I1, I2, and I3 Depend on Get the synthetic defect sample S d ,like Figure 5 As shown;

[0053] (6) S d and the defect sample R in the defect detection dataset d Splicing to form training data T d , the splicing method is to add S d Randomly add R d One or more defective samples;

[0054] (7) Select Focal Loss, Focal Tversky Loss and L1 Loss as loss functions, and perform the weighted regression on each prediction output of the U-Net++ network defect detection model. (Total 4), and label it with the defect label Using deep supervision, M a With R d The set of true value maps, then for each loss function:

[0055]

[0056]

[0057] (8) Calculate the target loss function Loss = FLoss + (FTLoss / λ1) + (L1Loss / λ2), where λ1 and λ2 are weight coefficients, λ1 is set to the gradient truncation of FTLoss / FLoss, and λ2 is set to the gradient truncation of L1Loss / FLoss, so that λ1 and λ2 are constants;

[0058] (9) Set the learning rate of the U-Net++ network defect detection model to 0.001, the training epoch to 2000, the learning rate adjustment strategy parameter to an exponential decay of 0.992, and the optimizer to select ADAM. d As input, As output, the training of the defect detection model is completed.

[0059] Step 4: Use the trained U-Net++ network defect detection model to perform CT image defect detection on composite components. Input the image of the component to be detected and output the defect detection result image of the image. If there is a defect, the location of the defect and its shape will be displayed in the result image, as shown in the following example: Figure 6 shown.

[0060] The above detailed description of the specific embodiments of the present invention serves only as examples. The present invention is not equivalent to the specific embodiments described above. Furthermore, the present invention is capable of detecting both surface and internal defects of components, including both surface and internal CT images. For those skilled in the art, any equivalent modifications and substitutions to the present invention are also within the scope of the present invention. Therefore, equivalent changes and modifications made without departing from the spirit and scope of the present invention are intended to be encompassed within the scope of the present invention.

Claims

1. A weakly supervised defect detection method for component images, characterized in that The steps include: Step 1: Based on the component defect detection task, a defect detection dataset is constructed with the same image type and resolution, and similar object characteristics and image quality. Normal samples are the majority, and defective samples account for only a small number. Step 2: Build a U-Net++ network defect detection model containing a triple convolutional attention module and a self-supervised convolutional attention module: including 15 convolution blocks, 4 downsampling (MaxPool2d), 10 deconvolution operations (ConvTranspose2d), and 20 jump connections; except for the convolution blocks at the output and input ends, the structure of each convolution block is a triple convolutional attention module CTAM + 2 convolution layers; the convolution block at the input end is 2 convolution layers; the convolution blocks at the four output ends are CTAM + 2 convolution layers + self-supervised convolutional attention module SSPCAB + 1 convolution layer, where the structure of each convolution layer is Conv2d + BatchNorm2d + ReLu, and the convolution kernel size is 3×3; Step 3: Use a weakly supervised training method to introduce the defect samples in the defect detection dataset into the training process of the U-Net++ network defect detection model. Together with the synthetic defect samples, the defect detection model is trained. The specific steps include: (1) Extracting the defect mask M from Perlin noise a , using the defect detection dataset as the defect data source E d ; (2) M a With E d Perform element-wise multiplication ⊙, by M a ⊙E d Obtain intermediate product I1; (3) M a Negate Compared with the normal sample N in the defect detection dataset d Perform element-wise multiplication ⊙, by Obtain intermediate product I3; (4) By M a ⊙N d The intermediate product I2 is obtained. I2 is only a The pixel value is 1, and N is displayed. d Texture data; (5) Perform element-wise addition on I1, I2, and I3 Depend on Get the synthetic defect sample S d ; (6) S d and the defect sample R in the defect detection dataset d Splicing to form training data T d , the splicing method is to add S d Randomly add R d One or more defective samples; (7) Select Focal Loss, Focal Tversky Loss and L1 Loss as loss functions, and perform the weighted regression on each prediction output of the U-Net++ network defect detection model. (Total 4), and label it with the defect label Using deep supervision, M a With R d The set of true value maps, then for each loss function: (8) Calculate the target loss function Loss = FLoss + (FTLoss / λ1) + (L1Loss / λ2), where λ1 and λ2 are weight coefficients, λ1 is set to the gradient truncation of FTLoss / FLoss, and λ2 is set to the gradient truncation of L1Loss / FLoss, so that λ1 and λ2 are constants; (9) Set the relevant parameters of the U-Net++ network defect detection model to the T d As input, For output, the training of the defect detection model is completed; Step 4: Use the trained U-Net++ network defect detection model to perform component defect detection. Input the component image to be inspected and output the defect detection result image of the image. If a defect exists, the location of the defect and its shape will be displayed in the result image.

2. The weakly supervised defect detection method for component images according to claim 1, characterized in that: In step 1, the image types in the component defect detection task include component surface images and component CT images. The image type of one detection task can only be one of the two.

3. The weakly supervised defect detection method for component images according to claim 1, characterized in that: In step 2, the triple convolutional attention module (CTAM) has three branches, each of which is essentially a spatial attention module consisting of a pooling layer + a convolution layer. The pooling layer includes maximum pooling and average pooling, and the convolution layer is Conv2d + BatchNorm2d + Sigmoid, with a convolution kernel size of 7 × 7. CTAM is located after the skip connection and upsampling fusion features in the U-Net++ network.

4. The weakly supervised defect detection method for component images according to claim 1, characterized in that: In step 2, the self-supervised convolutional attention module SSPCAB includes a masked convolution and a channel attention module (ECAnet); the masked convolution consists of four sub-convolutions, which process the upper left, upper right, lower left, and lower right areas of the target feature respectively, and their weighted results are the output values ​​of the target bit features; the feature map after the masked convolution is then weighted by the channel dimension by ECANet to obtain the final feature map; SSPCAB is located before the last convolution layer at the output end of the U-Net++ network.

Images