A deep learning-based defect detection method

Abstract

The application discloses a kind of defect detection methods based on deep learning, comprising the following steps: S1, obtain multiple images under the same scene;S2, according to the minimum defect detection size setting local search parameter;S3, construct network model Model1;S4, build the network model Model2 same with Model1 structure;S5, division dataset S6, image data in training set is converted into data enhanced image I from original image I by data enhancement mode A ;Image I is input into network model Model1 and is trained, and image I A It is input into network model Model2 and is trained;S7, constantly update network model Model1, network model Model2, confirm preferred model;S8, based on preferred model, construct prediction model;S9, image is input into prediction model, and feature map is obtained;S10, all feature maps are stacked to obtain prediction map;S11, based on the pixel value in mapping prediction map, obtain pixel distribution characteristics and then detect defect region.The method uses unsupervised training, can achieve pixel-level detection effect, robustness is strong, and implementability is high.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection, and more specifically to a defect detection method based on deep learning. Background Technology

[0002] With industrial development, inspecting product appearance has always been a common method to ensure high-quality delivery and maintenance. Due to the small size and high similarity of general defects to the background, manual defect inspection is difficult, inefficient, and inaccurate. Typically, cameras are used to photograph the product's appearance, and image analysis is performed for defect detection. The collected data samples usually consist of a large number of normal images and a very small number of images with defects, and the defect types are diverse, with significant differences in morphology between different types and large size variations even for the same type of defect. Traditional image processing methods detect defects based on their morphological characteristics, resulting in a high false negative rate. Deep learning methods can learn sample features from large amounts of data and are increasingly becoming the mainstream method for defect detection.

[0003] Traditional image processing methods can only detect defects with known features. For example, CN 116485789A provides a defect detection method based on the connected region features of binary images. First, a binary image is obtained by using an absolute threshold. Then, connected region features are set according to the defect features. If the features meet the set threshold, it is considered a defect. However, the image binarization threshold is set to a fixed value, the pixel grayscale is strongly affected by noise, and it can only detect one type of defect, resulting in low practicality.

[0004] Supervised neural network training requires a large amount of labeled data and demands high label quality. Labeling outliers from massive datasets is time-consuming and labor-intensive, and may result in labeling errors or omissions. For example, CN116485720A provides a deep learning-based defect detection method that can identify dozens of different defects by learning image features through deep convolutional networks. However, it requires labeling a large amount of training data, with many label types, resulting in a large workload and limited feasibility. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a defect detection method that employs unsupervised training, eliminating the need for costly data annotation processes; it achieves pixel-level detection results, provides good performance for defects of all sizes, and exhibits strong robustness, high feasibility, and high detection accuracy.

[0006] Therefore, the technical solution of the present invention is as follows:

[0007] A defect detection method based on deep learning includes the following steps:

[0008] S1. Obtain multiple images from the same scene, including both normal and defective images; resize all images to W×H, denoted as dataset D;

[0009] S2. Set the local search area width parameter P and the movement step size parameter S according to the minimum defect detection size, S≤P, and calculate the number of horizontal searches N1 and the number of vertical searches N2.

[0010] N1=(W-P+1) / S

[0011] N2=(H-P+1) / S

[0012] S3. Construct the network model Model1, where branch 1 consists of an input layer, an encoding layer, a feature layer, a feature recombination layer, a patch feature layer, and a patch max pooling layer; branch 2 consists of an input layer, an encoding layer, a feature layer, a feature recombination layer, a patch feature layer, a feature reconstruction layer, a recombined feature layer, a decoding layer, and an output layer; layers with the same name in branches 1 and 2 are shared.

[0013] S4. Construct a network model Model2 with the same structure as network model Model1;

[0014] S5. Divide the images in dataset D into a training set and a validation set;

[0015] S6. Transform the image data in the training set from the original image I into a data-augmented image I using data augmentation. A Input image I into the network model Model1 for training. A The input is fed into the network model Model2 for training, where the loss of the network model Model1 is denoted as Loss1 and the loss of the network model Model2 is denoted as Loss2;

[0016] Loss1=λ1×LossFunc1+λ2×LossFunc2

[0017] Loss2=λ1×LossFunc3+λ2×LossFunc2

[0018] Where LossFunc1 is the reconstruction loss between the input layer and the output layer of network model Model1; the loss coefficient of LossFunc1 is λ1; LossFunc2 is the bias loss between the Patch max pooling layer of network model Model1 and the Patch max pooling layer of network model Model2; the loss coefficient of LossFunc2 is λ2; and LossFunc3 is the reconstruction loss between the input layer and the output layer of network model Model2.

[0019] S7. Continuously update network model Model1 and network model Model2 using training set data, while also updating the model's Loss1 and Loss2 on the validation set; when training ends, select the network model Model1 and network model Model2 corresponding to the minimum value of Loss1 + Loss2 on the validation set as the preferred model.

[0020] S8. Extract branch 1 from network model Model1 as the prediction model;

[0021] S9. Input the images from the non-dataset D into the prediction model and obtain w in the Patch max pooling layer. n n = 1, 2...N, representing the weights of the nth Patch feature layer;

[0022] Extract the channel features f of the Patch feature layer n n = 1, 2 ... N, each f n The size is P×P; w n with f n By multiplying the corresponding features, we obtain a weighted N P×P feature map wf. n ;

[0023] S10, wf all feature maps n The features are arranged sequentially according to the step size S, with N1 features in each row and N2 features in each column. The feature maps are overlaid to obtain the prediction map. The pixel value of each point in the prediction map is calculated according to the following formula:

[0024]

[0025]

[0026] wf n ={Pixel uv u∈(0,P),v∈(0,P)

[0027] Where: Pre represents the prediction map; F is filled with N prediction maps. n Obtained by superposition calculation;

[0028] F n For an image of size W×H, F nxy F represents n The pixel value at position (x, y), Pixel uv wf n The pixel value at position (u,v) is indicated by [].

[0029] S11. Remap the pixel values ​​at each position in the prediction map to 0-255 to obtain the mapped prediction map. Then, detect the defect area by obtaining the pixel distribution characteristics of the mapped prediction map.

[0030] Furthermore, in step S3, the network model Model1 is set according to the following conditions:

[0031] Both the input and output layers are W×H or 3×W×H images;

[0032] The feature layer and the recombined feature layer have the same structure, which is C×W×H, where C is the number of channels;

[0033] The Patch feature layer structure is N×P×P, where N=N1×N2;

[0034] The Patch max pooling layer structure is P×P, and its output is N×1 Patch weights for loss calculation; the output of this layer is N×1.

[0035] The encoding layer, feature reorganization layer, feature reconstruction layer, and decoding layer are all convolutional layers with 3 to 5 layers.

[0036] Furthermore, LossFunc1 uses MAE loss, SSIM loss, MS-SSIM loss, or a combination of MS-SSIM loss and MAE loss.

[0037] Furthermore, LossFunc2 uses either MSE loss or MAE loss.

[0038] Furthermore, the data augmentation methods in step S5 are: flipping, rotating, deforming, adding noise, and / or affine transformation.

[0039] Furthermore, the method for detecting defective regions in step S11 is to binarize the prediction map according to a pre-set alarm threshold T, detect the connected regions of the binary map, and select the connected regions that meet the defect detection requirements as defective regions.

[0040] This method employs unsupervised training, requiring no data annotation, and achieves pixel-level detection results. It learns image features through neural networks to obtain the latent representation of the image, and obtains accurate predictions at small sizes through block operations. It can achieve good results for defects of all sizes, and is robust, feasible, and accurate. Attached Figure Description

[0041] Figure 1 This is a structural block diagram of the network model Model1 used in this invention;

[0042] Figure 2 The original image used in the embodiments;

[0043] Figure 3The example uses a 0-255 prediction map (a heatmap is used for easy demonstration).

[0044] Figure 4 The prediction results are for the example. Detailed Implementation

[0045] The technical solution of the present invention will be described in detail below with reference to the embodiments.

[0046] A defect detection method based on deep learning includes the following steps:

[0047] S1. Acquire multiple images of the wheel tread at the same location, including both normal and defective images; resize all images to W×H (W=H=1024), denoted as dataset D;

[0048] S2. If the minimum detection size of the wheel tread defect on the image is 20 pixels, set the local search area width parameter P=64 and the movement step parameter S=31, and calculate the horizontal search quantity N1 and the vertical search quantity N2.

[0049] N1=(W-P+1) / S=31

[0050] N2=(H-P+1) / S=31

[0051] S3. Construct the network model Model1, where branch 1 consists of an input layer, an encoding layer, a feature layer, a feature recombination layer, a patch feature layer, and a patch max pooling layer; branch 2 consists of an input layer, an encoding layer, a feature layer, a feature recombination layer, a patch feature layer, a feature reconstruction layer, a recombined feature layer, a decoding layer, and an output layer; layers with the same name in branches 1 and 2 are shared.

[0052] In this model, branch one is the branch that completes the prediction of defects, and branch two is the branch used for the network to learn image feature representations; neither can be omitted.

[0053] Wherein: both the input layer and the output layer are 1024×1024 or 3×1024×1024 images;

[0054] Wherein: the feature layer and the recombined feature layer have the same structure, which is C×1024×1024, where C is the number of channels; the value of C can be 128, 256, or 512; here, the value of C is 512.

[0055] Where: The Patch feature layer structure is N×64×64, N=N1×N2=961;

[0056] Among them: the Patch max pooling layer structure is 64×64, and its output is 961×1 Patch weights for loss calculation;

[0057] In this model, the encoding layer, feature reorganization layer, feature reconstruction layer, and decoding layer are all convolutional layers with 3 to 5 layers. It is sufficient to ensure the correct number and size of input and output channels. For example, the encoding layer can consist of a single convolutional layer with a kernel size of 3, padding of 1, stride of 1, 3 input channels, and C output channels. Alternatively, it can consist of a single convolutional layer with a kernel size of 3, padding of 1, stride of 1, 3 input channels, and C output channels, superimposed with two convolutional layers with a kernel size of 3, padding of 1, stride of 1, 3 input channels, and C output channels. The other layer structures can be derived similarly. In this embodiment, the model is built using the following network layers as an example:

[0058] The coding layer consists of 5 convolutional layers of 3×3, with padding=1 and stride=1;

[0059] The feature recombination layer consists of one 3×3 convolutional layer with padding=1 and stride=1 and four 7×7 convolutional layers with padding=3 and stride=2.

[0060] The feature reconstruction layer consists of four 3×3 convolutional layers with padding=1 and stride=1, plus one 3×3 convolutional layer with padding=1 and stride=1; bilinear interpolation is performed before the first four convolutional operations.

[0061] The decoding layer consists of 5 convolutional layers of 3×3, with padding=1 and stride=1;

[0062] S4. Construct a network model Model2 with the same structure as network model Model1;

[0063] S5. Divide the images in dataset D into a training set and a validation set in a 9:1 ratio;

[0064] S6. Transform the image data in the training set from the original image I into the data-augmented image I using data augmentation methods (such as flipping, rotating, deforming, adding noise, affine transformation, etc.; here, flipping, rotating, and affine transformation are chosen, but those skilled in the art can freely combine them for testing purposes). A Input image I into the network model Model1 for training. A The input is fed into the network model Model2 for training, where the loss of the network model Model1 is denoted as Loss1 and the loss of the network model Model2 is denoted as Loss2;

[0065] Loss1=λ1×LossFunc1+λ2×LossFunc2

[0066] Loss2=λ1×LossFunc3+λ2×LossFunc2

[0067] Where LossFunc1 is the reconstruction loss between the input layer and the output layer of network model Model1; the loss coefficient of LossFunc1 is λ1; LossFunc2 is the bias loss between the Patch max pooling layer of network model Model1 and the Patch max pooling layer of network model Model2; the loss coefficient of LossFunc2 is λ2; and LossFunc3 is the reconstruction loss between the input layer and the output layer of network model Model2.

[0068] S7. Continuously update network model Model1 and network model Model2 using training set data, while also updating the model's Loss1 and Loss2 on the validation set; when training ends, select the network model Model1 and network model Model2 corresponding to the minimum value of Loss1 + Loss2 on the validation set as the preferred model.

[0069] S8. Take branch one from the preferred model Model1 as the prediction model;

[0070] S9. Input the images from the non-dataset D into the prediction model and obtain w in the Patch max pooling layer. n n = 1, 2, ..., 961, representing the weights of the nth Patch feature layer;

[0071] Extract the channel features f of the Patch feature layer n n = 1, 2 ... 961, each f n The size is 64×64; w n with f n Corresponding multiplication yields 964 weighted 64×64 feature maps wf n ;

[0072] S10, wf all feature maps n The features are arranged sequentially with a step size of 31, with 31 features in each row and 31 features in each column. The feature maps are overlaid to obtain the prediction map. The pixel value of each point in the prediction map is calculated according to the following formula:

[0073]

[0074]

[0075] wf n ={Pixel uv u∈(0,P),v∈(0,P)

[0076] Where: Pre represents the prediction map; F is filled with N prediction maps. n Obtained by superposition calculation;

[0077] F n For an image of size W×H, F nxy F represents n The pixel value at position (x, y), Pixel uv Indicates wf n The pixel value at position (u,v) is indicated by [].

[0078] S11. Remap the pixel values ​​at each position in the predicted image to 0-255 to obtain a mapped predicted image. Defect areas are then detected by acquiring the pixel distribution characteristics of the mapped predicted image. Specifically, the predicted image can be binarized according to a pre-set alarm threshold T, and connected regions in the binary image can be detected. Connected regions that meet the defect detection requirements are recorded as defect areas. This is just one example; pixel distribution characteristics can also be grayscale histograms, gradient histograms, LBP features, etc.

[0079] When setting up the model, LossFunc1 can use MAE loss, SSIM loss, MS-SSIM loss, or a combination of MS-SSIM loss and MAE loss.

[0080] LossFunc2 uses either MSE loss or MAE loss.

[0081] This method employs unsupervised training, eliminating the need for costly data annotation processes; it achieves pixel-level detection results, demonstrating good performance for defects of all sizes, and exhibits strong robustness, high feasibility, and high detection accuracy.

[0082] The foregoing description of specific exemplary embodiments of the present invention is for illustrative and descriptive purposes. It is not intended to be exhaustive, nor to limit the invention to the precise forms disclosed; obviously, many changes and variations are possible in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to implement and utilize various exemplary embodiments of the invention, as well as their different alternatives and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A defect detection method based on deep learning, characterized in that... Includes the following steps: S1. Obtain multiple images from the same scene, including both normal and defective images; resize all images to W×H, denoted as dataset D; S2. Set the local search area width parameter P and the movement step size parameter S according to the minimum defect detection size, S≤P, and calculate the number of horizontal searches N1 and the number of vertical searches N2. N1=(W-P+1) / S N2=(H-P+1) / S S3. Construct the network model Model1, where branch 1 consists of an input layer, an encoding layer, a feature layer, a feature recombination layer, a patch feature layer, and a patch max pooling layer; branch 2 consists of an input layer, an encoding layer, a feature layer, a feature recombination layer, a patch feature layer, a feature reconstruction layer, a recombined feature layer, a decoding layer, and an output layer; layers with the same name in branches 1 and 2 are shared. S4. Construct a network model Model2 with the same structure as network model Model1; S5. Divide the images in dataset D into a training set and a validation set; S6. Transform the image data in the training set from the original image I into a data-augmented image I using data augmentation. A Input image I into the network model Model1 for training. A The input is fed into the network model Model2 for training, where the loss of the network model Model1 is denoted as Loss1 and the loss of the network model Model2 is denoted as Loss2; Loss1=λ1×LossFunc1+λ2×LossFunc2 Loss2=λ1×LossFunc3+λ2×LossFunc2 Where LossFunc1 is the reconstruction loss between the input layer and the output layer of network model Model1; the loss coefficient of LossFunc1 is λ1; LossFunc2 is the bias loss between the Patch max pooling layer of network model Model1 and the Patch max pooling layer of network model Model2; the loss coefficient of LossFunc2 is λ2; and LossFunc3 is the reconstruction loss between the input layer and the output layer of network model Model2. S7. Continuously update network model Model1 and network model Model2 using training set data, while also updating the model's Loss1 and Loss2 on the validation set; when training ends, select the network model Model1 and network model Model2 corresponding to the minimum value of Loss1 + Loss2 on the validation set as the preferred model. S8. Extract branch 1 from network model Model1 as the prediction model; S9. Input the images from the non-dataset D into the prediction model and obtain w in the Patch max pooling layer. n n = 1, 2...N, representing the weights of the nth Patch feature layer; Extract the channel features f of the Patch feature layer n n = 1, 2 ... N, each f n The size is P×P; w n with f n Corresponding multiplication yields N weighted P×P feature maps wf n ; S10, wf all feature maps n The features are arranged sequentially according to the step size S, with N1 features in each row and N2 features in each column. The feature maps are overlaid to obtain the prediction map. The pixel value of each point in the prediction map is calculated according to the following formula: wf n ={Pixel uv u∈(0,P),v∈(0,P) Where: Pre represents the prediction map; F is filled with N prediction maps. n Obtained by superposition calculation; F n For an image of size W×H, F nxy F represents n The pixel value at position (x, y), Pixel uv Indicates wf n The pixel value at position (u,v) is indicated by []. S11. Remap the pixel values ​​at each position in the prediction map to 0-255 to obtain the mapped prediction map. By obtaining the pixel distribution characteristics of the mapped prediction map, the defect area can be detected.

2. The defect detection method based on deep learning as described in claim 1, characterized in that: Step S3: Configure the network model Model1 according to the following conditions: Both the input and output layers are W×H or 3×W×H images; The feature layer and the recombined feature layer have the same structure, which is C×W×H, where C is the number of channels; The Patch feature layer structure is N×P×P, where N = N1×N2; The Patch max pooling layer structure is P×P, and its output is N×1 Patch weights for loss calculation; the output of this layer is N×1. The encoding layer, feature reorganization layer, feature reconstruction layer, and decoding layer are all convolutional layers with 3 to 5 layers.

3. The defect detection method based on deep learning as described in claim 1, characterized in that: LossFunc1 uses MAE loss, SSIM loss, MS-SSIM loss, or a combination of MS-SSIM loss and MAE loss.

4. The defect detection method based on deep learning as described in claim 1, characterized in that: LossFunc2 uses either MSE loss or MAE loss.

5. The deep learning-based defect detection method as described in claim 1, characterized in that: The data augmentation methods in step S5 are: flipping, rotating, deforming, adding noise, and / or affine transformation.

6. The defect detection method based on deep learning as described in claim 1, characterized in that: Step S11 involves detecting defective regions by binarizing the prediction map according to a pre-set alarm threshold T, detecting connected regions in the binary map, and selecting connected regions that meet the defect detection requirements as defective regions.

Images