Surface defect detection method based on deep learning algorithm

Abstract

A surface defect detection method based on a deep learning algorithm solves the problem of sample imbalance existing in the existing surface defect detection using a deep learning algorithm, and belongs to the field of detection technology in industrial production. The present invention includes: establishing a surface defect classification model based on a deep learning algorithm, and training the surface defect classification model using an initial training sample set D1; sampling various surface defect image samples in D1 to obtain a balanced image sample set D2, and continuing to train the surface defect classification model using D2. During the training process, freeze the parameters of the convolutional layer in the surface defect classification model, and adjust the parameters of the fully connected layer until the surface defect classification model converges; use the surface defect classification model to detect the image samples to be detected, calculate the weakly supervised localization information; and perform threshold segmentation processing, use the binary threshold to generate a position mask of the defect, and complete the surface defect detection.

Description

Technical Field

[0001] The present invention relates to a surface defect detection method based on a deep learning algorithm, and belongs to the technical field of detection in industrial production. Background Art

[0002] In the industrial production process, surface defect detection technology is crucial to product quality control, while manual quality inspection is time-consuming and laborious. In order to improve the level of production automation, machine vision methods are often used for detection. However, traditional machine vision methods are difficult to adapt to complex production environments and have poor robustness. Different solutions need to be designed for different problems. With the development of artificial intelligence technology, deep learning technology has gradually been applied to the field of surface defect detection, showing stronger performance.

[0003] Deep learning technology is based on high-performance neural network frameworks such as convolutional neural networks, attention mechanism models, transformers, etc., and has demonstrated amazing adaptability and vitality in fields such as computer vision, natural language processing, and even materials science and biology.

[0004] However, in actual industrial production environments, existing deep learning algorithms still face the following challenges in practical application: (1) Deep learning technology often requires the collection of a large number of samples, which is a challenge for early and rapid deployment in industrial inspection scenarios; (2) Industrial inspection scenarios often make it easier to obtain normal samples, while the number of defective samples is relatively small, which leads to sample imbalance and limits the performance of deep learning methods; (3) Even if a sufficient number of samples are collected, professionals or even experts in related fields are still needed to annotate the samples. Moreover, the more complex the task (such as segmentation and detection), the higher the degree of annotation required, which undoubtedly requires high time and labor costs.

[0005] In summary, in order to implement surface defect detection technology based on deep learning, it is necessary to overcome key problems such as small sample size, high labeling cost, and unbalanced samples. Summary of the invention

[0006] In view of the problems of small sample size, high labeling cost and unbalanced samples in the existing surface defect detection using deep learning algorithms, the present invention provides a surface defect detection method based on a deep learning algorithm.

[0007] A surface defect detection method based on a deep learning algorithm of the present invention comprises:

[0008] S1. Build a surface defect classification model based on deep learning algorithm and use the initial training sample set D 1 Training a surface defect classification model;

[0009] S2, for the initial training sample set D 1The various surface defect image samples in the image are sampled to obtain a balanced image sample set, and the balanced image sample set is used to continue training the surface defect classification model of S1. During the training process, the parameters of the convolution layer in the surface defect classification model are frozen, and the parameters of the fully connected layer are adjusted until the surface defect classification model converges;

[0010] S3, using the surface defect classification model to detect the image sample to be detected, and calculating the weakly supervised positioning information of the surface defect classification model;

[0011] S4. Perform threshold segmentation processing on the weakly supervised positioning information, use the binary threshold to generate the defect position mask, and complete the surface defect detection.

[0012] The present invention also provides a surface defect detection method based on a deep learning algorithm, comprising:

[0013] S1. Establish a surface defect classification model based on a deep learning algorithm and train the surface defect classification model;

[0014] S2, sampling each type of sample m times to obtain m balanced sample sets, using the m balanced sample sets to train the surface defect classification model of S1 respectively to obtain m surface defect classification models. During the training process, the parameters of the convolution layer in the surface defect classification model are frozen, and the parameters of the fully connected layer are adjusted until the surface defect classification model converges; α 1 Indicates the number of samples of the largest class, α n Indicates the minimum number of samples of one type;

[0015] Use the ensemble method to count the classification results of m surface defect classification models and output the final defect classification category;

[0016] S3, using m surface defect classification models to detect the image samples to be detected, determining the defect classification category, and calculating the weakly supervised positioning information of the surface defect classification model with the highest recall rate among the m surface defect classification models;

[0017] S4. Perform threshold segmentation processing on the weakly supervised positioning information calculated in S3, use the binary threshold to generate a defect position mask, and complete surface defect detection.

[0018] Preferably, in S3, the full gradient method is used to calculate the weakly supervised positioning information S of the surface defect classification model:

[0019]

[0020] Where F represents the surface defect classification model, is the output of the cth channel of the lth layer, is the corresponding bias, L is the total number of convolutional layers, c l is the number of channels of the convolution kernel of the lth layer, x represents the input image sample to be detected, ⊙ represents the XOR operation, and the function φ(·) means:

[0021] φ(·)=upsample(rescale(abs(·)))

[0022] abs(·) refers to the absolute value function, rescale(·) refers to the normalization operation, which normalizes the input to between 0 and 1, and upsample(·) represents the upsampling operation, which is used to interpolate the input to the same dimension as x.

[0023] Preferably, in S4, the method for determining the binarization threshold includes:

[0024] Select several samples from the training sample set, choose multiple binarization thresholds, and then make human visual judgments to determine a suitable value as the binarization threshold for detection.

[0025] The beneficial effect of the present invention is that the present invention can effectively train unbalanced defect samples and greatly improve their recall rate. The present invention reduces the difficulty of data labeling and only needs to provide image-level labels (category labels) to complete the positioning of defect positions. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0029] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0030] The surface defect detection method based on the deep learning algorithm of this embodiment includes:

[0031] Step 1: First stage training: Build a surface defect classification model based on deep learning algorithm and use the initial training sample set D 1Train the surface defect classification model; use common training methods, such as data enhancement, transfer learning, etc. For example, first use data enhancement methods (such as random flipping, random cropping, etc.) to enhance the data set. Then perform transfer learning on the model. Take the training of the VGG16 model as an example, abbreviated as f, transfer the parameters of the VGG16 model pre-trained based on imagenet to the model to be trained, and then perform fine-tuning training on the defect data after data enhancement.

[0032] Step 2: Second stage training: Initial training sample set D 1 We sample various surface defect image samples in the image set D to obtain a balanced image sample set D 2 , for the problem of sample imbalance, for example, for n types of samples, the number of samples in each type is α 1 ,α 2 ,…α n , let α 1 >α 2 >…>α n . For each type of sample, randomly select α m Zhang subsample, α m ≤α n , forming a sample set with balanced samples.

[0033] Using the balanced image sample set D 2 Continue to train the surface defect classification model f in step 1. During the training process, freeze the parameters of the convolutional layer in the surface defect classification model, and fine-tune the parameters of the fully connected layer until the surface defect classification model converges.

[0034] Step 3: Use the surface defect classification model to detect the image sample to be detected, and calculate the weakly supervised positioning information of the surface defect classification model;

[0035] Step 4: Perform threshold segmentation processing on the weakly supervised positioning information, use the binary threshold to generate the defect position mask, and complete the surface defect detection.

[0036] After the second stage of training in step 2, the recall rate of the model will be greatly improved, but some samples will not be fully utilized, such as Figure 1 As shown, in the second stage training of step 2 of this implementation mode, each type of sample can be sampled m times to obtain m balanced sample sets, and the surface defect classification model of step 1 is trained using the m balanced sample sets to obtain m surface defect classification models, which are recorded as f 1 ,f 2 ,…,f m ,During the training process, the parameters of the convolution layer in the surface ,defect classification model are frozen and the parameters of the ,fully connected layer are adjusted until the surface defect classification model converges; α 1 Indicates the number of samples of the largest class, α n Indicates the minimum number of samples of one type;

[0037] Using the ensemble method, you can use voting method, average method, etc. to select the category with the highest number of votes or the category with the highest average value, count the classification results of m surface defect classification models, and output the final defect classification category;

[0038] In step 3, m surface defect classification models are used to detect the image samples to be detected, the defect classification categories are determined, and the weakly supervised positioning information of the surface defect classification model with the highest recall rate among the m surface defect classification models is calculated.

[0039] In step 3 of this embodiment, the weakly supervised positioning information S of the surface defect classification model is calculated using a full-gradient method:

[0040]

[0041] Where F represents the surface defect classification model, is the output of the cth channel of the lth layer, is the corresponding bias, L is the total number of convolutional layers, c l is the number of channels of the convolution kernel of the lth layer, x represents the input image sample to be detected, ⊙ represents the XOR operation, and the function φ(·) means:

[0042] φ(·)=upsample(rescale(abs(·)))

[0043] abs(·) refers to the absolute value function, and rescale(·) refers to the normalization operation, which normalizes the input to between 0 and 1. For example, using upsample(·) represents an upsampling operation, which is used to interpolate the input to the same dimension as x.

[0044] In step 4 of this embodiment, the method for determining the binarization threshold includes:

[0045] Select several samples from the training sample set, select multiple binarization thresholds, such as δ = 0.1, 0.3, 0.5, 0.8, and then use human eye judgment to determine a suitable value as the binarization threshold during detection to generate a defect position mask.

[0046] This embodiment adopts a two-stage fine-tuning method in the process of training the surface defect classification model. The first stage is normal transfer learning and fine-tuning, and the second stage is to fine-tune the fully connected layer again. This training method can effectively train data with imbalanced categories and greatly improve the recall rate of the model. In addition, the full-gradient method is used for weakly supervised positioning of surface defects to reduce the difficulty of labeling and improve the accuracy of defect positioning. Through this positioning scheme, the defect location can be detected only by training the classification model, that is, only by providing the classification label.

[0047] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. Surface defect detection method based on deep learning algorithm, It is characterized in that The method comprises: S1. Establish a surface defect classification model based on a deep learning algorithm and train the surface defect classification model; S2, sampling each type of sample m times to obtain m balanced sample sets, using the m balanced sample sets to train the surface defect classification model of S1 respectively to obtain m surface defect classification models. During the training process, the parameters of the convolution layer in the surface defect classification model are frozen, and the parameters of the fully connected layer are adjusted until the surface defect classification model converges; a 1 Indicates the number of samples of the largest class, α n Indicates the minimum number of samples of one type; Use the ensemble method to count the classification results of m surface defect classification models and output the final defect classification category; S3, using m surface defect classification models to detect the image samples to be detected, determining the defect classification category, and calculating the weakly supervised positioning information of the surface defect classification model with the highest recall rate among the m surface defect classification models; S4, performing threshold segmentation processing on the weakly supervised positioning information calculated in S3, using the binary threshold to generate a defect position mask, and completing surface defect detection; In S2, the method for obtaining a balanced sample set includes: Initial training sample set D 1 There are n types of samples in total, and the number of samples in each type is α 1 ,α 2 ,…α n , α 1 >α 2 >…>α n , then for each type of sample, α is randomly selected m Zhang sub-samples to form a balanced sample set D 2 , α m ≤α n ; In S3, the full gradient method is used to calculate the weakly supervised localization information S of the surface defect classification model: Where F represents the surface defect classification model, is the output of the cth channel of the lth layer, is the corresponding bias, L is the total number of convolutional layers, c l is the number of channels of the convolution kernel of the lth layer, x represents the input image sample to be detected, ⊙ represents the XOR operation, and the function φ(·) means: φ(·)=upsample(rescale(abs(·))) abs(·) refers to the absolute value function, rescale(·) refers to the normalization operation, which normalizes the input to between 0 and 1, and upsample(·) represents the upsampling operation, which is used to interpolate the input to the same dimension as x.

2. The surface defect detection method based on deep learning algorithm according to claim 1, It is characterized in that The method further comprises: The classification results of m surface defect classification models are counted using the voting method or the average method, and the category with the highest number of votes or the highest average value is selected as the defect classification category for the final output.

3. The surface defect detection method based on deep learning algorithm according to claim 1, It is characterized in that In S4, the method for determining the binarization threshold includes: Select several samples from the training sample set, choose multiple binarization thresholds, and then make human visual judgments to determine a suitable value as the binarization threshold for detection.

Images