A method for detecting surface defects of industrial parts based on deep learning

By adopting a deep learning model with multi-level feature fusion in the detection of surface defects of industrial parts, combined with VGG16 backbone network and multi-scale feature fusion and attention mechanism module, the problems of low accuracy of traditional methods and large sample data requirements are solved, and high The detection effect of accuracy and strong generalization ability.

CN114187272BActive Publication Date: 2024-11-26ZHUHAI HUAXING INTELLIGENT MFG TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111520875.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-13
Publication Date
2024-11-26
Estimated Expiration
2041-12-13

AI Technical Summary

Technical Problem

The traditional industrial parts surface defect detection methods have low accuracy, require a large amount of sample data, and have poor generalization capabilities under small sample conditions, making it difficult to achieve high accuracy detection.

Method used

A deep learning-based method is adopted to establish a deep learning model with multi-level feature fusion, and a VGG16 backbone network is used to combine multi-scale feature fusion and attention mechanism modules to extract multi-scale features and fuse different levels of features to conduct surface defect detection of industrial parts.

Benefits of technology

It improves detection accuracy and generalization capabilities, can achieve high accuracy under small sample data sets, reduces product defect rate, and improves the overall quality of industrial parts.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114187272B_ABST
    Figure CN114187272B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for detecting surface defects of industrial parts based on deep learning. First, the VGG16 network is used as the backbone network to aggregate local features; then, the multi-scale feature fusion and attention mechanism modules are used to extract multi-scale features for superposition, and then the attention mechanism is used to obtain more key detail information, and then the different-level features of different depths of the network are fused; finally, the feature map is output for identification to distinguish the category of surface defects, and then the causes of various defects can be analyzed, the defective rate of products can be reduced, and the overall quality can be improved. Compared with the existing detection methods, this method has higher precision and strong generalization ability, and can also achieve a higher accuracy rate under small sample data sets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of surface defect detection, and in particular to a method for detecting surface defects of industrial parts based on deep learning. Background Art

[0002] In many industrial fields such as the steel industry, textile industry, semiconductor industry, automobile industry, and intelligent manufacturing, product surface defect detection has become a key link in industrial product quality control. However, traditional defect detection methods are easily affected by noise and scale, the accuracy of the algorithm cannot be guaranteed, and the robustness and effectiveness of defect detection are poor. In addition, in engineering applications, there is still a widespread problem of difficulty in collecting defect sample data, and the industrial data sets may be captured under different conditions, resulting in poor generalization ability of the network model. Collecting data sets is also an expensive and time-consuming process, so it is urgent to solve the problem of how to improve the accuracy of surface defect detection under small sample conditions. Summary of the invention

[0003] The present invention aims to solve the problem that traditional defect detection methods have low accuracy and require a large number of samples, and provides an industrial parts surface defect detection method based on deep learning.

[0004] To solve the above problems, the present invention is achieved through the following technical solutions:

[0005] A method for detecting surface defects of industrial parts based on deep learning comprises the following steps:

[0006] Step S1, collecting images of sample industrial parts, and preprocessing the collected images to obtain sample surface defect images;

[0007] Step S2, labeling the defect types of the sample surface defect images, and constructing a training set and a test set;

[0008] Step S3, establishing a deep learning model based on multi-level feature fusion; the deep learning model based on multi-level feature fusion is mainly composed of a backbone network, two multi-scale feature fusion and attention mechanism modules and a fully connected layer; wherein the input of the backbone network forms the input of the deep learning model based on multi-level feature fusion, the output of the backbone network is connected to the input of the first multi-scale feature fusion and attention mechanism module, the output of the first multi-scale feature fusion and attention mechanism module is connected to the input of the second multi-scale feature fusion and attention mechanism module, the output of the second multi-scale feature fusion and attention mechanism module is connected to the input of the fully connected layer, and the output of the fully connected layer forms the output of the deep learning model based on multi-level feature fusion;

[0009] Step S4, inputting the training set of step S2 into the deep learning model based on multi-level feature fusion constructed in step S3 for training, so as to obtain a trained deep learning model based on multi-level feature fusion;

[0010] Step S5, inputting the test set of step S2 into the surface defect detection model trained in step S4 for testing, and obtaining a final deep learning model based on multi-level feature fusion;

[0011] Step S6, collecting images of the industrial parts to be tested, and preprocessing the collected images to obtain surface defect images to be tested;

[0012] Step S7, sending the surface defect image to be tested in step S6 to the final deep learning model based on multi-level feature fusion in step S5 to detect the surface defects of industrial parts products.

[0013] In the above deep learning model based on multi-level feature fusion, the backbone network is the convolutional neural network VGG16.

[0014] In the above-mentioned deep learning model based on multi-level feature fusion, each multi-scale feature fusion and attention mechanism module is composed of a multi-scale feature fusion unit, a convolutional attention unit and a multi-level feature fusion unit; the input of the multi-scale feature fusion unit and one input of the multi-level feature fusion unit together form the input of the multi-scale feature fusion and attention mechanism module, the output of the multi-scale feature fusion unit is connected to the input of the convolutional attention unit and another input of the multi-level feature fusion unit, the output of the convolutional attention unit is connected to another input of the multi-level feature fusion unit, and the output of the multi-level feature fusion unit forms the output of the multi-scale feature fusion and attention mechanism module.

[0015] The above-mentioned multi-scale feature fusion unit consists of 4 branches and a concat connection layer; the first branch includes a 1×1 convolution layer; the second branch includes a 1×1 convolution layer and a 3×3 convolution layer connected in series in sequence; the third branch includes a 1×1 convolution layer and a 5×5 convolution layer connected in series in sequence; a 3×3 pooling layer and a 1×1 convolution layer connected in series in sequence; the inputs of the 4 branches together form the input of the multi-scale feature fusion unit, and the outputs of the 4-branch multi-scale feature fusion units are simultaneously connected to the input of the concat connection layer, and the output of the concat connection layer forms the output of the multi-scale feature fusion unit.

[0016] The above-mentioned convolutional attention unit is composed of a channel attention model, a spatial attention model, and two element-wise multiplication layers; the input of the channel attention model, one input of the first element-wise multiplication layer, and one input of the second element-wise multiplication layer together form the input of the convolutional attention unit, the output of the channel attention model is connected to another input of the first element-wise multiplication layer, the output of the first element-wise multiplication layer is respectively connected to the input of the channel attention model and another input of the second element-wise multiplication layer, the output of the spatial attention model is connected to another input of the second element-wise multiplication layer, and the output of the second element-wise multiplication layer forms the output of the convolutional attention unit.

[0017] Compared with the prior art, the present invention can detect the surface defects of industrial parts. First, the VGG16 network is used as the backbone network to aggregate local features; then the multi-scale feature fusion and attention mechanism modules are used to extract multi-scale features for superposition, and then the attention mechanism is used to obtain more critical detail information, and then the different levels of features at different depths of the network are fused; finally, the feature map is output for identification to distinguish the category of surface defects, and then the causes of various defects can be analyzed, the defective rate of products can be reduced, and the overall quality can be improved. Compared with the existing detection methods, this method has higher precision and strong generalization ability, and can also achieve a higher accuracy rate under small sample data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a block diagram of the principle of a deep learning model based on multi-level feature fusion.

[0019] Figure 2 The schematic diagram of the multi-scale feature fusion and attention mechanism module. DETAILED DESCRIPTION

[0020] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific examples.

[0021] A method for detecting surface defects of industrial parts based on deep learning comprises the following steps:

[0022] Step S1, collect images of sample industrial parts, and pre-process the collected images to obtain sample surface defect images.

[0023] The image preprocessing includes operations such as translation, inversion, brightness change, affine transformation, and noise enhancement to achieve data enhancement.

[0024] Step S2, labeling the defect types of the sample surface defect images, and constructing a training set and a test set.

[0025] The surface defect images of the samples are classified and labeled, and divided into training sets and test sets. The training set is used to train the model, and the test set is used to evaluate the performance of the model.

[0026] Step S3, establishing a deep learning model MSFA-Net based on multi-level feature fusion as a surface defect detection model.

[0027] The deep learning model MSFA-Net based on multi-level feature fusion established by the present invention is as follows Figure 1 As shown, it mainly consists of three parts: a convolutional neural network VGG16 as a local feature extractor of the backbone network, two multi-scale feature fusion and attention mechanism modules (MSFA) combined to extract features of different scales and obtain key detail information, and a fully connected layer to classify defects. The input of the backbone network forms the input of the deep learning model based on multi-level feature fusion, the output of the backbone network is connected to the input of the first multi-scale feature fusion and attention mechanism module, the output of the first multi-scale feature fusion and attention mechanism module is connected to the input of the second multi-scale feature fusion and attention mechanism module, the output of the second multi-scale feature fusion and attention mechanism module is connected to the input of the fully connected layer, and the output of the fully connected layer forms the output of the deep learning model based on multi-level feature fusion.

[0028] 1) Backbone network

[0029] The convolutional neural network VGG16 replaces the original larger convolution kernel by repeatedly stacking 3×3 small convolution kernels, which ensures the same receptive field, improves the network depth and reduces parameters. It improves the effect of the neural network to a certain extent, shows excellent performance in many local feature extraction tasks, and has strong transfer learning capabilities. Therefore, the pre-trained model of VGG16 is used as the backbone network.

[0030] 2) Multi-scale feature fusion and attention mechanism module (MSFA)

[0031] Each MSFA Figure 2 As shown in the figure, it mainly consists of three parts: a multi-scale feature fusion unit, a convolutional attention unit (CBAM), and a multi-level feature fusion unit.

[0032] 2.1) Multi-scale feature fusion unit

[0033] The multi-scale feature fusion unit consists of 4 branches and a concat connection layer; the first branch includes a 1×1 convolution layer; the second branch includes a 1×1 convolution layer and a 3×3 convolution layer connected in series; the third branch includes a 1×1 convolution layer and a 5×5 convolution layer connected in series; a 3×3 pooling layer and a 1×1 convolution layer connected in series; the inputs of the 4 branches together form the input of the multi-scale feature fusion unit, and the outputs of the 4-branch multi-scale feature fusion units are simultaneously connected to the input of the concat connection layer, and the output of the concat connection layer forms the output of the multi-scale feature fusion unit.

[0034] A multi-scale feature fusion module is constructed using the structural characteristics of the Inception network to make the network wider, reduce the number of parameters, and extract high-dimensional features. As shown in the figure, the first branch uses 1×1 convolution to identify small targets. The second and third branches first use 1×1 convolution to reduce the number of channels and gather useful information, and then perform 3×3 or 5×5 convolution to increase the receptive field. The fourth layer adds a pooling layer before the 1×1 convolution, which can reduce the dimension of the extracted feature information and increase the receptive field of the convolution kernel. Feature information of multiple scales can be obtained through the four branches, and finally the features are superimposed and output.

[0035] 2.2) Convolutional Attention Unit

[0036] The convolutional attention unit consists of a channel attention model, a spatial attention model, and two element-wise multiplication layers; the input of the channel attention model, one input of the first element-wise multiplication layer, and one input of the second element-wise multiplication layer together form the input of the convolutional attention unit, the output of the channel attention model is connected to the other input of the first element-wise multiplication layer, the output of the first element-wise multiplication layer is connected to the input of the channel attention model and the other input of the second element-wise multiplication layer respectively, the output of the spatial attention model is connected to another input of the second element-wise multiplication layer, and the output of the second element-wise multiplication layer forms the output of the convolutional attention unit.

[0037] CBAM is introduced to extract key features and suppress minor features to enhance feature expression. CBAM has two components, the first is the channel attention model and the second is the spatial attention model.

[0038] When extracting channel attention, the channel attention model inputs a multi-scale feature map F, performs maximum pooling and average pooling in the spatial dimension to generate two spatial information descriptions F c avg and F c max, and then send these two vectors to a two-layer shared neural network (MLP) to generate a channel attention feature map. Then use the element-wise operation to merge the feature vectors output by the MLP, and then pass the sigmoid function to generate the final channel attention feature, namely M c ∈R C×1×1 Finally, M c Perform element-wise multiplication with the input feature map F to generate the input feature F′ required by the spatial attention model. The calculation of channel attention is as follows:

[0039] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0040] =σ(W1(W0(F c avg ))+W1(W0(F c max )))

[0041]

[0042] In the formula, F c avg and F c max They represent the average pooling feature and the maximum pooling feature respectively, σ represents the sigmoid function, and W0 and W1 represent the weights of MLP.

[0043] When extracting spatial attention, the spatial attention model uses the feature map F' output by the channel attention model as the input feature map of the module, and divides it into two features F of size H×W×1 through maximum pooling and average pooling s avg and F s max , and then use the concat operation to splice the two together, and then use the 7×7 convolution operation to change the channel to 1 while keeping H and W unchanged, and then pass a sigmoid function to generate the spatial attention feature, i.e., M s ∈R H×W Finally, M s Perform element-wise multiplication with the input features F′ and F to obtain the final generated convolutional attention feature F". The calculation of spatial attention is as follows:

[0044] M s (F′)=σ(f 7×7 ([AvgPool(F′);MaxPool(F′)]))

[0045] =σ(f7×7 ([F s avg ; F s max ]))

[0046]

[0047] In the formula, F s avg and F s max Represent the average pooling feature and the maximum pooling feature respectively, σ represents the sigmoid function, f 7 ×7 Indicates a convolution operation of size 7×7.

[0048] 2.3) Multi-level feature fusion unit

[0049] The feature map F0 generated by the backbone network or the first multi-scale feature fusion and attention mechanism module, the feature map F generated by multi-scale feature fusion, and the feature map F" generated by CBAM are connected along the feature channel and fused into multi-level features, which are defined as follows:

[0050] F1=F0+F+F"

[0051] The multi-level features finally obtained in this way have different levels of features from different depths of the network. They not only have the semantic information of high-level features, which is conducive to defect classification, but also can retain more detailed information of low-level features, which is conducive to defect location and makes the detection results more accurate and reliable.

[0052] After completing multi-level feature fusion, the network obtains rich feature information, which may contain some unimportant repeated information. Therefore, we add a 3×3 maximum pooling layer between the two MSFA modules. On the one hand, it removes the redundant information in the features extracted by the first MFSA module and reduces the number of parameters. On the other hand, it prevents overfitting and improves the generalization ability of the model.

[0053] Fully connected layer: After going through the backbone network and multi-scale feature fusion and attention mechanism modules, rich and important features are successfully extracted. Then, the features are globally averaged pooled and fully connected to achieve the classification purpose. This is done to prevent the spatial information of the image from being destroyed, reduce the number of parameters, and have better anti-overfitting effects.

[0054] Step S4, inputting the training set of step S2 into the deep learning model based on multi-level feature fusion constructed in step S3 for training, so as to obtain a trained deep learning model based on multi-level feature fusion;

[0055] Step S5, inputting the test set of step S2 into the surface defect detection model trained in step S4 for testing, and obtaining a final deep learning model based on multi-level feature fusion;

[0056] Step S6, collecting images of the industrial parts to be tested, and preprocessing the collected images to obtain the surface defect images to be tested.

[0057] Step S7, sending the surface defect image to be tested in step S6 to the final deep learning model based on multi-level feature fusion in step S5 to detect the surface defects of industrial parts products.

[0058] The core of the present invention is to establish a deep multi-scale network, integrate multi-scale features and attention features, and obtain accurate feature information to detect complex and small surface defects on industrial parts. Feature images of different scales can improve the classification performance of different defects, thereby achieving a higher accuracy rate under a small amount of sample training, solving the problem of difficulty in collecting defect samples in industry.

[0059] The performance of the present invention is evaluated using the public NEU-CLS surface defect dataset published by Song et al. The NEU-CLS dataset contains a total of six types of defects, namely scratches (Sc), spots (Pa), pits (Ps), inclusions (in), cracks (Cr), and rolled scale (Rs). There are 300 images of each defect type, for a total of 1,800 images. We conducted three different types of experiments, randomly selecting 20%, 50%, and 90% of the data (60, 150, and 270 images) of each defect type in the NEU-CLS dataset as training samples, and the remaining samples as test datasets. All results were obtained through 10 runs to verify the performance of the proposed model on training datasets of different sizes.

[0060] Table 1 Comparison results of different proportions of training sample numbers

[0061]

[0062] In Table 1, the comparison results of MSFA-Net and the latest classification network are shown in detail with 80%, 50%, and 10% samples for testing, where Acc represents the accuracy. From this table, we can clearly see that the proposed MSFA-Net achieves the highest accuracy of 99.98% when the training sample data is 20%. ShuffleNet has the worst accuracy, only 91.32%, and the accuracy of GoogLeNet (97.64%), EfficientNet (98.47%) and ShuffleNet (91.32%) is not as good as MFSA-Net. When the proportion of training samples increases to 50% and 90%, the accuracy of our model is also higher than that of other models, reaching 100%. This is all due to the design of the fusion strategy of multi-scale feature fusion and attention mechanism, which enables the entire MSFA-Net to achieve good results in classification problems, especially in the problem of small sample learning. With the increase of training samples, the accuracy of each model has been significantly improved, which means that the classification accuracy is highly correlated with the number of training samples.

[0063] The experimental results show that the classification accuracy of the method proposed in this invention is better than other networks in the experiment, and it can accurately classify surface defects in real time. In particular, our method only needs a small number of training samples of specific defects to achieve accurate defect recognition. This model is of great significance to industrial parts production and quality control, and can solve the problem of difficult sample data collection in industry, significantly reducing costs.

[0064] It should be noted that although the embodiments of the present invention described above are illustrative, they are not intended to limit the present invention, and therefore the present invention is not limited to the above specific embodiments. Without departing from the principles of the present invention, any other embodiments obtained by those skilled in the art under the guidance of the present invention are deemed to be within the protection of the present invention.

Claims

1. A method for detecting surface defects of industrial parts based on deep learning, characterized in that: The steps include: Step S1, collecting images of sample industrial parts, and preprocessing the collected images to obtain sample surface defect images; Step S2, labeling the defect types of the sample surface defect images, and constructing a training set and a test set; Step S3, establishing a deep learning model based on multi-level feature fusion; the deep learning model based on multi-level feature fusion is mainly composed of a backbone network, two multi-scale feature fusion and attention mechanism modules and a fully connected layer; wherein the input of the backbone network forms the input of the deep learning model based on multi-level feature fusion, the output of the backbone network is connected to the input of the first multi-scale feature fusion and attention mechanism module, the output of the first multi-scale feature fusion and attention mechanism module is connected to the input of the second multi-scale feature fusion and attention mechanism module, the output of the second multi-scale feature fusion and attention mechanism module is connected to the input of the fully connected layer, and the output of the fully connected layer forms the output of the deep learning model based on multi-level feature fusion; The above backbone network is the convolutional neural network VGG16; Each of the above-mentioned multi-scale feature fusion and attention mechanism modules is composed of a multi-scale feature fusion unit, a convolutional attention unit and a multi-level feature fusion unit; the multi-scale feature fusion unit is composed of 4 branches and a concat connection layer; the first branch includes a 1×1 convolutional layer; the second branch includes a 1×1 convolutional layer and a 3×3 convolutional layer connected in series; the third branch includes a 1×1 convolutional layer and a 5×5 convolutional layer connected in series; a 3×3 pooling layer and a 1×1 convolutional layer connected in series; the inputs of the four branches together form the input of the multi-scale feature fusion unit, and the outputs of the four-branch multi-scale feature fusion units are simultaneously connected to the input of the concat connection layer, and the output of the concat connection layer forms the output of the multi-scale feature fusion unit; the convolutional attention unit is composed of a channel attention model, The convolutional attention unit consists of a spatial attention model and two element-wise multiplication layers; the input of the channel attention model, one input of the first element-wise multiplication layer and one input of the second element-wise multiplication layer together form the input of the convolutional attention unit, the output of the channel attention model is connected to the other input of the first element-wise multiplication layer, the output of the first element-wise multiplication layer is connected to the input of the channel attention model and the other input of the second element-wise multiplication layer respectively, the output of the spatial attention model is connected to another input of the second element-wise multiplication layer, and the output of the second element-wise multiplication layer forms the output of the convolutional attention unit; The input of the multi-scale feature fusion unit and one input of the multi-level feature fusion unit together form the input of the multi-scale feature fusion and attention mechanism module, the output of the multi-scale feature fusion unit is connected to the input of the convolutional attention unit and another input of the multi-level feature fusion unit, the output of the convolutional attention unit is connected to another input of the multi-level feature fusion unit, and the output of the multi-level feature fusion unit forms the output of the multi-scale feature fusion and attention mechanism module; Step S4, inputting the training set of step S2 into the deep learning model based on multi-level feature fusion constructed in step S3 for training, so as to obtain a trained deep learning model based on multi-level feature fusion; Step S5, inputting the test set of step S2 into the surface defect detection model trained in step S4 for testing, and obtaining a final deep learning model based on multi-level feature fusion; Step S6, collecting images of the industrial parts to be tested, and preprocessing the collected images to obtain surface defect images to be tested; Step S7, sending the surface defect image to be tested in step S6 to the final deep learning model based on multi-level feature fusion in step S5 to detect the surface defects of industrial parts products.