A Deep Learning-Based Real-Time Surface Defect Detection Method

Abstract

This invention relates to product surface defect detection technology, specifically a real-time surface defect detection method based on deep learning. The method comprises the following steps: Step 1: Creating a product surface defect dataset; Step 2: Constructing a MobileNetv3-CATR backbone feature extraction network; Step 3: Training the MobileNetv3-CATR backbone feature extraction network; Step 4: Inputting the product surface defect image to be detected into the MobileNetv3-CATR backbone feature extraction network for defect feature extraction; Step 5: Inputting the 13×13 scale feature map into a spatial pyramid pooling module; Step 6: Constructing a lightweight weighted bidirectional feature pyramid; Step 7: Inputting the processed 13×13 scale feature map, 26×26 scale feature map, and 52×52 scale feature map into the lightweight weighted bidirectional feature pyramid for defect feature fusion. This invention effectively solves the problem of poor real-time performance in existing product surface defect detection methods and is suitable for product surface defect detection.

Description

Technical Field

[0001] This invention relates to surface defect detection technology for products, specifically a real-time surface defect detection method based on deep learning. Background Technology

[0002] During production and transportation, products inevitably develop surface defects such as chips, spots, cracks, pits, inclusions, and scratches, which affect the product's performance and quality. Therefore, surface defect detection is necessary to ensure product performance and quality.

[0003] With the development of artificial intelligence technology, deep learning-based methods for detecting surface defects in products have become the mainstream approach due to their high detection accuracy. However, due to limitations in their underlying principles, these methods still suffer from slow detection speeds, resulting in poor real-time performance. Therefore, it is necessary to develop a deep learning-based real-time surface defect detection method to address the issue of poor real-time performance in existing methods. Summary of the Invention

[0004] To address the problem of poor real-time performance in existing product surface defect detection methods, this invention provides a real-time surface defect detection method based on deep learning.

[0005] This invention is achieved using the following technical solution:

[0006] A real-time surface defect detection method based on deep learning is implemented through the following steps:

[0007] Step 1: Create a product surface defect dataset; the product surface defect dataset includes a training set, a validation set, and a test set;

[0008] Step 2: Construct the MobileNetv3-CATR backbone feature extraction network;

[0009] Step 3: First, use the training set in the product surface defect dataset to train the MobileNetv3-CATR backbone feature extraction network, and then use the test set in the product surface defect dataset to test the detection accuracy of the MobileNetv3-CATR backbone feature extraction network.

[0010] Step 4: Input the surface defect image of the product to be inspected into the MobileNetv3-CATR backbone feature extraction network to extract defect features, thereby obtaining three feature maps of different scales; the three feature maps of different scales include: a 13×13 scale feature map, a 26×26 scale feature map, and a 52×52 scale feature map.

[0011] Step 5: Input the 13×13 scale feature map into the spatial pyramid pooling module. The spatial pyramid pooling module uses pooling kernels of different sizes to perform various scale processing on the 13×13 scale feature map, thereby obtaining the processed 13×13 scale feature map.

[0012] Step Six: Construct a lightweight weighted bidirectional feature pyramid;

[0013] Step 7: First, input the processed 13×13, 26×26, and 52×52 feature maps into a lightweight weighted bidirectional feature pyramid for defect feature fusion. Then, input the three fused feature maps of different scales into their respective detection heads, thus obtaining three detection heads of different scales. These three detection heads are then processed, resulting in three sets of adjustment parameters for each candidate bounding box. Finally, the coordinates and dimensions of the candidate bounding boxes are adjusted according to the adjustment parameters to generate predicted bounding boxes, thereby determining the information of product surface defects. The three detection heads of different scales include: a 13×13 scale detection head, a 26×26 scale detection head, and a 52×52 scale detection head. The adjustment parameters for each set of candidate bounding boxes include: one confidence parameter, four parameters for adjusting coordinates and dimensions, and multiple category parameters.

[0014] In step one, the product surface defect dataset includes a training set consisting of multiple product surface defect images with defect features, a validation set consisting of multiple product surface defect images with defect features, and a test set consisting of multiple product surface defect images with defect features; the ratio of the number of images in the training set, the number of images in the validation set, and the number of images in the test set is 8:1:1.

[0015] In step one, the defect features include milling chips, spots, cracks, pits, inclusions, and scratches.

[0016] In step two, the construction steps of the MobileNetv3-CATR backbone feature extraction network are as follows:

[0017] First, the CA-Bneck module is constructed using a coordinated attention module and a dynamic activation function;

[0018] Secondly, the Bneck module in the original MobileNetv3 backbone feature extraction network is replaced by the CA-Bneck module, thereby obtaining the reconstructed MobileNetv3 backbone feature extraction network.

[0019] Then, a Transformer module was added to the reconstructed MobileNetv3 backbone feature extraction network, thus obtaining the MobileNetv3-CATR backbone feature extraction network.

[0020] In step six, the construction steps of the lightweight weighted bidirectional feature pyramid are as follows:

[0021] First, delete two feature layers from the weighted bidirectional feature pyramid;

[0022] Secondly, additional edges are added between the original input nodes and output nodes located in the same feature layer, thus obtaining a lightweight weighted bidirectional feature pyramid.

[0023] In step seven, when performing defect feature fusion, the lightweight weighted bidirectional feature pyramid assigns the same weight to each feature map and performs a normalization process during the splicing operation.

[0024] In step seven, the information on the product surface defects includes the type of product surface defects and the location of the product surface defects.

[0025] Compared with existing surface defect detection methods, the deep learning-based real-time surface defect detection method described in this invention, by constructing a MobileNetv3-CATR backbone feature extraction network and a lightweight weighted bidirectional feature pyramid, not only has the advantage of high detection accuracy (experiments show that the average accuracy for detecting six types of surface defects can reach 85%), but also the advantage of fast detection speed (experiments show that the detection speed on a single Tesla T4 can reach 30 FPS), thus possessing the advantage of strong real-time detection.

[0026] This invention effectively solves the problem of poor real-time performance in existing product surface defect detection methods and is applicable to product surface defect detection. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the overall invention.

[0028] Figure 2 This is a schematic diagram of the CA-Bneck module in this invention.

[0029] Figure 3 This is a schematic diagram of the Transformer module in this invention.

[0030] Figure 4 This is a schematic diagram of the lightweight weighted bidirectional feature pyramid in this invention. Detailed Implementation

[0031] A real-time surface defect detection method based on deep learning is implemented through the following steps:

[0032] Step 1: Create a product surface defect dataset; the product surface defect dataset includes a training set, a validation set, and a test set;

[0033] Step 2: Construct the MobileNetv3-CATR backbone feature extraction network;

[0034] Step 3: First, use the training set in the product surface defect dataset to train the MobileNetv3-CATR backbone feature extraction network, and then use the test set in the product surface defect dataset to test the detection accuracy of the MobileNetv3-CATR backbone feature extraction network.

[0035] Step 4: Input the surface defect image of the product to be inspected into the MobileNetv3-CATR backbone feature extraction network to extract defect features, thereby obtaining three feature maps of different scales; the three feature maps of different scales include: a 13×13 scale feature map, a 26×26 scale feature map, and a 52×52 scale feature map.

[0036] Step 5: Input the 13×13 scale feature map into the spatial pyramid pooling module. The spatial pyramid pooling module uses pooling kernels of different sizes to perform various scale processing on the 13×13 scale feature map, thereby obtaining the processed 13×13 scale feature map.

[0037] Step Six: Construct a lightweight weighted bidirectional feature pyramid;

[0038] Step 7: First, input the processed 13×13, 26×26, and 52×52 feature maps into a lightweight weighted bidirectional feature pyramid for defect feature fusion. Then, input the three fused feature maps of different scales into their respective detection heads, thus obtaining three detection heads of different scales. These three detection heads are then processed, resulting in three sets of adjustment parameters for each candidate bounding box. Finally, the coordinates and dimensions of the candidate bounding boxes are adjusted according to the adjustment parameters to generate predicted bounding boxes, thereby determining the information of product surface defects. The three detection heads of different scales include: a 13×13 scale detection head, a 26×26 scale detection head, and a 52×52 scale detection head. The adjustment parameters for each set of candidate bounding boxes include: one confidence parameter, four parameters for adjusting coordinates and dimensions, and multiple category parameters.

[0039] In step one, the product surface defect dataset includes a training set consisting of multiple product surface defect images with defect features, a validation set consisting of multiple product surface defect images with defect features, and a test set consisting of multiple product surface defect images with defect features; the ratio of the number of images in the training set, the number of images in the validation set, and the number of images in the test set is 8:1:1.

[0040] In step one, the defect features include milling chips, spots, cracks, pits, inclusions, and scratches.

[0041] In step two, the construction steps of the MobileNetv3-CATR backbone feature extraction network are as follows:

[0042] First, the CA-Bneck module is constructed using a coordinated attention module and a dynamic activation function;

[0043] Secondly, the Bneck module in the original MobileNetv3 backbone feature extraction network is replaced by the CA-Bneck module, thereby obtaining the reconstructed MobileNetv3 backbone feature extraction network.

[0044] Then, a Transformer module was added to the reconstructed MobileNetv3 backbone feature extraction network, thus obtaining the MobileNetv3-CATR backbone feature extraction network.

[0045] In step six, the construction steps of the lightweight weighted bidirectional feature pyramid are as follows:

[0046] First, delete two feature layers from the weighted bidirectional feature pyramid;

[0047] Secondly, additional edges are added between the original input nodes and output nodes located in the same feature layer, thus obtaining a lightweight weighted bidirectional feature pyramid.

[0048] In step seven, when performing defect feature fusion, the lightweight weighted bidirectional feature pyramid assigns the same weight to each feature map and performs a normalization process during the splicing operation.

[0049] In step seven, the information on the product surface defects includes the type of product surface defects and the location of the product surface defects.

[0050] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of the present invention is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but all such changes and modifications fall within the scope of protection of the present invention.

Claims

1. A deep learning-based surface defect real-time detection method, characterized in that: The method is realized by the following steps: Step one: make a product surface defect data set; the product surface defect data set includes a training set, a validation set, and a test set; Step two: build a MobileNetv3-CATR backbone feature extraction network; Step three: first use the training set in the product surface defect data set to train the MobileNetv3-CATR backbone feature extraction network, and then use the test set in the product surface defect data set to test the detection accuracy of the MobileNetv3-CATR backbone feature extraction network; Step four: input the product surface defect picture to be detected into the MobileNetv3-CATR backbone feature extraction network for defect feature extraction, thereby obtaining three scale different feature maps; the three scale different feature maps include: a 13×13 scale feature map, a 26×26 scale feature map, and a 52×52 scale feature map; Step five: input the 13×13 scale feature map into the spatial pyramid pooling module, and the spatial pyramid pooling module uses different size pooling kernels to perform diversity scale processing on the 13×13 scale feature map, thereby obtaining the processed 13×13 scale feature map; Step six: build a lightweight weighted bidirectional feature pyramid; Step seven: first input the processed 13×13 scale feature map, 26×26 scale feature map, and 52×52 scale feature map into the lightweight weighted bidirectional feature pyramid for defect feature fusion, then input the fused three scale different feature maps into respective detection heads, thereby obtaining three scale different detection heads, then process the three scale different detection heads, each detection head thereby obtains three groups of candidate box adjustment parameters, and finally adjust the coordinates and sizes of the candidate boxes according to the candidate box adjustment parameters, thereby generating a prediction box to determine the information of the product surface defect; the three scale different detection heads include: a 13×13 scale detection head, a 26×26 scale detection head, and a 52×52 scale detection head; each group of candidate box adjustment parameters includes: one confidence parameter, four adjustment coordinate and size parameters, and multiple category parameters; In step two, the MobileNetv3-CATR backbone feature extraction network is constructed as follows: First, use the coordination attention module and the dynamic activation function to construct the CA-Bneck module; Second, replace the Bneck module in the original MobileNetv3 backbone feature extraction network with the CA-Bneck module, thereby obtaining a reorganized MobileNetv3 backbone feature extraction network; Then, add a Transformer module to the reorganized MobileNetv3 backbone feature extraction network, thereby obtaining the MobileNetv3-CATR backbone feature extraction network; In step six, the lightweight weighted bidirectional feature pyramid is constructed as follows: First, delete two feature layers in the weighted bidirectional feature pyramid; Secondly, additional edges are added between the original input nodes and output nodes in the same feature layer, thereby obtaining a lightweight weighted bidirectional feature pyramid; In step seven, when the defect feature fusion is performed, the lightweight weighted bidirectional feature pyramid gives a same weight to each feature map, and a normalization processing is performed when the splicing operation is performed.

2. The real-time surface defect detection method based on deep learning according to claim 1, characterized in that: In step one, the product surface defect data set includes a training set composed of multiple product surface defect pictures with defect features, a verification set composed of multiple product surface defect pictures with defect features, and a test set composed of multiple product surface defect pictures with defect features; the ratio of the number of pictures in the training set, the number of pictures in the verification set, and the number of pictures in the test set is 8:1:

1. 3.The deep learning-based surface defect real-time detection method of claim 2, wherein: In step one, the defect features include rolling scrap, spot, crack, pit, inclusion, and scratch.

4. The real-time surface defect detection method based on deep learning according to claim 1, characterized in that: In step seven, the information of the product surface defect includes the category of the product surface defect and the position of the product surface defect.

Images