A scratch detection method and system based on lightweight convolutional neural network

Abstract

The present invention proposes a scratch detection method and system based on a lightweight convolutional neural network. The method is as follows: collect samples and preprocess the samples; build a lightweight network model, wherein the lightweight network includes a backbone network composed of 2 to 4 stages for scratch feature recognition, the output of each stage is respectively input into a prediction module corresponding to each stage for preliminary confidence prediction, and the output of each prediction module is connected to a fusion layer for fusing a pair of preliminary predicted confidences; and scratch detection is performed using the preprocessed samples in the lightweight network model. This method removes the last stage of the VGG network, reduces the number of network parameters, and improves the detection speed.

Description

Technical Field

[0001] The present invention relates to the field of edge detection technology, and in particular to a scratch detection method and system based on a lightweight convolutional neural network. Background Art

[0002] The task of scratch defect detection on workpieces is based on image edge detection technology. Image edge detection is one of the most critical technical issues in the field of image processing and computer vision, and has strong practical value. Edge detection methods can be roughly divided into two categories: traditional methods and methods based on deep learning. Traditional methods use manual feature setting to extract edges, which has poor effect and poor robustness. Convolutional neural networks (CNNs) are favored by various fields due to their powerful feature extraction capabilities, and the rapid update of CNNs technology has further attracted more and more researchers to join in. Although CNNs-based methods have made great progress in edge detection tasks, there are still problems in the field of scratch defect detection that focuses on practicality and real-time performance. For example, most CNNs-based edge detection methods are trained using public natural image datasets with sufficient data volume, while scratch detection may have problems with limited data acquisition and insufficient sample diversity. The more serious problem is that the relevant research of researchers in the field focuses more on accuracy and tends to spend more parameters and reasoning time in exchange for a small amount of accuracy improvement. Therefore, the network is becoming more and more complex, the number of parameters is increasing, and the slow reasoning speed makes it difficult to deploy in actual scratch detection applications. Summary of the invention

[0003] In order to overcome the defects existing in the above-mentioned prior art, the object of the present invention is to provide a scratch detection method and system based on a lightweight convolutional neural network.

[0004] In order to achieve the above-mentioned object of the present invention, the present invention provides a scratch detection method based on a lightweight convolutional neural network, comprising the following steps:

[0005] Collect samples and pre-treat them;

[0006] Build a lightweight network model, which includes a backbone network composed of 2-4 stages for scratch feature recognition. The output of each stage is input into the prediction module corresponding to each stage for preliminary confidence prediction. The output of each prediction module is connected to a fusion layer for fusing a pair of preliminary predicted confidences.

[0007] The preprocessed samples are used to perform scratch detection in the lightweight network model.

[0008] Most of the current edge detection networks use the VGG network as the backbone network. The VGG network consists of 5 stages. This method reduces the number of network layers and speeds up the process.

[0009] The preferred solution of the scratch detection method is as follows: there are 4 stages, the first two stages are respectively composed of a dilated convolution layer and a ShuffleNet Bottleneck module layer; the last two stages are respectively composed of a dilated convolution layer and two ShuffleNet Bottleneck module layers; every two stages are connected by a downsampling operation, and the feature maps output by each stage are respectively added through the convolution compression channel and then output.

[0010] This method removes the last stage of the VGG network, reduces the number of network parameters, and improves the detection speed. At the same time, it uses dilated convolution instead of normal convolution to increase the receptive field size that is reduced by reducing the number of network layers, thereby improving the effect. The ShuffleNet Bottleneck module is used to replace some convolution modules to improve the detection speed.

[0011] The preferred solution of the scratch detection method is: after the feature output of the stage, the upsampling operation is used to enlarge the resolution of the features output by the high-level stage, and then combined with the features output by the low-level stage by channel, and the combined features are used for preliminary prediction of confidence. This can provide additional information to the underlying network and make the detection results more accurate.

[0012] The preferred solution of the scratch detection method is: when performing scratch detection, in the training phase, the output of the fusion layer is used as the prediction result, and in the test phase, the prediction module corresponding to the backbone network and the highest stage is used for prediction. The fusion layer can improve the accuracy to a certain extent, but it also increases the amount of calculation and reasoning time. Therefore, it is used in the training phase and not used in the test phase, that is, when actually detecting scratches. The fusion layer is discarded, thereby avoiding a large number of repeated operations, reducing the number of network parameters, and speeding up the network detection speed.

[0013] The preferred solution of the scratch detection method is: when the outputs of each prediction module are fused during the training phase, the output of the high-level prediction module is detached and added to the output of the low-level prediction module. This is equivalent to using different supervision information for different stages, so it can better guide the learning of different levels of the network and allow different stages of the network to automatically learn features of corresponding scales.

[0014] The preferred solution of the scratch detection method is: after the scratch is detected, the detection results are processed into connected domains, the minimum circumscribed rectangle of each connected domain is calculated, and the connected domain is retained or not according to the set length and width thresholds of the rectangle. This processing can effectively filter out erroneous results caused by network sensitivity.

[0015] The present invention also proposes a scratch detection system, which includes a processor and a memory, wherein the processor and the memory are communicatively connected, and the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute corresponding operations of the above-mentioned scratch detection method.

[0016] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0018] Figure 1 It is a schematic diagram of the scratch detection process;

[0019] Figure 2 It is a schematic diagram of the network structure of the present invention;

[0020] Figure 3 It is a schematic diagram of scratches;

[0021] Figure 4 It is a scratch mark diagram;

[0022] Figure 5 It is a schematic diagram of network training results;

[0023] Figure 6 Schematic diagram of test results. DETAILED DESCRIPTION

[0024] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0025] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0026] like Figure 1 As shown, the present invention provides an embodiment of a scratch detection method based on a lightweight convolutional neural network, which is as follows:

[0027] 1. Sample collection and pretreatment:

[0028] Use the image acquisition device to collect 10 images of different workpieces at different exposures for training, and 5 images for testing. Figure 3 The figure shows a workpiece with scratches. Figure 4 As shown, the scratches in the image are annotated using the annotation tool. The original image size is 5472x3648. Since the image is too large, the inference speed will be greatly reduced. Therefore, all images are scaled to 2736x1824 without losing accuracy. The scaled image is still too large to be used for graphics card training, so the training images and annotations are cropped from the scaled image to 456x456 size, and the pixel values ​​of the image are normalized to [0,1]. After cropping, there are a total of 240 training data and 120 test data. In order to enhance the generalization of the network, tools are used to create pseudo data, and the imgaug library is used for data expansion, and the images are rotated, translated, and distorted. The final data used for training is 1020 and the test data is 120.

[0029] 2. Network Construction

[0030] In this embodiment, the pytorch deep learning library is used to build a CNN network. The network structure is as follows: Figure 2As shown in the figure, dil_block represents the dilated convolution module, b_block represents the ShuffleNet Bottleneck module, 1x1_conv represents the convolution with a kernel size of one by one, which is used to compress channels and fuse information, up represents the upsampling module, which is used to improve the resolution of feature maps, and p_block represents the prediction module, which consists of a 1x1_conv and an upsampling module, and is used to predict and output confidence maps of a specific scale. The backbone network consists of 4 stages, and the output channels of each stage are 64, 128, 256, and 512 respectively. The first two stages consist of two module layers: a dilated convolution with a dilation rate of 2 and a ShuffleNet Bottleneck module with a grouping of 2; the last two stages consist of three modules: a dilated convolution with a dilation rate of 2, followed by two ShuffleNet Bottleneck modules with a grouping of 2. Every two stages are connected by a downsampling operation, which is maxpool, and the convolution layers within the stage are connected by a batch normal layer and a relu activation layer. All feature maps output by each stage are compressed through a 1x1 convolution channel, and then added to obtain multi-scale information to enhance edge information. The added features are used as the output of each stage. The feature outputs of each stage are sent to the prediction modules corresponding to each stage, and the prediction modules output the confidence of the scratches. In order to supplement the information of the underlying network, the features output by the high-level stage are enlarged to twice the resolution by upsampling operation, and then combined with the features of the low-level stage by channel. The combined features are then used for the preliminary prediction confidence in the training phase or the prediction in the test phase. In addition, in order to better guide the learning of different levels of the network, the cascade structure is used to train the network when the outputs of each prediction module are fused in the training phase. This structure detaches the output of the high-level prediction module and adds it to the output of the low-level prediction module, which is equivalent to using different supervision information for different stages, so that different stages of the network can automatically learn the features of the corresponding scales. The backbone network has a total of 4 stages, so 4 confidence maps will be generated, and finally these multiple results will be fused through a 1x1 convolution layer. Therefore, the model will produce 5 results, and the last fusion layer is omitted in the network structure diagram. The detach involved here refers to: Pytorch's detach operation, which stops the tensor variable from calculating the gradient, that is, the variable can no longer be back-propagated and is not used to update the network parameters.

[0031] 3. Model Training

[0032] The processed data is used for training, and the SGD optimizer is used to optimize the model. The batch size is set to 4 and the initial learning rate is 2×10 -4 The loss uses cross entropy loss, and the training is iterated for a total of 500 epochs. The training results are as follows Figure 5 As shown. During the training process of the model, each output confidence map of the model corresponds to a loss, a total of 5 losses, which are weighted summed and back-propagated to update the network parameters. The training and testing of the present invention are based on the Linux operating system and use a 1080 graphics card.

[0033] 4. Model Testing

[0034] Figure 1 This is a flowchart of scratch defect detection. Different from conventional methods, the present invention removes multiple stage prediction modules during the test phase and only uses the backbone network and the highest-level stage prediction modules for inference testing to speed up inference. The test uses 120 test data sets to detect scratches and then post-process the test results, i.e., connective domain processing, to calculate the minimum circumscribed rectangle of each connective domain. The length and width thresholds of the set rectangle are used to determine whether the connective domain should be retained. The results are shown in the figure below. Figure 6 Compared with the RCF and BDCN methods, the reasoning frame rate of the present invention reaches 64 frames per second, which is 32 frames per second and 44 frames per second higher than the two compared methods respectively.

[0035] V. Model Evaluation

[0036] As shown in Table 1, the proposed lightweight scratch detection method achieves better reasoning speed and accuracy than RCF. Compared with the BDCN method, it loses some accuracy, but the reasoning speed is more than 3 times that of BDCN. The test results prove the effectiveness of the method and meet the requirements of the scratch detection task.

[0037] Table 1 Test results

[0038] method DSC (%) Speed ​​(frames / second) RCF 33.7 32 BDCN 39.5 20 This method 37.8 64

[0039] The present invention also provides an embodiment of a scratch detection system, which includes a processor and a memory, wherein the processor and the memory are communicatively connected, and the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the corresponding operation of the above-mentioned scratch detection method.

[0040] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0041] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A scratch detection method based on a lightweight convolutional neural network, characterized in that: The following steps are involved: Collect image samples and preprocess the samples; A lightweight network model is constructed. The lightweight network includes a backbone network composed of 4 stages for scratch feature recognition, wherein the first two stages are composed of a dilated convolution layer and a ShuffleNet Bottleneck module layer respectively; the last two stages are composed of a dilated convolution layer and two ShuffleNet Bottleneck module layers respectively; every two stages are connected by a downsampling operation, and the feature maps output by each module layer of each stage are added through the convolution compression channel and then output; the output of each stage is input into the prediction module corresponding to each stage for preliminary confidence prediction, the prediction module is composed of a 1x1_conv and an upsampling module, and the output of each prediction module is connected to a fusion layer for fusing a pair of preliminary predicted confidences; The preprocessed samples are used to perform scratch detection in the lightweight network model.

2. The scratch detection method based on lightweight convolutional neural network according to claim 1, characterized in that: The expansion rate of the hole convolution layer is 2, the grouping of the ShuffleNet Bottleneck module is 2; and the convolution kernel size of the convolution compression channel is 1x1.

3. The scratch detection method based on lightweight convolutional neural network according to claim 1, characterized in that: After the features of the high-level stage are output, the upsampling operation is used to enlarge the resolution of the features output by the high-level stage, and then combined with the features output by the low-level stage by channel. The combined features are then used to preliminarily predict the confidence.

4. The scratch detection method based on lightweight convolutional neural network according to claim 1, characterized in that: When performing scratch detection, in the training phase, the output of the fusion layer is used as the prediction result. In the testing phase, the prediction module corresponding to the backbone network and the highest stage is used for prediction.

5. The scratch detection method based on lightweight convolutional neural network according to claim 1 or 4, characterized in that: When the outputs of each prediction module are fused during the training phase, the output of the high-level prediction module is detached and added to the output of the low-level prediction module.

6. The scratch detection method based on lightweight convolutional neural network according to claim 1, characterized in that: The convolutional layers within the stage are connected through batch normal layers and relu activation layers.

7. The scratch detection method based on lightweight convolutional neural network according to claim 1, characterized in that: After the scratches are detected, the detection results are processed into connected domains, and the minimum circumscribed rectangle of each connected domain is calculated. The length and width thresholds of the set rectangle are used to determine whether the connected domain should be retained.

8. The scratch detection method based on lightweight convolutional neural network according to claim 1, characterized in that: After collecting the samples, the scratches in the samples are marked, the samples are cropped, and the pixel values ​​of the samples are normalized to [0,1].

9. A scratch detection system, characterized in that: It comprises a processor and a memory, the processor and the memory are communicatively connected, the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the corresponding operation of the scratch detection method according to any one of claims 1-8.

Images