A steel coil surface defect detection method based on a convolutional neural network

Abstract

The application discloses a kind of steel coil surface defect detection methods based on convolutional neural network, comprising: taking out the feature of different positions of input feature map;The selected feature is input into the perceiver, and the category is set;The output of perceiver is used as the confidence score of each feature by Softmax function;The confidence score generated previously is weighted with the manually selected weight factor Summation;The weighted result is input into ReLu function to obtain mask value;The mask is multiplied by the input feature map by position to obtain the output feature map of attention layer;The application adopts attention layer structure, and the parameters are less, so that its calculation complexity and spatial complexity are lower, can more efficiently process high-resolution feature;And attention layer structure can be easily embedded into any existing network, has strong practical significance;While reducing the network parameter quantity, the accuracy of identification segmentation and classification is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of steel coil surface defect detection, in particular to a steel coil surface defect detection method based on a convolutional neural network. BACKGROUND

[0002] Attention layer structure is widely used in various fields of deep learning, whether it is image processing, speech recognition or various types of tasks in natural language processing, it is easy to encounter the figure of attention model. The core logic of this mechanism is from focusing on all to focusing on key. It can help the model focus on the most important part of the image, thereby improving the performance and accuracy of the model. The development prospect of attention mechanism is very broad. For example, graph neural networks can be divided into five categories: graph convolutional networks, graph autoencoders, graph generative networks, graph recurrent networks and graph attention networks. Each type of graph neural network has its own set of algorithms and systems for processing graph structure data, and the principles and applicable ranges are also different. These networks can recursively analyze local information and extract fine-grained information from the local area obtained.

[0003] Patent No. CN117593289 A "A lightweight strip steel surface defect prediction method based on multi-receptive field" discloses a strip steel surface defect detection method based on a convolutional neural network, including the following steps: step 1, constructing an LSDD model, the LSDD model includes an encoder and a decoder, the encoder is composed of 5 MHL modules; the decoder is composed of 5 LEF modules; step 2, applying a public data set to train the LSDD model; step 3, taking image data as input, and predicting through the trained LSDD model. This method can restore the defect information of the strip steel surface to a greater extent on the premise of maintaining light weight (few parameters, fast operation rate).

[0004] The patent number CN117392450 A "A steel material quality analysis method based on evolutionary multi-scale feature learning" provides a steel material quality analysis method based on evolutionary multi-scale feature learning, relating to the technical field of steel material defect image recognition. The method first establishes a steel material quality surface defect sample library, then constructs a steel material quality analysis model based on evolutionary multi-scale feature learning, and sets the initial hyperparameters and structure parameters of the quality analysis model; then adopts the Knee point strategy to select the relevant parameter values of the steel material quality analysis model with the best performance of multi-scale feature learning; finally, the optimal multi-scale feature learning quality analysis model generated by the optimization process is tested using the test data set. This method can significantly improve the accuracy and robustness of the steel strip surface defect recognition result, and the model parameter quantity is small and the computing resource is small, which can be effectively applied to the actual production process, providing a basis for the operator to timely grasp the steel strip quality, helping the steel enterprise to improve the product quality and reduce the production cost.

[0005] In the prior art, in order to pursue higher accuracy, the convolutional neural network has the problems of multiple model layers, large parameter quantity, slow picture processing speed and difficult deployment. SUMMARY

[0006] The purpose of the present application is to provide a steel coil surface defect detection method based on convolutional neural network, in order to solve the problem that the current neural network has multiple model layers, large parameter quantity, slow picture processing speed and difficult deployment in order to pursue higher accuracy.

[0007] To achieve the above purpose, the present application provides the following technical solutions:

[0008] A steel coil surface defect detection method based on convolutional neural network, comprising the following steps:

[0009] S1: taking out the features of different positions of the input feature map of the steel coil surface;

[0010] S2: inputting the selected features into the perception and setting the category;

[0011] S3: inputting the output of the perception into the Softmax function as the confidence score of each feature;

[0012] S4: generating a mask for the input feature map, the mask of the input feature map has the same width and height as the feature map input to the attention layer, and the value of each position is a scalar, the process of calculating each value mask is as follows: weighting and summing the confidence score generated previously and the manually selected weight factor;

[0013] S5: inputting the result of the previous weighting and summing into the ReLu function to obtain the mask value;

[0014] S6: multiplying the mask value with the input feature map by position to obtain the output feature map of the attention layer.

[0015] Further, the specific process in S1 is as follows:

[0016] Each position in the feature map corresponds to a local region of the input data, and the information in this local region is extracted and encoded by the convolutional layer to the corresponding position of the feature map, that is, different parts of the input data can be extracted by selecting different positions in the feature map.

[0017] Further, the specific process in S2 is as follows: in the process of setting the category, it is set according to the specific requirements of the problem, and in the prediction stage, given an input feature, the neural network will input the selected feature to the perceptron with input dimension D and output dimension C+1, where C is the number of categories in the classification task, and the additional category is regarded as invalid information.

[0018] Further, the specific process in S3 is as follows:

[0019] In the neural network, the output of the perceptron is processed by the Softmax function to obtain the confidence score of each feature, and the Softmax function is used to input an n-dimensional vector and normalize it to an n-dimensional probability distribution, where the value of each element is between 0 and 1, and the sum of all elements is 1.

[0020] Further, the specific process in S4 is as follows:

[0021] S401: the neural network processes the input data and outputs a confidence score, which represents the confidence of the network for a certain output;

[0022] S402: select the weight factor according to the specific requirements of the problem, and the weight factor is manually selected or calculated by related methods;

[0023] S403: weighted sum of the confidence score and the weight factor, that is, multiply each confidence score by the corresponding weight factor, and then sum the products to obtain the final result.

[0024] Further, the specific process in S5 is as follows:

[0025] When the weighted result is input into the ReLU function, a nonlinear transformation is performed on the weighted result, that is, all values greater than or equal to 0 in the weighted result are retained, and all values less than 0 are set to 0, that is, the ReLU function is used to suppress negative values in the weighted result and only positive values are retained. This process generates a mask, and a mask value greater than or equal to 0 is used to indicate that the feature of the position in the weighted result is activated, and a mask value less than 0 is used to indicate that the feature of the position is suppressed.

[0026] Further, the specific processing process in S6 is as follows:

[0027] The mask value is multiplied with the input feature map element by element, and the features in the input feature map corresponding to the positions with 0 in the mask are shielded, that is, become 0, and the features in the input feature map corresponding to the positions with 1 in the mask are retained, and the result obtained is a new feature map containing the features of the specific positions in the input feature map selected by the mask.

[0028] Compared with the prior art, the present application has the following beneficial effects:

[0029] The steel coil surface defect detection method based on the convolutional neural network has a simple attention layer structure and fewer parameters, so that the calculation complexity and spatial complexity are relatively low, and high-resolution features can be processed more efficiently; the attention layer structure can be easily embedded into any existing network, and has strong practical significance; while reducing the network parameter amount, the accuracy of recognition, segmentation and classification is improved. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 FIG. 1 is a flowchart of a steel coil surface defect detection method based on a convolutional neural network according to the present application;

[0031] Figure 2 FIG. 2 is a structural diagram of a steel coil surface defect detection method based on a convolutional neural network according to the present application;

[0032] Figure 3 FIG. 3 is a structural diagram of different stages of a RepVGG structure unit in Embodiment One of the present application;

[0033] Figure 4 FIG. 4 is a mask diagram of the output of the attention layer of the model Base-Model(A) in Embodiment One of the present application;

[0034] Figure 5 FIG. 5 is a confusion matrix diagram of the prediction result of the model Base-Model(A) in Embodiment One of the present application;

[0035] Figure 6 FIG. 6 is a structural diagram of a defect segmentation network in Embodiment Two of the present application;

[0036] Figure 7 Figure 2 is a schematic diagram of a deconvolution block structure in the second embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0038] As shown in the figure, the present application provides a steel coil surface defect detection method based on a convolutional neural network, which comprises the following steps: Figure 1

[0039] S1: extracting features at different positions of a feature map of a steel coil surface input; specifically, each position in the feature map corresponds to a local region of the input data, and the information in the local region is extracted and encoded into the corresponding position of the feature map by the convolutional layer, so that the features of different parts of the input data can be extracted by selecting different positions in the feature map;

[0040] S2: inputting the selected features into a perceptron and setting a category; specifically, in a neural network, a perceptron is a typical structure, which has the following main features: simple structure, convergence algorithm for the problems it can solve, and mathematical strictness in proving that, in the process of setting a category, the specific requirements of the problem are set, and in the prediction stage, a given input feature is input into the perceptron with an input dimension of D and an output dimension of C+1, where C is the number of categories in the classification task, and the additional category is regarded as invalid information, and the neural network will output the corresponding category.

[0041] S3: taking the output of the perceptron as a confidence score for describing each feature through a Softmax function; specifically, in a neural network, the output of a perceptron is usually processed through a Softmax function to obtain the confidence score of each feature, and the Softmax function can input an n-dimensional vector (usually used to represent a feature vector of an entity) and normalize it into an n-dimensional probability distribution, where each element has a value between 0 and 1, and the sum of all elements is 1;

[0042] ​As can be seen from S2 and S3 above, the learnable parameters of the attention layer are all in the perceptron during the initial calculation phase, but the C+1-dimensional confidence scores output by the perceptron do not have true labels. In other words, the attention layer cannot directly update the perceptron weights based on the confidence scores predicted in the initial phase. However, the confidence scores predicted by the perceptron affect the feature maps output by the attention layer, which are then propagated through the network, so the weights of the attention layer are updated during the network backpropagation.

[0043] S4: Generate a mask for the input feature map. The mask for the input feature map has the same width and height as the feature map input to the attention layer, and the value at each position is a scalar. The process for calculating each value of the mask position is as follows: weighted sum of the previously generated confidence score and the manually selected weight factor; the specific processing process is as follows:

[0044] S401: The neural network processes the input data and outputs a confidence score, which represents the network's confidence in a particular output. For example, in a classification problem, the confidence score may represent the neural network's confidence in the prediction of each class.

[0045] S402: Select a weight factor based on the specific requirements of the problem. The weight factor is selected manually or calculated using a related method.

[0046] S403: Performing a weighted summation of the confidence scores and the weight factors, that is, multiplying each confidence score by the corresponding weight factor, and then summing these products to obtain a final result;

[0047] In order to better understand why the attention layer is able to filter out invalid information from the feature map, the present invention further describes the role of the weight factor and its general selection method; Figure 2 It can be seen that the weight factor is a C+1 dimensional vector containing a unique negative number, which is used to weight the confidence of invalid information to weaken the value of the "attention layer mask". This means that if the attention layer needs to attenuate the features of a certain position in the feature map, it only needs to increase the confidence score corresponding to the negative position (for example, in Figure 2 At position P, the attention layer wants to attenuate the feature), and the attention layer is fully capable of completely invalidating the feature (because the weighted result of the confidence score is fed into the ReLu function). At the same time, the decision of the attention layer affects the probability value finally predicted by the model, so it can improve its own discrimination ability during the back propagation process of the model.

[0048] S5: input the result of the previous weighted summation into the ReLu function to obtain a mask value; the specific processing process is as follows: when the weighted result is input into the ReLu function, it is used for nonlinear transformation of the weighted result, that is, all values greater than or equal to 0 in the weighted result are retained, and all values less than 0 are set to 0, that is, the ReLu function is used to suppress the negative values in the weighted result, only the positive values are retained, and this process generates a mask, the mask value greater than or equal to 0 is used to represent that the feature of the position in the weighted result is activated, and the mask value less than 0 is used to represent that the feature of the position is suppressed.

[0049] S6: multiply the mask value with the input feature map by position to obtain the output feature map of the attention layer; the specific processing process is as follows: multiply the mask value with the input feature map element by element, the features in the input feature map corresponding to the positions of 0 in the mask will be shielded, that is, become 0, and the features in the input feature map corresponding to the positions of 1 in the mask will be retained, and the result obtained is a new feature map containing the features of the specific positions in the input feature map selected by the mask.

[0050] Based on the above description, in order to further better explain and illustrate the present application, the following specific embodiment one is also provided:

[0051] The steel coil surface defect recognition method based on the convolutional neural network comprises the following steps:

[0052] S1: build a defect recognition model backbone network composed of RepVGG unit; the specific process is as follows:

[0053] S11: based on the timeliness and model performance, this embodiment adopts the RepVGG reparameterization structure unit to form the backbone network of the defect recognition model, and the structure of the training stage and the inference stage is as shown in Figure 3

[0054] S12: the backbone network uses the topological structure to enhance the learning ability of the model in the training stage, and folds back all the branch topologies of the network structure in the inference stage (model structure reparameterization) to form a structure consistent with VGG, based on the design of topological network structure in the training stage and structure reparameterization in the inference stage, the RepVGG and the classic topological structure have achieved similar accuracy, and the model inference efficiency is greatly improved.

[0055] S2: perform basic experimental setting; the specific process is as follows:

[0056] ​S21: First, the basic experimental setup is carried out, and when instantiating the classification model, two kinds of basic model (Base-Model) and extended model (Base-Model-E) are selected, the convolution layer parameters of the 5 "RepVGG Block" of the basic model are 16, 24, 24, 32 and 32, and the convolution layer parameters of the 5 "RepVGG Block" of the extended model are 24, 32, 32, 40 and 40;

[0057] S22: The influence of the convolution layer width on the classification accuracy is explored by comparing the basic model and the extended model.

[0058] S3: The model is trained using the Adam optimizer; specifically as follows:

[0059] S31: The Adam optimizer is used when training the model, and the initial learning rate is 0.001, and the cosine decay is used to adjust the learning rate during training;

[0060] S32: In order to enable the classification model to clearly distinguish images without defects (which can further reduce the false detection rate), 610 images without defects are added to the data set when training the classification model, so at this time the classification data set contains 7 classes and 4754 images (the output of the convolution layer of the attention layer does not need to be changed, the additional images are not defects), and the images in the training set are subjected to data enhancement before being input into the model, and the enhancement methods include random brightness, random contrast, and random horizontal and vertical flipping.

[0061] S4: The results of the defect recognition are analyzed; specifically as follows:

[0062] S41: The experimental results are shown in Table 1, which shows the accuracy of different models in identifying defects, the inference efficiency of the model, and the parameter amount of the model, wherein the model accuracy shown is the average accuracy, which is obtained by training the model on a randomly divided data set 5 times and averaging the accuracy of the 5 models, and the standard deviation of the 5 accuracies is also given in Table 1;

[0063] S42: Model Base-Model(A) indicates that the model contains an attention layer, Base-Model indicates that the model only contains "RepVGG Block", and "7:3" in the table indicates that the images used for training and the images used for verification respectively account for 70% and 30% of the image data set;

[0064] S43: The base model (Base-Model(A)) with the attention layer achieved the best classification accuracy (99.47% when trained on 70% of the data and 98.87% when trained on 50% of the data) and the most stable performance (standard deviation of 0.16 when trained on 70% of the data, the lowest among all models).

[0065] Table 1 Performance of different models in classification accuracy, inference speed and model parameter quantity

[0066]

[0067]

[0068] In this embodiment 1, in S4 Figure 5 The confusion matrix shown shows that: for defect-free images, the model can accurately identify them, that is, it does not classify defect-free images as defects of any category, and only identifies the two scratched images as defect-free. Therefore, it is feasible to correct the misdetection in the defect detection stage during the defect recognition stage. At the same time, it can be found that: increasing the width of the convolutional layer in the model does not improve the performance of the model (the performance of Base-Model-E is lower than that of Base-Model). When the attention layer is included, the model's ability to identify defects is significantly improved. At the same time, the attention layer does not have a significant impact on the model's reasoning efficiency. In order to intuitively demonstrate the role of the attention layer, Figure 4 Figure 3 shows some mask images output by the attention layer during model inference.

[0069] Example 2:

[0070] This embodiment provides a technical solution: a steel coil surface defect segmentation method based on a convolutional neural network, comprising the following steps:

[0071] S1: The Unet-based segmentation scheme accurately segments the detected defects; the details are as follows:

[0072] S11: Defect measurement requires accurate segmentation of detected defects. A Unet-based segmentation solution was used. Experiments demonstrated that a lightweight network can accurately segment surface defects in steel coils.

[0073] In this embodiment, the network structure of Unet is as follows: Figure 6The encoder and decoder are shown in FIG. 1 (the dimensions in the figure are the width and height of the feature map), which includes two parts of encoding (left) and decoding (right), where the "convolution block" is used to represent different convolution structure units (ResNet structure unit, VGG structure unit and RepVGG structure unit are tested), and the deconvolution block is composed of upsampling unit and deconvolution layer (Transposed Convolution Layer) and other parts, and its structure is as shown in FIG. 2. Figure 7 Figure 6 The input feature map of the deconvolution block at the rear in the above is the feature map output by the deconvolution block in front and the corresponding convolution block output feature map connected in the channel direction, which is to recover the edge pixel information lost in convolution;

[0074] S12: An optional module of adding an attention layer in the encoder part is selected to be inserted after the second convolution block, because the feature map at this time contains more rich defect information, and the feature map output by the attention layer can act on each stage in the subsequent network.

[0075] S13: Related settings when instantiating the model are explained. The structure of the deconvolution block in the decoder part is fixed (as shown in FIG. 2), and for the convolution block in the encoder part, the unit structures of ResNet, VGG and RepVGG are compared by experiment to select the appropriate convolution block structure. The output channel number of the convolution layer in each convolution block in the encoder is set to 16, 24, 32 and 40 (from top to bottom in FIG. 1), and the output channel number of the convolution layer and the deconvolution layer in each deconvolution block in the decoder is set to 32, 24, 16 and 8 (from bottom to top in FIG. 1, and the output of the convolution layer in the top layer is fixed to 1, which is the pixel category output). If the convolution block structure is selected as ResNet, the model name is ResNet-Unet. Figure 7 Figure 6 Figure 6

[0076] S14: When training the model, the loss function uses the BCE loss function, where N represents the number of pixels in the input image, y true and y pred represent the true label and the predicted label (i.e. whether the pixel belongs to a defect) of the pixel at different positions, respectively. The Adam optimizer is still used, the initial learning is 0.01, and the learning rate is adjusted using cosine decay.

[0077] S2: The results of the segmentation experiment are analyzed as follows:

[0078] ​​​​S21: The experimental results are shown in Table 2, wherein ResNet-Unet-1.5 represents that the width of the convolution layer and the deconvolution layer is simultaneously expanded to 1.5 times the initial width (the width of the ResNet-Unet structure as the initial width), ResNet-Unet(A) represents that the attention layer is used; when evaluating the model performance, mIoU (mean Intersection over Union, mIoU) is used as the model segmentation accuracy evaluation index, and the model parameter amount and the inference speed are also given, the mIoU experimental results in the table are the average values of 5 experimental results, and the corresponding standard deviation is given;

[0079] Table 2 Performance and inference efficiency comparison of different defect segmentation models

[0080]

[0081]

[0082] S22: All models have achieved good segmentation accuracy, and the RepVGG-Unet-1.5(A) model has achieved the best segmentation accuracy of 97.63 mIoU on average. In addition, from the inference speed of each model, it can be seen that all models have very good real-time performance. By comparing the models using the same scaling ratio and not containing the attention layer, it can be found that the RepVGG structure has the best performance, the VGG structure is second, and the ResNet structure has the worst performance. By comparing the performance of the same model using different scaling ratios, it can be found that expanding or reducing the width of the base model does not bring very obvious performance changes. After comparing all models, it can be found that the attention layer significantly improves the performance of the model, and if the model contains the attention layer, its performance increases significantly with the increase of the model width. Therefore, using the attention layer can effectively improve the defect segmentation performance of the model.

[0083] In summary, the embodiment of the present application is aimed at the characteristics of simple background and structure of the steel coil surface defect image, and the attention layer structure is used to effectively improve the accuracy of model defect recognition and segmentation without increasing the model calculation complexity (the parameter amount occupied by the attention layer can be calculated). This fully illustrates the effectiveness of the attention layer designed by the present application in extracting simple background and structure image features.

[0084] The above is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can make equivalent replacement or change according to the technical solution and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for detecting surface defects of steel coils based on convolutional neural networks, characterized in that: The following steps are involved: S1: Extract the features at different positions of the input feature map on the surface of the steel coil; S2: Input the selected features into the perceptron and set the category; S3: The output of the perceptron is passed through the Softmax function as the confidence score describing each feature; S4: Generate a mask of the input feature map. The mask of the input feature map has the same width and height as the feature map input to the attention layer, and the value of each position is a scalar. The process of calculating each value of the mask position is as follows: the confidence score generated previously is weighted summed with a manually selected weight factor; S5: Input the previous weighted summation result into the ReLu function to obtain the mask value; S6: Multiply the mask value with the position-wise input feature map to obtain the output feature map of the attention layer.

2. The method for detecting surface defects of a steel coil based on a convolutional neural network according to claim 1, wherein: The specific processing process in S1 is as follows: Each position in the feature map corresponds to a local area of ​​the input data. The information in this local area is extracted by the convolutional layer and encoded into the corresponding position of the feature map. That is, the features of different parts of the input data can be extracted by selecting different positions in the feature map.

3. The method for detecting surface defects of a steel coil based on a convolutional neural network according to claim 1, wherein: The specific processing process in S2 is as follows: in the process of setting categories, it is set according to the specific requirements of the problem. In the prediction stage, given an input feature, the neural network inputs the selected feature into a perceptron with an input dimension of D and an output dimension of C+1, where C is the number of categories in the classification task, and additional categories are regarded as invalid information.

4. The method for detecting surface defects of a steel coil based on a convolutional neural network according to claim 1, wherein: The specific processing process in S3 is as follows: In a neural network, the output of the perceptron is processed by the Softmax function to obtain the confidence score of each feature. The Softmax function is used to input an n-dimensional vector and normalize it into an n-dimensional probability distribution, where the value of each element is between 0 and 1 and the sum of all elements is 1.

5. The method for detecting surface defects of a steel coil based on a convolutional neural network according to claim 1, wherein: The specific processing process in S4 is as follows: S401: The neural network processes the input data and outputs a confidence score, which represents the network's confidence in a specific output; S402: Select a weight factor based on the specific requirements of the problem. The weight factor is selected manually or calculated using a related method. S403: Perform weighted summation on the confidence scores and the weight factors, that is, multiply each confidence score by the corresponding weight factor, and then sum these products to obtain the final result.

6. The method for detecting surface defects of a steel coil based on a convolutional neural network according to claim 1, wherein: The specific processing process in S5 is as follows: When the weighted result is input into the ReLU function, it is used to perform a nonlinear transformation on the weighted result, that is, all values ​​greater than or equal to 0 in the weighted result will be retained, and all values ​​less than 0 will be set to 0, that is, the ReLU function is used to suppress negative values ​​in the weighted result and only retain positive values. This process generates a mask. A mask value greater than or equal to 0 is used to indicate that the feature of the position in the weighted result is activated, and a mask value less than 0 is used to indicate that the feature of the position is suppressed.

7. The method for detecting surface defects of a steel coil based on a convolutional neural network according to claim 1, wherein: The specific processing process in S6 is as follows: The mask value is multiplied element-by-element by the input feature map. The features in the input feature map corresponding to the positions where the mask is 0 will be shielded, that is, changed to 0, while the features in the input feature map corresponding to the positions where the mask is 1 will be retained. The result is a new feature map that contains the features of the specific positions in the input feature map selected by the mask.

Images