High-precision steel surface defect detection method suitable for complex industrial environment

Abstract

The invention relates to a high-precision steel surface defect detection method suitable for a complex industrial environment, and aims to solve the problems of feature coupling and background interference caused by data scarcity, long tail distribution and multi-defect coexistence. According to the method, a conditional generative adversarial network (CGAN) and a convolutional neural network (CNN) are combined, an SE-Net channel attention mechanism is integrated, and the method is used for automatic detection and classification of multiple defects on the steel surface. By introducing a physical constraint CGAN data generation method, a multi-defect coexistence composite image conforming to industrial reality can be generated, so that the diversity and accuracy of a training data set are effectively enhanced. The model optimizes the expression of defect features through multi-level feature extraction and an attention mechanism, and enhances the distinguishing ability and recognition precision of multiple defects. Experiments show that the method is excellent in performance in a multi-defect identification task, the accuracy rate of 98.89% and the F1 score of 99.72% are achieved, the method is remarkably superior to a traditional detection method and other deep learning models, and the method has high robustness and practical value in a complex industrial environment.

Description

A high-precision steel surface defect detection method suitable for complex industrial environments Technical Fields

[0001] This invention belongs to the field of industrial automation and computer vision technology, specifically relating to an intelligent detection method and system for steel surface defects, and a multi-defect recognition technology for steel surfaces based on conditional generative adversarial networks and attention mechanisms. Background Technology:

[0002] Surface defect detection in steel is a core component of industrial non-destructive testing and quality control. The accuracy and reliability of the detection results directly determine the mechanical properties, structural integrity, and service life of steel products, and are of vital importance to ensuring the safe operation of major equipment and improving the economic benefits and market competitiveness of production enterprises.

[0003] Traditional inspection methods have long relied on manual visual inspection, which is not only inefficient and labor-intensive, but also highly susceptible to factors such as worker fatigue, experience differences, and subjective emotions, leading to high rates of missed detections and false positives. Furthermore, it is difficult to standardize inspection criteria, making it unsuitable for the high-speed, continuous, and intelligent production trends of modern industry. To overcome the limitations of manual inspection, automated inspection technologies based on traditional digital image processing have emerged. These methods process images using manually designed feature extractors such as edge detection and texture analysis, achieving a certain degree of automated defect identification. However, when faced with complex and variable lighting conditions, steel plate surface reflections, and significant differences in the shape, size, and contrast of defects in actual industrial environments, their generalization ability and robustness are severely insufficient, and inspection accuracy decreases significantly in complex scenarios.

[0004] In recent years, breakthroughs in deep learning technology, especially the powerful end-to-end feature learning capabilities of Convolutional Neural Networks (CNNs), have brought revolutionary progress to defect detection. Two-stage detection algorithms, represented by Faster R-CNN, achieve high detection accuracy through a "region proposal + classification regression" mechanism. However, these advanced methods still have significant limitations when facing the unique challenges of steel surface defects: two-stage algorithms have high computational complexity, making it difficult to meet the real-time requirements of high-speed production lines; while single-stage algorithms are faster, they generally suffer from insufficient feature fusion capabilities, a lack of effective feature decoupling mechanisms, and poor suppression of complex background interference when dealing with small defects, low-contrast defects, and complex scenarios where multiple defects coexist in the same area. This results in low detection rates for dense, small defects and a high likelihood of feature confusion and false detections. Crucially, in actual production environments, the distribution of defect samples is extremely uneven, exhibiting a typical long-tail distribution. Severe defects such as "cracks" are scarce, while common defects such as "scratches" are abundant, leading to weak identification capabilities of data-driven deep learning models for scarce defect types. Meanwhile, most NEU-DET public datasets only provide annotations for single defects, failing to fully cover the common industrial scenario of "coexistence of multiple defects." This results in insufficient generalization ability of models trained on such datasets, leading to severe feature confusion and a sharp deterioration in detection performance when faced with complex samples in real production.

[0005] In summary, existing technologies have not yet provided effective solutions to the two core challenges of data scarcity and feature coupling caused by the coexistence of multiple defects. Therefore, the industry urgently needs a new technical solution that can enhance sample diversity from the data source and accurately decouple and identify coexisting defects at the model level. Invention content:

[0006] The purpose of this invention is to overcome the problems of feature coupling and background interference caused by data scarcity, long-tail distribution and coexistence of multiple defects in existing steel surface defect detection technologies, and to provide a steel surface defect detection method that can effectively generate multiple defect samples that conform to physical laws and achieve high-precision identification and classification in complex scenarios.

[0007] To solve the above technical problems, the specific steps are as follows:

[0008] Step 1: Construct a dataset of steel surface defect images containing rolled-in scales, inclusions, patches, cracks, pitting, and scratches;

[0009] Step 2: Expand and enhance the dataset, use conditional generative adversarial network to generate multi-defect coexistence synthetic images that meet physical constraints, and divide all images into training set, validation set and test set;

[0010] Step 3: Construct the CGAN-CNN-SE model: use a conditional generative adversarial network as the data synthesis module and a convolutional neural network integrating the SE-Net attention mechanism as the defect recognition module;

[0011] Step 4: Train the CGAN-CNN-SE model using the training set obtained in Step 2;

[0012] Step 5: Use the validation set to monitor and optimize the model performance during training, and use the test set to evaluate the final detection performance of the model.

[0013] Step 6: Input the steel image to be detected into the trained model and output the defect category, location, and confidence level.

[0014] Furthermore, in step 2, the physical constraints introduced when generating the synthetic image using a conditional generative adversarial network include:

[0015] (1) Defect quantity constraint: the maximum number of defects per picture is limited based on production line statistics;

[0016] (2) Spatial distribution constraints: Defining the positional relationships of defects based on the material stress distribution law;

[0017] (3) Morphological constraints: quantify the roundness, aspect ratio, and roughness parameters of defects through metallographic analysis;

[0018] Furthermore, in step 3, the workflow of the convolutional neural network integrating the SE-Net attention mechanism is as follows:

[0019] (1) The input image is subjected to multi-level feature extraction through convolutional layers;

[0020] (2) The extracted feature maps are fed into the SE-Net module, and channel-level global information is obtained through compression.

[0021] (3) Generate channel attention weights through incentive operations;

[0022] (4) Multiply the weights with the original feature map by scaling transformation, and output the optimized feature map to the classifier.

[0023] Furthermore, in step 3, the operation steps of the SE-Net module are as follows:

[0024] (1) In the compression stage, global average pooling is performed on the input feature map. The calculation formula is as follows:

[0025]

[0026] Among them, z cLet i be the global average value for each channel, and u be the global average value for each channel. c (i,j) is the value of channel c at position (i,j);

[0027] (2) During the activation phase, channel weights are generated through two fully connected layers and a nonlinear activation function. The calculation formula is as follows:

[0028] s=σ(W2δ(W1z)) (2)

[0029] Where W1 and W2 both represent the dimensions of the fully connected layer, and σ and δ represent the Sigmoid function and the ReLU function, respectively;

[0030] (3) During the scaling stage, the generated weights are multiplied channel by channel with the original feature map to complete the feature recalibration. The calculation formula is as follows:

[0031]

[0032] Where O represents the output feature, and I represents the output feature. i For the i-th input feature, w i Its corresponding weight is ò, which is a small constant to prevent numerical instability.

[0033] Beneficial effects: This invention has the following effects through four innovative points:

[0034] (1) By introducing the CGAN data generation method with physical information constraints, the problem of scarcity and long-tail distribution of multi-defect samples is solved at the source. The generated data is rich and diverse and conforms to industrial reality, which significantly improves the generalization ability of the model.

[0035] (2) By constructing a recognition network that combines CNN and SE-Net, and using the channel attention mechanism to dynamically optimize the feature response, we have achieved effective decoupling of multiple defect features and enhancement of key information, which has significantly improved the ability to distinguish coexisting defects and the recognition accuracy.

[0036] (3) By integrating CGAN data generation with CNN-SE recognition model end-to-end, a complete solution was formed. In the detection of multiple defects on steel surface under complex background, the accuracy rate of 98.89% and the F1 score of 99.72% were achieved, demonstrating excellent robustness and engineering application value. Attached Figure Description

[0037] Figure 1 is a diagram of the overall framework of the CGAN-CNN-SE model proposed in this invention.

[0038] Figure 2 is a schematic diagram of the CGAN conditional generative adversarial network in this invention;

[0039] Figure 3 is a structural block diagram of the feature extraction network (CNN-SE) module in this invention;

[0040] Figure 4 is a schematic diagram of the SE-Net channel attention module in this invention;

[0041] Figure 5 is a schematic diagram of the coexistence of a single type of defect and multiple types of defects on the steel surface;

[0042] Figure 6 shows a comparison of the accuracy curves of different models on the test set;

[0043] Figure 7 is a comparison of the loss curves of different models on the test set. Detailed implementation method:

[0044] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0045] The following are explanations of the English abbreviations used in this invention: CGAN: Conditional Generative Adversarial Network; SE-Net: Squeeze and Activate Network; CNN: Convolutional Neural Network; ECA: Efficient Channel Attention; RFAConv: Receptive Field Attention Convolution; Pre: Precision; Recall: Recall; Acc: Accuracy.

[0046] As shown in Figure 1, this invention provides a method for detecting steel surface defects based on CGAN and CNN-SE in complex industrial environments. The process is as follows:

[0047] 1. Acquire and process steel surface defect datasets;

[0048] (1) Dataset Construction. The steel surface defect dataset used in the experiment adopts the publicly available dataset NEU-CLS, which includes six types of defects: rolled-in scales (RS), inclusions (In), patches (Pa), cracks (Cr), pitting (PS), and scratches (Sc). There are 300 images for each type of defect.

[0049] (2) Dataset augmentation. CGAN was used to generate 900 synthetic images containing multiple defects, including 15 combinations such as RS with In, RS with Pa, and RS with Cr. Simultaneously, all images (1800 real images and 900 synthetic images, totaling 2700 images) were randomly divided into training, validation, and test sets in an 8:1:1 ratio.

[0050] 2. As shown in Figures 1 and 2, construct the CGAN-CNN-SE model:

[0051] (1) In the data synthesis stage, a CGAN network is constructed. The generator receives a real image of a single defect and conditional information containing the number of defects, spatial distribution, and morphological features, and generates a synthesized image through an encoder-decoder structure. The discriminator is responsible for distinguishing between the real image and the synthesized image.

[0052] (2) In the defect identification stage, a feature extraction network (CNN-SE) is constructed, the structure of which is shown in Figure 3. This network contains four convolutional layers with the number of convolutional kernels being 64, 128, 256, and 512 respectively, and uses the Leaky ReLU activation function. The SE-Net module is then connected after the convolutional features.

[0053] As shown in Figures 3 and 4, the workflow of the SE-Net module introduced in this invention is as follows:

[0054] (1) The input feature map first compresses the spatial information through the Squeeze operation (global average pooling) to obtain channel statistics.

[0055] (2) The correlation between channels is learned through the Excitation operation (two fully connected layers) to generate channel weights.

[0056] (3) The weights are multiplied with the original feature map channel by channel through the Scale operation to output the optimized feature map.

[0057] The CGAN-CNN-SE model was trained using the processed steel defect dataset. The training parameters were configured as follows: 100 training epochs, batch size of 75, initial learning rate of 0.003, using the Adam optimizer, with the learning rate decreasing to 0.1 times its original value every 10 epochs. After training, the trained weights were saved.

[0058] The test set is used to classify steel surface defects. The optimal weight file obtained from training is used to perform inference verification on the test set images. Finally, the model is applied to a real steel surface defect detection system.

[0059] Explanation of experimental data in this embodiment:

[0060] (1) Comparison experiment of single defect models

[0061] To comprehensively evaluate the performance of different models in single defect detection on steel surfaces, CGAN-CNN-SE was compared with various models such as Faster R-CNN, VGG16, ResNet, DNN, DenseNet, MobileNet, EfficientNet, and InceptionV3 under the same dataset and experimental environment. The specific results are shown in Table 1.

[0062] Table 1 Comparison of Single Defect Identification Performance Experiments

[0063] Model Precision (%) Recall (%) F1 Score (%) Accuracy (%) Faster R-CNN 94.07 94.26 94.37 95.83 VGG16 91.36 90.60 91.28 91.67 ResNet 94.61 94.54 94.68 96.11 DNN 93.50 88.76 88.11 94.72 DenseNet 91.49 86.41 89.87 93.61 MobileNet 93.33 92.02 92.28 94.17 EfficientNet 95.28 94.15 94.15 95.56 InceptionV3 95.20 91.80 94.04 96.39 CGAN-CNN-SE 98.14 97.27 97.64 98.89 surface

[0064] As shown in Table 1, the experimental results indicate that in the single defect identification task, the CGAN-CNN-SE model proposed in this invention achieves the highest precision, recall, F1 score, and accuracy, at 98.14%, 97.27%, 97.64%, and 98.89%, respectively, which are significantly better than other comparative models, demonstrating its excellent basic defect identification ability.

[0065] (2) Comparison experiment of multiple defect models

[0066] To verify the effectiveness of the method of this invention, a dataset with multiple defects was constructed by combining images generated by CGAN with the original images. CGAN-CNN-SE was compared with various models such as Faster RCNN, VGG16, ResNet, DNN, DenseNet, MobileNet, EfficientNet, and InceptionV3 under the same dataset and experimental environment. The specific results are shown in Table 2.

[0067] Table 2 Comparison of various defect identification performance experiments

[0068] Model Accuracy (%) Recall (%) F1 Score (%) Precision (%) Faster R-CNN 95.83 95.83 95.83 96.25 VGG16 91.67 91.67 91.67 93.34 ResNet 96.11 94.53 95.31 97.22 DNN 94.72 93.95 94.33 95.69 DenseNet 93.61 92.07 92.83 95.00 MobileNet 94.17 93.39 93.77 93.75 EfficientNet 95.56 95.56 95.56 96.12 InceptionV3 96.39 98.92 97.62 97.50 CGAN-CNN-SE 98.89 99.44 99.72 98.61 surface

[0069] As shown in Table 2, the experimental results demonstrate that the method of this invention exhibits even greater advantages in the more challenging scenario of multiple defects coexisting, achieving an F1 score of 99.72% and an accuracy of 98.89%, comprehensively surpassing other models. This fully proves that the method of this invention effectively solves the problem of feature coupling and recognition difficulties caused by the coexistence of multiple defects through CGAN data augmentation and SE-Net feature optimization.

[0070] Figures 6 and 7 show the training process curves for Faster R-CNN, VGG16, ResNet, DNN, DenseNet, MobileNet, EfficientNet, InceptionV3, and the method of this invention. The pink curve represents the method of this invention, the gray curve represents the InceptionV3 model, the green curve represents the DenseNet model, the red curve represents the VGG16 model, the dark blue curve represents the ResNet model, the brown curve represents the MobileNet model, the light blue curve represents the DNN model, the dark yellow curve represents the Faster R-CNN model, and the light yellow curve represents the EfficientNet model.

[0071] As shown in Figure 6, the accuracy of each model increases and then stabilizes with increasing training cycles. Among them, the method of this invention converges the fastest, essentially converging after 250 training cycles, achieving a final accuracy of 98.89%. In contrast, InceptionV3 requires 550 cycles to stabilize, with a final accuracy of 96.39%. VGG16 converges quickly in the early stages but has the lowest final accuracy (91.67%). Experiments demonstrate that the method of this invention outperforms the comparative models in both convergence speed and final performance.

[0072] As shown in Figure 7, the method of this invention performs best in terms of training loss, converging quickly to a stable state after 250 epochs with a final loss value of 0.113. Among the comparative models, InceptionV3 achieved the lowest loss (0.070), but its convergence speed was slow; the final losses of ResNet, MobileNet, and Faster R-CNN were 0.132, 0.149, and 0.165, respectively; VGG16 had the highest loss value (0.268) due to its simple structure. The results demonstrate that the method of this invention outperforms other models in both convergence efficiency and loss control.

[0073] (3) Model ablation experiment

[0074] To evaluate the performance contribution of each improved module, systematic ablation experiments were conducted. The experiments were carried out in two phases: benchmarking was performed using only the existing dataset, followed by expanding the training dataset with synthetic images generated by CGAN to verify its performance improvement effect. The experimental results are shown in Table 3.

[0075] Table 3 Ablation Experiment

[0076]

[0077] As shown in Table 3, the ablation experiment data reveals that the baseline models CNN, DenseNet, and EfficientNetB7 achieved accuracies of 92.50%, 90.88%, and 91.28%, respectively, without any data augmentation, and their F1 scores were all below 91%. After introducing CGAN for data augmentation, the performance of each model significantly improved: CGAN-CNN achieved an accuracy of 95.17% and an F1 score of 94.16%; CGAN-DenseNet and CGAN-EfficientNetB7 also achieved accuracies of 94.29% and 93.61%, respectively. Ultimately, the CGAN-CNN-SE model, which incorporates the SE-Net module, achieved the best performance, with an accuracy of 98.89%, an F1 score of 99.72%, and recall and precision also improving to 99.44% and 98.61%, respectively, fully validating the synergistic enhancement effect of the CGAN and SE-Net modules on model performance.

[0078] The above embodiments are merely illustrative of the technical concept and features of the present invention, intended to enable those skilled in the art to understand and implement the invention, and are not intended to limit the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects in steel based on conditional generative adversarial networks and attention mechanisms, characterized in that, Includes the following steps: Acquire an image of the steel surface to be detected; input the steel surface image into a trained target detection model, the target detection model being constructed based on a convolutional neural network (CNN) using a conditional generative adversarial network (CGAN) and an integrated channel attention mechanism (SE-Net); output the category and location information of the defect target in the steel surface image.

2. The method according to claim 1, characterized in that, The training data for the target detection model includes multi-defect coexistence synthetic images generated by CGAN that conform to physical information constraints.

3. The method according to claim 2, characterized in that, The physical information constraints include constraints on the number of defects, spatial distribution constraints based on material stress distribution, and morphological feature constraints based on metallographic analysis.

4. The method according to claim 1, characterized in that, The workflow of the SE-Net convolutional neural network (CNN) with integrated channel attention mechanism includes: extracting feature maps of the input image through convolutional layers; inputting the feature maps into the SE-Net module and compressing the spatial dimension of the feature maps through global pooling to obtain channel-level statistical information; generating channel attention weights based on the channel-level statistical information through a network structure containing fully connected layers and activation functions; and multiplying the channel attention weights with the original feature maps to complete feature recalibration, thereby enhancing key features and suppressing background interference.

5. The method according to claim 4, characterized in that, The activation function includes the Sigmoid function.

6. The method according to claim 1, characterized in that, The target detection model is trained using a dataset of steel surface defect images containing rolled-in scales, inclusions, patches, cracks, pitted surfaces, and scratches.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images