Metal surface defect detection method based on deep learning

Abstract

The invention discloses a metal surface defect detection method based on deep learning, and relates to the field of metal surface defect detection, the method is based on an improved U-Net model, and the accuracy and robustness of defect detection are improved through multiple innovative technologies. The method mainly comprises the following steps: S1, data acquisition; s2, data preprocessing; s3, innovative design of the model; s4, performing model training and evaluation; and S5, defect detection. Experimental results show that the provided model is superior to existing U-Net, U-Net 3 +, TransUNet and VM-U-Net models in evaluation indexes such as defect detection precision, IoU and F1 scores and the like. The method can be widely applied to metal surface defect segmentation tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of metal surface defect detection, and specifically provides a metal surface defect detection method based on deep learning. Background Art

[0002] Surface defect detection is crucial in industrial processes, and its accuracy and efficiency directly affect product quality and manufacturing costs. Common defects such as cracks, pores, and scratches usually exhibit complex shapes and variable textures. These characteristics make it difficult for traditional detection methods to meet the requirements of high precision and high efficiency in actual production.

[0003] In recent years, deep learning, especially convolutional neural networks, has made remarkable progress in image segmentation tasks. As a network structure widely used in medical image segmentation, U-Net has been attempted to be applied to metal surface defect detection. However, in the detection of metal surface defects, the traditional U-Net model often suffers from problems such as poor spatial information flow and mismatch between high-dimensional semantic information and low-level details. These problems pose challenges to the model in complex metal surface defect detection tasks, especially in multi-scale feature extraction and defect area focusing. Summary of the Invention

[0004] The purpose of the present invention is to provide a metal surface defect detection method based on deep learning to solve the problems raised in the above background art.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A metal surface defect detection method based on deep learning, comprising the following steps:

[0006] S1. Data collection: Collect metal surface defect image data;

[0007] S2. Data preprocessing: Preprocess the collected metal surface defect images to meet the input requirements of the model;

[0008] S3. Model design: Use the improved U-Net to segment the images and extract the metal surface defect areas;

[0009] S4. Model training and evaluation: Use the preprocessed training data set to train the model for multiple rounds, and at the same time use the validation set to monitor the training progress, evaluate the performance of each training cycle, and adjust the hyperparameters according to the validation results;

[0010] S5. Defect detection: Use an independent test set to evaluate the trained model, detect metal surface defects, and calculate evaluation metrics such as intersection over union (IoU), mean intersection over union (mIoU), and F1 score to verify the detection effect of the model.

[0011] Preferably, in step S1, metal surface defect image data is collected, and the image data includes different types of metal surface defects.

[0012] Preferably, in step S2, the collected metal surface defect images are preprocessed, which specifically includes the following steps:

[0013] S21: Data augmentation: Horizontally flip the image, randomly adjust the brightness and contrast, and apply Gaussian blur to enhance data diversity;

[0014] S22: Image cropping and size adjustment: For images with larger sizes, use a 256x256 window for cropping with a step size of 256 to ensure that all image sizes are unified to 256x256;

[0015] S23: Normalization processing: Normalize the image to standardize the pixel values to a mean of 0 and a variance of 1 to help accelerate model training;

[0016] S24: Dataset division: Divide the processed dataset into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0017] Preferably, in step S3, an improved U-Net is used to segment the images and extract the metal surface defect regions, which specifically includes the following steps:

[0018] S31: Encoder processing: The input metal surface defect image passes through an encoder with progressive downsampling to extract multi-level features of the image. Each layer of convolutional operation gradually abstracts the information of the image into feature maps at different levels, providing rich low-level and high-level semantic features;

[0019] S32: Spatial position perception: Based on the feature map of the last layer extracted by the encoder, a spatial position perception module is introduced;

[0020] S33: Decoder processing: The processed feature map enters the decoder stage. The decoder upsamples layer by layer to restore the high resolution of the image. After each upsampling, the feature map of the decoder is fused with the feature map of the corresponding layer of the encoder through skip connections;

[0021] S34: Cross-scale attention fusion: For the feature map after upsampling in the decoder stage, combined with the output of the skip connection of the encoder feature map, perform high-level feature extraction and adaptive feature weighting.

[0022] Preferably, in step S32, the spatial position perception specifically includes the following steps:

[0023] S321. Multi-scale Context Information Capture: Rich context information is captured under different receptive fields through multi-scale dilated convolutions, and then the extracted multi-scale features are fused through channel concatenation;

[0024] S322. Spatial Location Semantic Enhancement: To enhance the modeling ability of spatial location information, the module introduces position encoding and spatial attention mechanisms in features at different levels. The position encoding embeds position dependencies into intermediate feature maps by explicitly calculating the global semantic correlations of pixel positions. Then, the fused features are further refined through the spatial attention mechanism.

[0025] S323. Multi-scale Feature Fusion: The original features, multi-scale features, and spatially enhanced features are fused. Finally, after subsequent convolutional optimization, it becomes the input for downstream tasks.

[0026] Preferably, in the step S34, the specific steps of cross-scale attention fusion are as follows:

[0027] S341. Advanced Feature Extraction: Based on direction-aware context modeling driven by asymmetric convolutions, the directional context information in advanced features is fully captured;

[0028] By gradually performing 1x3 and 3x1 asymmetric convolution operations, more refined spatial information extraction is achieved, while significantly reducing the number of parameters and computational overhead.

[0029] S342. Adaptive Attention Mechanism: By dynamically adjusting the importance of features at different scales, the complementarity of cross-scale information is accurately captured;

[0030] Specifically, it includes advanced feature attention modeling, low-level feature attention modeling, and cross-scale feature weighted fusion;

[0031] Dynamic weight calculations are also performed on advanced features and low-level features respectively, and local and global information are captured through the same convolutional layer design, thereby enhancing the expressive power of advanced features and low-level features;

[0032] Finally, by combining the advanced and low-level attention weights, the dynamic fusion of cross-scale features is completed.

[0033] S343. Multi-scale Feature Fusion: The collaborative integration of global context is achieved. The fused cross-scale features and low-level features are concatenated in the channel dimension.

[0034] S344. Output Feature Enhancement: Fine-grained representation is achieved using asymmetric convolutions.

[0035] Preferably, in step S4, the preprocessed training dataset is used to train the model for multiple rounds. At the same time, the validation set is used to monitor the training progress, evaluate the performance of each training cycle, and adjust the hyperparameters according to the validation results. The specific steps are as follows:

[0036] S41: Data Loading and Batching: Batch load the preprocessed training dataset and validation set into the model;

[0037] S42: Training Settings: Set the initial learning rate, batch size, optimizer training hyperparameters, and select a loss function to measure the performance of the model;

[0038] S43: Validation Set Evaluation: In each training cycle, calculate the loss through forward propagation and perform backpropagation to update the model parameters. At the same time, use the validation set to evaluate the loss value and evaluation metrics of the model;

[0039] S44: Hyperparameter Adjustment: Adjust the hyperparameters according to the evaluation results of the validation set;

[0040] S45: Weight Saving: Whenever the evaluation metrics of the validation set improve, save the currently trained model parameters for subsequent loading and further training.

[0041] Preferably, an independent test set is used to evaluate the trained model, detect metal surface defects, and calculate evaluation metrics such as IoU, MIoU, and F1 score to verify the detection effect of the model.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] 1. Through the improved U-Net model of the present invention, combined with scale convolution, spatial attention mechanism, and position encoding, the accuracy and robustness of metal surface defect detection are improved.

[0044] 2. The present invention solves the problem of poor spatial information flow in metal surface defect detection, can effectively focus on the defect area and extract multi-scale features, and optimizes the detection effect.

[0045] 3. Compared with existing models, the present invention performs better in evaluation metrics such as IoU, MIoU, and F1 score, improving the accuracy of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flowchart of the present invention;

[0047] Figure 2 is a cropping effect diagram with a window of 256×256 and a step size of 256;

[0048] Figure 3 is a structural diagram of our model;

[0049] Figure 4 and Figure 5 are the segmentation effect diagrams of the comparison model. Specific embodiments

[0050] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0051] The present invention selects two publicly available datasets for verification.

[0052] Dataset

[0053] ① The NEU-Seg dataset contains 3,630 labeled training images and is suitable for segmentation tasks. It includes defect types on the surface of hot-rolled steel strips, such as inclusions, patches, and scratches. The size of each image is 200×200 pixels.

[0054] ② The USB-Seg dataset is used for surface defect detection of USB structural parts and is also suitable for segmentation tasks. The dataset contains a total of 430 images. The defect types include dents, deformations, scratches, stains, and spots. The size of each image is 200×200 pixels.

[0055] Embodiment

[0056] The dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1. During the training process, the size of the input image is 256×256 pixels. The optimizer uses Adaptive Moment Estimation with Weight Decay, the base learning rate is set to 0.001, and the learning rate decay method is Cosine Annealing. The total number of training epochs is 400, and the batch size is set to 16.

[0057] Experimental environment: The experiment is based on Ubuntu 22.04, uses PyTorch 2.1.2 and Python 3.10, and the CUDA version is 11.8. The hardware configuration includes an Intel(R) Xeon(R) Platinum 8375C processor with 12-core vCPU and an NVIDIA RTX 3090 GPU with 24GB video memory.

[0058] Use an independent test set to evaluate the trained model, detect metal surface defects, and calculate evaluation metrics such as IoU, MIoU, and F1-score to verify the detection effect of the model. We also compared several different models, including U-Net, Attention U-Net, U-Net 3+, Attention U-Net, Transunet, and VM-unet, and compared the evaluation metrics (such as IoU, MIoU, F1-score, etc.) of each model on the publicly available NEU-Seg [1] and USB-Seg [2] datasets, as shown in Tables 1 and 2.

[0059] The results show that the model of the present invention performs excellently in the defect detection task.

[0060] In addition, the present invention conducted ablation experiments to verify the impact of the improved modules (spatial position perception module and cross-scale attention fusion module) on the model performance, as shown in Table 3.

[0061] Table 1 Comparison results in the NEU-Seg dataset

[0062]

[0063] Table 2 Comparison results in the USB-Seg dataset

[0064]

[0065] Table 3 Ablation experiment results

[0066]

[0067] As is known by common technical knowledge, the present invention can be implemented by other embodiments that do not depart from its spiritual essence or essential features. Therefore, the above-disclosed embodiments are illustrative in all aspects and not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are encompassed by the present invention.

Claims

1. A method for detecting metal surface defects based on deep learning, characterized in that: It includes the following steps: S1. Data collection: Collect metal surface defect image data; S2. Data preprocessing: Preprocess the collected metal surface defect images to meet the input requirements of the model; S3. Model design: Use the improved U-Net to segment the images and extract the metal surface defect regions; S4. Model training and evaluation: Use the preprocessed training dataset to train the model for multiple rounds. At the same time, use the validation set to monitor the training progress, evaluate the performance of each training cycle, and adjust the parameters according to the validation results; S5. Defect detection: Use an independent test set to evaluate the trained model, detect metal surface defects, and calculate evaluation metrics such as intersection over union (IoU), mean intersection over union (mIoU), and F1 score to verify the detection effect of the model.

2. The method for detecting metal surface defects based on deep learning according to claim 1, wherein: In step S1, collect metal surface defect image data, and the image data includes different types of metal surface defects.

3. A method for preparing the method for detecting metal surface defects based on deep learning according to claim 1, characterized in that: In step S2, preprocess the collected metal surface defect images, which specifically includes the following steps: S21. Data augmentation: Horizontally flip the images, randomly adjust the brightness and contrast, and apply Gaussian blur to enhance data diversity; S22. Image cropping and size adjustment: For images with larger sizes, use a 256x256 window for cropping with a step size of 256 to ensure that all image sizes are unified to 256x256; S23. Normalization processing: Normalize the images to standardize the pixel values to a mean of 0 and a variance of 1 to help accelerate model training; S24. Dataset division: Divide the processed dataset into a training set, a validation set, and a test set in a ratio of 8:1:

1.

4. The method of a metal surface defect detection method based on deep learning according to claim 1, characterized in that: In step S3, use the improved U-Net to segment the images and extract the metal surface defect regions, which specifically includes the following steps: S31. Encoder processing: The input metal surface defect images pass through an encoder with successive downsampling to extract multi-level features of the images. The convolutional operations at each layer gradually abstract the information of the images into feature maps at different levels, providing rich low-level and high-level semantic features; S32. Spatial position perception: Based on the last layer of feature maps extracted by the encoder, a spatial position perception module is introduced; S33. Decoder processing: The processed feature maps enter the decoder stage. The decoder successively upsamples to restore the high resolution of the images. After each upsampling, the feature maps of the decoder are fused with the corresponding layer of feature maps of the encoder through skip connections; S34. Cross-scale attention fusion: For the feature maps after upsampling in the decoder stage, combined with the output of the skip connections of the encoder feature maps, perform high-level feature extraction and adaptive feature weighting.

5. The method for detecting metal surface defects based on deep learning according to claim 4, characterized in that: In step S32, the spatial position perception specifically includes the following steps: S321. Multi-scale context information capture: Capture rich context information under different receptive fields through multi-scale dilated convolutions, and then fuse the extracted multi-scale features through channel concatenation; S322. Spatial Location Semantic Enhancement: To enhance the modeling ability of spatial location information, the module introduces position encoding and spatial attention mechanism in different-level features. The position encoding embeds position dependence into the intermediate feature map by explicitly calculating the global semantic correlation of pixel positions. Then, the fused features are further refined by the spatial attention mechanism. S323. Multi-scale Feature Fusion: The original features, multi-scale features, and spatially enhanced features are fused. Finally, after subsequent convolutional optimization, it becomes the input for downstream tasks.

6. The method of a metal surface defect detection method based on deep learning according to claim 4, characterized in that: In step S34, the specific steps of cross-scale attention fusion are as follows: S341. High-level Feature Extraction: Based on direction-aware context modeling driven by asymmetric convolution, it fully captures the directional context information in high-level features; By gradually performing asymmetric convolution operations of 1x3 and 3x1, more refined spatial information extraction is achieved, while significantly reducing the number of parameters and computational overhead. S342. Adaptive Attention Mechanism: By dynamically adjusting the importance of features at different scales, it accurately captures the complementarity of cross-scale information; Specifically, it includes high-level feature attention modeling, low-level feature attention modeling, and cross-scale feature weighted fusion; Dynamic weight calculations are also performed on high-level and low-level features respectively, and local and global information are captured through the same convolutional layer design, thereby enhancing the expressive power of high-level and low-level features; Finally, by combining high-level and low-level attention weights, the dynamic fusion of cross-scale features is completed. S343. Multi-scale Feature Fusion: It realizes the collaborative integration of global context. The fused cross-scale features and low-level features are concatenated in the channel dimension. S344. Output Feature Enhancement: It uses asymmetric convolution to achieve refined representation.

7. The method for detecting metal surface defects based on deep learning according to claim 1, characterized in that: In step S4, the preprocessed training dataset is used to train the model for multiple rounds. At the same time, the validation set is used to monitor the training progress, evaluate the performance of each training cycle, and adjust the hyperparameters according to the validation results. The specific steps are as follows: S41: Data Loading and Batching: The preprocessed training dataset and validation set are loaded into the model in batches; S42: Training Settings: Set the initial learning rate, batch size, optimizer training hyperparameters, and select a loss function to measure the performance of the model; S43: Validation Set Evaluation: In each training cycle, calculate the loss through forward propagation and update the model parameters through backward propagation. At the same time, use the validation set to evaluate the loss value and evaluation metrics of the model; S44: Hyperparameter Adjustment: Adjust the hyperparameters according to the evaluation results of the validation set; S45: Weight Saving: Whenever the evaluation metrics of the validation set improve, save the currently trained model parameters for subsequent loading and further training.

8. The method for detecting metal surface defects based on deep learning according to claim 1, wherein: The trained model is evaluated using an independent test set to detect metal surface defects, and evaluation metrics such as IoU, MIoU, and F1 score are calculated to verify the detection effect of the model.

Images