Metal surface flaw detection method based on single-chip simulation and deep learning

Abstract

The invention discloses a metal surface flaw detection method based on monolithic simulation and deep learning, which comprises the following steps: firstly, decomposing an image into components of multiple scales by using a multi-scale Gaussian filter, and enhancing each illumination component based on a Retinex algorithm; secondly, generating an HDR image from the enhanced single LDR image by using an HDR simulation technology, and obtaining a single HDR image combining illumination distortion correction and HDR simulation; and finally, building a surface defect detection model, organically combining global context information and local detail information by adopting GCT attention, separately processing classification and regression tasks through Officient Decouled Head, optimizing the feature learning ability of each task, and adjusting the confidence score of a detection frame by adopting a Soft-NMS, so as to obtain a surface defect detection result. And keeping a high-confidence detection frame and reducing a confidence score of an overlapped detection frame. According to the method, the precision and reliability of metal surface flaw detection are improved, and the robustness is higher in the face of uneven illumination and complex backgrounds.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a metal surface defect detection method based on single-chip simulation and deep learning. Background Art

[0002] In actual industrial scenarios, due to the complexity of the lighting environment, the lighting conditions on the surface of the metal objects to be inspected often change significantly, resulting in lighting distortion in the captured images. Lighting distortion manifests itself in uneven brightness, heavy shadows, partial overexposure or underexposure, etc., making it impossible to accurately present the surface details of the metal objects to be inspected. This not only affects the visual quality of the image, but also affects subsequent image processing, analysis, and detection steps, making defect detection difficult.

[0003] In recent years, the development of machine vision has triggered a large number of studies on detecting surface defects under lighting distortion scenes, but they rely heavily on expert knowledge, and their detection accuracy and robustness are not ideal. In addition, these methods are also limited in their application scenarios and detection range. With the rapid development of deep learning, object detection technology has achieved remarkable success and has achieved remarkable results in image detection and segmentation, but it still faces challenges in practical applications, such as high algorithm complexity, high computational cost, and reduced accuracy and robustness of defect detection.

[0004] Therefore, the prior art needs to be further improved and developed. Summary of the invention

[0005] In view of the above-mentioned deficiencies of the prior art, the present invention provides a metal surface defect detection method based on single-chip simulation and deep learning to solve the problem of lighting distortion in industrial defect detection, comprising the following steps:

[0006] S1, uses a multi-scale Gaussian filter to decompose the image into components of multiple scales, and enhances each illumination component based on the Retinex algorithm.

[0007] S2, using HDR simulation technology to generate an HDR image from the enhanced single LDR image, expand the dynamic range to ensure the accuracy and stability of HDR simulation, and obtain a single HDR image combining illumination distortion correction and HDR simulation.

[0008] S3, builds a surface defect detection model, which includes an input end, a backbone network, a neck and a head. The input end uses mosaic data enhancement and comprehensive training samples to enhance the model's adaptability to complex scenes and improve the training speed; the backbone network consists of Focus, Conv, C3 and SPP modules for robust feature extraction; the neck utilizes the FPN+PANet framework with a CSP2 structure to effectively integrate and fuse various features; the head generates detection output results, where the three detection heads correspond to feature maps of different scales.

[0009] The metal surface defect detection method based on single-chip simulation and deep learning is characterized in that the step S1 specifically also includes: using a multi-scale Gaussian function to convolve the image with Gaussian functions of different scales to estimate the illumination component, after obtaining the illumination component, constructing a corresponding correction function, weighted summing the Retinex algorithm results of different scales to obtain the final enhancement result, the weight of the weighted fusion is determined according to the local contrast and color information of the pixel, and then the enhanced image is color restored, and after adjusting the brightness value, the hue (H) and saturation (S) of the color image are resynthesized, and RGB conversion is performed to obtain an image after illumination correction.

[0010] The metal surface defect detection method based on single-chip simulation and deep learning is characterized in that step S2 specifically also includes: using an S-curve equation to enhance the contrast between bright areas and dark areas with brightness in the middle range, converting a normally exposed image into an image with different brightness, thereby obtaining more image information.

[0011] The metal surface defect detection method based on monolithic simulation and deep learning is characterized in that the step S3 specifically also includes: using the GCT (Gaussian Context Transformer) attention mechanism to perform global average pooling on the input feature map to obtain the global information of each channel, and then using the Gaussian function to process the global information, and calculate the weight of each channel, so that the network is more focused on the features of the defect area while suppressing the interference of irrelevant information; using the Efficient Decoupled Head to process the classification and regression tasks separately, the classification head focuses on global features such as the shape and color of the defect, uses the feature map to capture the semantic information of the image, and classifies different types of defects, and the regression head focuses on local features such as the position and boundary of the defect, and uses a more fine-grained local feature map to locate the defect to ensure the accuracy of the position and size; using Soft-NMS (Soft Non-Maximum The confidence score of the detection frame is decreased by the Suppression function, which retains the high-confidence detection frame and reduces the confidence score of the overlapping detection frame. This solves the defects of the traditional NMS in processing overlapping detection frames and reduces the problems of false detection and missed detection caused by lighting.

[0012] The present invention provides a metal surface defect detection method based on single-chip simulation and deep learning, which combines Retinex lighting component correction, single-chip HDR simulation technology and deep learning method to solve problems such as uneven lighting, image lighting distortion and distinction of surface defect features, thereby improving the accuracy, stability and practicality of metal surface defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of a metal surface defect detection method based on single-chip simulation and deep learning in the present invention.

[0014] Figure 2 Network architecture diagram of the surface defect detection algorithm built DETAILED DESCRIPTION

[0015] The present invention provides a metal surface defect detection method based on single chip simulation and deep learning. In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0016] The present invention provides a metal surface defect detection method based on single chip simulation and deep learning, such as Figure 1 As shown, it includes the following steps:

[0017] S1, uses a multi-scale Gaussian filter to decompose the image into components of multiple scales, and enhances each illumination component based on the Retinex algorithm to eliminate local overexposure.

[0018]

[0019] in is convolution, (x, y) represents the image pixel coordinates, ω i is the corresponding weight parameter on scale i.

[0020] After obtaining the illumination component, the corresponding correction function is constructed, and the results of the Retinex algorithm at different scales are weighted and summed to obtain the final enhancement result. The weight of the weighted fusion is determined according to the local contrast and color information of the pixel. Then, the enhanced image is restored to enhance its color vividness and naturalness. After adjusting the brightness value, the hue (H) and saturation (S) of the color image are resynthesized, and RGB conversion is performed to obtain the image after illumination correction.

[0021] S2, in order to solve the problem of illumination distortion, uses HDR simulation technology to generate an HDR image from the enhanced single LDR image to expand the dynamic range to ensure the accuracy and stability of HDR simulation. Finally, a single HDR image combining illumination distortion correction and HDR simulation is obtained, providing richer and more accurate input for subsequent defect detection.

[0022] The S-curve is used to enhance the contrast between bright and dark areas in the middle brightness range, converting a normally exposed image into an image with different brightness, thereby obtaining more image information.

[0023] The S-curve equation is:

[0024]

[0025] L wk represents the brightness of the kth multi-exposure image generated; P K Used to control the brightness difference between adjacent exposure images; L d (i, j) represents the normalized brightness of the input image at pixel (i, j); L max，k represents the maximum brightness value in the kth different exposure images generated; L smax is a constant and is set to 382 in the simulation; L ad，k It is used to control the average brightness of the kth image with different exposures. The equation is:

[0026]

[0027] in, represents the exposure value of the Kth image, μ = 0.85. and P K This will produce different exposure images.

[0028] S3, build a surface defect detection model based on the YOLOv5 algorithm, such as Figure 2 As shown in the figure, the structure includes an input end, a backbone network, a neck and a head, wherein the input end utilizes mosaic data enhancement and comprehensive training samples to enhance the model's adaptability to complex scenes and improve the training speed; the backbone network consists of Focus, Conv, C3 and SPP modules for robust feature extraction; the neck utilizes the FPN+PANet framework with a CSP2 structure to effectively integrate and fuse various features; the head generates detection output results, wherein the three detection heads correspond to feature maps of different scales.

[0029] The GCT (Gaussian Context Transformer) attention mechanism is used to perform global average pooling on the input feature map to obtain the global information of each channel. Then, the Gaussian function is used to process this global information and calculate the weight of each channel, so that the network can focus more on the features of the defect area while suppressing the interference of irrelevant information. When processing images with uneven illumination, GCT can effectively deal with the impact of illumination changes on feature extraction. By dynamically adjusting the weights of each channel, the network can focus more on the features of the defect area, thereby improving the accuracy and reliability of detection. The design of GCT is very lightweight and has low computational complexity. It is suitable for application in real-time detection systems and ensures detection efficiency.

[0030] The classification and regression tasks are processed separately using the Efficient Decoupled Head. The classification head focuses on global features such as the shape and color of the defect, uses feature maps to capture the semantic information of the image, and classifies different types of defects. The regression head focuses on local features such as the location and boundaries of the defect, and uses more fine-grained local feature maps to locate the defect to ensure the accuracy of the location and size. The Efficient Decoupled Head optimizes the feature learning ability of each task, enabling the network to more accurately identify defects in the classification task and more accurately locate defects in the regression task, thereby improving the overall detection accuracy and efficiency.

[0031] In the inference stage, the Soft-NMS (Soft Non-Maximum Suppression) function is used to reduce the confidence scores of the detection frames, retain high-confidence detection frames and reduce the confidence scores of overlapping detection frames. This solves the defects of traditional NMS in processing overlapping detection frames and reduces false detections and missed detections caused by lighting.

[0032] The traditional NMS algorithm iteratively performs IoU operations with the box with the maximum score and other boxes, and filters out those boxes with larger IoU. Although this method is effective, it has two main problems: first, for adjacent detection boxes, NMS forces their scores to zero, resulting in failure to detect real objects in overlapping areas, thereby reducing the average detection rate; second, the threshold of NMS is difficult to determine. If it is set too small, it will be mistakenly deleted, and if it is set too high, it is easy to misdetect. Soft-NMS adjusts the confidence score of the detection box through function operation, rather than simply deleting the box with IoU greater than the threshold. Soft-NMS can be easily integrated into existing object detection algorithms without retraining the model.

[0033] The model performance is evaluated by three coefficients: precision P (precision), recall R (recall), and mean AP (mean AP).

[0034]

[0035]

[0036]

[0037] TP (true positive) is the positive sample that is correctly marked as a positive sample in the prediction result, TN (true negative) is the negative sample that is correctly marked as a negative sample in the prediction result, FP (false positive) is the positive sample that is incorrectly marked as a negative sample in the prediction result, FN (false negative) is the negative sample that is incorrectly marked as a positive sample in the prediction result, AP i represents the average precision of the i-th category.

[0038] The hot-rolled strip surface defect dataset NEU-DET published by Northeastern University is used for training. The dataset contains 1,800 grayscale images, including cracks, inclusions, plaques, pitting surfaces, rolling scales, and scratches. Each defect has 300 images. The results are shown in the following table:

[0039]

[0040] From the above results, it can be seen that the optimized YOLOv5 algorithm has significant improvements in precision, recall and mAP. The overall mAP50 of the optimized model reaches 72.9%, which is 4.2% higher than 68.7% before optimization. This shows that this method is a comprehensive and effective defect detection method, which improves the accuracy, stability and practicality of metal surface defect detection and provides valuable insights for further research and application in the field of industrial defect detection.

Claims

1. A metal surface defect detection method based on monolithic simulation and deep learning, characterized in that: The following steps are involved: S1, uses a multi-scale Gaussian filter to decompose the image into components of multiple scales, and enhances each illumination component based on the Retinex algorithm to eliminate local overexposure; S2, using HDR simulation technology to generate an HDR image from the enhanced single LDR image, expanding the dynamic range, and obtaining a single HDR image that combines illumination distortion correction and HDR simulation; S3, build a surface defect detection model, which includes an input end, a backbone network, a neck, and a head. The input end is enhanced with mosaic data and comprehensive training samples to enhance the model's adaptability to complex scenarios and improve training speed; The backbone network consists of Focus, Conv, C3 and SPP modules for robust feature extraction; the neck utilizes the FPN+PANet framework with CSP2 structure to effectively integrate and fuse various features; the head generates detection output results, where the three detection heads correspond to feature maps of different scales.

2. The metal surface defect detection method based on single chip simulation and deep learning according to claim 1, characterized in that: The step S1 specifically also includes: using a multi-scale Gaussian function to convolve the image with Gaussian functions of different scales to estimate the illumination component, after obtaining the illumination component, constructing a corresponding correction function, weighting and summing the results of the Retinex algorithms of different scales to obtain the final enhancement result, the weight of the weighted fusion is determined according to the local contrast and color information of the pixel, and then color restoration is performed on the enhanced image, after adjusting the brightness value, the hue (H) and saturation (S) of the color image are resynthesized, and RGB conversion is performed to obtain an image after illumination correction.

3. The metal surface defect detection method based on single chip simulation and deep learning according to claim 1, characterized in that: The step S2 specifically also includes: using an S-curve equation to enhance the contrast between bright areas and dark areas with brightness in the middle range, converting a normally exposed image into an image with different brightness, thereby obtaining more image information.

4. The metal surface defect detection method based on single chip simulation and deep learning according to claim 1 is characterized in that: The step S3 specifically also includes: using the GCT (Gaussian Context Transformer) attention mechanism to perform global average pooling on the input feature map to obtain the global information of each channel, and then using the Gaussian function to process the global information and calculate the weight of each channel, so that the network can focus more on the features of the defect area while suppressing the interference of irrelevant information; using the Efficient Decoupled Head to process the classification and regression tasks separately, the classification head focuses on global features such as the shape and color of the defect, uses the feature map to capture the semantic information of the image, and classifies different types of defects, and the regression head focuses on local features such as the position and boundary of the defect, and uses a more fine-grained local feature map to locate the defect to ensure the accuracy of the position and size; in the inference stage, the Soft-NMS (Soft Non-Maximum Suppression) function operation is used to reduce the confidence score of the detection box, retain the high confidence detection box and reduce the confidence score of the overlapping detection box.

Images