Surface defect detection method for multi-scale and small target features

Abstract

A surface defect detection method for multi-scale and small target features comprises the following steps: step 1, acquiring a surface defect data set, converting the format of a labeling file into a required format, and dividing an image into a training set, a verification set and a test set; step 2, constructing a DMS-YOL O surface defect detection model; 3, training the constructed DMS-YOLO surface defect detection model, and training the surface defect detection model by using the training set and the verification set to obtain an optimal surface defect detection model; 4, performing evaluation on the test data set through the trained model; 5, fully testing and debugging the model trained in the step 3 to ensure the accuracy and stability of the surface defect detection method in different scenes; and step 6, deploying the trained DMS-YOLO surface defect detection model in an actual industrial production environment, performing defect detection on the surface of a real product, and recording the accuracy rate, the false detection rate and the missing detection rate of a detection result. And 7, summarizing the whole detection process, forming a standardized solution, and popularizing the standardized solution to different industrial fields for application verification to ensure that the method has practicability and stability.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, in particular to computer vision and target detection technology, and specifically to a method for detecting road surface defects. Background Art

[0002] As modern society relies more and more on various infrastructure, industrial equipment and public facilities, ensuring the safety, reliability and efficient operation of these facilities has become an urgent problem to be solved in various industries. Surface defects of facilities, especially small cracks, corrosion and wear, are often potential hidden dangers that cause major safety accidents. Therefore, timely detection and effective treatment of these defects are of vital importance to maintaining the normal operation of facilities, extending service life, ensuring personnel safety and reducing maintenance costs.

[0003] Surface defect detection refers to the use of technical means to identify and locate various defects on the surface of an object, such as cracks, scratches, and bubbles. These defects may have a significant impact on the quality of products and the safety of facilities. For example, microcracks or corrosion may appear on the surface of steel structures and bridges. If not discovered in time, it will lead to structural damage and potential safety hazards; there may be cracks or potholes on the road surface of highways, which not only affects driving safety, but may also accelerate further damage to the road surface; in semiconductor manufacturing, if there are tiny scratches or particle contamination on the surface of the wafer, it will affect the production quality of the chip and even cause the entire production batch to be discarded; similarly, if there are tiny cracks or bubbles on the surface of contact lenses, it will directly affect the wearer's comfort and eye health, and even cause visual damage in severe cases.

[0004] Early defect detection mainly relied on manual inspection, which has the problems of long monitoring cycle, low efficiency and high cost. Traditional automated defect detection relies on image processing technology. Researchers use traditional image processing technologies such as threshold and edge detection for defect detection. However, these methods rely on manual extraction of defect features and have poor robustness when facing complex and changing scenes.

[0005] In recent years, with the development of computer vision and artificial intelligence technology, target detection algorithms based on deep learning have been widely used in various defect detection fields. The paper "Pavement Crack Detection Method with Effective Feature Extraction and Cascade Optimization" discloses a highway crack detection method that combines effective multi-scale feature fusion and cascade optimization structure. In response to the problems of crack polymorphism and noise interference, this method first constructs a multi-scale feature extraction structure to enhance the ability to capture crack features of different scales; secondly, it introduces spatial and channel attention mechanisms to suppress background noise interference and strengthen crack feature expression; finally, by jointly optimizing and learning decoding features at different levels, the clarity and integrity of crack segmentation patterns are improved. Although many defect detection models have been proposed, the following technical problems still need to be solved:

[0006] 1) The defect shapes and sizes vary greatly, and it is difficult for the model to accurately extract defect features;

[0007] 2) When the model processes features of different scales, it is difficult to retain both global information and detailed features, resulting in limited ability to detect targets in complex backgrounds, and prone to false detection and missed detection;

[0008] 3) The model cannot strike a balance between low computational complexity and high accuracy, making it difficult to achieve efficient deployment in practical applications.

[0009] Therefore, the applicant proposes a surface defect detection method for multi-scale and small target features to solve the above technical problems. Summary of the invention

[0010] The purpose of the present invention is to solve the technical problems in the prior art that multi-scale and deformable target detection models are difficult to simultaneously take into account global semantic information and detail feature fusion in complex road scenes, and have low recognition rate for small targets and complex background defects, and to provide a surface defect detection method for multi-scale and small target features.

[0011] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0012] A surface defect detection method for multi-scale and small target features, comprising the following steps:

[0013] Step 1: Obtain the surface defect dataset, convert the format of the annotation file to the required format and divide the images into training set, validation set and test set, set the initial learning rate and training rounds;

[0014] Step 2: Build a DMS-YOLO surface defect detection model;

[0015] Step 3: Train the constructed DMS-YOLO surface defect detection model, and use the training set and the validation set to train the surface defect detection model to obtain the optimal surface defect detection model;

[0016] Step 4: Evaluate the trained model on the test dataset;

[0017] Step 5: Fully test and debug the model trained in step 3 to ensure the accuracy and stability of the surface defect detection method in different scenarios;

[0018] Step 6: Deploy the trained DMS-YOLO surface defect detection model to the actual industrial production environment, perform defect detection on the surface of real products, and record the accuracy, false detection rate, and missed detection rate of the detection results.

[0019] It also includes step 7: summarize the entire detection process, form a standardized solution, and promote it to different industrial fields for application verification to ensure the practicality and stability of the method.

[0020] In step 2, the constructed DMS-YOLO surface defect detection model includes a backbone feature extraction network Backbone and a neck network Neck; specifically:

[0021] The input image first undergoes image enhancement preprocessing, and then passes through the first and second convolution Conv layers of the backbone feature extraction network Backbone in sequence, and the feature map is passed to the third C2f module of the backbone feature extraction network Backbone; the output feature map passes through the fourth convolution layer of the backbone feature extraction network Backbone and is passed to the fifth C2f module, and then passes through the sixth convolution layer, the seventh C2f module, the eighth convolution layer, and the ninth C2f module in sequence; finally, the feature map is input to the tenth SPPF module of the backbone feature extraction network Backbone to complete the feature extraction of the backbone feature extraction network Backbone; in this process, the fifth, seventh, and tenth layers of the backbone feature extraction network Backbone output feature maps respectively as the input of the subsequent neck network Neck module;

[0022] In the neck network Neck part, the feature maps output by the fifth, seventh and tenth layers of the backbone feature extraction network Backbone are first input together into the first layer PCE module in the neck network Neck for preliminary feature fusion; then, the feature map of the tenth layer of the backbone feature extraction network Backbone and the output feature map of the first layer PCE module are input together into the second layer HWFEM module, and more refined feature expression is achieved through high-dimensional feature weighting; then, the feature map output by the second layer HWFEM module and the feature map of the seventh layer of the backbone feature extraction network Backbone are combined into a single feature map.Figure 1 Then, the feature map output by the first-layer PCE module of the neck network Neck is combined with the output feature map of the third-layer DIF module. Figure 1 The output feature maps are input into the fourth-layer HWFEM module together with the feature maps of the fifth layer of the backbone feature extraction network Backbone, and then the output feature maps are input into the fifth-layer DIF module together with the output of the fifth-layer DIF module, which combines the deep semantics and local detail information. Subsequently, the output feature maps of the first-layer PCE module and the fifth-layer DIF module are input into the sixth-layer HWFEM module together. The output feature maps of the sixth layer are passed to the detection head as the first feature maps, and are also input into the seventh-layer convolution module to further adjust the feature scale. The feature maps of the seventh-layer convolution output and the output feature maps of the fourth-layer HWFEM module are fused through the Concat operation and then passed to the ninth-layer C2f module to obtain the second feature maps for the detection head.

[0023] In addition, the feature map of the ninth layer of the neck network Neck is processed by the tenth layer convolution module, and then fused with the output feature map of the second layer HWFEM module through the Concat operation. The fusion result is input into the twelfth layer C2f module to generate the third feature map for the detection head. Finally, the three feature maps from the neck part of the neck network are sent to the AMS-Dyhead detection head.

[0024] The three-way feature map obtained by the neck network Neck part is input to the AMS-Dyhead detection head part to start detection. During the detection process, the three-way feature map is processed by the scale attention, spatial attention and task attention modules respectively to achieve multi-scale and multi-resolution feature optimization. The scale attention module first performs global pooling on the feature map to extract global context information, and then generates a feature weight map through the convolution Conv layer and the activation function layer (ReLU and Sigmoid). The spatial attention module uses differentiated convolution operations to process feature maps of different resolutions, where high-resolution feature maps use DCNv2 convolution and low-resolution feature maps use DCNv3 convolution to adapt to different feature structures; the processed feature map calculates the offset through the offset module, and then adjusts the feature expression through the Sigmoid activation function; the task attention module first extracts features through global pooling, then passes through two layers of fully connected layers and activation function (ReLU), and outputs the task weight map in the Normalize layer; finally, the output of the scale attention part is combined with the task attention module to achieve more fine-grained and multi-level feature expression, thereby optimizing the target detection effect.

[0025] The CIoU loss function used is:

[0026]

[0027] L Focaler-IoU =1-IoU focaler

[0028] Where IoU refers to the intersection over union ratio, IoU focaler It refers to the intersection-over-union ratio corrected according to the Focaler-IoU idea, L Focaler-IoU Refers to the final loss function value; when IoU is lower than the lower threshold d, the loss is 0; when IoU exceeds the upper threshold u, the loss is 1; and when IoU is between these two thresholds, the loss increases linearly with the increase of IoU value. This design makes the loss function more sensitive to the change of IoU value in a specific interval, so as to pay more attention to samples with moderate bounding box overlap, which helps the model to extract features from medium-difficulty samples more effectively, rather than just focusing on easy or difficult samples.

[0029] The PCE module structure is as follows Figure 4 As shown, the PCE module structure is as follows:

[0030] The input feature map first undergoes an average pooling Avg Pool operation to extract global feature information, and then uses a depth-separable convolution Dw_Conv in two directions to extract context information in the height direction (Pool_h) and the width direction (Pool_w). The features in these two directions are fused into a new feature map by adding the elements layer by layer. The fused feature map is further enhanced by the Excite module and enters the Self_Attention mechanism part to generate global feature weights and perform weighted adjustment on important features. Subsequently, the feature map processed by the self-attention mechanism is separated into two-directional channels through the Split operation, and batch normalization BatchNorm2d and activation ReLU are performed respectively to improve the stability and nonlinear ability of feature expression. Finally, these feature channels are further adjusted to a feature value range of [0,1] through the Sigmoid activation function to generate a more refined feature representation for use in subsequent network layers.

[0031] The HWFEM module structure is as follows Figure 5As shown in the figure, the two input feature maps are respectively subjected to the convolution Conv operation to extract feature expressions of different scales, and then feature alignment is performed through specific operations. The aligned feature maps are fused through point-by-point operations and enter the act module for activation processing; in the act module, the feature map first generates weight features through bilinear operations, and is adjusted to the [0,1] range through the Sigmoid function to generate adaptive feature weights. Finally, the weights are weighted and fused with the input feature map through point-by-point multiplication operations, and the enhanced feature map is output, retaining key information for subsequent processing.

[0032] AMS-Dyhead structure diagram Figure 6 As shown in the figure, the three-way feature maps obtained by the neck network Neck are processed by the scale attention, spatial attention and task attention modules respectively to achieve multi-scale and multi-resolution feature optimization. The scale attention module first performs global pooling on the feature map to extract global context information, and then generates a feature weight map through the convolution Conv layer and the activation function layer (ReLU and Sigmoid). The spatial attention module uses differentiated convolution operations to process feature maps of different resolutions, where high-resolution and medium-resolution feature maps use DCNv2 convolution, and low-resolution feature maps use DCNv3 convolution to adapt to different feature structures; the processed feature map calculates the offset through the offset module, and then adjusts the feature expression through the Sigmoid activation function; the task attention module first extracts features through global pooling, then passes through two layers of fully connected layers and activation function (ReLU), and outputs the task weight map in the Normalize layer; finally, the output of the scale attention part is combined with the task attention module to achieve more fine-grained and multi-level feature expression, thereby optimizing the target detection effect.

[0033] Compared with the prior art, the present invention has the following technical effects:

[0034] 1) This paper reconstructs the neck network of the YOLO model and designs a pyramid context extraction module to integrate feature information at different levels, improve the model's ability to perceive global information, and emphasize key features. In addition, a hierarchical weighted fusion enhancement module is designed to dynamically adjust the strength of the feature map, further enhancing the model's ability to represent multi-scale features in complex backgrounds;

[0035] 2) The present invention designs an adaptive multi-scale dynamic detection head, which introduces deformable convolution to enhance the model's ability to capture complex shapes and irregular boundaries, and designs a collaborative attention mechanism that combines scale attention with task attention to improve the model's adaptability to changes in target shape and posture;

[0036] 3) The present invention applies the Focaler-IoU idea to improve the CIoU loss function, improve the category imbalance problem, and enhance the model's ability to detect small targets, thereby improving the overall detection performance;

[0037] 4) Compared with the YOLOv8n network, the mAP50 of the DMS-YOLO method of the present invention reaches 87.9%, which is 3% higher than the original benchmark model. At the same time, the number of parameters is 3.67M, the amount of calculation is 8GFLOPs, and the model size is only 7.3MB, which has lightweight characteristics and easy deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0039] Figure 1 This is a schematic diagram of the unimproved YOLOv8 network structure.

[0040] Figure 2 Schematic diagram of the network structure of the DMS-YOLO method of the present invention.

[0041] Figure 3 Schematic diagram of the structure of ACA-FPN in the present invention.

[0042] Figure 4 This is a schematic diagram of the PCE module structure in the present invention.

[0043] Figure 5 It is a schematic diagram of the HWFEM module structure in the present invention.

[0044] Figure 6 It is a schematic diagram of the structure of AMS-Dyhead in the present invention. DETAILED DESCRIPTION

[0045] A surface defect detection method for multi-scale and small target features, comprising the following steps:

[0046] Step 1: Obtain the surface defect dataset, convert the format of the annotation file to the required format and divide the images into training set, validation set and test set, set the initial learning rate and training rounds;

[0047] Step 2: Build a DMS-YOLO surface defect detection model;

[0048] Step 3: Train the constructed DMS-YOLO surface defect detection model, and use the training set and the validation set to train the surface defect detection model to obtain the optimal surface defect detection model;

[0049] Step 4: Evaluate the trained model on the test dataset;

[0050] Step 5: Fully test and debug the model trained in step 3 to ensure the accuracy and stability of the surface defect detection method in different scenarios;

[0051] Step 6: Deploy the trained DMS-YOLO surface defect detection model to the actual industrial production environment, perform defect detection on the surface of real products, and record the accuracy, false detection rate, and missed detection rate of the detection results;

[0052] Step 7: Summarize the entire testing process, form a standardized solution, and promote it to different industrial fields for application verification to ensure the practicality and stability of the method;

[0053] In step 2, the constructed DMS-YOLO surface defect detection model includes a backbone feature extraction network Backbone and a neck network Neck; specifically:

[0054] The input image first undergoes image enhancement preprocessing, and then passes through the first and second convolution Conv layers of the backbone feature extraction network Backbone in sequence, and the feature map is passed to the third C2f module of the backbone feature extraction network Backbone; the output feature map passes through the fourth convolution layer of the backbone feature extraction network Backbone and is passed to the fifth C2f module, and then passes through the sixth convolution layer, the seventh C2f module, the eighth convolution layer, and the ninth C2f module in sequence; finally, the feature map is input to the tenth SPPF module of the backbone feature extraction network Backbone to complete the feature extraction of the backbone feature extraction network Backbone; in this process, the fifth, seventh, and tenth layers of the backbone feature extraction network Backbone output feature maps respectively as the input of the subsequent neck network Neck module;

[0055] In the neck network Neck part, the feature maps output by the fifth, seventh and tenth layers of the backbone feature extraction network Backbone are first input together into the first layer PCE module in the neck network Neck for preliminary feature fusion; then, the feature map of the tenth layer of the backbone feature extraction network Backbone and the output feature map of the first layer PCE module are input together into the second layer HWFEM module, and more refined feature expression is achieved through high-dimensional feature weighting; then, the feature map output by the second layer HWFEM module and the feature map of the seventh layer of the backbone feature extraction network Backbone are combined into a single feature map. Figure 1 Then, the feature map output by the first-layer PCE module of the neck network Neck is combined with the output feature map of the third-layer DIF module. Figure 1The output feature maps are input into the fourth-layer HWFEM module together with the feature maps of the fifth layer of the backbone feature extraction network Backbone, and then the output feature maps are input into the fifth-layer DIF module together with the output of the fifth-layer DIF module, which combines the deep semantics and local detail information. Subsequently, the output feature maps of the first-layer PCE module and the fifth-layer DIF module are input into the sixth-layer HWFEM module together. The output feature maps of the sixth layer are passed to the detection head as the first feature maps, and are also input into the seventh-layer convolution module to further adjust the feature scale. The feature maps of the seventh-layer convolution output and the output feature maps of the fourth-layer HWFEM module are fused through the Concat operation and then passed to the ninth-layer C2f module to obtain the second feature maps for the detection head.

[0056] In addition, the feature map of the ninth layer of the neck network Neck is processed by the tenth layer convolution module, and then fused with the output feature map of the second layer HWFEM module through the Concat operation. The fusion result is input into the twelfth layer C2f module to generate the third feature map for the detection head. Finally, the three feature maps from the neck part of the neck network are sent to the AMS-Dyhead detection head.

[0057] The three-way feature map obtained by the neck network Neck part is input to the AMS-Dyhead detection head part to start detection. During the detection process, the three-way feature map is processed by the scale attention, spatial attention and task attention modules respectively to achieve multi-scale and multi-resolution feature optimization. The scale attention module first performs global pooling on the feature map to extract global context information, and then generates a feature weight map through the convolution Conv layer and the activation function layer (ReLU and Sigmoid). The spatial attention module uses differentiated convolution operations to process feature maps of different resolutions, where high-resolution feature maps use DCNv2 convolution and low-resolution feature maps use DCNv3 convolution to adapt to different feature structures; the processed feature map calculates the offset through the offset module, and then adjusts the feature expression through the Sigmoid activation function; the task attention module first extracts features through global pooling, then passes through two layers of fully connected layers and activation function (ReLU), and outputs the task weight map in the Normalize layer; finally, the output of the scale attention part is combined with the task attention module to achieve more fine-grained and multi-level feature expression, thereby optimizing the target detection effect.

[0058] The CIoU loss function used is:

[0059]

[0060] L Focaler-IoU =1-IoU focaler

[0061] Where IoU refers to the intersection over union ratio, IoU focaler It refers to the intersection-over-union ratio corrected according to the Focaler-IoU idea, L Focaler-IoU Refers to the final loss function value; when IoU is lower than the lower threshold d, the loss is 0; when IoU exceeds the upper threshold u, the loss is 1; and when IoU is between these two thresholds, the loss increases linearly with the increase of IoU value. This design makes the loss function more sensitive to the change of IoU value in a specific interval, so as to pay more attention to samples with moderate bounding box overlap, which helps the model to extract features from medium-difficulty samples more effectively, rather than just focusing on easy or difficult samples.

[0062] After the network construction is completed, the trained model is used to test on the test set. The test uses indicators such as precision (Precision, P), recall (Recall, R), mean average precision (mean Average Precision, mAP), F1-score, parameters (Parame ters), and computational complexity (GFLOPs) to evaluate the performance of the proposed method DMS-YOLO.

[0063] The calculation formula of Precision is as follows:

[0064]

[0065] The calculation formula of Recall is as follows:

[0066]

[0067] Among them, TP represents the number of target objects correctly detected by the model, FN represents the number of target objects missed by the model, and FP represents the number of target objects misdetected.

[0068] The calculation formula of mAP is as follows:

[0069]

[0070] The average precision (AP) of each category is usually regarded as the area under the precision-recall curve, and mAP is the average of APs of all categories. mAP50 refers to the accuracy that the model can achieve when the prediction confidence is 0.5, and mAP50-95 refers to the accuracy that the model can achieve when the prediction confidence is between 0.5 and 0.95.

[0071] The calculation formula of F1-score is as follows:

[0072]

[0073] F1-score is a weighted average of precision and recall, which is used to measure the stability of the model.

[0074] The PCE module structure is as follows Figure 4 As shown, the PCE module structure is as follows:

[0075] The input feature map first undergoes an average pooling Avg Pool operation to extract global feature information, and then uses a depth-separable convolution Dw_Conv in two directions to extract context information in the height direction (Pool_h) and the width direction (Pool_w). The features in these two directions are fused into a new feature map by adding the elements layer by layer. The fused feature map is further enhanced by the Excite module and enters the Self_Attention mechanism part to generate global feature weights and perform weighted adjustment on important features. Subsequently, the feature map processed by the self-attention mechanism is separated into two-directional channels through the Split operation, and batch normalization BatchNorm2d and activation ReLU are performed respectively to improve the stability and nonlinear ability of feature expression. Finally, these feature channels are further adjusted to a feature value range of [0,1] through the Sigmoid activation function to generate a more refined feature representation for use in subsequent network layers.

[0076] The HWFEM module structure is as follows Figure 5 As shown in the figure, the two input feature maps are respectively subjected to the convolution Conv operation to extract feature expressions of different scales, and then feature alignment is performed through specific operations. The aligned feature maps are fused through point-by-point operations and enter the act module for activation processing; in the act module, the feature map first generates weight features through bilinear operations, and is adjusted to the [0,1] range through the Sigmoid function to generate adaptive feature weights. Finally, the weights are weighted and fused with the input feature map through point-by-point multiplication operations, and the enhanced feature map is output, retaining key information for subsequent processing.

[0077] AMS-Dyhead structure diagram Figure 6As shown in the figure, the three-way feature maps obtained by the neck network Neck are processed by the scale attention, spatial attention and task attention modules respectively to achieve multi-scale and multi-resolution feature optimization. The scale attention module first performs global pooling on the feature map to extract global context information, and then generates a feature weight map through the convolution Conv layer and the activation function layer (ReLU and Sigmoid). The spatial attention module uses differentiated convolution operations to process feature maps of different resolutions, where high-resolution and medium-resolution feature maps use DCNv2 convolution, and low-resolution feature maps use DCNv3 convolution to adapt to different feature structures; the processed feature map calculates the offset through the offset module, and then adjusts the feature expression through the Sigmoid activation function; the task attention module first extracts features through global pooling, then passes through two layers of fully connected layers and activation function (ReLU), and outputs the task weight map in the Normalize layer; finally, the output of the scale attention part is combined with the task attention module to achieve more fine-grained and multi-level feature expression, thereby optimizing the target detection effect.

[0078] In summary, the DMS-YOLO method of the present invention realizes the global fusion and dynamic weighting of multi-scale features by designing an adaptive context-aware feature pyramid network, which significantly improves the model's perception and expression capabilities of complex defects. Compared with the existing mainstream feature pyramid network, it shows obvious advantages in accuracy and computational efficiency. Secondly, an adaptive multi-scale dynamic detection head is proposed, and deformable convolution (DCNv3) is used to improve the model's ability to capture complex shape features, and a collaborative attention mechanism is designed to fuse scale and task attention to enhance the model's understanding of multi-scale information. Finally, the CIoU loss function is improved using the Focaler-IoU idea to improve the detection capability of small targets.

[0079] Example:

[0080] The present invention will show some examples of using the improved model to detect road surface defects. The DMS-YOLO surface defect detection method includes the following steps.

[0081] Step 1: Select 4378 images from the road defect image RDD2022 dataset, convert them into the required YOLO format, and divide them into training set, validation set, and test set in a ratio of 8:1:1. The initial learning rate is 0.01 and the number of training rounds is 300.

[0082] Step 2: Based on YOLOv8n, a DMS-YOLO network is constructed, the neck network is reconstructed, and an ACA-FPN feature fusion network is designed. The PCE module is designed to adopt a global multi-scale fusion strategy, directly input feature maps from different scales at the same time, and process them as a whole to retain more context information; the HWFEM module is designed to adopt a weighted guidance and layer-by-layer stacking strategy, and through adaptive weighting and efficient information transfer, the expression of low-resolution global semantic information in multi-scale features is enhanced to ensure the integrity and discrimination of features in the fusion process; and a new network is constructed using a better Focaler-IoU loss function to further improve the detection accuracy of the network;

[0083] Step 3: Use the training set and validation set divided in step 1 to train the model;

[0084] Step 4: Detect surface defects on the test data set and evaluate the model through evaluation indicators;

[0085] Step 5: Test and debug the model obtained in step 3 on the SVRDD dataset to ensure the accuracy and robustness of the surface defect detection algorithm in different scenarios;

[0086] Step 6: Deploy the trained DMS-YOLO model to the actual industrial production environment, perform defect detection on the surface of real products, and record the accuracy, false detection rate, and missed detection rate of the detection results;

[0087] Step 7: Summarize the entire testing process, form a standardized solution, and promote it to different industrial fields for application verification to ensure the practicality and stability of the method.

[0088] The above steps can achieve accurate detection of surface defects.

[0089] The present invention selects road defect images from China in the RDD2022 dataset. The selected images cover five types of road defects, namely D00 (longitudinal cracks), D10 (transverse cracks), D20 (reticular cracks), D40 (potholes) and Repair, with a total of 4378 images and 8764 instances. The DMS-YOLO model is trained 300 times, with a batch_size of 32, an initial learning rate of 0.01, and an SGD optimizer is used to optimize the weight parameters. The momentum parameter is set to 0.937 and the decay coefficient is 0.005.

[0090] In order to verify the effectiveness of various improvements in the DMS-YOLO method of the present invention in surface defect detection, the ablation experiment results with various improvements are shown in Table 1 below.

[0091] Table 1 Comparison of ablation experiment results

[0092]

[0093] As shown in Table 1, compared with the original YOLOv8n, the ACA-FPN network proposed in the present invention has a slight increase in the number of parameters, but the mAP50 is improved by 1.4%, and the amount of calculation is reduced by 5%; the improved detection head AMS-Dyhead can greatly improve the detection accuracy of the model, the mAP50 is improved by 1.6%, and the number of parameters is reduced by 6%; the bounding box regression loss function in the original YOLOv8 network is redesigned as Focaler-IoU, while keeping the number of model parameters and the amount of calculation unchanged, the mAP50 is improved by 0.7%; the two improved modules of ACA-FPN and AMS-Dyhead are integrated, the mAP50 is improved by 2.5%, and the number of parameters is slightly improved; finally, the three improved strategies of ACA-FPN, AMS-Dyhead and Focaler-IoU are introduced at the same time to construct the DMS-YOLO model, the average accuracy of the model reaches 87.9%, and the accuracy is improved by 3% while the amount of calculation is reduced.

[0094] In order to further evaluate the effectiveness of the proposed method DMS-YOLO, the improved model was compared with other mainstream target detection models on the RDD2022 dataset, including the two-stage anchor box model Faster R-CNN, the one-stage anchor box model SSD, YOLOv5n, the one-stage anchor box-free model YOLOv8s, YOLOv10n and YOLOv11n, and the Transformer-based RT-detr model. The experiments were conducted under the same dataset and experimental conditions, and the models have converged. The specific experimental results are shown in Table 2.

[0095] Table 2 Experimental comparison of mainstream models on the RDD2022 dataset

[0096]

[0097] From the experimental results in Table 2, the detection performance of DMS-YOLO on the RDD2022 dataset is better than other mainstream detection models. The experimental results show that on the basis of reducing the amount of calculation, the improved model has a mAP50 of 87.9% on the RDD2022 dataset, which is 3% higher than the original baseline model. At the same time, the number of parameters is 3.67M, the amount of calculation is 8GFLOPs, and the model size is only 7.3MB. It has lightweight characteristics and easy deployment, which has practical application significance for surface defect detection.

[0098] In order to verify the accuracy and robustness of the DMS-YOLO method in different scenarios, this study selected the public dataset SVRDD for testing. The dataset covers 8,000 images of road damage in five districts of Beijing, covering different environmental interferences and various lighting conditions. There are 7 categories of road damage types in the dataset, namely longitudinal cracks, transverse cracks, cracks, potholes, longitudinal repairs, transverse repairs and manhole covers. Table 3 shows the experimental results of the improved model on this dataset.

[0099] It can be seen from Table 3 that the improved model has significantly improved mAP50, mAP50-95, Recall and F1-score by 1.7%, 0.8%, 2.7% and 1% respectively in the SVRDD dataset under the premise of reducing the amount of calculation. This verifies the universality and robustness of the proposed method DMS-YOLO.

[0100] Table 3 Experimental results of the improved model on the SVRDD dataset

[0101]

[0102] The method DMS-YOLO of the present invention proposes a series of improvement strategies to cope with the above-mentioned shortcomings and improve its performance in defect detection. First, in view of the shortcomings of the original model in multi-scale feature fusion and target deformation adaptation, an adaptive context-aware feature pyramid network (ACA-FPN) is designed. ACA-FPN mainly consists of two parts: a pyramid context extraction module (PCE) and a hierarchical weighted fusion enhancement module (HWFEM). The PCE module is responsible for integrating multi-scale features, extracting global context information, and enhancing the model's ability to capture key features by adaptively weighting features of different channels. At the same time, it uses the self-attention mechanism to establish long-range dependencies, further improving the ability to capture global and local information. The HWFEM module adaptively assigns weights when fusing features of each layer to achieve hierarchical weighted fusion, ensuring that the features can effectively capture and highlight important semantic information during the fusion process. In addition, a dynamic interpolation fusion module (DIF) is designed, which further promotes the information interaction between multi-scale features by upsampling the low-resolution feature map and adding it element-by-element with the high-resolution feature map.

[0103] Secondly, an adaptive multi-scale dynamic detection head (AMS-Dyhead) is designed. AMS-Dyhead introduces deformable convolution DCNv3 in the detection head and designs a collaborative attention mechanism to fuse scale and task attention to improve the model's adaptability to target shape changes.

[0104] In addition, in order to improve the processing effect of the existing regression loss function on medium-difficulty samples, the present invention replaces the CIoU loss function with the Focaler-IoU loss function, thereby further enhancing the model's detection performance for small targets and complex scenes.

Claims

1. A surface defect detection method for multi-scale and small target features, characterized in that: The following steps are involved: Step 1: Obtain the surface defect dataset, convert the format of the annotation file to the required format and divide the images into training set, validation set and test set, set the initial learning rate and training rounds; Step 2: Build a DMS-YOLO surface defect detection model; Step 3: Train the constructed DMS-YOLO surface defect detection model, and use the training set and the validation set to train the surface defect detection model to obtain the optimal surface defect detection model; Step 4: Evaluate the trained model on the test dataset; Step 5: Fully test and debug the model trained in step 3 to ensure the accuracy and stability of the surface defect detection method in different scenarios; Step 6: Deploy the trained DMS-YOLO surface defect detection model to the actual industrial production environment, perform defect detection on the surface of real products, and record the accuracy, false detection rate, and missed detection rate of the detection results.

2. The method according to claim 1, characterized in that It also includes step 7: summarize the entire detection process, form a standardized solution, and promote it to different industrial fields for application verification to ensure the practicality and stability of the method.

3. The method according to claim 1, characterized in that In step 2, the constructed DMS-YOLO surface defect detection model includes a backbone feature extraction network Backbone and a neck network Neck; specifically: The input image first undergoes image enhancement preprocessing, and then passes through the first and second convolution Conv layers of the backbone feature extraction network Backbone in sequence, and the feature map is passed to the third C2f module of the backbone feature extraction network Backbone; the output feature map passes through the fourth convolution layer of the backbone feature extraction network Backbone and is passed to the fifth C2f module, and then passes through the sixth convolution layer, the seventh C2f module, the eighth convolution layer, and the ninth C2f module in sequence; finally, the feature map is input to the tenth SPPF module of the backbone feature extraction network Backbone to complete the feature extraction of the backbone feature extraction network Backbone; in this process, the fifth, seventh, and tenth layers of the backbone feature extraction network Backbone output feature maps respectively as the input of the subsequent neck network Neck module; In the neck network Neck part, the feature maps output by the fifth, seventh and tenth layers of the backbone feature extraction network Backbone are first input together to the first layer PCE module in the neck network Neck for preliminary feature fusion; then, the feature map of the tenth layer of the backbone feature extraction network Backbone and the output feature map of the first layer PCE module are input together to the second layer HWFEM module to achieve more refined feature expression through high-dimensional feature weighting; then, the feature map output by the second layer HWFEM module is passed to the third layer DIF module together with the feature map of the seventh layer of the backbone feature extraction network Backbone to further fuse and enhance cross-layer feature information; then, the output feature map of the first layer PCE module of the neck network Neck and the output feature map of the third layer DIF module are input into the fourth layer HWFEM module together. After completing the fusion of multi-resolution features, the output feature map and the feature map of the fifth layer of the backbone feature extraction network Backbone are input into the fifth layer DIF module together, combining the deep semantics and local detail information; then, the output of the first layer PCE module and the output of the fifth layer DIF module are input into the sixth layer HWFEM module together; the output feature map of the sixth layer is passed to the detection head as the first feature map, and is also input into the seventh layer convolution module to further adjust the feature scale; The feature map output by the seventh convolution layer and the output feature map of the fourth HWFEM module are fused through the Concat operation and then passed to the ninth C2f module to obtain the second feature map for use by the detection head. In addition, the feature map of the ninth layer of the neck network Neck is processed by the tenth layer convolution module, and then fused with the output feature map of the second layer HWFEM module through the Concat operation. The fusion result is input into the twelfth layer C2f module to generate the third feature map for the detection head to use; finally, the three feature maps from the Neck part of the neck network are sent to the AMS-Dyhead detection head.

4. The method according to claim 4, characterized in that: The three-way feature map obtained from the Neck part of the neck network is input into the AMS-Dyhead detection head part to start detection. During the detection process, the three-way feature map is processed by the scale attention, spatial attention and task attention modules respectively to achieve multi-scale and multi-resolution feature optimization; the scale attention module first performs global pooling on the feature map to extract global context information, and then generates a feature weight map through the convolution Conv layer and the activation function layer ReLU and the activation function layer Sigmoid; the spatial attention module uses differentiated convolution operations to process feature maps of different resolutions, where high-resolution feature maps use DCNv2 convolution and low-resolution feature maps use DCNv3 convolution to adapt to different feature structures; the processed feature map calculates the offset through the offset module, and then adjusts the feature expression through the Sigmoid activation function; The task attention module first extracts features through global pooling, then passes through two fully connected layers and the activation function ReLU, and outputs the task weight map in the Normalize layer; finally, the output of the scale attention part is combined with the task attention module to achieve a more fine-grained and multi-level feature expression, thereby optimizing the target detection effect.

5. The method according to claim 1, characterized in that The CIoU loss function used is: L Focaler-IoU =1-IoU focaler Where IoU refers to the intersection over union ratio, IoU focaler It refers to the intersection-over-union ratio corrected according to the Focaler-IoU idea, L Focaler-IoU Refers to the final loss function value; when IoU is lower than the lower threshold d, the loss is 0; when IoU exceeds the upper threshold u, the loss is 1; and when IoU is between these two thresholds, the loss increases linearly with the increase of IoU value; this design makes the loss function more sensitive to changes in IoU value in a specific interval, thereby paying more attention to samples with moderate bounding box overlap, which helps the model to more effectively extract features from medium-difficulty samples, rather than just focusing on easy or difficult samples.

6. The method according to any one of claims 4 to 6, characterized in that The PCE module structure is as follows: The input feature map first undergoes an average pooling Avg Pool operation to extract global feature information, and then uses a depth-separable convolution Dw_Conv in two directions to extract context information in the height direction Pool_h and the width direction Pool_w respectively. The features in these two directions are fused into a new feature map by adding elements layer by layer. The fused feature map is further enhanced by the Excite module and then enters the Self_Attention mechanism part to generate global feature weights and perform weighted adjustment on important features. Subsequently, the feature map processed by the self-attention mechanism is separated into two channels through the Split operation, and batch normalization BatchNorm2d and activation ReLU are performed respectively to improve the stability and nonlinear ability of feature expression; finally, these feature channels are further adjusted to a feature value range of [0,1] through the Sigmoid activation function to generate a more refined feature representation for use in subsequent network layers.

7. The method according to any one of claims 4 to 6, characterized in that The two input feature maps are respectively subjected to convolution Conv operations to extract feature expressions of different scales, and then feature alignment is performed through specific operations. The aligned feature maps are fused through point-by-point operations and enter the act module for activation processing; in the act module, the feature map first generates weight features through bilinear operations, and is adjusted to the [0,1] range through the Sigmoid function to generate adaptive feature weights; finally, the weights are weighted and fused with the input feature map through point-by-point multiplication operations, and the enhanced feature map is output, retaining key information for subsequent processing.

8. The method according to any one of claims 4 to 6, characterized in that The three-way feature maps obtained by the neck network Neck are processed by the scale attention, spatial attention and task attention modules respectively to achieve multi-scale and multi-resolution feature optimization; the scale attention module first performs global pooling on the feature map to extract global context information, and then generates a feature weight map through the convolution Conv layer and the activation function layer ReLU and the activation function layer Sigmoid; the spatial attention module uses differentiated convolution operations to process feature maps of different resolutions, where high-resolution and medium-resolution feature maps use DCNv2 convolution, and low-resolution feature maps use DCNv3 convolution to adapt to different feature structures; the processed feature map calculates the offset through the offset module, and then adjusts the feature expression through the Sigmoid activation function; The task attention module first extracts features through global pooling, then passes through two fully connected layers and the activation function ReLU, and outputs the task weight map in the Normalize layer; finally, the output of the scale attention part is combined with the task attention module to achieve a more fine-grained and multi-level feature expression, thereby optimizing the target detection effect.

Images