Steel defect detection method, system, electronic equipment and storage medium

Abstract

The present invention provides a steel defect detection method, system, electronic device, and storage medium. The method comprises collecting a steel surface defect dataset, and sequentially organizing, converting, and processing the surface defect dataset; introducing an SPD-Conv module into a YOLOv8 feature extraction backbone network, and embedding an EMA module to obtain a feature extraction model; fusing the Neck portion into the feature extraction model, adding a lightweight cross-scale feature fusion module, and introducing an Inner-IoU loss function to obtain an updated feature extraction model; training the updated feature extraction model to obtain a steel defect detection model and a weight file; and testing a test set based on the weight file and the steel defect detection model to obtain a detection result. The present invention can better extract detailed features and better handle defects of different scales, thereby improving the accuracy and robustness of detection.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a steel defect detection method, system, electronic equipment and storage medium. Background Art

[0002] Traditional steel defect detection methods, such as manual visual inspection, ultrasonic testing, magnetic particle inspection, and radiographic testing, have long played an important role in industrial production. Manual visual inspection relies on the inspector's experience and visual ability. While intuitive, it is susceptible to subjective factors and difficult to ensure consistent inspection results. Ultrasonic testing utilizes the propagation characteristics of high-frequency sound waves within a material to identify defects. While suitable for detecting hidden defects such as internal cracks, it has limited ability to detect surface defects. Magnetic particle inspection uses a magnetic field to attract ferromagnetic powder, revealing surface and near-surface defects. While suitable for crack detection in ferromagnetic materials, it is ineffective against non-ferromagnetic materials. Radiographic testing (X-rays or gamma rays) detects internal defects through transmission imaging of the material. While suitable for high-precision inspection, it is costly, complex, and requires strict radiation safety precautions. While these traditional methods still play a role in industrial inspection, they often rely on manual labor or specialized equipment. Their results are significantly affected by operator experience, making standardization and automation difficult. Their efficiency is low, making them unable to meet the demand for efficient and accurate inspection in modern production lines.

[0003] With the development of machine learning technology, statistical learning-based methods have begun to be applied to steel defect detection, improving the level of automation in detection. Support vector machines (SVMs) achieve accurate defect classification by constructing the optimal classification hyperplane in a high-dimensional feature space, and perform well in small-sample learning scenarios. Random forests (RFs) enhance the generalization ability of the model through ensemble learning of multiple decision trees, and have certain advantages in dealing with complex defect patterns. The K-nearest neighbor (KNN) method classifies samples based on similarity and is suitable for defect detection tasks with relatively concentrated feature distributions. These traditional machine learning methods perform well in steel surface defect classification, but they usually rely on manual feature extraction, have difficulty processing high-dimensional, nonlinear, and complex data, and have limited detection capabilities for multiple types and scales of defects.

[0004] In recent years, the rapid development of deep learning has further promoted advancements in steel defect detection technology. Deep neural networks can automatically learn multi-level feature representations of data, eliminating the reliance on manual feature design in traditional methods. In particular, object detection technology, which extracts spatial features using deep convolutional neural networks (CNNs), has improved the accuracy and efficiency of defect detection. Currently, object detection methods are mainly divided into two categories: two-stage detectors and single-stage detectors. Two-stage detectors (such as Faster R-CNN) first generate candidate regions and then perform classification and regression. While they offer high detection accuracy, they suffer from slow inference speed and are unable to meet high real-time requirements. Single-stage detectors (such as the YOLO series and SSD) perform end-to-end detection directly on the input image, offering low computational cost and high detection speed, making them suitable for real-time inspection in steel production lines.

[0005] In summary, among the existing technologies, traditional machine learning methods still rely on manual feature extraction in the classification of steel surface defects, which makes it difficult to process high-dimensional, nonlinear and complex data, and has limited ability to detect multiple types and multi-scale defects. Summary of the Invention

[0006] Based on this, the purpose of the present invention is to provide a steel defect detection method, system, electronic device and storage medium to solve the deficiencies in the above-mentioned prior art.

[0007] In a first aspect, the present invention provides a method for detecting steel defects, the method comprising:

[0008] Collecting steel surface defect data sets, and sequentially arranging and converting the surface defect data sets;

[0009] The sorted and converted surface defect dataset is processed in sequence based on ContextGuided convolution and PConv convolution to obtain a processed surface defect dataset;

[0010] An SPD-Conv module is introduced into the YOLOv8 feature extraction backbone network, and an EMA module is embedded between the Split layer and the Bottleneck layer in the YOLOv8 feature extraction backbone network to construct a feature extraction model.

[0011] The Neck part is integrated into the feature extraction model, a lightweight cross-scale feature fusion module is added, and an Inner-IoU loss function is introduced into the feature extraction model to obtain an updated feature extraction model;

[0012] Dividing the processed surface defect data set into a training set and a test set, and training the updated feature extraction model based on the training set to obtain a steel defect detection model and a weight file;

[0013] The test set is tested based on the weight file and the steel defect detection model to obtain a test result.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: through ContextGuided convolution and PConv convolution, the detailed features of the steel surface can be extracted, thereby eliminating the need to rely on manual extraction of detailed features. By introducing the SPD-Conv module and embedding the EMA module in the YOLOv8 feature extraction backbone network, the detection effect of the model in complex backgrounds is effectively improved, and defects of different scales can be better handled, thereby improving the accuracy and robustness of detection. By fusing the Neck part and adding a lightweight cross-scale feature fusion module, the feature information of different scales can be fully integrated, thereby improving the adaptability of the model in multi-scale defect detection, especially the detection of small target defects. In addition, by introducing the Inner-IoU loss function, the target positioning capability can be optimized.

[0015] Furthermore, the step of sequentially arranging and converting the surface defect data set includes:

[0016] organizing the surface defect dataset to mark defects in each image in the surface defect dataset;

[0017] The format of the organized surface defect data set is converted.

[0018] Furthermore, the step of sequentially processing the sorted and converted surface defect dataset based on ContextGuided convolution and PConv convolution includes:

[0019] Performing local feature extraction on the sorted and converted surface defect dataset based on the ContextGuided convolution to obtain local features;

[0020] The context information of the local features is extracted based on the PConv convolution, and the co-occurrence relationship between the local features and the surrounding environment is integrated with feature fusion, and the co-occurrence relationship is dynamically adjusted through global semantic information.

[0021] Furthermore, the steps of introducing the SPD-Conv module into the YOLOv8 feature extraction backbone network and embedding the EMA module between the Split layer and the Bottleneck layer in the YOLOv8 feature extraction backbone network include:

[0022] Replace the CBS module in the YOLOv8 feature extraction backbone network and introduce the SPD-Conv neural network building block;

[0023] Design a C2f_EMA module with an attention mechanism, and introduce the C2f_EMA module into the YOLOv8 feature extraction backbone network to replace the C2f module in the YOLOv8 feature extraction backbone network;

[0024] The C2f_EMA module is embedded between the Split layer and the Bottleneck layer in the YOLOv8 feature extraction backbone network.

[0025] Furthermore, the step of fusing the Neck part into the feature extraction model and adding a lightweight cross-scale feature fusion module includes:

[0026] Adding a lightweight cross-scale feature fusion module to the Neck part, and fusing the Neck part after adding the lightweight cross-scale feature fusion module into the feature extraction model;

[0027] A convolution operation is added before and after each upsampling in the feature extraction model, and the upsampling is connected with a splicing operation.

[0028] Furthermore, the step of dividing the processed surface defect data set to obtain a training set and a test set includes:

[0029] Dividing the processed surface defect dataset based on a stratified sampling strategy to obtain a training set, a validation set, and a test set;

[0030] The training set, the validation set, and the test set are stored in corresponding folders respectively.

[0031] Furthermore, after the step of testing the test set based on the weight file and the steel defect detection model to obtain a test result, the method further includes:

[0032] Evaluate the detection results based on the target detection model and evaluation indicators, wherein the evaluation indicators include accuracy, recall rate and average accuracy value;

[0033] The calculation expression of the accuracy is:

[0034] ;

[0035] Where, represents the accuracy, represents the number of samples correctly classified as positive by the steel defect detection model, Represents the number of samples that are incorrectly classified as positive by the steel defect detection model but are actually negative;

[0036] The calculation expression of the recall rate is:

[0037] ;

[0038] Where, represents the recall rate, Indicates the number of samples that are incorrectly classified as negative by the steel defect detection model but are actually positive;

[0039] The calculation expression of the average accuracy value is:

[0040] ;

[0041] Where, represents the average accuracy value, represents the total number of categories, represents the average precision of the i-th category, Indicates the average precision.

[0042] In a second aspect, the present invention further provides a steel defect detection system, comprising:

[0043] A collection module is used to collect steel surface defect data sets, and to organize and convert the surface defect data sets in sequence;

[0044] A processing module, configured to process the sorted and converted surface defect dataset in sequence based on ContextGuided convolution and PConv convolution to obtain a processed surface defect dataset;

[0045] An introduction module is used to introduce an SPD-Conv module into the YOLOv8 feature extraction backbone network, and embed an EMA module between the Split layer and the Bottleneck layer in the YOLOv8 feature extraction backbone network to construct a feature extraction model;

[0046] A fusion module is used to fuse the Neck part in the feature extraction model, add a lightweight cross-scale feature fusion module, and introduce an Inner-IoU loss function into the feature extraction model to obtain an updated feature extraction model;

[0047] A training module, configured to divide the processed surface defect data set into a training set and a test set, and train the updated feature extraction model based on the training set to obtain a steel defect detection model and a weight file;

[0048] The detection module is used to detect the test set based on the weight file and the steel defect detection model to obtain a detection result.

[0049] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned steel defect detection method when executing the computer program.

[0050] In a fourth aspect, the present invention further provides a storage medium storing a computer program, which implements the above-mentioned steel defect detection method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of a steel defect detection method according to a first embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the feature extraction backbone network process in the first embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the SPD-Conv process in the first embodiment of the present invention;

[0054] Figure 4 Schematic diagram of the C2f_EMA process in the first embodiment of the present invention;

[0055] Figure 5 A schematic diagram of the CCFM process flow for the feature fusion module in the first embodiment of the present invention;

[0056] Figure 6 is a structural block diagram of a steel defect detection system in a second embodiment of the present invention;

[0057] Figure 7 FIG. 4 is a schematic diagram of the hardware structure of an electronic device in a third embodiment of the present invention.

[0058] Description of main component symbols:

[0059] 10. Collection module; 20. Processing module; 30. Introduction module; 40. Fusion module; 50. Training module; 60. Detection module;

[0060] 70. Bus; 71. Processor; 72. Memory; 73. Communication interface.

[0061] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0062] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The drawings illustrate several embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.

[0063] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0065] Example 1

[0066] See also Figure 1 , which shows a steel defect detection method in a first embodiment of the present invention, the method includes steps S1 to S6:

[0067] S1, collecting steel surface defect datasets, and sequentially arranging and converting the surface defect datasets;

[0068] Specifically, the step S1 includes steps S11 to S12:

[0069] S11, organizing the surface defect dataset to mark defects in each image in the surface defect dataset;

[0070] It is understandable that a public surface defect dataset of steel is collected. In this embodiment, the surface defect dataset includes six typical surface defects, specifically rolling scale (Rs), plaque (Pa), crack (Cr), pitting surface (Ps), inclusion (In), and scratch (Sc). The surface defect dataset includes 1800 grayscale images of six different types of typical surface defects, and each type of surface defect includes 300 samples.

[0071] When organizing the surface defect dataset, the original image data and the corresponding annotation files are first organized to ensure that each image has the corresponding defect annotation and to remove damaged or invalid data;

[0072] S12, converting the format of the sorted surface defect dataset;

[0073] It is understandable that the surface defect dataset is uniformly converted into YOLO format, and each image corresponds to a .txt label file with the same name, and the annotation content is<class_id> 、<x_center> 、<y_center> 、 <width> 、 <height>All coordinates are normalized relative to the image width and height. During the conversion process, the coordinates of the upper left and lower right corners of the target bounding box are extracted from the original annotations, the coordinates of the target's center point, width, and height are calculated, and normalization is performed. Finally, the images and corresponding labels are organized into the "train / images", "train / labels", "val / images", "val / labels", "test / images", and "test / labels" folders according to the directory structure required by the YOLO model. The "data.yaml" file is configured to specify the category names and paths for each dataset.

[0074] In addition, it is worth mentioning that the relevant experimental environment needs to be configured. For specific experimental environment configuration, please refer to Table 1:

[0075] Table 1

[0076]

[0077] In the model's default.yaml file, the batch size is set to 16 based on system resources, model, and dataset size. The input image resolution is resized to 640×640 pixels, the number of worker threads for data loading is set to 8, the initial learning rate is 0.01, the optimizer weight decay is adjusted to 0.0005 to prevent overfitting, the intersection over union (IoU) threshold for non-maximum suppression (NMS) is set to 0.7, and the total number of cycles for all model training is 300 epochs.

[0078] S2, processing the sorted and converted surface defect dataset in sequence based on ContextGuided convolution and PConv convolution to obtain a processed surface defect dataset;

[0079] Specifically, step S2 includes steps S21 to S22:

[0080] S21, performing local feature extraction on the sorted and converted surface defect dataset based on the ContextGuided convolution to obtain local features;

[0081] S22, extracting context information of the local features based on the PConv convolution, and combining feature fusion to fuse the co-occurrence relationship between the local features and the surrounding environment, and dynamically adjusting the co-occurrence relationship through global semantic information;

[0082] It can be understood that local feature extraction of surface defect dataset based on ContextGuided convolution is used for local detail features of steel surface, and the context information of local features is extracted through PConv convolution, which can extract the information of the surrounding area of ​​​​the layout feature and solve the problem of local occlusion. By combining feature fusion to fuse the co-occurrence relationship between local features and the surrounding environment, the expressive ability of features can be effectively enhanced, and global context weighting can be achieved by using global semantic information to dynamically adjust the importance of features.

[0083] S3, introducing the SPD-Conv module into the YOLOv8 feature extraction backbone network, and embedding the EMA module between the Split layer and the Bottleneck layer in the YOLOv8 feature extraction backbone network to construct a feature extraction model;

[0084] Specifically, step S3 includes steps S31 to S33:

[0085] S31, replacing the CBS module in the YOLOv8 feature extraction backbone network and introducing the SPD-Conv neural network building block;

[0086] S32, designing a C2f_EMA module with an attention mechanism, and introducing the C2f_EMA module into the YOLOv8 feature extraction backbone network to replace the C2f module in the YOLOv8 feature extraction backbone network;

[0087] S33, embedding the C2f_EMA module between the Split partitioning layer and the Bottleneck neck layer in the YOLOv8 feature extraction backbone network;

[0088] It is understandable that the SPD-Conv module is introduced into the YOLOv8 feature extraction backbone network to replace the original CBS module to enhance the perception of low-resolution areas and small targets. Based on the original C2f module, the C2f_EMA module integrating the EMA attention mechanism is designed. By embedding the C2f_EMA module between the Split layer and the Bottleneck layer, redundant feature information is effectively suppressed, the network's attention to key target areas is improved, and a more efficient feature extraction model is constructed. The network structure diagram is shown in the figure below. Figure 2 shown.

[0089] In specific implementation, the following steps 101 to 113 are included:

[0090] Step 101: Input the training set images into the feature extraction model and process them through the SPD-Conv module. The process of the SPD-Conv module is as follows. Figure 3 , Figure 3 The size of the input feature map in (a) is (C1, S, S), that is, (64, 640, 640), where C1 is the number of channels and S is the spatial resolution;

[0091] Step 102, then, Figure 3 (b) in the figure is transformed into Figure 3 In (c), each 2×2 spatial area is rearranged to the channel dimension. Each 2×2 spatial block is split into 4 pixels. The information of these 4 pixels is placed in the 4 positions of the channel dimension. The original 2×2 small area of ​​each channel now becomes a pixel corresponding to each of the 4 channels. Each original channel contributes 4 new channels. In the width and height directions, every 2 pixels are merged into 1, that is, the spatial size is reduced to half of the original size, making Figure 3 The size of the feature map in (d) becomes (4C1, S / 2, S / 2) or (256, 320, 320). Figure 3 (a) to (e) represent the process of SPD-Conv;

[0092] Step S103: Subsequently, these rearranged feature maps are divided into four sub-regions, each representing the local information of the original feature map at a different spatial position. The size of each sub-region is (C1, S / 2, S / 2), i.e., (64, 320, 320). These four sub-feature maps are concatenated in the channel dimension to obtain a new feature map with a size of (4C1, S / 2, S / 2), i.e., (256, 320, 320);

[0093] Step S104: Finally, a standard convolution operation with a stride of 1 is applied to the feature map, and the output feature map is of size (C2, S / 2, S / 2), i.e., (64, 320, 320). The output feature map has richer semantic information and a larger receptive field. The value of C2 is set according to demand. Through this "space-for-channel" strategy, SPD-Conv can retain local structural information while reducing spatial resolution, thereby improving the feature expression ability of low-resolution images and small objects.

[0094] Step S105: The output feature map size (64, 320, 320) in step 104 is processed by SPD-Conv and output as a feature map size (128, 160, 160), which is then used as input to the C2f_EMA module that embeds EMA between the Split layer and the Bottleneck layer based on C2f. Figure 4 As shown, the C2f_EMA module uses the original feature map size of 128×160×160, which means the number of channels is 128 and the spatial size is 160×160, which contains the feature information of the original local area;

[0095] In step S106, the input feature map undergoes a convolution operation, namely the initial CBS (Conv+BN+SiLU): a 1×1 convolution is used to linearly combine the input feature map, which helps to reorganize information between channels and compress features. It is then normalized by BatchNorm and finally the SiLU activation function is used to enhance the nonlinear expression capability. The output is still 128×160×160, which is conducive to subsequent feature extraction.

[0096] Step S107, then perform channel splitting (Split): divide the 128 channels into two parts along the channel dimension: 64×160×160 and 64×160×160, respectively named and .in The shallow information is used for subsequent splicing. as an entry feature enhancement path;

[0097] Step S108, then enter the EMA attention module: First, global average pooling is used to extract channel-level global context information (output 64×1×1), and then a two-layer fully connected network is used to generate compressed channel attention weights (1×1×64). Perform channel-by-channel weighting operations to enhance important channels and suppress redundant channels. The output is 64×160×160. The enhanced feature map has stronger discriminative ability.

[0098] Step S109, the feature map 64×160×160 passes through the bottleneck layer (Bottleneck ×n): enhanced The nth bottleneck layer residual module is then used for deep feature extraction. Each block consists of two layers of convolution, combined with residual connections, to extract multi-scale semantic features while preserving the spatial distribution of the original features. Each output is 64×160×160, for a total of n.

[0099] Step S110, then concatenate the features: , attention enhancement features, n bottleneck layer outputs are spliced ​​in the channel dimension to form a (n+2)×64×160×160 high-dimensional feature map, realizing the fusion of shallow + deep + enhanced features, effectively improving the expression ability of detecting small and medium-sized targets and edge information;

[0100] In step S111, the high-dimensional feature map generated is then processed by a CBS (Conv+BN+SiLU): the high-dimensional concatenated feature map is subjected to a 1×1 convolution to compress the number of channels back to the original input channel number (128), and BN and SiLU are used to ensure feature stability and nonlinear expression. The output size is 128×160×160. This operation integrates the fusion features and maintains the original size to facilitate subsequent connection to the backbone network. The output feature map has the same size as the input, 128×160×160, but contains multi-scale and multi-dimensional fusion features extracted from shallow layers, deep layers and the attention mechanism, and has stronger expression and discrimination capabilities.

[0101] In step S112, the output feature map of size 128×160×160 in step 111 is processed by three SPD-Conv and three C2f_EMA modules. The representation capability of the feature map is enhanced. Each C2f_EMA module gradually increases the number of channels in the feature map. Then, through feature concatenation, a higher-dimensional feature map is formed, which contains more information. The dimension of the final output feature map is generally the same as the input. After processing by three SPD-Conv and three C2f_EMA modules, the number of channels and feature representation capability of the feature map are enhanced, which can better handle low-resolution, small objects, and features in complex backgrounds. The detection accuracy of details and small objects is improved. The final output feature map of size 1024×20×20 has stronger representation capability and the ability to adapt to complex scenes.

[0102] Finally, in step S113, spatial pyramid pooling (SPPF) is performed. SPPF maintains the spatial size and number of channels of the feature map (the input and output are of the same dimension), but enhances the multi-scale spatial context information, especially for multi-scale object detection. The feature map becomes more diverse and contains more spatial hierarchical information.

[0103] S4, fusing the Neck part into the feature extraction model, adding a lightweight cross-scale feature fusion module, and introducing the Inner-IoU loss function into the feature extraction model to obtain an updated feature extraction model;

[0104] Specifically, step S4 includes steps S41 to S42:

[0105] S41, adding a lightweight cross-scale feature fusion module to the Neck part, and fusing the Neck part after adding the lightweight cross-scale feature fusion module into the feature extraction model;

[0106] S42, adding a convolution operation before and after each upsampling in the feature extraction model, and connecting the upsampling with a splicing operation;

[0107] It is understandable that a lightweight cross-scale feature fusion module is added to the feature fusion Neck part, combining multi-scale extraction, convolution fusion and channel compression;

[0108] In the specific implementation, the feature fusion Neck part of the original YOLOv8 model is introduced, and the multi-scale feature fusion module CCFM is introduced. For details, please refer to Figure 5 , a convolution operation is added before and after each upsampling, and the upsampling is connected to the splicing operation. After the high-dimensional feature map 1024×20×20 is output from the feature extraction network, it enters the feature fusion network, first performing a CBS convolution operation, and then performing an upsampling operation. If upsampling is performed directly, there will be two problems. First, there are too many channels, which will increase the computational burden. Second, there is semantic information redundancy, and directly splicing the underlying features is not conducive to fusion. After the upsampling operation, a feature map with a larger spatial scale is output, which includes the features extracted from the input feature map and has improved spatial resolution.

[0109] In addition, the upsampling operation is connected to the splicing operation. The upsampling operation aligns the high-level feature maps with the low-level features in the spatial dimension, while the splicing operation realizes the aggregation of multi-scale information. In the fusion process after splicing, the CBS convolution operation is combined to further integrate and reconstruct the features, and the output feature map is 512×40×40, which alleviates the problem of inconsistent feature distribution.

[0110] After a series of feature fusion operations (C2f, Conv, Upsample, and Concat), the model converts the output feature maps of the last three C2f layers into input feature maps for defect detection. The first output of these three C2f layers becomes the input feature map for detecting small objects, the second becomes the input feature map for detecting medium objects, and the final output becomes the input feature map for detecting large objects. After detecting the small, medium, and large feature maps, the model outputs a set of predicted boxes (position, confidence, and category) at the corresponding scales. These boxes are then merged and filtered using non-maximum suppression (NMS) to form the final object detection result.

[0111] S5, dividing the processed surface defect data set into a training set and a test set, and training the updated feature extraction model based on the training set to obtain a steel defect detection model and a weight file;

[0112] Specifically, step S5 includes steps S51 to S52:

[0113] S51, dividing the processed surface defect dataset based on a stratified sampling strategy to obtain a training set, a validation set, and a test set;

[0114] S52, storing the training set, the validation set, and the test set in corresponding folders respectively;

[0115] It should be explained that the surface defect dataset is divided into training set, validation set, and test set in a ratio of 6:2:2. A stratified sampling strategy is used during the division to ensure that the proportion of each type of defect sample in each subset is consistent. During the division process, sklearn's train_test_split is added to randomly divide the random seed setting to ensure the reproducibility of each division. At the same time, random.shuffle is used to randomly shuffle the data to avoid partial order problems in the sample set. After the division is completed, the images and annotation files of different subsets are stored in corresponding folders (images / train, images / val, images / test, and labels / train, etc.). The distribution of labels in the training set, validation set, and test set can be found in Table 2:

[0116] Table 2

[0117]

[0118] It can be understood that the experiment is carried out according to the configuration-related experimental environment and experimental configuration, the input image size of the training set image of the steel defect dataset is set to 640*640, and then input into the updated feature extraction model for training, and a model for steel defect detection is obtained, and a weight file is obtained.

[0119] S6, testing the test set based on the weight file and the steel defect detection model to obtain a test result;

[0120] It is understandable that after obtaining the detection results, the detection results need to be evaluated. In this embodiment, the detection results are evaluated based on the target detection model and evaluation indicators. The evaluation indicators include accuracy, recall rate and average accuracy value; the calculation expression of the accuracy is:

[0121] ;

[0122] Where, represents the accuracy, represents the number of samples correctly classified as positive by the steel defect detection model, Represents the number of samples that are incorrectly classified as positive by the steel defect detection model but are actually negative;

[0123] The calculation expression of the recall rate is:

[0124] ;

[0125] Where, represents the recall rate, Indicates the number of samples that are incorrectly classified as negative by the steel defect detection model but are actually positive;

[0126] The calculation expression of the average accuracy value is:

[0127] ;

[0128] Where, represents the average accuracy value, represents the total number of categories, represents the average precision of the i-th category, represents the average accuracy;

[0129] It should be noted that the average precision is the area under the precision-recall curve. AP comprehensively reflects the overall performance of the model by integrating precision and recall at different thresholds. The calculation expression of average precision is:

[0130] ;

[0131] Where, Indicates exact value, Recall is the recall rate, and AP is obtained by integrating precision over recall from 0 to 1 and calculating the area under the precision-recall curve. This reflects the overall performance of the model at different thresholds.

[0132] The average accuracy value is a comprehensive indicator for measuring the performance of multi-class classification or detection models. It calculates the average precision of each category and then takes the average AP of all categories. In this example, mAP@0.5 and mAP@0.5:0.95 are selected as evaluation indicators to verify the effectiveness of the detection results.

[0133] In addition, FPS (Frames Per Second) is an important indicator for measuring video or graphics processing performance. It represents the number of frames processed or displayed per second and is used to evaluate the smoothness, real-time processing capabilities, and performance of the model system. The calculation expression for the number of frames processed or displayed per second is:

[0134] ;

[0135] Where FPS represents the number of frames processed or displayed per second, and Processing time per frame represents the processing time per frame;

[0136] Furthermore, an ablation experiment was conducted on the steel defect detection model. The experimental results are shown in Table 3:

[0137] Table 3

[0138]

[0139] Table 3 shows that in Experimental Group 1, the introduction of the SPD-Conv module alone resulted in a slight reduction in model parameters, but both mAP@0.5 and FPS decreased. This indicates that while SPD-Conv reduces model complexity, it does not significantly improve overall performance when used alone. In Experimental Group 2, the introduction of the C2f_EMA module significantly improved detection accuracy, but the frame rate decreased. This indicates that while this module excels in enhancing feature extraction capabilities, it also increases computational overhead. Experimental Group 3, which only introduced the CCFM module, also improved detection accuracy, but the increased computational overhead led to a decrease in FPS. Experimental Group 4, which combined the SPD-Conv and C2f_EMA modules, significantly improved detection accuracy and frame rate while slightly reducing model parameters. mAP@0.5 increased by 2.1% compared to the original model, demonstrating a synergistic effect between the two in terms of performance optimization. Experimental Group 5, which combined the SPD-Conv and CCFM modules, achieved limited improvement in detection accuracy despite a reduction in parameters. This suggests that this combination offers some advantages in terms of model lightweighting, but its performance improvement is not significant. Experimental Group 6, combining the C2f_EMA and CCFM modules, achieved improvements in both detection accuracy and FPS, but the overall performance was still inferior to the optimal combination. Ultimately, Experimental Group 7, which combined SPD-Conv, C2f_EMA, and CCFM modules, achieved the best performance across all metrics, achieving the highest mAP@0.5 (78.6%), the lowest number of parameters (1.68M), and a high frame rate (270.2). Although the FPS decreased compared to the original model, SPD-Conv improved feature extraction efficiency, C2f_EMA optimized the smoothness of the prediction results, and CCFM enhanced the ability to detect multi-scale objects. The synergistic effect of these three effectively balanced detection accuracy, model complexity, and inference speed.

[0140] In summary, the steel defect detection method in the embodiments of the present invention: 1. The convolution of the YOLOv8 feature extraction backbone network is replaced with the SPD-Conv (Sparse Partial Depthwise Convolution) module. This module adaptively adjusts the weights of the convolution kernel to enable the model to focus on key defect areas in the image, thereby significantly improving the detection accuracy of low-resolution and small-target defects, especially for detecting small-scale defects;

[0141] 2. Design the C2f_EMA (C2f with Efficient Multi-scale Attention) module. This module enhances the expressiveness of multi-scale features. By introducing a multi-scale attention mechanism, it effectively improves the model's detection performance in complex backgrounds, enabling it to better handle defects of different scales and enhancing detection accuracy and robustness.

[0142] 3. SCCI-YOLO introduces a lightweight cross-scale feature fusion module (CCFM) into the Neck network. This module can fully integrate feature information at different scales, improving the model's adaptability in multi-scale defect detection, especially for the detection of small target defects.

[0143] 4. Replace the loss function with the Inner-IoU loss function. To address the shortcomings of the traditional IoU loss function, SCCI-YOLO introduced the Inner-IoU loss function, which can more accurately measure the degree of matching between target boxes, thereby improving regression accuracy, accelerating model convergence, and optimizing target positioning capabilities.

[0144] 5. Balancing high precision and high efficiency: Through the aforementioned optimization modules, SCCI-YOLO improves detection accuracy while maintaining a high inference speed. This makes it suitable for the real-time detection needs of industrial production lines, and it excels in detecting defects in complex backgrounds and small targets.

[0145] Example 2

[0146] The present invention also proposes a steel defect detection system, please refer to Figure 6 , which shows a steel defect detection system in a second embodiment of the present invention, the system includes:

[0147] The collection module 10 is used to collect steel surface defect data sets, and to organize and convert the surface defect data sets in sequence;

[0148] A processing module 20 is configured to process the sorted and converted surface defect dataset in sequence based on ContextGuided convolution and PConv convolution to obtain a processed surface defect dataset;

[0149] An introduction module 30 is used to introduce an SPD-Conv module into the YOLOv8 feature extraction backbone network, and embed an EMA module between the Split layer and the Bottleneck layer in the YOLOv8 feature extraction backbone network to construct a feature extraction model;

[0150] A fusion module 40 is used to fuse the Neck part in the feature extraction model, add a lightweight cross-scale feature fusion module, and introduce an Inner-IoU loss function into the feature extraction model to obtain an updated feature extraction model;

[0151] A training module 50 is configured to divide the processed surface defect data set into a training set and a test set, and train the updated feature extraction model based on the training set to obtain a steel defect detection model and a weight file;

[0152] The detection module 60 is used to detect the test set based on the weight file and the steel defect detection model to obtain a detection result.

[0153] In some optional embodiments, the collection module 10 includes:

[0154] an arranging unit, configured to arrange the surface defect dataset to mark defects in each image in the surface defect dataset;

[0155] A conversion unit is used to convert the format of the sorted surface defect data set.

[0156] In some optional embodiments, the processing module 20 includes:

[0157] An extraction unit, configured to extract local features from the sorted and converted surface defect dataset based on the ContextGuided convolution to obtain local features;

[0158] A fusion unit is used to extract context information of the local features based on the PConv convolution, and to fuse the co-occurrence relationship between the local features and the surrounding environment through joint feature fusion, and to dynamically adjust the co-occurrence relationship through global semantic information.

[0159] In some optional embodiments, the introduction module 30 includes:

[0160] A replacement unit, used to replace the CBS module in the YOLOv8 feature extraction backbone network and introduce an SPD-Conv neural network building block;

[0161] A design unit is used to design a C2f_EMA module with an attention mechanism, and introduce the C2f_EMA module into the YOLOv8 feature extraction backbone network to replace the C2f module in the YOLOv8 feature extraction backbone network;

[0162] The embedding unit is used to embed the C2f_EMA module between the Split partitioning layer and the Bottleneck neck layer in the YOLOv8 feature extraction backbone network.

[0163] In some optional embodiments, the fusion module 40 includes:

[0164] adding a fusion unit, configured to add a lightweight cross-scale feature fusion module to the Neck part, and fusing the Neck part after adding the lightweight cross-scale feature fusion module into the feature extraction model;

[0165] A connection unit is used to add a convolution operation before and after each upsampling in the feature extraction model, and connect the upsampling with the splicing operation.

[0166] In some optional embodiments, the training module 50 includes:

[0167] A partitioning unit, configured to partition the processed surface defect dataset based on a stratified sampling strategy to obtain a training set, a validation set, and a test set;

[0168] A storage unit is used to store the training set, the validation set, and the test set in corresponding folders respectively.

[0169] In some optional embodiments, the detection module 60 includes:

[0170] An evaluation unit, configured to evaluate the detection results based on a target detection model and evaluation indicators, wherein the evaluation indicators include accuracy, recall rate, and average accuracy value;

[0171] The calculation expression of the accuracy is:

[0172] ;

[0173] Where, represents the accuracy, represents the number of samples correctly classified as positive by the steel defect detection model, Represents the number of samples that are incorrectly classified as positive by the steel defect detection model but are actually negative;

[0174] The calculation expression of the recall rate is:

[0175] ;

[0176] Where, represents the recall rate, Indicates the number of samples that are incorrectly classified as negative by the steel defect detection model but are actually positive;

[0177] The calculation expression of the average accuracy value is:

[0178] ;

[0179] Where, represents the average accuracy value, represents the total number of categories, represents the average precision of the i-th category, Indicates the average precision.

[0180] The functions or operation steps implemented when the above modules and units are executed are substantially the same as those in the above method embodiments and will not be repeated here.

[0181] The steel defect detection system provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the system embodiment, reference can be made to the corresponding content in the aforementioned method embodiment.

[0182] Example 3

[0183] The third embodiment of the present invention further provides an electronic device, see Figure 7 , which is a schematic diagram of the hardware structure of the electronic device in the third embodiment of the present invention.

[0184] The electronic device may include a processor 71 and a memory 72 storing computer program instructions.

[0185] Specifically, the processor 71 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the present application.

[0186] The memory 72 may include a large-capacity memory for data or instructions. By way of example, and not limitation, the memory 72 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 72 may include removable or non-removable (or fixed) media. Where appropriate, the memory 72 may be internal or external to the data processing device. In certain embodiments, the memory 72 is non-volatile memory. In certain embodiments, the memory 72 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0187] The memory 72 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 71 .

[0188] The processor 71 implements the steel defect detection method of the first embodiment by reading and executing computer program instructions stored in the memory 72 .

[0189] In some embodiments, the electronic device may further include a communication interface 73 and a bus 70. Figure 7 As shown, the processor 71, the memory 72, and the communication interface 73 are connected via a bus 70 and communicate with each other.

[0190] The communication interface 73 is used to implement communication between the various modules, devices, units and / or equipment in this application. The communication interface 73 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0191] The bus 70 includes hardware, software, or both, and couples the components of the device to each other. The bus 70 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 70 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Bus 70 may include one or more buses, where appropriate. Although this application describes and illustrates a particular bus, this application contemplates any suitable bus or interconnect.

[0192] The electronic device can obtain the steel defect detection system and execute the steel defect detection method of the first embodiment.

[0193] In addition, in conjunction with the steel defect detection method in the first embodiment, the present application may provide a storage medium for implementation. The storage medium stores computer program instructions; when the computer program instructions are executed by a processor, the steel defect detection method in the first embodiment is implemented.

[0194] Throughout this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0195] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.< / height> < / width>

Claims

1. A steel defect detection method, characterized in that: The method comprises: Collecting steel surface defect data sets, and sequentially organizing and converting the surface defect data sets; The sorted and converted surface defect dataset is processed in sequence based on ContextGuided convolution and PConv convolution to obtain a processed surface defect dataset; The SPD-Conv module is introduced into the YOLOv8 feature extraction backbone network, and the EMA module is embedded between the Split layer and the Bottleneck layer in the YOLOv8 feature extraction backbone network to construct a feature extraction model. This step specifically includes: Replacing the CBS module in the YOLOv8 feature extraction backbone network and introducing the SPD-Conv neural network building block, wherein the SPD-Conv neural network building block is used to sequentially rearrange, partition, and convolve the feature maps; Design a C2f_EMA module with an attention mechanism, and introduce the C2f_EMA module into the YOLOv8 feature extraction backbone network to replace the C2f module in the YOLOv8 feature extraction backbone network; The C2f_EMA module is embedded between the Split partitioning layer and the Bottleneck neck layer in the YOLOv8 feature extraction backbone network, wherein the Split partitioning layer is used for channel partitioning and the Bottleneck neck layer is used for deep feature extraction; The Neck part is fused into the feature extraction model, a lightweight cross-scale feature fusion module is added, and the Inner-IoU loss function is introduced into the feature extraction model to obtain an updated feature extraction model. This step specifically includes: Adding a lightweight cross-scale feature fusion module to the Neck part, and fusing the Neck part after adding the lightweight cross-scale feature fusion module into the feature extraction model; Adding convolution operations before and after each upsampling in the feature extraction model, and connecting the upsampling with a splicing operation, wherein the upsampling aligns the high-level feature map with the low-level feature map in the spatial dimension, and the splicing operation fuses multi-scale information; Dividing the processed surface defect data set into a training set and a test set, and training the updated feature extraction model based on the training set to obtain a steel defect detection model and a weight file; The test set is tested based on the weight file and the steel defect detection model to obtain a test result.

2. The steel defect detection method according to claim 1, characterized in that: The steps of sequentially arranging and converting the surface defect data sets include: organizing the surface defect dataset to mark defects in each image in the surface defect dataset; The format of the organized surface defect data set is converted.

3. The steel defect detection method according to claim 1, characterized in that: The step of dividing the processed surface defect data set to obtain a training set and a test set includes: Dividing the processed surface defect dataset based on a stratified sampling strategy to obtain a training set, a validation set, and a test set; The training set, the validation set, and the test set are stored in corresponding folders respectively.

4. The steel defect detection method according to claim 1, characterized in that: After the step of detecting the test set based on the weight file and the steel defect detection model to obtain a detection result, the method further includes: Evaluate the detection results based on the target detection model and evaluation indicators, wherein the evaluation indicators include accuracy, recall rate and average accuracy value; The calculation expression of the accuracy is: ； Where, represents the accuracy, represents the number of samples correctly classified as positive by the steel defect detection model, Represents the number of samples that are incorrectly classified as positive by the steel defect detection model but are actually negative; The calculation expression of the recall rate is: ； Where, represents the recall rate, Indicates the number of samples that are incorrectly classified as negative by the steel defect detection model but are actually positive; The calculation expression of the average accuracy value is: ； Where, represents the average accuracy value, represents the total number of categories, represents the average precision of the i-th category, Indicates the average precision.

5. A steel defect detection system, characterized in that: The system comprises: A collection module is used to collect steel surface defect data sets, and to organize and convert the surface defect data sets in sequence; A processing module, configured to process the sorted and converted surface defect dataset in sequence based on ContextGuided convolution and PConv convolution to obtain a processed surface defect dataset; An introduction module is used to introduce an SPD-Conv module into the YOLOv8 feature extraction backbone network, and embed an EMA module between the Split layer and the Bottleneck layer in the YOLOv8 feature extraction backbone network to construct a feature extraction model; The introduction module includes: a replacement unit, configured to replace the CBS module in the YOLOv8 feature extraction backbone network and introduce an SPD-Conv neural network building block, wherein the SPD-Conv neural network building block is configured to sequentially rearrange, partition, and convolve the feature map; A design unit, configured to design a C2f_EMA module with an attention mechanism, and introduce the C2f_EMA module into the YOLOv8 feature extraction backbone network to replace the C2f module in the YOLOv8 feature extraction backbone network; An embedding unit, configured to embed the C2f_EMA module between a Split partitioning layer and a Bottleneck neck layer in the YOLOv8 feature extraction backbone network, wherein the Split partitioning layer is used for channel partitioning, and the Bottleneck neck layer is used for deep-level feature extraction; A fusion module is used to fuse the Neck part in the feature extraction model, add a lightweight cross-scale feature fusion module, and introduce an Inner-IoU loss function into the feature extraction model to obtain an updated feature extraction model; The fusion module includes: adding a fusion unit, configured to add a lightweight cross-scale feature fusion module to the Neck part, and fusing the Neck part after adding the lightweight cross-scale feature fusion module into the feature extraction model; a connection unit configured to add convolution operations before and after each upsampling in the feature extraction model, and connect the upsampling with a splicing operation, wherein the upsampling aligns the high-level feature maps with the low-level features in the spatial dimension, and the splicing operation fuses multi-scale information; A training module, configured to divide the processed surface defect data set into a training set and a test set, and train the updated feature extraction model based on the training set to obtain a steel defect detection model and a weight file; The detection module is used to detect the test set based on the weight file and the steel defect detection model to obtain a detection result.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steel defect detection method according to any one of claims 1 to 4 is implemented.

7. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steel defect detection method according to any one of claims 1 to 4 is implemented.

Images