A steel surface defect classification method for long-tail distribution scenarios

Abstract

The application discloses a steel surface defect classification method for long-tail distribution scenarios, comprising the following steps: S1, obtaining a steel surface original image sample set; S2, constructing an active model set, including a classification model and a detection model; S3, training the classification model and the detection model respectively; S4, inputting a steel surface image to be identified into each model to obtain an original output score; S5, judging the output of each model according to the confidence threshold corresponding to each model to obtain a binary judgment result; S6, weighting and integrating the judgment results of each model according to the corresponding weights to obtain a fusion score; S7, generating a final steel surface defect classification result according to the fusion score and a preset decision rule; and S8, jointly optimizing the confidence threshold set and the model weight set. The application can significantly improve the tail category recognition performance while maintaining the head category recognition accuracy, and improve the industrial deployment efficiency through parallel training and parallel reasoning.

Description

Technical Field

[0001] This invention relates to the fields of industrial machine vision, intelligent surface quality inspection, deep learning integrated recognition, and intelligent manufacturing technologies, and particularly to a method, system, electronic device, and storage medium for classifying steel surface defects in long-tailed distribution scenarios. More specifically, this invention addresses defect images of industrial surfaces such as steel strips, hot-rolled steel plates, cold-rolled steel plates, and coated steel plates. Under conditions of uneven category sample height, sparse tail defects, complex and varied defect morphologies, and high real-time requirements, it proposes a multi-model integrated recognition scheme that fuses classification and detection models. Background Technology

[0002] Automatic identification of surface defects in steel is a crucial link in the quality control chain of steel production. Steel plates may exhibit various defects such as pitting, cracks, scratches, indentations, patches, inclusions, and oxide scale. If these defects are not detected in time, they will not only reduce the surface quality grade of the steel, but may also lead to failure risks during subsequent stamping, welding, painting, and service processes, resulting in significant economic losses and safety hazards.

[0003] Traditional image processing methods mainly rely on artificial features such as thresholding, edge detection, texture analysis, clustering, or support vector machines, along with shallow learning models. However, their adaptability to complex backgrounds, irregular contours, and fine-grained texture differences is limited. With the development of deep learning, CNN-based classification models and object detection-based localization models have been widely applied to steel surface defect identification. However, when using only classification models, it is easier to focus on the overall appearance, resulting in insufficient perception of small-tailed defects and local irregularities; when using only detection models, although the local localization ability is strong, it is not always optimal for some large-area defects or defects with significant overall appearance differences.

[0004] On the other hand, existing methods for addressing the long-tail problem include data augmentation, loss function modification, contrastive learning, and generative synthesis, but their effectiveness often depends on the data scale, the clarity of defect texture boundaries, and the adaptability of the augmentation strategy. In industrial settings, defect images collected from different shifts, under different lighting conditions, and with different rolling processes on the same production line will exhibit significant domain shifts, further reducing the generalization stability of a single model.

[0005] Therefore, there is an urgent need for a new steel surface defect classification scheme that, without significantly increasing the complexity of industrial deployment, simultaneously takes into account the ability to identify long-tail categories, the stable accuracy of head categories, the ability to perceive small and irregular defects, industrial real-time performance, and the ability to generalize across data distributions. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies where a single model cannot simultaneously achieve tail-class enhancement, local detail recognition, global semantic understanding, and real-time industrial deployment, and to provide a steel surface defect classification method. This method integrates classification and detection models by constructing a multi-model fusion framework, and significantly improves the classification performance of steel surface defects under long-tail distributions through a two-stage classification strategy and a threshold-weight joint optimization mechanism, particularly enhancing the recognition ability of tail-class defects and small-area defects.

[0007] The objective of this invention is achieved through the following technical solution: a method for classifying steel surface defects in long-tailed distribution scenarios, comprising the following steps:

[0008] S1. Obtain a sample set of original images of the steel surface and preprocess the original images; the preprocessing includes size normalization, pixel normalization, and data augmentation; divide the preprocessed sample set into a training set and a validation set;

[0009] S2. Select an inference efficiency that meets the requirements for industrial deployment. The models constitute the activity model set. The activity model set includes at least one classification model and one detection model;

[0010] S3. Train the classification model and detection model in the activity model set respectively;

[0011] S4. Input the image of the steel surface to be identified into each model in the active model set trained in step S3, and obtain the original output score of each model. The model for the first The output of the class defect is denoted as ;

[0012] Map the multiple bounding box level probabilities output by the detection model to image-level category scores: Let the first... Each detection model outputs The candidate detection box, the first The detection box belongs to the first... The predicted probability of class defects is The image-level classification score corresponding to this detection model is defined as:

[0013] ;

[0014] S5. Based on the confidence thresholds corresponding to each model. The outputs of each model are evaluated to obtain a binary decision result:

[0015] ;

[0016] in, Indicates the first The model for the first The judgment result of the class defect;

[0017] S6. The decision results of each model are assigned according to their corresponding weights. Perform weighted integration to obtain the fusion scores for each category; let the first category be... The fusion weights of the models are Then for the first The fusion score for class defects is defined as follows:

[0018] ;

[0019] S7. Based on the fusion score The system generates the final steel surface defect classification results using preset decision rules; it employs a preset decision rule that combines background priority decision, majority voting decision, and default background fallback.

[0020] First, determine if the background priority condition is met: if at least one... If a model determines that a sample has a defect-free background, it will directly output a defect-free result and will not continue to perform specific defect category determination.

[0021] If the background priority condition is not met, then a majority vote is performed for each defect category; for any defect category When its fusion score satisfies:

[0022] ;

[0023] Judgment No. Class defects exist;

[0024] If none of the defect categories meet the majority voting criteria, the default background fallback strategy is adopted to classify the sample as defect-free.

[0025] S8. Set the confidence thresholds for each model on the validation set. and model weight set Joint optimization is performed to obtain the optimal threshold set. and the optimal weight set .

[0026] The beneficial effects of this invention are as follows: Compared with existing technologies, this invention firstly achieves a complementary fusion of classification and detection models, enabling the model to simultaneously possess global semantic recognition capabilities and local spatial positioning sensitivity. This allows it to better handle scenarios such as complex textures on steel surfaces, local fine cracks, micro-tail defects, and irregular patch defects. This invention does not simply average multiple networks; instead, it employs a combined design of "heterogeneous model complementarity + two-stage classification + threshold weight joint optimization + parallel efficient deployment" to form a complete solution with practical engineering value in the long-tail classification scenario of industrial steel surface defects.

[0027] Secondly, this invention significantly alleviates the confusion between long-tail categories and defect-free background through a two-stage classification structure. By first screening out defective samples and then performing fine classification, the adverse effects of background noise on tail category identification can be reduced, thereby improving the recall rate of a few defective samples while ensuring overall accuracy.

[0028] Furthermore, this invention optimizes the threshold and weights on the validation set together, enabling multi-model integrated decision-making to no longer rely on fixed averages or simple voting, but to adaptively match the specific steel defect data distribution, thereby improving adaptability and adjustability during industrial deployment.

[0029] Finally, this invention supports selecting lightweight and fast inference models to enter the active model set, and supports parallel training and parallel inference. Therefore, it is beneficial to balance recognition accuracy and production line real-time performance, and is suitable for scenarios such as online quality inspection of steel plate surfaces, defect early warning, graded screening and production process optimization. Attached Figure Description

[0030] Figure 1 This is a flowchart of the steel surface defect classification method of the present invention;

[0031] Figure 2 These are some examples of images showing defects on the steel surface in this embodiment. Detailed Implementation

[0032] This invention addresses the challenges of long-tailed category distribution, sparse tail defect samples, weak local textures, complex background interference, and high real-time requirements in industrial online inspection of surface defect images of steel strips, hot-rolled steel plates, cold-rolled steel plates, and coated steel plates. It proposes a steel surface defect classification method for long-tailed distribution scenarios. The overall idea is as follows: First, an active model set composed of a classification model and a detection model is constructed. Then, a two-stage classification strategy enhances the classification model's ability to distinguish the presence and hierarchical classification of defects. Simultaneously, the detection model's ability to locate and classify fine-grained local defect regions compensates for the pure classification model's insufficient perception of small targets and tail-type defects. Next, the thresholds and weights of each model are jointly optimized on a validation set, allowing different models to leverage their respective strengths within a unified decision framework. Finally, a decision mechanism that prioritizes background, uses majority voting, and uses a default background fallback to output the final defect category. This maintains the accuracy of head-type category recognition while improving the recognition stability of tail-type defects, small-area defects, and complex background conditions.

[0033] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0034] like Figure 1 As shown, the present invention provides a method for classifying steel surface defects in long-tailed distribution scenarios, comprising the following steps:

[0035] S1. Data Acquisition and Preprocessing: Acquire a sample set of original images of the steel surface and preprocess the original images; the constructed sample set is represented as follows. ,in, Indicates the first Original image of a steel surface. This indicates the label information corresponding to the image. This represents the total number of images. For classification models, the label... Primarily represents image-level category labels; for detection models, labels It also includes the target bounding box and the corresponding category, i.e. , in, Indicates the first The first image The bounding box of a defective target. Indicates its defect category, This indicates the number of targets in the image.

[0036] To reduce the impact of imaging noise, brightness variations, and scale differences on subsequent model training, preprocessing of the input image is required. This preprocessing includes size normalization, pixel normalization, and data augmentation. The preprocessed training samples are represented as follows: ,in, This indicates that the image size is uniform. Indicates pixel normalization, This refers to data augmentation transformation. Preferably, data augmentation includes one or more of the following: random flipping, random cropping, brightness perturbation, cutout, random scaling, and translation. This is done because defect images collected from different shifts, under different lighting conditions, and during different rolling processes in industrial production lines exhibit significant domain shifts. Standardization and appropriate augmentation can enhance the model's robustness to appearance changes and improve the generalization ability of subsequent models.

[0037] The preprocessed sample set Divided into training set and verification set .

[0038] S2. Constructing an Activity Model Set: To simultaneously utilize global semantic information and local spatial positioning information, this invention does not employ a single model, but instead selects models whose inference efficiency meets the requirements of industrial deployment. The models constitute the activity model set. The active model set includes at least one classification model and one detection model. The classification model preferably employs four convolutional neural network models: ResNet, VGG, EfficientNet, and ConvNeXt. Their advantage lies in their ability to learn overall texture, shape, and appearance patterns from the entire image. The detection model preferably employs three object detection network models: Faster R-CNN, YOLO8, and YOLO11. Their advantage lies in their ability to extract candidate boxes, spatially locate, and classify local defect regions, making them more suitable for handling localized minor defects and tail-class samples. This multi-model ensemble framework leverages the differences and complementarities in feature extraction across different network structures to improve the classification performance of long-tail defects, particularly enhancing the recognition performance of low-frequency defect categories.

[0039] S3. Model Training: The classification and detection models in the active model set are trained separately. This invention employs an independent training method for each model, rather than a hybrid model routing structure requiring joint training of the gating network. Its advantages are: firstly, it allows for full reuse of existing pre-trained models and mature training processes; secondly, it decouples the training processes of each model, facilitating parallel training and flexible subsequent replacement, thus making it more suitable for industrial deployment.

[0040] The classification model employs a two-stage classification strategy for training. The first stage trains a binary classification model to determine whether the input image has defects. The second stage trains a multi-label classification model to identify the specific defect category of samples determined to be defective in the first stage. The ResNet, VGG, EfficientNet, and ConvNeXt models used in this invention can be used as either binary classification models or defect classification models. For example, a binary classification model can be trained independently using ResNet, followed by a defect classification model, and then the two models can be fine-tuned for use in both stages of recognition. The specific training steps for the classification model include the following:

[0041] (1) Construct the first-stage binary classification model, and classify the input... Image of a steel surface Perform defect / no-defect discrimination to obtain binary classification output probabilities:

[0042] ;

[0043] in, This represents the logit output by the first-stage binary classification model. This represents the Sigmoid activation function. This represents the probability that the input image is a defective sample;

[0044] (2) Output the probability of binary classification Compared with the preset first-stage decision threshold In comparison, when If the input image is defective, it is determined to be a defective sample and sent to the second-stage classification model; otherwise, it is determined to be a defect-free sample.

[0045] (3) For samples determined to be defective, the defect category is identified using the second-stage multi-label classification model. The second-stage multi-label classification model is defined as follows:

[0046] ;

[0047] in, Indicates the first Scoring function for class defects, This represents the total number of defect categories; for samples entering the second stage, the second-stage classifier outputs a defect category prediction vector. ; Indicates that the input image belongs to the first... The predicted probability of each type of defect. This two-stage classification structure is adopted because defect-free background samples typically constitute a large proportion of long-tail steel defect data. If a single-stage multi-class classification is performed directly, background noise will compress the effective decision space for tail-class defects. Separating "whether there is a defect" from "which type of defect it belongs to" helps reduce interference from background categories in tail-class defect identification, thus mitigating class imbalance. Finally, the predicted probability is determined based on the predicted category vector. Output the defect category classification results of this classification model.

[0048] (4) The following loss functions are used for model training in both stages:

[0049] ;

[0050] ;

[0051] in, This represents the loss function for the first stage. This represents the loss function for the second stage; Indicates the total number of training samples; Represents the cross-entropy loss, for a sample Category In other words, ; This indicates the set of samples that were deemed defective in the first phase. Indicates the first The binary classification labeling of the samples in the first stage. This represents the corresponding predicted probability. Indicates the first The sample about the first Class defect labeling, This represents the corresponding predicted probability. The advantage of this setup is that the first stage emphasizes rapid screening for defect existence, while the second stage emphasizes fine differentiation of defect types. Through two-stage hierarchical discrimination and targeted supervision, it helps improve the overall model's ability to identify tail-type defects.

[0052] The detection model is trained independently, in parallel with the classification model. The detection model is trained using a defect detection strategy.

[0053] For the Faster R-CNN object detection model, its training objective includes both classification and regression losses at the Region Candidate Network (RPN) stage and the Region of Interest (RoI) stage. The total loss function can be written as:

[0054] ;

[0055] in, This represents the network classification loss for candidate regions, with Focal Loss specifically employed to mitigate foreground-background imbalance. This represents the regression loss of the candidate region network. This represents the classification loss for candidate regions. This represents the regression loss of the candidate region. This represents the entire set of candidate boxes (all anchors). This represents the set of positive samples (anchors whose IoU with the real target is high enough to be considered positive samples). These represent the offsets of the four coordinates of the regression target. This represents the set of candidate regions after ROI Pooling. This represents the set of positive samples after ROI Pooling. Indicates the first The predicted probability of each candidate anchor in the ground truth class. Indicates the true label of the category. Indicates the first Each ROI sample is predicted as a category. The probability, As a class balance factor, To focus parameters, and These represent the parameters of the predicted bounding box and the target bounding box, respectively. This represents the regression loss function.

[0056] For single-stage detection models such as YOLO8 and YOLO11, their loss function can be expressed as:

[0057] ;

[0058] in, This represents the bounding box regression loss. Represents the category classification loss. This indicates a loss of the target's existence. , , These represent the weight coefficients of the corresponding loss terms; u represents the u-th candidate prediction unit output by the detector head. This represents the set of positive sample prediction units that successfully match the real defect target. This represents the entire set of prediction units; therefore, the bounding box regression loss and class classification loss are calculated only for positive prediction units, while the target existence loss is calculated for all prediction units. In the bounding box regression term, Represents the prediction box With real frame The crossover ratio between them

[0059] and These represent the coordinates of the center point of the predicted bounding box and the center point of the ground truth bounding box, respectively. This represents the length of the diagonal of the smallest bounding rectangle that simultaneously encloses the predicted bounding box and the ground truth bounding box. This indicates the consistency between the predicted bounding box and the ground truth bounding box in terms of aspect ratio. The balance factor representing the aspect ratio consistency term; This represents the square of the Euclidean distance divided by the denominator. This indicates that normalization is performed, resulting in the normalized squared error; in the categorical classification items, Indicates the first The positive sample prediction unit regarding the first... The true label of class defects This indicates that the prediction unit belongs to the first... The predicted probability of class defects; in the target existence term, Indicates the first Each prediction unit has a labeled target. This indicates the confidence probability that the prediction unit predicts the presence of a target. The above settings enable the detection model to simultaneously learn the spatial localization, category discrimination, and foreground-background distinction of the defect target, making it more suitable for detection tasks such as steel surface defects that are irregular in shape, have large scale differences, and are subject to strong background interference.

[0060] By employing a synchronous and independent training approach for the classification and detection models, the advantages of each model architecture can be fully utilized during the training phase, while avoiding the problem of mutual interference between gradients of different tasks during joint training. In this invention, the maximum number of training epochs for the classification model is set to 60 epochs. For the detection models, Faster R-CNN is trained for 24 epochs, while YOLO8 and YOLO11 are trained for 100 epochs. An early stopping mechanism is also implemented: training stops when a preset early stopping condition is met, resulting in the trained classification and detection models.

[0061] S4. Input the image of the steel surface to be identified into each model in the active model set trained in step S3, and obtain the original output score of each model. The model for the first The output of the class defect is denoted as .

[0062] Since this invention ultimately focuses on image-level classification of steel surface defects, rather than using bounding boxes as the final output, it is necessary to map the multiple bounding box-level probabilities output by the detection model to image-level category scores. The classification model directly outputs image-level category classifications, thus requiring no further processing. For a single image to be detected, let the... Each detection model outputs The candidate detection box, the first The detection box belongs to the first... The predicted probability of class defects is The image-level classification score corresponding to this detection model can be defined as:

[0063] ;

[0064] This mapping method means that if a candidate box has a high confidence level for a certain type of defect, then the entire image is considered to have high support for that category. The advantages of this approach are twofold: firstly, maximum value mapping is simple to calculate and stable; secondly, it is highly consistent with the working mechanism of detection models that "capture the most salient local defects."

[0065] S5. Based on the confidence thresholds corresponding to each model. The outputs of each model are used to make a decision, resulting in a binary decision. After calculating the image-level scores for the classification and detection models, let the... The model for the first Image-level scores for class defects are Then, based on the specific threshold corresponding to this model... Binarize its output to obtain:

[0066] ;

[0067] in, Indicates the first The model for the first The decision regarding class defects. This step is used because different model networks often have different output scales, confidence distributions, and positive / negative sample preferences. Simply summing continuous probabilities directly can easily lead to some models with excessively high or low outputs having an inappropriate impact on the final result. By introducing an independent threshold for each model... This can be achieved by first unifying all models into binary opinions of "support / not support", thereby reducing the bias caused by inconsistent output scaling of different networks.

[0068] S6. The decision results of each model are assigned according to their corresponding weights. Weighted ensemble analysis is performed to obtain the ensemble scores for each category. Let the size of the activity model set be... , No. The fusion weights of the models are ,satisfy Then for the first... The fusion score for class defects is defined as follows:

[0069] ;

[0070] in, Indicates the first The fusion score for class defects. Unlike simple averaging fusion, this invention allows different models to contribute differently to the final decision. For example, a detection model that is better at identifying local tail class defects can have its influence on the corresponding class enhanced by higher weights; a classification model that is better at identifying head class or background class defects can have its overall accuracy stabilized by appropriate weights.

[0071] S7. Based on the fusion score The invention generates the final steel surface defect classification result using preset decision rules. To balance false detection control and tail-class defect identification capabilities in industrial applications, this invention employs a preset decision rule combining background-priority decision, majority voting decision, and default background fallback. Specifically, it first determines whether the background-priority condition is met: assuming the number of active model sets used is... If there are at least one If a model determines that a sample has a defect-free background, it directly outputs a defect-free result and does not proceed with further defect category determination. This rule can be expressed as:

[0072] ;

[0073] in, For indicator functions, Indicates the first The model's decision on the background class. This indicates a background image that is defect-free. The advantage of this setting is that false positives for defects in industrial scenarios can lead to additional costs for re-inspection and quality grading. Therefore, when the vast majority of models consider the sample to be defect-free, prioritizing backgrounds can significantly reduce the false positive rate.

[0074] If the background priority condition is not met, then a majority vote is performed for each defect category. For any defect category... When its fusion score satisfies:

[0075] ;

[0076] Judgment No. The rule essentially states that if the weighted opinions supporting a category exceed half of the total weights of all models used, then that category is considered to have gained sufficient consensus and can be used as the final output. The advantage of this approach is that it preserves the stability of multi-model ensembles while allowing the opinions of highly reliable models to gain greater influence through the weighting mechanism.

[0077] If none of the defect categories meet the majority voting criteria, a default fallback strategy is adopted to classify the sample as defect-free. The purpose of this fallback strategy is to process the sample in a more conservative manner when the opinions of the various models are scattered and a consistent consensus on defects has not been formed, thereby improving the overall reliability of the system.

[0078] S8. The threshold set used in steps S5 and S6 and weight set Joint optimization is performed; to adapt the thresholds and fusion weights of each model to the current distribution of steel defect data, this invention does not directly use fixed thresholds or uniform weights, but instead optimizes the confidence threshold set corresponding to each model on the validation set. and model weight set Perform joint parameter optimization to obtain the optimal threshold set. and the optimal weight set .

[0079] Its joint optimization problem is written as:

[0080] ;

[0081] ;

[0082] in, This represents the validation set evaluation metric. The preferred definition of this metric is the sum of Precision and Recall, i.e. The reason for adopting this joint optimization approach is that different models have significant differences in output distribution, confidence intervals, and positive / negative sample preferences. If the same threshold or the same set of average weights is used uniformly, the advantages of each model cannot be fully utilized. By optimizing the threshold and weights for each model separately, the threshold of a more conservative model can be appropriately lowered, the threshold of a more aggressive model can be appropriately raised, and the model that performs better on the validation set can be given a higher fusion weight.

[0083] Furthermore, this invention employs an alternating optimization method to solve for the aforementioned parameters. First, the current weight set is fixed. Search for the optimal set of thresholds: Then fix the updated threshold set. Search for the optimal set of weights: Repeat the above process until the convergence condition is met or the maximum number of iterations is reached. This iterative optimization process typically converges within 2 to 3 cycles, thus possessing good computational feasibility and meeting the needs of practical industrial applications.

[0084] After completing the joint optimization of threshold and weight, the optimal parameters are obtained. and Then, these are used respectively for the model binary decision in step S5 and the weighted integration in step S6, thus forming a complete integrated reasoning process.

[0085] In summary, this invention organically combines the classification model's ability to perceive the global appearance with the detection model's sensitivity to local details through a process of "original image preprocessing—multi-model pool construction—two-stage training of the classification model and simultaneous training of the detection model—detection box-level probability mapping—model threshold decision—weight fusion—preset decision rule output—validation set joint optimization." Furthermore, the validation set joint optimization of thresholds and weights enhances the overall system's adaptability to long-tail defect distributions. Compared to existing technologies, this invention, by fusing classification and detection models and combining a two-stage classification strategy with a threshold-weight joint optimization mechanism, enables the system to simultaneously possess global semantic modeling capabilities and local defect perception capabilities. It can effectively improve the recognition accuracy and stability of long-tail defects, micro-defects, and complex background defects on steel surfaces without significantly increasing the complexity of industrial deployment, while reducing the false detection rate and meeting the real-time detection needs of production lines. It has good engineering practical value and promising prospects for widespread application.

[0086] Experimental results

[0087] The proposed method was systematically validated using the Severstal dataset. This dataset contains 12,568 images of steel surfaces with a resolution of 1600×256 and pixel-level defect annotations. Representative examples of the four defect categories in the Severstal dataset are shown below. Figure 2 As shown, the defects are categorized into four types: pitted surfaces, scratches, and patches. Pitted surfaces appear as small, discrete areas on the steel surface; scratches are long, thin, longitudinal lines with a relatively uniform shape, but occur less frequently; scratches are continuous wear marks covering a large area; and patches consist of irregularly shaped raised or recessed areas, with significant variations in appearance and morphology within the same category. Scratches account for less than 2% of the total, exhibiting a significant long-tail distribution. To verify the technical effectiveness of this invention, the sample set was divided into training, validation, and test sets in an 8:1:1 ratio. The model was trained on the training set, and hyperparameters, thresholds, and weights were optimized on the validation set. The model's prediction performance was evaluated using the test set. The experiment used four classification models: EfficientNet-B3, ConvNeXt-Small, ResNet101, and VGG19, and three detection models: Faster R-CNN, YOLO8, and YOLO11, forming an ensemble framework. To balance computational efficiency, models with high inference efficiency were chosen. Table 1 shows the average accuracy (ACC) for each class under different tail-class augmentation strategies on the test set.

[0088] Table 1. Average accuracy of each category under different tail-enhancing strategies

[0089]

[0090] Note: Conventional data augmentation methods include conventional data augmentation methods such as rotation, cropping, and brightness adjustment; "∆%" represents the percentage performance improvement relative to the baseline method (Base).

[0091] As shown in Table 1, among various tail class augmentation strategies, the two-stage strategy proposed in this invention and its combination with augmentation and data augmentation are the most robust overall on all classification backbones, generally outperforming texture augmentation, contrastive learning, and conventional data augmentation. Among them, the two-stage strategy proposed in this invention is the only method that can simultaneously improve the accuracy (ACC) of tail class Class 2 and head class Class 3 on all backbone networks, indicating that it can effectively alleviate class confusion and suppress the bias caused by long-tail distribution.

[0092] Categories on the Severstal dataset Fraction( ) and average F1 score ( The results are shown in Table 2.

[0093] Table 2. Categories on the Severstal dataset Score and average F1 score

[0094]

[0095] Note: These represent the individual F1 scores for the background class and the four types of defects, respectively.

[0096] As shown in Table 2, on the Severstal test set, the method of this invention, using a two-stage strategy for the classification branch, achieved the best average F1 score of 0.9498 and improved the F1 score of tail class Class 2 to 0.9123, which is significantly better than the single classification model, the single detection model, and the simple voting method. This indicates that the advantages of this invention are not only reflected in the overall performance, but more importantly, it significantly improves the identification of the most difficult and easily missed tail class defects, while maintaining high accuracy for head and background classes. This demonstrates the practical effectiveness of the complementary fusion of classification and detection models, two-stage classification, and threshold-weight joint optimization in the identification of long-tail steel defects.

[0097] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for classifying steel surface defects in long-tailed distribution scenarios, characterized in that, Includes the following steps: S1. Obtain a sample set of original images of the steel surface and preprocess the original images; the preprocessing includes size normalization, pixel normalization, and data augmentation; divide the preprocessed sample set into a training set and a validation set; S2. Select an inference efficiency that meets the requirements for industrial deployment. The models constitute the activity model set. The activity model set includes at least one classification model and one detection model; S3. Train the classification model and detection model in the activity model set respectively; S4. Input the image of the steel surface to be identified into each model in the active model set trained in step S3, and obtain the original output score of each model. The model for the first The output of the class defect is denoted as ; Map the multiple bounding box level probabilities output by the detection model to image-level category scores: Let the first... Each detection model outputs The candidate detection box, the first The detection box belongs to the first... The predicted probability of class defects is The image-level classification score corresponding to this detection model is defined as: ； S5. Based on the confidence thresholds corresponding to each model. The outputs of each model are evaluated to obtain a binary decision result: ； in, Indicates the first The model for the first The judgment result of the class defect; S6. The decision results of each model are assigned according to their corresponding weights. Perform weighted integration to obtain the fusion scores for each category; let the first category be... The fusion weights of the models are Then for the first The fusion score for class defects is defined as follows: ； S7. Based on the fusion score The system generates the final steel surface defect classification results using preset decision rules; it employs a preset decision rule that combines background priority decision, majority voting decision, and default background fallback. First, determine if the background priority condition is met: if at least one... If a model determines that a sample has a defect-free background, it will directly output a defect-free result and will not continue to perform specific defect category determination. If the background priority condition is not met, then a majority vote is performed for each defect category; for any defect category When its fusion score satisfies: ； Judgment No. Class defects exist; If none of the defect categories meet the majority voting criteria, the default background fallback strategy is adopted to classify the sample as defect-free. S8. Set the confidence thresholds for each model on the validation set. and model weight set Joint optimization is performed to obtain the optimal threshold set. and the optimal weight set .

2. The method for classifying steel surface defects according to claim 1, characterized in that, The classification model uses four convolutional neural network models: ResNet, VGG, EfficientNet, and ConvNeXt; the detection model uses three object detection network models: Faster R-CNN, YOLO8, and YOLO11.

3. The method for classifying steel surface defects according to claim 2, characterized in that, The classification model in step S4 is trained using a two-stage classification strategy. The first stage trains a binary classification model to determine whether the input image has defects, and the second stage trains a multi-label classification model to identify the specific defect category of samples identified as defective in the first stage. The classification model training specifically includes the following steps: (1) Construct a binary classification model for the input of the first class. Image of a steel surface Perform defect / no-defect discrimination to obtain binary classification output probabilities: ； in, This represents the logit output by the first-stage binary classification model. This represents the Sigmoid activation function. This represents the probability that the input image is a defective sample; (2) When If the input image is defective, it is determined to be a defective sample and sent to the second-stage classification model; otherwise, it is determined to be a defect-free sample. (3) Defect category identification is performed using a multi-label classification model, which is defined as follows: ； in, Indicates the first Scoring function for class defects, Indicates the total number of defect categories; (4) The following loss functions are used for model training in both stages: ； ； in, This represents the loss function for the first stage. This represents the loss function for the second stage; Indicates the total number of training samples; Represents cross-entropy loss, This indicates the set of samples that were deemed defective in the first phase. Indicates the first The binary classification labeling of the samples in the first stage. Indicates the first The sample about the first Labeling of class defects.

4. The method for classifying steel surface defects according to claim 2, characterized in that, For the Faster R-CNN object detection model, its training objective includes both classification and regression losses at the Region Candidate Network (RPN) stage and the Region of Interest (RoI) stage. The total loss function is: ； in, This represents the network classification loss for candidate regions. This represents the regression loss of the candidate region network. This represents the classification loss for candidate regions. Indicates the regression loss of the candidate region; For the YOLO8 and YOLO11 detection models, the loss function is expressed as: ； in, This represents the bounding box regression loss. Represents the category classification loss. This indicates a loss of the target's existence. , , These represent the weighting coefficients of the corresponding loss terms.

Images