Surface defect detection method based on multi-scale feature fusion and mixed supervision

Abstract

The invention discloses a surface defect detection method based on multi-scale feature fusion and mixed supervision, and the method specifically comprises the following steps: 1, inputting a normal sample of an image-level label and an original defect sample of a pixel-level label, carrying out the nonlinear edge soft data enhancement of the normal sample, and generating a synthetic defect sample of the pixel-level label; 2, constructing a segmentation sub-network, extracting a multi-scale feature map, fusing a defect segmentation mask, and generating a multi-scale fusion feature representation; 3, constructing a classification sub-network, and introducing a cross attention module focusing shallow layer features to recover micro defect details represented by multi-scale fusion features; 4, optimizing a joint loss function of segmentation and classification tasks by adopting a mixed supervised training strategy; and outputting defect probability prediction by using the trained classification sub-network. According to the method, the labeling workload can be reduced, the cost is reduced, and the accuracy and robustness of defect detection are improved.

Description

TECHNICAL FIELD

[0001] The technical field of the application is the technical field of industrial surface defect detection, and particularly relates to a surface defect detection method based on multi-scale feature fusion and mixed supervision. BACKGROUND

[0002] In large-scale industrial production, surface defect detection faces two core challenges: defect sample scarcity and high labeling cost. Existing research shows that the defect occurrence rate in industrial scenarios is often as low as one in ten thousand, and pixel-level labeling requires a lot of manpower, especially when the defect boundary is blurred, the labeling difficulty increases significantly. To solve this problem, the research field has developed unsupervised, weakly supervised, and mixed supervision methods. Unsupervised methods only use normal samples to train the model, and identify defects through anomaly detection; weakly supervised techniques rely on image-level labels to avoid detailed pixel-level labeling. However, the detection accuracy of these methods is generally lower than that of fully supervised models, prompting researchers to explore mixed supervision frameworks that combine a small amount of pixel-level labeling with a large number of image-level labels to reduce labeling costs while improving accuracy. The limitations of existing mixed supervision methods mainly lie in the inadequate processing of multi-scale features. Therefore, the existing technology article (Sun Boyan, Wang Hongyuan, Liu Qian, Feng Zundeng, Tang Ying. Mixed supervision metal surface defect detection based on multi-scale and attention mechanism [J]. Journal of Intelligent Systems, 2023, 18(4): 886-893.) uses an improved multi-scale feature fusion module to enhance the model's ability to extract defect features of different shapes and explore the impact of scale on accuracy; and fuses the Non-Local Block attention mechanism unit to enhance the model's ability to recognize defect features and improve the accuracy of defect detection; uses mixed supervision training to explore the impact of the number of pixel-level labels on accuracy. However, this method mainly focuses on optimizing global features with attention, and lacks targeted enhancement of shallow detail features. For example, the global context attention module used in this method tends to lose edge and texture information of shallow defects when fusing multi-scale features, resulting in a decrease in the detection accuracy of small defects. In addition, the limitations of data augmentation strategies cannot be ignored. Traditional data augmentation methods (such as copying and pasting defect areas) often produce artificial edge artifacts, affecting the model's generalization ability. This method does not explicitly mention data augmentation strategies, but existing mixed supervision methods generally lack processing of natural transitions of defect edges, resulting in distribution differences between synthetic samples and real defects. Although existing mixed supervision methods combine multi-scale features and attention mechanisms, they have not yet addressed the following issues: (1) loss of shallow detail features during multi-scale feature fusion; (2) insufficient targeted enhancement of attention modules for small defects; (3) natural transition processing of defect edges in data augmentation. SUMMARY

[0003] The present application aims to provide a surface defect detection method based on multi-scale feature fusion and mixed supervision. The present application can reduce the labeling workload, reduce the cost, and improve the accuracy and robustness of defect detection.

[0004] The technical solution of the present application is a surface defect detection method based on multi-scale feature fusion and mixed supervision, which specifically comprises the following steps: Step 1: input normal samples with image-level labels and original defect samples with pixel-level labels, obtain defect segmentation masks from the original defect samples, then perform nonlinear edge soft data enhancement on the normal samples to generate synthetic defect samples with pixel-level labels; Step 2: construct a segmentation subnetwork, mix the synthetic defect samples with the normal samples to obtain a dataset, extract multi-scale feature maps from the dataset through the segmentation subnetwork and fuse the defect segmentation masks to generate multi-scale fusion feature representations; Step 3: construct a classification subnetwork, introduce a cross-attention module focusing on shallow features in the classification subnetwork, fuse the shallow detail features and deep semantic features of the multi-scale fusion feature representations using the cross-attention module to restore the tiny defect details of the multi-scale fusion feature representations; Step 4: adopt a mixed supervision training strategy, combine the pixel-level labels and image-level labels in the dataset to optimize the joint loss function of the segmentation and classification tasks; use the trained classification subnetwork to output defect probability predictions.

[0005] In the foregoing surface defect detection method based on multi-scale feature fusion and mixed supervision, in step 1, the process of obtaining defect segmentation masks from the original defect samples with pixel-level labels is to crop the defect regions and perform geometric transformation, then perform 0 element padding and size adjustment to restore the original size.

[0006] In the foregoing surface defect detection method based on multi-scale feature fusion and mixed supervision, the process of performing nonlinear edge soft data enhancement on the normal samples is to calculate the distance of each pixel to the defect center based on the defect mask to generate a distance matrix, then map the distance matrix to a weight matrix, the formula is: ; wherein, is the weight matrix of the defect region, is the normalized distance from the defect center to the edge; Then, the weight matrix is used to perform smooth fusion on the defect regions and background images of the normal samples according to the weight : to obtain the synthetic defect samples.

[0007] In the foregoing surface defect detection method based on multi-scale feature fusion and mixed supervision, the segmentation sub-network extracts multi-scale features using a 5x5 convolution kernel, and receives at least three feature maps of different scales 、 、 and the corresponding defect segmentation mask 、 、 The feature maps of different scales and the corresponding defect segmentation masks are integrated by layer-by-layer fusion.

[0008] In the foregoing surface defect detection method based on multi-scale feature fusion and mixed supervision, in step 3, the classification sub-network includes a convolution block, a down-sampling layer, a cross-attention module based on focused shallow features, and a fully connected layer; the classification sub-network starts with a convolution block, then performs down-sampling, and recombines with the previous defect segmentation mask using the cross-attention module based on focused shallow features, and fuses in the channel dimension through the convolution block, repeated three times, and finally outputs the defect probability through the fully connected layer.

[0009] In the foregoing surface defect detection method based on multi-scale feature fusion and mixed supervision, the specific process of fusing shallow detail features and deep semantic features using the cross-attention module based on focused shallow features is as follows: Step 3.1: input shallow feature map and deep feature map ; Step 3.2: calculate the channel average and maximum values of the shallow feature map , obtaining two single-channel maps of shallow features; calculate the channel average and maximum values of the deep feature map after spatial pyramid pooling, obtaining two single-channel maps of deep features; Step 3.3: concatenate, convolve and weight fuse all single-channel maps to calculate the fusion output: ; wherein, is the weight coefficient.

[0010] In the foregoing surface defect detection method based on multi-scale feature fusion and mixed supervision, in step 4, the training loss function of the mixed supervision network is: ; wherein, is the cross-entropy loss of the segmentation sub-network, is the cross-entropy loss of the classification sub-network, is the static classification loss coefficient input at the beginning of training, and is a dynamic balance coefficient.

[0011] In the foregoing surface defect detection method based on multi-scale feature fusion and mixed supervision, the dynamic balance coefficient and The calculation formula is as follows: ; ; Among them, represents the current training round, represents the total training round.

[0012] Compared with the prior art, the present application has the following beneficial effects: The present application can realize efficient defect detection under limited labeled data, reduce the labeling workload and cost, while ensuring detection accuracy, and is suitable for the case of defect sample scarcity in industrial environment. The present application also fuses multi-scale features to capture defect features of different sizes, effectively improves the accuracy of defect detection, overcomes the limitations of traditional single-scale methods, and enhances the detection ability of complex defects. The present application introduces a cross-attention module based on focused shallow features, which further improves the recovery ability of small defect details by fusing shallow detail features and deep semantic features, ensuring that even in the case of small defects, high detection accuracy can be maintained. The present application trains the mixed supervision network and dynamically adjusts the balance coefficient, optimizes the cooperation of segmentation and classification tasks, so that high-precision defect detection can be realized under sparse pixel-level annotation, and has strong robustness and practicality. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is the overall network architecture diagram of the present application; Figure 2 is the non-linear edge soft data enhancement flowchart of the present application; Figure 3 is a multi-scale feature and mask level fusion module diagram of the present application; Figure 4 is a CAMFSF module structure diagram of the present application; Figure 5 is a cross-attention classification subnetwork based on focused shallow features of the present application; Figure 6 is a defect schematic diagram in the data set used by the present application; Figure 7 is a DAGM result comparison diagram of the present application when N=5; Figure 8 is a defect sample of KolektorSDD2 when N=16 shown by the present application. DETAILED DESCRIPTION

[0014] The application will be further described below in conjunction with the accompanying drawings and examples, but not as the basis for limiting the application.

[0015] Example: A surface defect detection method based on multi-scale feature fusion and mixed supervision, the overall network architecture of the method is as shown in Figure 1 The architecture runs under mixed supervision, uses a small amount of pixel-level annotation and a large number of image-level labels, increases the diversity of defect samples through a data augmentation stage, and then generates multi-scale feature maps and defect segmentation masks by a segmentation subnetwork. These outputs are then processed by a classification subnetwork, which uses a cross-attention module based on focused shallow features (CAMFSF) to restore the fine details lost in deep layers, and finally obtains a defect probability prediction, which includes the following steps: Step 1: input image-level label normal samples and pixel-level annotated original defect samples, obtain defect segmentation masks from the original defect samples, then perform non-linear edge soft data augmentation on the normal samples to generate pixel-level annotated synthetic defect samples; In this step, as shown in Figure 2 The process of obtaining defect segmentation masks from pixel-level annotated original defect samples is to crop the defect area and perform geometric transformation (such as scaling, rotation) to diversify the size and position, and then perform 0 element padding and size adjustment to restore the original size.

[0016] The process of performing non-linear edge soft data augmentation on normal samples is to calculate the distance from each pixel to the center of the defect based on the defect mask, generate a distance matrix, then map the distance matrix to a weight matrix through a Sigmoid function, the formula is: ; Wherein, is the weight matrix of the defect area, is the normalized distance from the defect center to the edge, ranging from 0 to 20; Then, the weight matrix is used to smooth the fusion of the defect area and the background image of the normal sample according to the weight : to obtain the synthetic defect sample.

[0017] Step 2: build a segmentation subnetwork, mix the synthetic defect samples with the normal samples to obtain a dataset, extract multi-scale feature maps and fuse defect segmentation masks through the segmentation subnetwork to generate multi-scale fusion feature representations; In this step, the segmentation subnetwork is used to extract multi-scale features and fuse them with the defect segmentation mask, so as to realize accurate positioning of the anomaly. The segmentation subnetwork adopts a 5 × 5 convolution kernel to reduce the depth of the layer, while retaining the global feature understanding while reducing the computational complexity and training time. As shown in Figure 3 , the multi-scale feature and mask level fusion module receives at least three feature maps of different scales 、 and from the funnel-shaped network 、 and , together with the corresponding defect segmentation masks . These feature maps are fused one by one by scale, spatially optimized by CAMFSF, and down-sampled to integrate information at all levels, thereby generating a multi-scale fusion feature representation.

[0018] This method solves the problem of different sizes of industrial defects (from large scratches to tiny spots) and overcomes the limitations of single-scale methods. By retaining the feature details of each convolutional layer and outputting defect segmentation masks at multiple resolutions, the segmentation subnetwork enhances the generalization ability and supports subsequent classification, meeting the demand for accurate defect mapping in industry while requiring only minimal labeled data.

[0019] Step 3: Constructing a classification subnetwork, introducing a cross-attention module focusing on shallow features in the classification subnetwork, and using the cross-attention module to fuse the shallow detail features and deep semantic features of the multi-scale fusion feature representation, to restore the tiny defect details of the multi-scale fusion feature representation; In this step, the classification subnetwork includes a convolution block, a down-sampling layer, a cross-attention module based on focusing on shallow features, and a fully connected layer; as shown in Figure 5 , the classification subnetwork starts with a convolution block (two-dimensional convolution, normalization, and ReLU), followed by down-sampling, and then recombines with the previous defect segmentation mask using the cross-attention module based on focusing on shallow features, and fuses them in the channel dimension through a convolution block, repeated three times, and finally outputs the defect probability through a fully connected layer.

[0020] As shown in Figure 4 , the specific process of using the cross-attention module based on focusing on shallow features to fuse shallow detail features and deep semantic features is as follows: Step 3.1: Input shallow feature map and deep feature map . Step 3.2: Calculate the channel average and maximum of the shallow feature map to obtain two single-channel maps of the shallow features; calculate the channel average and maximum of the deep feature map After the hollow space pyramid pooling enhances the representation ability, the average and maximum values of the channel are calculated to obtain two single-channel images of deep features; Step 3.3: All single-channel images are fused by splicing, convolution and weight weighting generated by the Sigmoid function, and the fusion output is calculated: ; Among them, represents the weight coefficient.

[0021] This process re-introduces the shallow features into the deep representation according to their relevance, enhancing the detection of small defects.

[0022] Step 4: A hybrid supervision training strategy is adopted to combine pixel-level labeling and image-level labels in the dataset to optimize the joint loss function of segmentation and classification tasks; the trained classification subnetwork is used to output defect probability prediction.

[0023] In this embodiment, the hybrid supervision network is a supervision method combining a small amount of pixel-level labeling and image-level labels, and includes the cooperative training of the segmentation subnetwork and the classification subnetwork.

[0024] The training loss function of the hybrid supervision network is: ; Among them, is the cross-entropy loss of the segmentation subnetwork, is the cross-entropy loss of the classification subnetwork, is the static classification loss coefficient input at the beginning of training, and is a dynamically adjusted balance coefficient, and the dynamically adjusted balance coefficient and The calculation formula is as follows: ; ; Among them, represents the current training round, represents the total training round.

[0025] For normal samples and defect samples with pixel-level labels, the cross-entropy loss of the segmentation subnetwork is activated; otherwise, it is set to zero, which embodies the hybrid supervision.

[0026] The method is evaluated on four benchmark datasets (DAGM, KolektorSDD, KolektorSDD2 and Sevestal Steel Dataset), and is compared with the latest unsupervised, hybrid supervision and fully supervised methods.

[0027] The evaluation used four datasets, each reflecting real-world industrial defect detection challenges, ensuring a comprehensive testing of the method: DAGM Dataset: This dataset contains grayscale images simulating ten different surface types, synthetically generated to mimic industrial materials. Each type contains various defects—scratches, spots, or dents—as a binary classification problem (defective vs. non-defective). Originally defined with six classes, four additional classes were later introduced, totaling ten. Images are typically 512x512 pixels, with varying sizes and visibility of defects, providing a controlled yet diverse testing environment.

[0028] Kolektor SDD: This dataset originates from real-world production environments, containing grayscale images of industrial surfaces primarily showcasing pronounced cracks. Due to the small sample size (e.g., only a few dozen images per fold), a three-fold cross-validation approach was employed to mitigate overfitting issues and ensure statistical reliability. Each image captures real manufacturing anomalies, making it a rigorous test for practical application deployment.

[0029] Kolektor SDD2: This dataset, provided by Kolektor Group d.o.o., contains color images (approximately 230x630 pixels) taken by a vision detection system under controlled conditions. It is divided into a training set (2085 negative samples, 246 positive samples) and a test set (894 negative samples, 110 positive samples). Defects are annotated through fine segmentation masks, ranging from minor scratches and spots to large surface irregularities, varying in shape, size, and color intensity, reflecting industrial variability.

[0030] Severstal Steel Defect Dataset: This extensive dataset contains 12568 grayscale images (1600x256 pixels) spanning four defect classes, representing anomalies in steel production such as scratches, dents, and excess material. Due to its size, the study used a subset: all defective samples and normal samples from class 3 (the most common), ensuring both manageability and representativeness for evaluating the scalability of the method.

[0031] Figure 6 Representative defect samples from these datasets are shown. For example, DAGM images display subtle scratches, Kolektor SDD presents pronounced cracks, Kolektor SDD2 showcases various color anomalies, and Severstal Steel contains a diverse range of steel imperfections, collectively challenging the adaptability of the invention across different types of defects and imaging conditions.

[0032] Experiments were conducted on a Linux system equipped with an NVIDIA GeForce RTX 4090 GPU, using Python 3.8, CUDA 12.1, and the PyTorch framework implementation. Hyperparameters were adjusted for each dataset to optimize training: DAGM: batch size of 1, learning rate of 0.05, and δ = 1.

[0033] KolektorSDD: batch size of 1, learning rate of 1, and δ = 0.01.

[0034] KolektorSDD2: batch size of 1, learning rate of 0.01, and δ = 1.

[0035] Severstal Steel: batch size of 10, learning rate of 0.1, and δ = 0.1.

[0036] The training employed a stochastic gradient descent method with either no momentum or weight decay, and gradient flow from the classification to the segmentation subnetwork was disabled to isolate task-specific learning. To address dataset imbalance, an undersampling strategy was employed to match the number of defect samples to the normal sample subset within each training cycle. Nonlinear edge smoothing was used to augment normal samples, ensuring diversity while keeping the frequency of defect sample patches equal. Morphological dilation was applied to narrow annotations, expanding the labels for distance transform weighting.

[0037] The method was evaluated under mixed supervision by varying the number of pixel-level labeled normal samples (N), while using image-level labels for all remaining samples. It was compared to unsupervised methods (F-AnoGAN, Uninformed Students), mixed supervision (SegDec-Net), and fully supervised methods (Seg / Dec, Extended Seg / Dec). Performance evaluation prioritized AP (Area Under the Precision-Recall Curve), a key metric in industrial quality control for assessing detection reliability at different thresholds. For DAGM and KolektorSDD2, F1 score (Harmonic Mean of Precision and Recall) and AUC (Area Under the ROC Curve) were supplemented to AP to provide a comprehensive view of classification performance. These metrics are crucial for reducing false positives (waste reduction) and false negatives (quality assurance), aligning with industrial operational needs.

[0038] Performance of each method on the DAGM dataset: Table 1 shows the AP values for N = {0, 5, 15, 45, N allAP scores for the ten classes. When N = 0, the average precision (AP) of the present method reached 65.5%, significantly outperforming F-AnoGAN (19.5%) and Uninformed Students (66.8%), which reflects its ability to effectively leverage weak labels. When N = 5, the AP rose to 99.9%, an 8.4 percentage point (9.2% relative) improvement over SegDec-Net (91.5%), with an F1 score of 99.6% (versus 92.3%) and an AUC of 100% (versus 94.9%). When N = 15, the AP reached 100% for all classes, surpassing the fully supervised baselines, indicating that performance saturation can be achieved with minimal annotation effort.

[0039] Table Performance of each method on the DAGM dataset

[0040] Figure 7 The detection results at N = 5 are shown, which demonstrate the ability of the present invention to accurately identify subtle defects in textured backgrounds (e.g., slight scratches), while SegDec-Net performs poorly in this regard, highlighting the value of multi-scale fusion and CAMFSF.

[0041] Performance of each method on the Kolektor SDD dataset: Table 2 lists the AP values at N = {5, 10, 15, 20, 33}. At N = 5, the present invention achieved an accuracy of 99.0%, surpassing Uninformed Students (57.1%) and SegDec-Net (99.1%), demonstrating its robustness under sparse labels.

[0042] Table 2 AP progression for Kolektor SDD

[0043] From Table 2, it can be found that the AP steadily rises, reaching 100% at N = 20, surpassing Seg / Dec (98.8%), demonstrating that a small-scale labeled set is sufficient to achieve top performance in real-world crack detection tasks. Three-fold cross-validation ensures that these results are statistically reliable, reflecting the feasibility of real-world industrial applications.

[0044] Performance of each method on the Kolektor SDD2 dataset: Table 3 shows the AP values for N = {0, 16, 53, 126}. At N = 0, the average precision (AP) of the present invention is 88.3%, which is 15 percentage points higher than 73.3% of SegDec-Net (20.5% relative improvement), its F1 score is 84.1% (compared to 72.7%), and the AUC value is 94.9% (compared to 90.3%), which highlights its effectiveness without pixel-level labels.

[0045] Table 3 Kolektor SDD2 AP gain and index improvement

[0046] When N = 16, the AP climbs to 91.4%, surpassing the extended Seg / Dec (91.0%) and reducing the labeling time by about 80%. Figure 8 Defect sample results for N = 16 are shown, the present invention can accurately classify defect-free images and minimize false positives, which is a key industrial advantage.

[0047] Table 4 analyzes the components of the present invention on Kolektor SDD2, we experiment with weak supervision (N = 0) and mixed supervision (N = 16) respectively. When N = 0, the data augmentation module can improve 1.4% relative to the final model, the multi-scale fusion module can improve 3%, and the CAMFSF can improve 10.2%. When all three modules are added to the baseline model, the AP increases to 88.3% (+19.6%). When N = 16, the data augmentation module can improve 3.3% relative to the final model, the multi-scale fusion module can improve 2.1%, and the CAMFSF can improve 1.8%. When all three modules are added to the baseline model, the AP increases to 91.4% (+2.8%).

[0048] Table 4 ablation experiment results

[0049] The complete model (accuracy of 91.4%) confirms the key role of each module: edge smoothing reduces artifacts, multi-scale fusion captures various abnormalities, CAMFSF (center attention-based multi-scale feature fusion) optimizes classification, and overall performance exceeds the benchmark model.

[0050] Performance of each method on the Severstal Steel dataset: Table 5 evaluates the AP when N all = {750, 1500} and N = {10, 50, 150, 300, 750, 1500}. For N all= 1500, N = 750, which outperforms SegDec-Net (98.6%) and the extended segmentation / decoding method (99.0%) while reducing the annotation effort by half. For N all = 750, N = 50, which outperforms N all = 1500, N = 10 (92.1%), indicating that it also performs efficiently on smaller datasets. This scalability is suitable for large-scale industrial scenarios with varying data volumes.

[0051] Table 5. Performance of Severstal Steel AP on N all / N

[0052] The inference speed of the present invention is as follows: 15 fps (DAGM, 512x512), 6 fps (Kolektor SDD, 512x1408), 22 fps (Kolektor SDD2, 232x640), and 10 fps (Severstal Steel, 1600x256). Real-time detection can be achieved for images with less than 410,000 pixels, meeting the industrial throughput requirements.

[0053] The superior performance of the present invention is attributed to its ability to address two key challenges in surface defect detection: the scarcity of annotated defect samples and the multi-scale nature of defects. The experimental results in Section 4.4 (Tables 1-4) show that its performance is consistently superior compared to unsupervised methods (e.g., F-AnoGAN, Uninformed Students), hybrid supervised baselines (e.g., SegDec-Net), and fully supervised methods (e.g., Seg / Dec, Extended Seg / Dec). For example, on the DAGM dataset, the present invention achieves an average precision (AP) of 99.9% with only five pixel-level labels (N = 5), which is a relative improvement of 9.2% over SegDec-Net's 91.5% (Table 1). This leap is also evident on the Kolektor SDD2 dataset, where with zero pixel-level labels (N = 0), the AP reaches 88.3%, which is a relative improvement of 20.5% over SegDec-Net's 73.3% (Table 3); on the Severstal Steel dataset, with N = 750 and N all = 1500, which exceeds the 99.0% of the fully supervised Extended Seg / Dec (Table 4).

[0054] Non-linear edge-smoothing data augmentation plays a crucial role by generating realistic synthetic defect samples. Adding this module to the baseline model on the Kolektor SDD2 dataset improves the AP from 86.9% to 88.3% (+1.4%), mitigating the artificial edge artifacts commonly found in copy-paste methods. This augmentation ensures that the training samples are close to the real-world distribution, enhancing the model's robustness - a key advantage over previous augmentation strategies that neglect edge blending. The multi-scale feature and defect segmentation mask fusion subnetwork further amplifies performance by capturing defects at different scales, from small scratches to large cracks. Table 5 shows that adding this module improves the AP from 85.3% to 88.3% (+3%), reflecting its ability to preserve low-level details and high-level semantics, while SegDec-Net struggles with small anomalies due to its single-scale processing.

[0055] The CAMFSF module perfects this synergy by restoring deep, lost fine features. Its impact is evident in Table 5, improving the AP from 78.1% to 88.3% (+10.2%) on the Kolektor SDD2 dataset and from 85.3% to 88.3% (+3%) on the DAGM dataset. Figure 7 In this invention, the SegDec-Net's overlooked subtle DAGM defects are detected. By dynamically weighting shallow features across attention, CAMFSF ensures sensitivity to small defects, which is crucial for industrial applications, as neglecting minor defects can compromise product integrity. The loss function balances segmentation and classification with dynamic coefficients, optimizing training under mixed supervision, enabling high precision even with sparse pixel-level annotations, such as the saturated performance on the DAGM dataset with N=15 (100% AP, Table 1).

[0056] This invention offers transformative potential for industrial quality control, focusing on practical computational solutions that achieve near-full supervision with minimal pixel-level annotations (e.g., achieving a 9.2% average precision improvement on DAGM using <10% of the labeled data (Table 1) and reducing the labeling amount by 80% on Kolektor SDD2 (Table 3)), directly addressing the labor-intensive labeling bottleneck. In practice, this means significant cost savings: a factory that inspects thousands of items daily can reduce manual labeling time from hours to minutes, reallocating resources to production or maintenance. Non-linear edge-smoothing augmentation (Table 5, +1.4% AP) ensures that synthetic data reflects real-world defects, enabling deployment in scenarios involving rare anomalies, such as aerospace composites or precision steel manufacturing.

[0057] Multi-scale fusion and CAMFSF (Table 5, +3% and +10.2% AP) enhance the ability to detect defects of different sizes, from small scratches to large cracks, as shown in Figure 7 and Figure 8The versatility of the proposed approach applies to different industrial contexts (e.g., color anomalies for Kolektor SDD2 or gray-scale depressions for Sevastal Steel), reducing false positives ( Figure 8 ) and material waste, a key economic factor. Moreover, the real-time inference speed further enhances its feasibility: 15 fps on 512x512 images support conveyor belt inspection, while 22 fps on Kolektor SDD2 are suitable for high-speed production lines, although optimization for larger images (e.g., 6 fps on Kolektor SDD) can expand its range of applications.

[0058] However, Figure 8 The limitations in the prior art highlight a practical challenge: for low-contrast defects industries (e.g., polished metals), pre-processing enhancements like contrast stretching can be required to fully exploit the present invention.

[0059] The present invention, through nonlinear edge soft data enhancement, is suitable for the case of defect sample scarcity in industrial environments while ensuring detection accuracy, enabling efficient defect detection with limited labeled data, reducing labeling workload and cost. The present invention also captures defect features of different sizes by fusing multi-scale features, effectively improving the accuracy of defect detection, overcoming the limitations of traditional single-scale methods, and enhancing the detection capability of complex defects. The present invention introduces a cross-attention module based on focused shallow features, which further improves the recovery ability of small defect details by fusing shallow detail features and deep semantic features, ensuring high detection accuracy even in the case of small defects. The present invention optimizes the synergy of segmentation and classification tasks by training a hybrid supervision network and dynamically adjusting the balance coefficient, enabling high-precision defect detection under sparse pixel-level annotations, with strong robustness and practicality.

[0060] In summary, the present invention can reduce labeling workload, reduce cost, and improve the accuracy and robustness of defect detection.

Claims

1. A surface defect detection method based on multi-scale feature fusion and hybrid supervision, characterized in that: The specific steps include: Step 1: Input the normal sample with image-level labels and the original defect sample with pixel-level annotations, obtain the defect segmentation mask from the original defect sample, and then perform nonlinear edge softening data enhancement on the normal sample to generate a synthetic defect sample with pixel-level annotations; Step 2: Construct a segmentation subnetwork, mix the synthetic defect samples with normal samples to obtain a dataset, extract multi-scale feature maps from the dataset through the segmentation subnetwork and fuse the defect segmentation masks to generate a multi-scale fusion feature representation; Step 3: Build a classification subnetwork and introduce a cross-attention module that focuses on shallow features into the classification subnetwork. The cross-attention module is used to fuse shallow detail features represented by multi-scale fusion features with deep semantic features to restore tiny defect details represented by multi-scale fusion features. Step 4: Adopt a hybrid supervised training strategy to optimize the joint loss function of the segmentation and classification tasks by combining the pixel-level annotations and image-level labels in the dataset; use the trained classification subnetwork to output the defect probability prediction.

2. The surface defect detection method based on multi-scale feature fusion and hybrid supervision according to claim 1 is characterized in that: In step 1, the process of obtaining the defect segmentation mask from the original defect sample annotated at the pixel level is to crop the defect area and perform geometric transformation, and then perform zero-element filling and size adjustment to restore the original size.

3. The surface defect detection method based on multi-scale feature fusion and hybrid supervision according to claim 1 is characterized in that: The process of nonlinear edge soft data enhancement for normal samples is to calculate the distance from each pixel to the defect center based on the defect mask, generate a distance matrix, and then map the distance matrix to a weight matrix. The formula is: ； in, is the weight matrix of the defect area, is the normalized distance from the defect center to the edge; Then, the weight matrix is ​​used to weight the defect area of ​​the normal sample and the background image. : Perform smooth fusion to obtain synthetic defect samples.

4. The surface defect detection method based on multi-scale feature fusion and hybrid supervision according to claim 1 is characterized in that: In step 2, the segmentation sub-network uses a 5×5 convolution kernel to extract multi-scale features and receives feature maps of at least three different scales. 、 、 and the corresponding defect segmentation mask 、 、 , feature maps of different scales are integrated with the corresponding defect segmentation masks through layer-by-layer fusion.

5. The surface defect detection method based on multi-scale feature fusion and hybrid supervision according to claim 1 is characterized in that: In step 3, the classification subnetwork includes a convolution block, a downsampling layer, a cross-attention module based on focused shallow features, and a fully connected layer; the classification subnetwork starts with a convolution block, then performs downsampling, and uses the cross-attention module based on focused shallow features to recombine with the previous defect segmentation mask, and fuses it through the convolution block in the channel dimension, repeating three times, and finally outputs the defect probability through the fully connected layer.

6. The surface defect detection method based on multi-scale feature fusion and hybrid supervision according to claim 1 is characterized in that: The specific process of fusing shallow detail features and deep semantic features using the cross attention module based on focusing on shallow features is as follows: Step 3.1: Input shallow feature map and deep feature maps ; Step 3.2: Shallow feature map Calculate the channel average and maximum values ​​to obtain two single-channel images of shallow features; for deep feature maps After the dilated spatial pyramid pooling, the channel average and maximum values ​​are calculated to obtain two single-channel images of deep features. Step 3.3: All single-channel images are concatenated, convolved, and weighted to fuse and calculate the fusion output: ； in, Represents the weight coefficient.

7. The surface defect detection method based on multi-scale feature fusion and hybrid supervision according to claim 1, characterized in that: In step 4, the training loss function of the hybrid supervision network is: ； in, is the cross entropy loss of the segmentation subnetwork, is the cross entropy loss of the classification subnetwork, is the static classification loss coefficient input at the beginning of training, and It is the dynamically adjusted balance coefficient.

8. The surface defect detection method based on multi-scale feature fusion and hybrid supervision according to claim 7, characterized in that: The dynamically adjusted balance coefficient and The calculation formula is as follows: ； ； in, Indicates the current training round, Represents the total number of training rounds.

Images