Modular defect detection method based on data heterogeneous meta-learning
By using modular deep neural networks and data heterogeneous learning, the problems of sample scarcity and task diversity in the detection of surface defects in industrial products are solved, and efficient and accurate cross-task detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN INST OF TECH
- Filing Date
- 2025-07-24
- Publication Date
- 2026-07-07
AI Technical Summary
In the detection of surface defects in industrial products, there are problems such as difficulty in obtaining high-quality samples, high labeling costs, and diverse types of defect detection tasks. Existing methods are unable to meet the needs of multiple tasks.
A modular deep neural network approach based on heterogeneous meta-learning is adopted, including a reusable feature backbone, a switchable neck module, and a task-specific head. Through pre-training on large-scale image data and meta-learning on multi-source heterogeneous defect datasets, shared parameters are optimized to achieve rapid adaptation across tasks.
It significantly improves the accuracy and speed of defect detection under conditions with few samples, can quickly adapt to various heterogeneous defect tasks, and improves detection accuracy and generalization ability.
Smart Images
Figure CN120876429B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a modular defect detection method based on heterogeneous data learning, belonging to the fields of industrial automation and machine vision. Background Technology
[0002] Surface defect detection of industrial products is crucial for ensuring product quality, but in practical applications, it often faces two main challenges: First, high-quality defect samples are difficult to obtain and labeling costs are high, resulting in sample scarcity; second, defect detection tasks are diverse, such as classification, detection, and segmentation, and the data structures and labeling forms for different tasks are different, making it difficult for existing methods to simultaneously meet the needs of multiple tasks.
[0003] Traditional deep learning methods, such as U-Net-based segmentation or Faster R-CNN-based detection, while achieving some progress, typically require large amounts of labeled data, and their models are usually optimized for single tasks, lacking cross-task generalization ability. Existing few-shot meta-learning frameworks (such as MAML) are mainly optimized for homogeneous tasks and do not consider the heterogeneity of data structures in defect detection tasks. Therefore, there is an urgent need for a new technical solution that can fully utilize large-scale image data to obtain general features and achieve rapid adaptation and high-precision detection under various heterogeneous defect tasks. Summary of the Invention
[0004] This invention aims to solve the challenges of limited samples, cross-task requirements, and heterogeneous data in industrial surface defect detection, and proposes a modular deep neural network method based on heterogeneous data learning.
[0005] The technical solution of this invention:
[0006] A modular surface defect detection method based on heterogeneous data learning includes the following steps:
[0007] S1: Prepare a large-scale image classification dataset ImageNet-1k, a multi-source heterogeneous defect detection dataset, and a target industrial surface defect dataset;
[0008] S2: Construct a modular defect detection model UDINet, including a reusable feature backbone, a switchable neck module, and a task-specific head, where...
[0009] Reusable feature backbone network: used to extract multi-scale features from input images;
[0010] The switchable neck module includes:
[0011] The classification neck is connected to the classification head through a global average pooling layer for global feature extraction;
[0012] The neck localization module, consisting of the interest feature selection module (FS) and the feature pyramid network (FPN), is used to achieve multi-scale feature fusion.
[0013] Task-specific headers include:
[0014] A classification head, used to output image-level defect category predictions;
[0015] The inspection head is used to output the coordinates of the defect bounding box and the defect category;
[0016] The segmentation head is used to output pixel-level defect masks and their categories.
[0017] S3: The backbone network of the modular defect detection model UDINet is pre-trained using the large-scale image classification dataset ImageNet-1k to learn general image feature representations;
[0018] S4: Use a multi-source heterogeneous defect detection dataset to perform data heterogeneous learning on the reusable parameters of the modular defect detection model UDINet, and optimize the shared parameters of the reusable feature backbone and neck module;
[0019] S5: Evaluate the cross-domain defect detection performance of the modular defect detection model UDINet using a target industrial surface defect dataset.
[0020] Specifically, the multi-source heterogeneous defect detection dataset mentioned in step S1 includes: a defect classification dataset, a defect target detection dataset, and a defect semantic segmentation dataset;
[0021] The target industrial surface defect dataset includes: NEU-CLS, a dataset for classifying six types of defects on steel surfaces; PCB-DET, a dataset for detecting six types of defects on printed circuit boards; and CCM-SEG, a dataset for segmenting four types of defects in camera modules.
[0022] Specifically, in step S2: the reusable feature backbone of the modular defect detection model UDINet consists of four cascaded convolutional stages for multi-scale feature extraction; the classification neck connects to the classification head and extracts global features through a global average pooling layer; the localization neck uses the interest feature selection module FS to filter potential defect region features and connects to the feature pyramid network FPN for multi-scale feature fusion; the detection head implements multi-scale defect location regression and classification based on the anchorless FCOS structure; and the segmentation head performs pixel-by-pixel defect mask prediction on the highest resolution feature map.
[0023] Specifically, in step S3: the backbone network of the modular defect detection model UDINet is pre-trained on the large-scale image classification dataset ImageNet-1k to learn the low-level features and high-level semantic information of the image and obtain initialization parameters with good transferability to industrial defect features.
[0024] Specifically, in step S4: the modular defect detection model UDINet is constructed into multiple task-specific models, and K-shot samples are used for gradient in-process updates on each defect task; the meta-loss is calculated on multiple tasks and the shared parameters are updated, and gradient descent optimization is iteratively performed to achieve rapid adaptation to heterogeneous defect tasks.
[0025] The beneficial effects of this invention are:
[0026] 1. The modular network architecture design allows for the sharing of backbone and neck parameters, while also enabling flexible plug-and-play adaptation of task-specific headers for different defective tasks, achieving feature reuse and task-specific adaptation.
[0027] 2. Two-stage transfer learning: Meta-learning is used to optimize shared parameters, enabling the model to quickly adapt to new defect detection tasks.
[0028] 3. Region of Interest Feature Selection Mechanism: The FS module processes only potential defect regions through region clipping and feature filtering, which significantly improves inference speed while enhancing the accuracy of defect localization and segmentation.
[0029] 4. Under limited labeled sample conditions, the method of the present invention performs few-shot evaluations on various tasks such as steel surface defect classification NEU-CLS, printed circuit board defect detection PCB-DET, and camera module defect segmentation CCM-SEG. The results show that it has significant improvements in classification accuracy, detection mAP, and segmentation mIoU compared with existing technologies. Attached Figure Description
[0030] Figure 1 This is a schematic diagram of the architecture of the modular defect detection model UDINet.
[0031] Figure 2 Example output diagrams of the modular defect detection model UDINet in the PCB-DET and CCM-SEG tasks. Detailed Implementation
[0032] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments, but is not limited thereto. For clarity, the following embodiments are described in order of steps:
[0033] Step 1: Dataset Preparation
[0034] The training and evaluation datasets are prepared as follows: As shown in Table 1, the network backbone is initialized using the large-scale image classification dataset ImageNet-1k during the pre-training phase. In the heterogeneous meta-learning phase, multiple heterogeneous defect detection datasets are prepared: datasets for defect classification tasks (such as DAGM2007 and MvTecAD), datasets for defect target detection tasks (such as NEU-DET and Aero-engine), and datasets for defect semantic segmentation tasks (such as the tile defect dataset and KolektorSDD2). These datasets have different task types and data structures, providing comprehensive information on various defect patterns. In the final model evaluation phase, three target scene datasets are used: NEU-CLS (six-class defect classification dataset for steel surfaces), PCB-DET (six-class defect detection dataset for printed circuit boards), and CCM-SEG (four-class defect segmentation dataset for camera modules). The NEU-CLS dataset contains 1800 images of 6 types of steel surface defects, with a resolution of 200×200; the PCB-DET dataset contains 1386 images of 6 types of PCB defects, with a resolution of 3034×1586; and the CCM-SEG dataset contains 480 images of 4 types of camera module surface defects, with a resolution of 2048×2448.
[0035] During the evaluation phase, each dataset is typically divided into a training set, a validation set, and a test set in a 7:2:1 ratio to ensure class balance.
[0036] Table 1
[0037]
[0038] Step 2: Constructing the modular network UDINet
[0039] The UDINet model is constructed based on defect detection requirements. UDINet includes a shared backbone feature extraction network, a neck feature fusion network, and multiple task head modules. The backbone network can employ pre-trained network structures such as ResNet-50, extracting multi-scale features from the image through multiple convolutional layers. The neck fusion module aggregates features from different scales to accommodate the detection needs of both small targets and large structures; for classification tasks, it can be simplified to Global Average Pooling (GAP), while for detection and segmentation tasks, structures such as Feature Pyramid Network (FPN) can be added. Furthermore, this invention introduces a Region of Interest (FOI) feature selection (FS) module in the neck area, which concentrates computational resources by cropping potential defect regions and discarding the background, improving the processing efficiency of high-resolution images. The head modules are customized according to the task type: the classification head consists of fully connected layers or Softmax layers outputting class probabilities; the target detection head outputs bounding boxes and confidence scores; and the segmentation head outputs pixel-level masks. Through this modular design, UDINet can flexibly switch between classification, detection, and segmentation tasks while maintaining structural consistency.
[0040] Step 3: Model Pre-training
[0041] The parameters of the constructed UDINet backbone network are pre-trained. Specifically, the backbone network is trained using the ImageNet-1k large-scale image classification dataset to obtain rich general visual features such as edges, textures, and shapes. In this way, the model already possesses good feature extraction capabilities before entering the few-shot defect task, providing favorable initialization for subsequent meta-learning.
[0042] Step 4: Data Heterogeneous Meta-Learning
[0043] After pre-training, meta-learning is performed on the shared parameters of the model using a multi-source heterogeneous defect dataset. The specific algorithm steps are shown in Table 2: Following the meta-learning algorithm, a defect detection task is randomly sampled each time, which can be a classification, detection, or segmentation task. The corresponding neck and head modules are selected based on the task to form the current UDINet sub-model. Then, the data from this task is used for one or more gradient updates. For example, each task uses NK samples, where K=10, to obtain the model parameters adapted to that task. Next, the loss of the updated model on this task is calculated, and the shared parameters are adjusted using the cumulative loss across all tasks. Meta-gradient updates are performed. By repeating this process across different tasks, the model learns a set of parameters that allow it to quickly adapt to new tasks with a small number of gradient iterations. This shared backbone and neck parameters retain general visual knowledge from ImageNet while incorporating feature patterns from various defective tasks, improving cross-task generalization ability.
[0044] Table 2
[0045]
[0046] Step 5: Model Testing and Evaluation
[0047] After meta-learning is completed, the model undergoes cross-domain few-shot performance evaluation. The UDINet model trained in step four is tested on the NEU-CLS, PCB-DET, and CCM-SEG datasets. During testing, the model is fine-tuned using only a very small number of samples (e.g., 5-shot, 10-shot, etc.) for each task, and then the detection performance is verified on the test set.
[0048] Evaluation results on the NEU-CLS, PCB-DET, and CCM-SEG datasets show that the proposed method achieves an accuracy improvement of approximately 7.16% compared to the randomly initialized baseline in a 5-shot setting. Compared to the ImageNet pre-trained baseline alone, it further improves accuracy and evaluation metrics such as mAP / mIoU in various scenarios, significantly accelerating training convergence. Experimental results demonstrate that the proposed method achieves excellent performance on all three tasks: even with very few labeled samples, UDINet can accurately identify and locate various types of surface defects. Comparative experiments with random initialization and pre-training only the backbone network verify that the two-stage transfer learning strategy, including data heterogeneous learning, can effectively improve the model's detection accuracy and generalization ability. Furthermore, the introduced feature selection module, while maintaining high accuracy, improves the inference speed of the detection task from 5.62 FPS to 24.83 FPS and the inference speed of the segmentation task from 4.91 FPS to 20.76 FPS. The above results verify that the steps can effectively achieve high-precision detection and segmentation of heterogeneous defects and have good practical application value.
[0049] Table 3
[0050]
[0051] Table 4
[0052]
[0053] In summary, this invention achieves efficient and accurate defect detection in industrial defect detection scenarios characterized by scarce data and diverse tasks through a modular network structure and a two-stage learning strategy. The method is highly versatile; the UDINet network structure can be adapted to new defect tasks by replacing different neck and head modules, fully demonstrating flexibility and scalability, and providing an effective solution for industrial defect detection.
Claims
1. A modular surface defect detection method based on heterogeneous data learning, characterized in that, Includes the following steps: S1: Prepare a large-scale image classification dataset ImageNet-1k, a multi-source heterogeneous defect detection dataset, and a target industrial surface defect dataset; S2: Construct a modular defect detection model UDINet, including a reusable feature backbone, a switchable neck module, and a task-specific head, where... Reusable feature backbone network: used to extract multi-scale features from input images; The switchable neck module includes: The classification neck is connected to the classification head through a global average pooling layer for global feature extraction; The neck localization module, consisting of the interest feature selection module (FS) and the feature pyramid network (FPN), is used to achieve multi-scale feature fusion. Task-specific headers include: A classification head, used to output image-level defect category predictions; The inspection head is used to output the coordinates of the defect bounding box and the defect category; The segmentation head is used to output pixel-level defect masks and their categories. S3: The backbone network of the modular defect detection model UDINet is pre-trained using the large-scale image classification dataset ImageNet-1k to learn general image feature representations; S4: Use a multi-source heterogeneous defect detection dataset to perform data heterogeneous learning on the reusable parameters of the modular defect detection model UDINet, and optimize the shared parameters of the reusable feature backbone and neck module; S5: Evaluate the cross-domain defect detection performance of the modular defect detection model UDINet using a target industrial surface defect dataset.
2. The modular surface defect detection method based on heterogeneous element learning according to claim 1, characterized in that, The multi-source heterogeneous defect detection dataset mentioned in step S1 includes: a defect classification dataset, a defect target detection dataset, and a defect semantic segmentation dataset; The target industrial surface defect dataset includes: NEU-CLS, a dataset for classifying six types of defects on steel surfaces; PCB-DET, a dataset for detecting six types of defects on printed circuit boards; and CCM-SEG, a dataset for segmenting four types of defects in camera modules.
3. The modular surface defect detection method based on heterogeneous element learning according to claim 1, characterized in that, In step S2: the reusable feature backbone of the modular defect detection model UDINet consists of four cascaded convolutional stages for multi-scale feature extraction; the classification neck connects to the classification head and extracts global features through a global average pooling layer; the localization neck uses the interest feature selection module FS to filter potential defect region features and connects to the feature pyramid network FPN for multi-scale feature fusion; the detection head implements multi-scale defect location regression and classification based on the anchorless FCOS structure; and the segmentation head performs pixel-by-pixel defect mask prediction on the highest resolution feature map.
4. The modular surface defect detection method based on heterogeneous element learning according to claim 1, characterized in that, In step S3: the backbone network of the modular defect detection model UDINet is pre-trained on the large-scale image classification dataset ImageNet-1k to learn the low-level features and high-level semantic information of the image and obtain initialization parameters with good transferability to industrial defect features.
5. The modular surface defect detection method based on heterogeneous element learning according to claim 1, characterized in that, In step S4: the modular defect detection model UDINet is constructed into multiple task-specific models, and K-shot samples are used for gradient update on each defect task; the meta-loss is calculated on multiple tasks and the shared parameters are updated, and gradient descent optimization is iteratively performed to achieve rapid adaptation to heterogeneous defect tasks.