Self-Supervised CNN Encoder for Semiconductor Defect Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based defect classification in the semiconductor industry requires a large amount of data for model training, leading to long data collection cycles and costly data annotation, with existing methods providing limited performance boosts when using limited labeled sample data.
Innovation Solution
A method combining a CNN classification model and an anomaly detection model via location ensemble, where the CNN encoder is trained using multi-task self-supervised learning with real non-defect and synthetic defect images, and the anomaly detection model learns the distribution of real non-defect images, allowing for high accuracy defect classification and localization with limited labeled samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning-based defect classification is used, then defect classification accuracy is improved, but data collection cycle and annotation cost increase
Solution Approach 1:
The patent applies self-supervised pretraining before the main defect classification task. The model is first trained on large amounts of unlabeled semiconductor images to learn general defect patterns and representations. This preliminary training enables the model to achieve high classification accuracy on the target defect types using only a small number of labeled samples, thereby reducing the data collection cycle and annotation cost while maintaining high measurement precision.
2Measurement precision
If deep learning-based defect classification is used, then defect classification accuracy is improved, but data annotation cost increases
Solution Approach 1:
The patent employs self-supervised learning where the model generates its own training data and labels from unlabeled images. The self-supervised pretraining module creates synthetic defect images and generates defect type predictions automatically without requiring human annotation. This self-service approach enables the model to learn meaningful representations and achieve high classification accuracy with minimal labeled data, significantly reducing the quantity of annotated data needed while maintaining high measurement precision.
3Measurement precision
If semi-supervised learning is used, then performance is improved, but stability decreases due to data assumptions
Solution Approach 1:
The patent divides the training process into distinct segments: self-supervised pretraining on unlabeled data, followed by supervised fine-tuning on labeled data. The self-supervised pretraining module handles the general defect pattern learning from unlabeled images, while the subsequent supervised fine-tuning adapts the model to specific defect types using labeled data. This segmentation separates the learning tasks, allowing the model to benefit from both unlabeled data volume and labeled data accuracy without the instability caused by mixing assumptions in semi-supervised approaches.
4Measurement precision
If anomaly detection model is combined with CNN classification model via location ensemble, then localization accuracy is improved, but model complexity increases
Solution Approach 1:
The patent merges the CNN classification model and anomaly detection model into a unified architecture where both models share the same encoder backbone. The classification model processes images for defect type identification, while the anomaly detection model simultaneously generates localization maps. By combining these two models through location ensemble that integrates their outputs, the system achieves high localization accuracy without requiring completely separate models, thus managing the complexity through shared computational resources and coordinated processing.
Data Source
AI summary
A system and a method for classifying and localizing surface defects with a high accuracy rate when only a small number of labeled defect data is available. The method comprises predicting a defect class for an input image using a convolutional neural network (CNN) classification model; and predicting a defect location for the input image via location ensemble with a coarse localization map and an anomaly map, which are generated using the CNN classification model and anomaly detection model respectively. The CNN classification model is built based on a CNN encoder and trained with labeled real defect images. The CNN encoder is trained by multi-task self-supervised learning with real non-defect and synthetic defect images.