Wafer Defect Image Classification Using Multi-Modal Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for classifying defects in semiconductor wafers fail to accurately determine defects by considering different aspects/modalities of defect images, leading to inaccurate classification due to the reliance on single deep learning models and limited consideration of various imaging modalities.
Innovation Solution
A method and system utilizing a deep learning network that combines multiple modalities of wafer defect images, including color, Inner Crack Imaging, and black and white images, with a reference image to focus on defect classification, employing a Directed Acyclic Graph (DAG) architecture to integrate and classify defects effectively, reducing the need for labeled images and training epochs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single deep learning model is used for defect classification, then the system complexity is reduced, but the classification accuracy deteriorates due to inability to consider different aspects/modalities of defect images
Solution Approach 1:
The patent divides the defect classification system into multiple specialized deep learning models, each trained to analyze specific imaging modalities (e.g., optical images, SEM images, AFM images). Each model segment focuses on particular defect aspects, and their predictions are aggregated to achieve comprehensive and accurate classification, resolving the contradiction between system simplicity and classification accuracy.
Solution Approach 2:
The patent combines multiple deep learning models into an integrated classification system where predictions from different models analyzing different imaging modalities are merged. This merging of multiple specialized models achieves superior classification accuracy compared to a single model, while maintaining manageable system complexity through modular architecture.
2Measurement precision
If multiple modalities of defect images are analyzed, then the classification accuracy is improved, but the computational resources and training time increase
Solution Approach 1:
The patent segments the training process by training separate deep learning models on different imaging modalities independently. This allows parallel training of specialized models on smaller, modality-specific datasets rather than training one large model on all data, reducing overall training time while maintaining the benefit of multi-modality analysis for improved accuracy.
Solution Approach 2:
The patent applies partial action by training each specialized model on a subset of the total data relevant to its specific modality, rather than training all models on the complete dataset. This reduces the computational burden and training time for each model while still achieving accurate classification through the aggregation of specialized predictions.
3Measurement precision
If multiple modalities of defect images are analyzed, then the classification accuracy is improved, but the device complexity increases due to multiple models and processing paths
Solution Approach 1:
The patent segments the classification system into independent, specialized deep learning models for different imaging modalities. Each model has a simple, focused architecture optimized for its specific modality, avoiding the complexity of a single monolithic model. The segmented architecture maintains accuracy while improving manageability and reducing overall system complexity.
Solution Approach 2:
The patent creates a universal classification framework where multiple specialized models work together through a common aggregation mechanism. This multi-functional system handles different imaging modalities uniformly, achieving high classification accuracy across various defect types while maintaining a standardized, manageable system architecture.
Data Source
AI summary
The present disclosure provides method and system 100 for classifying defects in wafer using wafer defect images, based on deep learning network. Embodiments herein uses synergy between several modalities of the wafer defect images for the classification decision. Further, by adding a mixture of modalities, information may be obtained from different sources such as color image, ICI, the black and white image, to classify the defect image. In addition to mixture of modalities, a reference image may be used for each modality. The reference image of each modality image is provided to deep learning models to concentrate on the defect itself and not on the related underlying lithography of the defect image. Further, the reference image may be provided to the training process of the deep learning models that may significantly reduce the number of labelled images and the training epochs required for convergence of the deep learning model.