Semi-Supervised Defect Localization With Unaligned Multi-Modal Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automated defect classification systems in display panel manufacturing are time-consuming and lack consistency due to the complexity of classifying high-resolution images, with existing machine learning approaches struggling to achieve desired speed and accuracy, especially when dealing with unaligned data from different sources.
Innovation Solution
A system utilizing a fusion neural network that combines independently trained neural network branches, each processing unaligned data from different sources, with attention modules to focus on defect areas and channels, allowing for more accurate and human-understandable defect classification by overlaying spatial and channel attention maps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex machine learning approaches are used for automated defect classification, then classification accuracy can be improved, but processing speed and system complexity worsen
Solution Approach 1:
The system divides the defect classification task into multiple independent neural network branches, each specialized for specific defect types or imaging modalities. These branches process data in parallel and their outputs are fused to generate the final classification, enabling both high accuracy and efficient processing
Solution Approach 2:
The system processes unaligned data from multiple imaging modalities (optical, electron microscopy, spectroscopy) simultaneously through separate neural network branches, adding a dimension of multi-modal processing that improves accuracy without requiring time-consuming data alignment
2Measurement precision
If multiple imaging modalities are used to improve defect detection accuracy, then measurement precision improves, but data processing complexity and time increase
Solution Approach 1:
The system creates separate neural network branches for each imaging modality (optical images, electron microscopy images, spectroscopy data), allowing each branch to independently process its specialized data type without requiring complex integration or alignment procedures
Solution Approach 2:
A fusion layer acts as an intermediary that combines the outputs from multiple independent neural network branches, integrating information from different imaging modalities while maintaining processing independence and reducing overall system complexity
3Ease of operation
If manual defect classification is performed by operators, then classification can be performed with human understanding, but time consumption and inconsistency increase
Solution Approach 1:
The neural network system performs defect classification autonomously by learning from training data, eliminating the need for manual operator intervention. The system consistently applies learned classification rules across all defects, providing both speed and consistency without human time investment
Data Source
AI summary
A system for manufacturing defect classification is presented. The system includes a first neural network receiving a first data as input and generating a first output, a second neural network receiving a second data as input and generating a second output, wherein first neural network and the second neural network are trained independently from each other, and a fusion neural network receiving the first output and the second output and generating a classification. The first data and the second data do not have to be aligned. Hence, the system and method of this disclosure allows various type of data that are collected during manufacturing to be used in defect classification.