Lifelong Deep Neural Networks for Unknown Defect Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional artificial neural networks (ANNs) and deep neural networks (DNNs) struggle with real-world scenarios requiring a combination of supervised and unsupervised learning, as they are typically designed for known classes and cannot efficiently handle unknown defects or anomalies, especially in industrial quality control where precise defect detection is needed with limited training data.
Innovation Solution
Lifelong Deep Neural Network (L-DNN) technology enables real-time learning and adaptation to recognize both known and unknown defects without extensive training, using a combination of a representation-rich DNN-based subsystem and a fast-learning subsystem to process data at the edge, allowing for flexible operation and efficient anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional supervised DNNs are trained to recognize known defects with high precision, then manufacturing precision is improved, but the system cannot detect unknown defects and requires extensive retraining when new defect types appear
Solution Approach 1:
The system segments the neural network into two distinct components: a backbone network that extracts features from input data, and a head network that performs classification. This segmentation allows the backbone to learn general defect features while the head can be quickly adapted to specific defect types, enabling both high precision for known defects and adaptability to unknown defects through incremental learning.
Solution Approach 2:
The system implements dynamic adaptability by allowing the head network to be incrementally updated with new defect types without retraining the entire network. This dynamic structure enables the system to evolve and detect new defect types as they appear in production, maintaining versatility while preserving the precision learned by the backbone network.
2Measurement precision
If supervised ANNs are trained using backpropagation on large amounts of labeled data, then classification accuracy is improved, but training time and computational cost increase significantly
Solution Approach 1:
The backbone network is pre-trained on a large dataset to learn general defect features and representations. This preliminary action allows the network to capture essential patterns without requiring extensive training whenever new defect types are introduced, significantly reducing subsequent training time while maintaining high classification accuracy.
Solution Approach 2:
Instead of retraining the entire network for new defect types, the system applies partial training only to the head network with minimal labeled data. This partial action approach achieves high classification accuracy for new defects without the excessive time cost of full network retraining.
3Reliability
If traditional DNNs are designed for specific known classes, then performance on those classes is high, but the system cannot accommodate unknown classes without costly retraining
Solution Approach 1:
The backbone network serves as a universal feature extractor that can handle multiple defect types and classes. By separating feature extraction from classification, the system achieves multi-functionality where the same backbone can support various head networks for different defect types, ensuring reliable performance on known classes while accommodating unknown classes through incremental head training.
4Adaptability or versatility
If unsupervised learning methods are used to detect unknown patterns without labeled data, then adaptability to unknown defects is improved, but classification precision and recall decrease
Solution Approach 1:
The system merges supervised learning (for the backbone network trained on labeled data to ensure high precision) with unsupervised or incremental learning capabilities (in the head network that can adapt to new defect types with minimal or no labeled examples). This combination achieves both high precision through the supervised backbone and adaptability to unknown patterns through the flexible head.
Data Source
AI summary
Industrial quality control is challenging for artificial neural networks (ANNs) and deep neural networks (DNNs) because of the nature of the processed data: there is an abundance of consistent data representing good products, but little data representing bad products. In quality control, the task is changed from conventional DNN task of “recognize what I learned best” to “recognize what I have never seen before.” Lifelong DNN (L-DNN) technology is a hybrid semi-supervised neural architecture that combines the ability of DNNs to be trained, with high precision, on known classes, while being sensitive to any number of unknown classes or class variations. When used for industrial inspection, L-DNN exploits its ability to learn with little and highly unbalanced data. L-DNN's real-time learning capability takes advantage of rare cases of poor-quality products that L-DNN encounters after deployment. L-DNN can be applied to industrial inspections and manufacturing quality control.