Defect Classification with Outlier Filtering for Noisy Manufacturing Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning models for detecting manufacturing defects face challenges due to erroneous labeling and small datasets, leading to reduced accuracy and overfitting, which can result in poor performance on new data.
Innovation Solution
A two-pass approach is employed, where a first machine learning model is trained using clean data samples after filtering out noisy data, and a second model, known as an outlier filter, is trained to reject unconfident data during deployment, improving prediction accuracy by ensuring only high-confidence data is used for classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional machine learning models are trained on available manufacturing data, then the models can be deployed for defect detection, but the accuracy is reduced due to erroneous labeling and small dataset size
Solution Approach 1:
The training process is segmented into two distinct phases: first training a base model on available data, then training an outlier filter model separately to identify and reject low-confidence predictions. This segmentation allows each model to specialize - the base model learns defect patterns while the outlier filter learns to recognize uncertainty, resolving the contradiction between using available data and maintaining reliability on new data.
Solution Approach 2:
The outlier filter model acts as an intermediary between the base model and the final prediction output. It receives predictions from the base model and filters out low-confidence cases before they are finalized. This intermediary layer protects the system from erroneous predictions caused by labeling errors or small dataset limitations, thereby improving both accuracy and reliability.
2Productivity
If machine learning models are trained with small datasets, then training time and computational resources are reduced, but overfitting occurs leading to poor performance on new data
Solution Approach 1:
The outlier filter model is trained in advance on the same small dataset to learn characteristics of low-confidence predictions. By performing this preliminary action during the training phase, the system prepares a filtering mechanism that will generalize better to unseen data, compensating for the limitations of small training datasets while maintaining training efficiency.
Solution Approach 2:
The system implements feedback through the outlier filter that identifies when the base model is uncertain about its predictions. This feedback mechanism allows the system to recognize overfitting patterns and reject predictions that don't generalize well, thereby improving reliability on unseen data without requiring additional training data or increased computational resources.
3Quantity of substance
If all data samples are used for training, then the training dataset size is maximized, but noisy and erroneously labeled data reduces model accuracy
Solution Approach 1:
The outlier filter model extracts and identifies low-confidence predictions from the base model's output. By taking out these uncertain predictions, the system effectively removes the influence of noisy or erroneously labeled data from the final decision-making process, thereby maintaining the benefits of using all available training data while protecting prediction accuracy.
Solution Approach 2:
The system changes the parameter of confidence threshold by using the outlier filter to dynamically identify which predictions meet the confidence criteria. This parameter change allows the system to adaptively accept or reject predictions based on their confidence level, maximizing the use of available data while maintaining high prediction accuracy by filtering out noisy results.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and method for classifying manufacturing defects are disclosed. In one embodiment, a first data sample satisfying a first criterion is identified from a training dataset, and the first data sample is removed from the training dataset. A filtered training dataset including a second data sample is output. A first machine learning model is trained with the filtered training dataset. A second machine learning model is trained based on at least one of the first data sample or the second data sample. Product data associated with a manufactured product is received, and the second machine learning model is invoked for predicting confidence of the product data. In response to predicting the confidence of the product data, the first machine learning model is invoked for generating a classification based the product data.