Synthetic Abnormal Data Generation for Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The imbalance in data quantity between normal and abnormal data poses challenges in training machine learning models for anomaly detection, leading to potential misclassification of normal data as abnormal, especially in scenarios where collecting defective product images is difficult.
Innovation Solution
A method is introduced to generate abnormal data from normal data using a random walk process, where pixel data is exchanged within a kernel based on predefined movement directions and distances, and an unsupervised learning approach is used to create a reconstruction model for anomaly detection, reducing computational load and enhancing detection performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If unsupervised learning is used to train anomaly detection models, then the need for abundant abnormal data is reduced, but misclassification of normal data as abnormal occurs
Solution Approach 1:
The patent applies preliminary action by generating synthetic abnormal data before training the anomaly detection model. The random walk process is executed in advance to create labeled abnormal data samples from normal data, which are then used to train the model. This preliminary data generation resolves the contradiction by providing sufficient abnormal data for training while maintaining detection accuracy through controlled data transformation.
2Difficulty of detecting and measuring
If machine vision and artificial intelligence are used for defect detection, then detection capability is improved, but computational load and equipment requirements increase
Solution Approach 1:
The patent applies copying by creating synthetic abnormal data samples that replicate the characteristics of actual defective products. Instead of requiring complex computational models to detect rare defects directly, the system copies normal data and transforms it into synthetic abnormal data through the random walk process. This allows the model to learn from replicated patterns rather than requiring high computational power to detect rare real defects.
3Reliability
If the neural network size is increased to improve anomaly detection performance, then detection accuracy improves, but computational requirements and equipment complexity increase
Solution Approach 1:
The patent applies parameter changes by modifying the data distribution parameters through the random walk process. Instead of increasing neural network complexity, the system changes the parameters of the training data by exchanging pixel values according to predefined movement directions and distances. This transforms normal data into synthetic abnormal data with specific statistical characteristics, allowing the model to achieve high detection performance with simpler network architecture.
Data Source
AI summary
An electronic device and method are provided for generating abnormal data among learning data required for training a machine learning model. In addition, an electronic device and method are provided for a product anomaly detection using a model trained based on unsupervised learning. Embodiments may solve the data imbalance problem for learning by generating as much abnormal data as necessary based on normal data and provide an unsupervised learning-based anomaly detection method and electronic device that can provide stable performance without increasing the size or complexity of the artificial neural network and thus requiring relatively high-performance computing equipment.