Anomaly Detection Training With Iterative Outlier Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection analytics for wind farms are not robust to outlier data, leading to poor performance when training data contains unhealthy asset states, requiring manual filtering or customization.
Innovation Solution
A system and method for automatically detecting and excluding outlier data during training using iterative algorithms like RANSAC and GroupSAC, followed by training anomaly detection analytics on healthy data patterns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing anomaly detection analytics are trained on historical data assuming healthy operation, then the analytics can identify normal data variation patterns, but the training data may contain outliers from unhealthy asset states causing poor performance
Solution Approach 1:
The system performs preliminary outlier detection and removal from training data before training the anomaly detection analytics. An iterative algorithm processes the training data to identify and remove outliers corresponding to unhealthy asset states, ensuring that only clean data from healthy operations is used for training, thereby improving both accuracy and robustness
Solution Approach 2:
The system employs self-service by using the anomaly detection analytics themselves to identify outliers in the training data. The analytics are iteratively applied to the training set to detect anomalies, which are then removed, creating a self-cleaning process that prepares the training data without external intervention
2Reliability
If manual filtering or customization of outlier rejection algorithms is performed, then training data quality improves, but the process becomes time-consuming and requires customization for each analytic
Solution Approach 1:
The system automates the outlier removal process by employing the anomaly detection analytics themselves to identify and flag outliers in the training data. This self-service approach eliminates the need for manual filtering or customization of rejection algorithms for each analytic, significantly reducing preparation time while maintaining high data quality
Solution Approach 2:
The system creates a universal outlier removal process that can be applied to any anomaly detection analytic without customization. The iterative algorithm works with any analytics package, making the data preparation process universally applicable across different analytics and asset types, eliminating the need for analytic-specific customization
3Productivity
If all training data is used without filtering, then the training process is simple and fast, but the anomaly detection analytics perform poorly when trained on contaminated data
Solution Approach 1:
The system performs preliminary cleaning of the training data by iteratively applying the anomaly detection analytics to identify and remove outliers before the actual training process. This preliminary action ensures that only high-quality data from healthy asset operations is used for training, improving detection performance while maintaining training efficiency through automation
Solution Approach 2:
The system uses feedback from the anomaly detection analytics themselves to identify outliers in the training data. The analytics are applied to the training set, their outputs are analyzed to identify outliers, and this feedback loop continues iteratively until the training data is cleaned, ensuring high detection performance without sacrificing training efficiency
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for detecting anomalies during operation of an asset to improve performance of the asset includes collecting, via a server, data relating to operation of the asset or a group of assets containing the asset. The data includes normal and abnormal asset behavior of the asset or the group of assets containing the asset. Further, the method includes automatically removing, via an iterative algorithm programmed in the server that utilizes one or more inputs or outputs of an anomaly detection analytic, portions of the data containing the abnormal asset behavior to form a dataset containing only the normal asset behavior. The method also includes training, via a computer-based model programmed in the server, the anomaly detection analytic using, at least, the dataset containing only the normal asset behavior. Moreover, the method includes applying, via the server, the anomaly detection analytic to the asset so as to monitor for anomalies during operation thereof.