Unsupervised Sensor Anomaly Detection With Majority Voting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional anomaly detection methods for industrial IoT applications face challenges in scaling due to ill-conditioned sensor data with missing, corrupted, and noisy values, lacking labeled data, and requiring manual data pre-processing, which leads to unreliable anomaly detection.
Innovation Solution
A system that preprocesses sensor data using data exploration techniques, including data cleansing and feature engineering, to select relevant feature sensors, builds pair-wise univariate models, and applies unsupervised machine-learning with weighted majority voting to detect anomalies in industrial machines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional anomaly detection methods are applied to industrial IoT sensor data, then anomaly detection can be performed, but the reliability is low due to ill-conditioned data with missing, corrupted, and noisy values
Solution Approach 1:
The patent applies data pre-processing techniques including imputation for missing values, outlier detection and treatment, and noise filtering before anomaly detection. These preliminary actions clean the ill-conditioned sensor data, transforming it into a suitable format for reliable anomaly detection while addressing the harmful data quality issues proactively
Solution Approach 2:
The patent introduces intermediate processing layers including feature engineering, dimensionality reduction, and data transformation techniques that act as mediators between the raw ill-conditioned sensor data and the anomaly detection algorithms. These intermediaries transform the data into a more suitable representation that preserves anomaly information while reducing the impact of data quality issues
2Reliability
If manual data pre-processing is performed with prior domain knowledge, then data quality can be improved, but the automation level is low and scalability is poor
Solution Approach 1:
The patent implements automated data pre-processing pipelines that self-adjust and self-optimize based on the characteristics of the input sensor data. The system automatically selects appropriate pre-processing techniques, tunes parameters, and adapts to different data sources without requiring manual domain knowledge input, thereby achieving both high reliability and full automation
Solution Approach 2:
The patent employs adaptive parameter adjustment mechanisms that automatically modify pre-processing parameters based on data characteristics. The system dynamically changes parameters such as imputation methods, outlier thresholds, and filtering settings to optimize performance for different sensor data types and anomaly patterns, eliminating the need for manual domain knowledge configuration
3Measurement precision
If labeled sensor data are obtained for supervised learning, then model accuracy can be improved, but the cost and time for data labeling are high
Solution Approach 1:
The patent inverts the conventional supervised learning approach by using unsupervised anomaly detection methods that do not require labeled data. Instead of training models with labeled normal and abnormal data, the system learns from unlabeled sensor data alone, automatically identifying anomalies based on deviations from normal patterns, thereby achieving accurate anomaly detection without the time-consuming data labeling process
Solution Approach 2:
The patent implements self-supervised learning mechanisms where the model automatically generates its own training signals from unlabeled sensor data. The system identifies normal patterns and detects anomalies as deviations from these patterns without external labeling, enabling accurate anomaly detection while eliminating the need for manual data annotation
4Extent of automation
If unsupervised learning is applied without labeled data, then automation can be achieved, but the performance trade-off between precision and recall reduces reliability
Solution Approach 1:
The patent segments the anomaly detection process into multiple independent detection models that each focus on different aspects or types of anomalies. By dividing the detection task into multiple specialized models operating in parallel, the system achieves both full automation and improved reliability through diverse detection perspectives that compensate for individual model limitations
Solution Approach 2:
The patent combines multiple unsupervised anomaly detection models using ensemble methods and voting mechanisms. By merging the results from multiple models with different strengths, the system achieves more reliable and accurate anomaly detection than any single model could provide alone, while maintaining full automation
Data Source
AI summary
One embodiment can provide a system for detecting anomaly for high-dimensional sensor data associated with one or more machines. During operation, the system can obtain sensor data from a set of sensor associated with one or machines, apply data exploration techniques on the sensor data to automatically process sensor data to identify a subset of feature sensors from the available set of feature sensors, apply an unsupervised machine-learning technique to the identified subset of feature sensors and the target sensor to learn a set of pair-wise univariate models, and determine whether and how an anomaly occurs in the operation of the one or more machines based on the set of pair-wise univariate models.