Probabilistic Graphical Model Construction with Anomaly Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection methods struggle when historical data contains undetected anomalies, as they assume all data is normal, leading to missed alerts in systems like health monitoring for rotorcraft or patient monitoring, where prior knowledge is lacking or multivariate features are complex.
Innovation Solution
A method for constructing a probabilistic graphical model that learns parameters, includes latent variables, iteratively associates components with normal data, constructs an association matrix, detects abnormal components, and deletes them to refine the model, reducing build time and improving detection efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional anomaly detection methods are used on historical data containing undetected anomalies, then the model build time is long and computational resources are consumed, but the detection accuracy deteriorates because the methods assume all data is normal
Solution Approach 1:
The patent segments the historical data into normal data and abnormal data using a semi-supervised learning approach. By dividing the dataset and applying different learning strategies to each segment (supervised learning for normal data, unsupervised learning for abnormal data), the method achieves both high detection accuracy and reduced computational time compared to processing all data uniformly.
Solution Approach 2:
The patent changes the learning parameters and approaches based on data characteristics. For normal data, it uses supervised learning with labeled examples to learn precise anomaly patterns. For abnormal data, it uses unsupervised learning to identify unusual patterns without labels. This parameter adaptation enables accurate detection while optimizing build time.
2Measurement precision
If semi-supervised learning is applied to separate normal and abnormal data, then detection accuracy improves, but the model complexity increases
Solution Approach 1:
The patent segments the learning process into two distinct phases: supervised learning on normal data and unsupervised learning on abnormal data. This segmentation allows each phase to use optimized, simpler algorithms appropriate for its specific task, reducing overall model complexity while maintaining high detection accuracy.
Solution Approach 2:
The patent extracts and removes abnormal data from the training set before applying supervised learning. By taking out the abnormal components that would interfere with learning normal patterns, the model becomes less complex and more focused, achieving better accuracy without the complexity of handling mixed data types simultaneously.
3Quantity of substance
If all historical data is used for training without filtering anomalies, then more data is available for learning, but the model learns incorrect patterns from undetected anomalies
Solution Approach 1:
The patent extracts abnormal data from the historical dataset using unsupervised learning techniques before the main training process. By removing these problematic data points, the remaining training data can be used effectively without teaching the model incorrect patterns, thus preserving detection accuracy while still utilizing the majority of available data.
Solution Approach 2:
The patent applies different quality standards and processing methods to different parts of the dataset. Normal data receives supervised learning with high quality labels, while abnormal data is separately identified and processed with unsupervised methods. This local quality differentiation ensures that each data type is handled appropriately, maintaining overall model accuracy.
Data Source
Figure 1
Figure 2
AI summary
A method (100) of constructing a probabilistic graphical model (10) of a system from data that includes both normal and anomalous data includes the step of learning parameters of a structure for the probabilistic graphical model (10). The structure includes at least one latent variable (26) on which other variables (12, 14, 16, 18, 20, 22, 24) are conditional, and has a plurality of components. The method further includes the steps of: iteratively associating one or more of the plurality of components of the latent variable (26) with normal data;constructing a matrix of the associations;detecting abnormal components of the latent variable (26) based on one of a low association with the normal data or the matrix of associations; and deleting the abnormal components of the latent variable (26) from the probabilistic graphical model (10).