Passive Signal-Following Detection in Multivariate Anomaly Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing ML time series prognostics are susceptible to the 'following' phenomenon, where anomalies are not detected due to ML models erroneously predicting values that match anomalous behavior, leading to missed alerts and potential catastrophic failures in safety-critical industries.
Innovation Solution
A passive inferencing method using an inferential following metric (iFM) that calculates the ratio of the standard deviation of measured values to the standard deviation of residuals in residuals between predicted and actual values, identifying signal following without requiring multiple training iterations, scalable to dense monitoring scenarios with billions of observations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If ML models are trained to predict signal values, then prediction accuracy improves, but signal following occurs where anomalies are missed
Solution Approach 1:
The patent introduces an inferential following metric (iFM) as an intermediary diagnostic tool that analyzes the relationship between signal standard deviation and residual standard deviation. This metric acts as a mediator to detect when the ML model is following signals erroneously, allowing the system to maintain high prediction accuracy while monitoring for reliability issues through the iFM ratio.
2Object-generated harmful factors
If traditional anomaly detection methods are used, then false alarms increase, but missed detections occur due to signal following
Solution Approach 1:
The patent implements a feedback mechanism where the iFM is continuously calculated and compared against a threshold. When the iFM exceeds the threshold, it provides feedback indicating signal following is occurring, which triggers alerts. This feedback loop allows the system to adaptively respond to model behavior changes, reducing both false alarms and missed detections by maintaining appropriate detection sensitivity.
3Quantity of substance
If ML models process dense monitoring data with billions of observations, then monitoring coverage improves, but computational complexity increases
Solution Approach 1:
The patent extracts only the essential statistical features (standard deviation of signals and standard deviation of residuals) needed to calculate the iFM, rather than processing the entire billion-point datasets. This extraction approach maintains comprehensive monitoring coverage while significantly reducing computational complexity by focusing only on the critical metrics needed for following detection.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems, methods, and other embodiments associated with passive inferencing of signal following in multivariate anomaly detection are described. In one embodiment, a method for inferencing signal following in a machine learning (ML) model includes calculating an average standard deviation of measured values of time series signals in a set of time series signals; training the ML model to predict values of the signals; predicting values of each of the signals with the trained ML model; generating a time series set of residuals between the predicted values and the measured values; calculating an average standard deviation of the sets of residuals; determining that signal following is present in the trained ML model where a ratio of the average standard deviation of measured values to the average standard deviation of the sets of residuals exceeds a threshold; and presenting an alert indicating the presence of signal following in the trained ML model.