Signal Masking Certification for Multivariate Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models struggle with masking analysis, making it difficult to determine if signal degradation is masked by combined degradations in other signals, which is critical for safety-critical applications like transportation and healthcare, where undetected anomalies can have severe consequences.
Innovation Solution
A masking certification system that performs a propagation of uncertainty analysis to ensure that signal degradation is not masked by degradations in other signals, using a multivariate machine learning model to generate a Jacobian matrix and determine masking bias, certifying whether a signal is un-maskable by analyzing partial derivatives and finite differences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models are used for anomaly detection in safety-critical systems, then detection capability is improved, but the ability to perform rigorous masking analysis deteriorates because NNs employ stochastic optimization of weights
Solution Approach 1:
The patent introduces an intermediary certification system that bridges the gap between ML anomaly detection and rigorous masking analysis. This system uses surrogate models (such as Gaussian processes or neural networks with known bounds) to approximate the behavior of the primary ML model, enabling mathematical certification of masking properties without directly analyzing the stochastic weights of the original model. The intermediary provides a mathematically tractable representation that preserves essential detection capabilities while enabling formal verification.
2Reliability
If propagation of uncertainty analysis is performed to detect masked degradations, then reliability is improved, but computational complexity increases because it requires generating and analyzing Jacobian matrices
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing Jacobian matrices at representative operating points before actual anomaly detection occurs. These pre-computed matrices capture the sensitivity relationships between signals at different operating conditions. During runtime, the system can quickly query these pre-analyzed matrices rather than computing them on-demand, significantly reducing the computational burden of propagation of uncertainty analysis while maintaining detection reliability.
3Measurement precision
If a multivariate machine learning model monitors multiple signals, then detection coverage is improved, but the risk of masking increases because degradations in other signals can conceal degradation in one signal
Solution Approach 1:
The patent implements feedback through the propagation of uncertainty analysis that continuously evaluates how degradations in multiple signals interact. The system computes the Jacobian matrix to determine the sensitivity of each signal to changes in others, then uses this information to assess whether degradation in one signal could be masked by combined degradations in other signals. This feedback mechanism provides quantitative masking risk assessment that informs whether the multivariate model's detection results are trustworthy or if further investigation is needed.
Data Source
AI summary
Systems, methods, and other embodiments associated with ensuring that detection of degraded signals is not able to be masked by degradation in other signals are described. In one example embodiment, a method accesses a multivariate machine learning model that is trained to monitor a set of time series signals and a test observation of the signals has a degradation on an m-th signal. A baseline bias for the m-th signal is determined using the test observation and a model-estimated value of the m-th signal. A Jacobian matrix of the model is generated based on a finite difference smaller than an observed value of the m-th signal. A masking bias is determined based on the baseline bias and a next K largest entries of an m-th row of the Jacobian matrix. And, the degradation is certified to be un-maskable or not based on the masking bias and the finite difference.