Transformer-Based Time Series Anomaly Detection With Sensor Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex physical systems generate vast amounts of time series data from sensors, leading to significant processing and storage challenges, and existing methods are inefficient in detecting anomalies in real-time.
Innovation Solution
A computer system utilizing a machine learning model with transformers to process sensor signal data, transforming raw time series into tiles and embeddings for efficient processing, and employing self-attention mechanisms to analyze relationships between sensor signals, enabling timely detection of anomalies and corrective actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional methods are used to process sensor signal data from complex systems, then processing and storage overhead increases significantly, but anomaly detection capability remains insufficient
Solution Approach 1:
The patent extracts only the essential features from raw sensor signal data by transforming them into fixed-size embeddings that capture the most relevant information for anomaly detection. This extraction process removes redundant data while preserving the critical patterns needed for detecting anomalies, thereby reducing processing and storage overhead while maintaining detection capability.
Solution Approach 2:
The patent transforms sensor signal data from its original high-dimensional form into a compressed embedding representation with fixed size. This parameter transformation changes the data from variable-length raw signals to fixed-size vectors, enabling efficient processing and storage while retaining the essential characteristics needed for anomaly detection through the trained machine learning model.
2Loss of time
If raw time series data is processed in real-time for anomaly detection, then detection timeliness improves, but processing complexity increases
Solution Approach 1:
The patent performs preliminary processing of sensor data by transforming raw time series into fixed-size embeddings before anomaly detection. This preliminary transformation creates a standardized, compressed representation that simplifies subsequent real-time processing. The embeddings are pre-computed with fixed dimensions, reducing the complexity of real-time analysis while enabling timely anomaly detection through the trained model.
3Measurement precision
If multiple sensor signals are analyzed to improve detection accuracy, then anomaly detection precision improves, but data volume increases
Solution Approach 1:
The patent merges multiple sensor signal inputs into a unified embedding representation that captures relationships among all sensors. The machine learning model processes multiple sensor signals simultaneously and compresses them into a fixed-size embedding that integrates information from all sources. This merging approach maintains comprehensive multi-sensor analysis for improved detection accuracy while reducing the overall data volume through efficient compression.
Data Source
AI summary
A computer system for managing a machine learning model that detects potential anomalies in the operation of a complex system is disclosed. In some embodiments, the computer system is programmed to receive sensor signal data originally produced by sensors of the complex system. The sensor signal data can include values for multiple sensor signals at multiple resolutions. The computer system is programmed to train, from given sensor signal data, the machine learning model that comprises one or more transformers, each transformer capturing a set of relationships between signals in a predetermined group of signals. During training, the computer system is programmed to also establish an expected range for an indicator of the relationship. The computer system is programmed to then execute the machine learning model on new sensor signal data and take remedial steps when any computed indicator falls outside the expected range, indicating a potential anomaly in the operation of the complex system.