Transformer Labeler for Time Series Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI/machine learning models struggle with training accuracy when faced with imprecise labels for rare events in time series data, such as data loss or hardware failures, due to the inefficiency of current methods that rely on few, low-quality labels and computationally expensive hyperparameter tuning.
Innovation Solution
An automatic semi-supervised labeler model, utilizing a time series transformer with self-attention, is employed to generate more precise labels by parameterizing the uncertainty in imprecise labels and coupling it with a machine learning model through a feedback loop, optimizing a loss function to determine the actual time of anomalous events, thus producing high-quality training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If fewer labels generated by subject matter experts are used, then the complexity of label generation is reduced, but the quantity and quality of training data deteriorates
Solution Approach 1:
The system uses the transformer model to automatically generate labels for time series data without requiring manual annotation by subject matter experts. The model processes telemetry data and autonomously determines event timestamps, enabling self-service label generation that scales with data volume while maintaining consistency through the learned temporal patterns.
Solution Approach 2:
The transformer model learns from existing labeled data and replicates the labeling process by applying the same temporal reasoning and pattern recognition to unlabeled data. This copying mechanism allows the system to generate large quantities of consistent labels by replicating the expert annotation logic that was initially hand-crafted.
2Ease of operation
If hand-crafting heuristics/rules to generate labels is done, then the ease of operation is improved, but the quality and scalability of labels deteriorates
Solution Approach 1:
The system replaces manual heuristic rules with a transformer-based neural network that processes time series data. This substitution eliminates the need for explicit hand-crafted logic while maintaining ease of operation through automated processing. The model captures temporal patterns and generates precise labels by learning from training data rather than relying on predetermined rules.
Solution Approach 2:
The transformer model dynamically adjusts its internal parameters during training to optimize label generation. Instead of using fixed heuristics, the model learns optimal temporal windows, event detection thresholds, and pattern recognition parameters from the training data, enabling it to adapt to different data characteristics while maintaining high label quality.
3Reliability
If hyperparameter tuning through empirical study is performed, then the accuracy of the classifier is improved, but the computational cost and time required deteriorates
Solution Approach 1:
The system uses a feedback mechanism where the transformer model generates labels that are then used to train the classifier, and the classifier's performance feedback is used to refine the labeling process. This iterative feedback loop eliminates the need for manual hyperparameter tuning by automatically optimizing the temporal parameters through gradient descent and backpropagation, reducing computational cost while maintaining accuracy.
Solution Approach 2:
Instead of manually tuning hyperparameters through empirical studies, the system automatically optimizes temporal parameters such as sampling window length and event detection thresholds through gradient-based optimization. The transformer model learns optimal parameter values during training by minimizing loss functions that measure classification accuracy, eliminating the need for exhaustive hyperparameter search.
Data Source
AI summary
The technology described herein describes training an automatic semi-supervised labeler model, such as including a time series transformer with self-attention encoder, in conjunction with classifier training to produce more precise labels describing when anomalous, rare events occurred. The automatic labeler assigns a probability distribution parameterized by distribution parameters over a sample window. A classifier outputs an approximation of distribution parameters for an imprecise label (secondary event) correlated with the anomalous event. The approximation distribution along with the secondary event distribution are input into a loss function, which couples the automatic labeler to the classifier in a feedback loop. The loss function is optimized over iterations of the loop, with the loss minimized when the automatic labeler outputs the correct label. Once trained, additional labels can be automatically generated for further training. A model trained with more precisely labeled events can then predict an anomalous event given previously unseen data.