Unsupervised Anomaly Clustering for Variable-Length KPI Time Series
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning models struggle to effectively cluster anomalies in computerized systems due to the challenge of ensuring that representation learning captures temporal and multi-dimensional dynamics of variable length timeseries, particularly in unsupervised settings where category labels are scarce.
Innovation Solution
A fully unsupervised cognitive model with a two-stage architecture, utilizing an encoder to learn fixed-size representations and a clustering stage, optimized using a composed loss function that combines a triplet loss and a clustering objective, allowing for iterative and alternated training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning models are used for supervised anomaly clustering, then clustering accuracy is improved, but the requirement for category labels increases system complexity and reduces applicability in unsupervised settings
Solution Approach 1:
The patent extracts and removes the dependency on category labels from the anomaly clustering system. By formulating the problem as unsupervised learning, the system achieves clustering accuracy without requiring external label information, thereby reducing system complexity while maintaining effectiveness.
Solution Approach 2:
The system performs self-service by automatically learning anomaly representations and clustering structures without external supervision. The unsupervised learning framework enables the model to discover patterns and group anomalies based solely on intrinsic data characteristics, eliminating the need for manual labeling processes.
2Measurement precision
If representation learning is applied to capture temporal dynamics, then clustering quality is improved, but computational cost increases
Solution Approach 1:
The patent segments the learning process into distinct stages: representation learning for capturing temporal dynamics, followed by clustering. This segmentation allows efficient processing by focusing computational resources on extracting meaningful temporal features first, then performing clustering on the learned representations, thereby reducing overall computational cost while maintaining clustering quality.
Solution Approach 2:
The system performs preliminary action by pre-learning temporal representations before clustering. This preliminary representation learning captures essential temporal dynamics in advance, allowing the subsequent clustering stage to operate more efficiently on compact, meaningful features rather than raw timeseries data.
3Productivity
If fixed-size representations are learned from variable length timeseries, then clustering efficiency is improved, but information loss may occur
Solution Approach 1:
The patent transitions from variable-length timeseries to fixed-size representations by introducing a new dimensional space for clustering. This dimensionality change allows efficient processing of fixed-size vectors while the representation learning process preserves essential temporal and multi-dimensional dynamics through careful feature extraction, minimizing information loss.
Solution Approach 2:
The system changes parameters by transforming variable-length inputs into fixed-size representations through learned encoding. This parameter transformation maintains clustering efficiency while the representation learning process is designed to preserve critical temporal patterns and multi-dimensional relationships, balancing efficiency with information retention.
Data Source
AI summary
The invention is notably directed to a computer-implemented method of clustering anomalies detected in a computerized system. The proposed method makes use of an unsupervised cognitive model, executed based on input datasets to obtain clusters of anomalies. The method accesses input datasets, which correspond to detected anomalies of the computerized system. These anomalies span respective time windows. Each input dataset comprises a set of timeseries of key performance indicators. The key performance indicators of each input dataset extend over a respective time window. That is, each anomaly corresponds to a respective time window. This model includes a first stage, which includes an encoder designed to learn fixed-size representations of input datasets, and a second stage, which is a clustering stage. The model is executed based on the input datasets accessed, the first stage learning fixed-size representations of the input datasets and the second stage clustering the learned representations.