Autoencoder KPI Characterization for Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for anomaly detection in computerized systems fail to provide deep root cause analysis and are inefficient when dealing with large datasets, lacking explainability and scalability.
Innovation Solution
A computer-implemented method using a cognitive model implemented as an autoencoder by a trained neural network, which processes key performance indicators (KPIs) through temporal convolutional and deconvolutional layer blocks to obtain reconstruction errors, enabling characterization and anomaly detection in computerized systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If baseline causality methods are used for anomaly detection, then anomaly detection capability is provided, but efficiency and scalability deteriorate when running on large datasets
Solution Approach 1:
The patent segments the KPIs into n different types and channels them through separate buffer channels, allowing parallel processing of different KPI categories. This segmentation enables the system to handle large datasets more efficiently by dividing the processing workload into manageable streams that can be processed concurrently through the autoencoder's multiple input channels.
Solution Approach 2:
The patent introduces a multi-channel dimension by creating n separate input channels corresponding to different KPI types. This dimensional transformation allows the autoencoder to process multiple KPI streams simultaneously, improving scalability and efficiency when dealing with large datasets while maintaining comprehensive anomaly detection coverage.
2Ease of manufacture
If explainability methods are used for anomaly detection, then model prediction explanation is provided, but root cause analysis capability deteriorates
Solution Approach 1:
Instead of only explaining model predictions post-hoc, the patent inverts the approach by using the autoencoder's reconstruction process to directly identify root causes. The encoder-decoder architecture naturally reveals which KPIs contribute most to reconstruction error, providing both explanation and root cause analysis simultaneously through the same computational process.
Solution Approach 2:
The patent uses reconstruction error as an intermediary metric that bridges explainability and root cause analysis. By measuring the discrepancy between input KPIs and their reconstructions, the system identifies which KPIs are most responsible for anomalies, thereby providing both explanation of model behavior and identification of root causes through a single intermediary measurement approach.
3Adaptability or versatility
If multiple KPIs are processed through a cognitive model, then comprehensive system characterization is achieved, but processing complexity increases
Solution Approach 1:
The patent segments KPIs into n types and processes them through separate buffer channels and input channels in the autoencoder. This segmentation allows comprehensive characterization of multiple KPIs while managing processing complexity through modular architecture, where each channel handles a specific KPI type independently before consolidation in the encoder.
Solution Approach 2:
The patent creates a universal autoencoder architecture that can handle multiple KPI types through a common encoder-decoder framework. The model's multi-input-channel design allows it to universally process different KPI categories (CPU, memory, disk, network) using the same computational blocks, thereby achieving comprehensive characterization without proportionally increasing processing complexity.
Data Source
AI summary
The invention is directed to characterizing a computerized system. Access key performance indicators (KPIs), for the computerized system. Each of the KPIs is a timeseries of KPI values and is categorized into one of n types. KPI values are channeled through n buffer channels. Each buffer channel buffers KPI values of one of n types. Finally, reconstructions errors are obtained by feeding initial KPI values to n respective input channels of a cognitive model, implemented as an autoencoder by a trained neural network including an encoder and a decoder. Encoder has temporal convolutional layer blocks connected by each input channel. Decoder has deconvolution layer blocks connected by encoder. Initial KPI values are independently processed in n input channels, then compressed by encoder, prior to being reconstructed by decoder. Reconstruction errors are obtained by comparing reconstructed KPI values with initial KPI values. Computerized system is characterized based on reconstruction errors obtained.