Hierarchical Anomaly Detection for Virtual Machine Resource Monitoring

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing systems for monitoring virtual machine resources face inefficiencies in identifying anomalies due to the difficulty in managing large numbers of individual metrics and failing to consider relationships between them, leading to potential false positives and reduced reliability.

Innovation Solution

A hierarchical anomaly detection system that utilizes multiple iterations of machine learning techniques at different levels, starting from group-level to system-level, to process collected operations information, reducing false positives by refining training sets and identifying anomalies more accurately.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If individual metrics are monitored separately in existing systems, then the system can collect comprehensive operations information, but the difficulty in managing large numbers of individual metrics increases and relationships between metrics are not considered

Engineering Contradiction:
Improveanomaly detection reliabilityVSAvoidcomplexity of managing individual metrics
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent combines multiple individual metrics into hierarchical groups (component-level, region-level, system-level) to reduce the complexity of managing large numbers of individual metrics while preserving the ability to detect anomalies. Metrics are aggregated into groups that represent logical relationships, making the system more manageable without losing detection capability.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The patent segments the monitoring system into multiple hierarchical levels (component, region, system) where each level handles specific metrics. This segmentation allows the system to manage complexity by dividing metrics into manageable groups while maintaining comprehensive monitoring through the hierarchical structure.

Inventive Principle:
Principle #1Segmentation

2Measurement precision

If machine learning techniques are applied to individual metrics separately, then comprehensive coverage is achieved, but false positives increase due to lack of context from relationships between metrics

Engineering Contradiction:
Improveanomaly detection precisionVSAvoidreliability of anomaly detection
Core Design Contradiction:
Measurement precisionVSReliability

Solution Approach 1:

The patent adds a hierarchical dimension to anomaly detection by evaluating metrics at multiple levels (component, region, system) rather than individually. This dimensional change allows the system to consider relationships between metrics across different levels, improving detection precision while reducing false positives through contextual understanding.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The patent introduces hierarchical groups as intermediaries between individual metrics and the anomaly detection process. These groups aggregate metrics and provide contextual relationships, acting as mediators that improve detection precision by considering metric relationships while reducing false positives through broader context.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Ease of operation

If all metrics are analyzed at the same level, then processing is simplified, but the system fails to identify relationships between metrics at different hierarchical levels

Engineering Contradiction:
Improveease of metric processingVSAvoidloss of relationships between metrics
Core Design Contradiction:
Ease of operationVSLoss of information

Solution Approach 1:

The patent segments metrics into hierarchical groups (component, region, system levels) that preserve relationships between metrics while simplifying processing at each level. Each level handles a specific subset of metrics, making processing easier while the hierarchical structure maintains the relationships between metrics across levels.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces a hierarchical dimension to organize metrics, allowing relationships between metrics at different levels to be preserved. This dimensional organization enables the system to process metrics more easily at each level while maintaining awareness of relationships across the full hierarchy.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS10884805B2Dynamically configurable operation information collection
Publication Date: 2021.01.05 AMAZON TECH INC
  • US10884805B2 patent drawing
  • US10884805B2 patent drawing
  • US10884805B2 patent drawing

AI summary

Systems and methods are described for the collection and transmission of virtual machine resource operations information. Individual agents on virtual machine resources collect and store operations information in accordance with a current operations information collection configuration. The individual agents will initiate a transmission of the collected operations information. Responsive to the receipt of the transmission of the collected operations information, the monitoring processing service calculates a hierarchy of anomaly scores utilizing machine learning techniques. The monitoring processing service can generate a processing result.