Storage I/O Path Anomaly Detection with Task-Sequence Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face challenges in detecting and identifying anomalous events, such as software bugs, misconfigurations, or hardware failures, which can lead to abnormal behavior, without generating unnecessary alerts for non-anomalous operations.
Innovation Solution
A method and system utilizing a machine learning model, like LogBERT, to classify sequences of tasks in input-output (I/O) operations as anomalous or non-anomalous, and generate alerts only when anomalous events are detected, based on a distance comparison with an anomaly catalog.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional anomaly detection methods are used in storage systems, then anomaly detection capability is provided, but false positive alerts are generated for non-anomalous operations
Solution Approach 1:
The patent replaces traditional rule-based anomaly detection mechanisms with a machine learning-based classification system. The machine learning model analyzes sequences of tasks and compares them against an anomaly catalog to distinguish between genuine anomalies and normal variations in storage operations, thereby reducing false positive alerts while maintaining detection accuracy.
Solution Approach 2:
The patent changes the detection parameters from static rules to dynamic machine learning models that can adapt to different storage system behaviors. By using sequences of tasks and comparing distances to anomaly patterns, the system adjusts its detection sensitivity to avoid false positives while maintaining reliability in identifying actual anomalies.
2Reliability
If comprehensive anomaly detection is implemented, then detection coverage is improved, but system complexity increases
Solution Approach 1:
The patent segments the anomaly detection process into distinct components: task sequence extraction, machine learning classification, and anomaly catalog comparison. This segmentation allows each component to be optimized independently and simplifies the overall system architecture while maintaining comprehensive detection coverage through the coordinated interaction of these modular components.
Data Source
AI summary
A method, comprising: obtaining a parsed trace log that is generated by parsing a raw trace log of a storage system, the parsed trace log identifying a first sequence of tasks that are executed as part of a same input-output (I/O) operation; classifying the first sequence of tasks by using a machine learning model, the first sequence of tasks being classified into one of a first category and a second category, the first category corresponding to anomalous events, and the second category corresponding to non-anomalous events; when the first sequence of tasks is classified in the first category: identifying an anomaly code that corresponds to the first sequence of tasks and generating an alert that is associated with the anomaly code, the anomaly code being identified based on a distance between the first sequence of tasks and a second sequence of tasks that is mapped to the anomaly code.