Failure Analysis System for Distributed Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computing systems, traditional monitoring tools generate a large number of system events and alarms for a single root cause of failure, making it difficult for administrators to identify and remediate the issue manually.
Innovation Solution
A failure analysis system using model-based reasoning and decision tree analysis to diagnose the root cause of failures and provide remediation recommendations, automating the process and enabling adaptation over time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional monitoring tools are used to detect and report all aspects of system health, then comprehensive system monitoring is achieved, but a large number of system events and alarms are generated for a single root cause, making it difficult for administrators to identify the root cause manually
Solution Approach 1:
The system segments the complex analysis task into distinct functional modules: an inference engine that performs model-based reasoning to generate hypotheses, a decision tree analysis component that systematically evaluates hypotheses, and a recommendation generation module. This segmentation allows each component to handle specific aspects of root cause analysis, reducing the overall complexity for administrators.
Solution Approach 2:
The patent introduces an automated failure analysis system as an intermediary between the monitoring tools and administrators. This intermediary processes the large volume of events and alarms, performs sophisticated analysis using multiple methodologies, and presents simplified results to administrators, thereby reducing the complexity of manual analysis while maintaining comprehensive monitoring.
2Loss of information
If traditional monitoring tools report all system health aspects, then complete failure information is captured, but the large amount of information is difficult for administrators to evaluate and analyze manually
Solution Approach 1:
The failure analysis system performs self-service by automatically collecting, analyzing, and interpreting failure information without requiring manual administrative intervention. The system autonomously evaluates events and alarms, generates hypotheses about root causes, and provides actionable recommendations, thereby maintaining complete failure information while eliminating the burden of manual evaluation for administrators.
Solution Approach 2:
The patent replaces the manual mechanical process of administrator analysis with automated computational systems. The inference engine uses model-based reasoning algorithms, and the decision tree analysis employs systematic computational evaluation, substituting human cognitive effort with automated mechanical processes that can handle complete failure information efficiently.
3Measurement precision
If manual analysis of all alarms and events is performed to identify root cause, then thorough diagnosis is achieved, but excessive time and effort are required
Solution Approach 1:
The system performs preliminary actions by pre-building models of system behavior and pre-configuring decision trees with possible failure modes and their characteristics. When failures occur, the inference engine quickly matches observed events against these pre-prepared models, and the decision tree rapidly evaluates pre-defined hypotheses, achieving thorough diagnosis without requiring time-consuming manual analysis of all alarms and events.
Data Source
AI summary
A failure analysis system identifies a root cause of a failure (or other health issue) in a virtualized computing environment and provides a recommendation for remediation. The failure analysis system uses a model-based reasoning (MBR) approach that involves building a model describing the relationships/dependencies of elements in the various layers of the virtualized computing environment, and the model is used by an inference engine to generate facts and rules for reasoning to identify an element in the virtualized computing environment that is causing the failure. Then, then the failure analysis system uses a decision tree analysis (DTA) approach to perform a deep diagnosis of the element, by traversing a decision tree that was generated by combining the rules for reasoning provided by the MBR approach, in conjunction with examining data collected by health monitors. The result of the DTA approach is then used to generate the recommendation for remediation.