Three Phase Exception Handling for Service Availability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern system landscapes with multiple services from different vendors and versions pose challenges in maintaining high availability, as the increasing complexity leads to longer investigation and resolution times for issues, resulting in critical service downtimes and high maintenance costs due to manual intervention.
Innovation Solution
A three-phase exception handling system where inspector instances observe service behavior via a unified API, executing commands to resolve abnormal behavior, notifying a centralized engine, and involving administrators for customized rule execution to minimize downtime and reduce human intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual monitoring and intervention is used to maintain service availability, then administrators can handle abnormal service behavior, but the time needed to investigate and solve issues increases with system complexity
Solution Approach 1:
The exception handling process is segmented into three distinct phases: automated inspection phase, rule-based automated resolution phase, and administrator intervention phase. This segmentation allows each phase to handle specific complexity levels independently, reducing overall resolution time while maintaining reliability.
Solution Approach 2:
Predefined rules are prepared in advance and stored in the centralized inspection engine. When abnormal service behavior is detected, the system immediately executes the most relevant predefined rules without waiting for administrator analysis, significantly reducing the time to resolve common issues.
2Productivity
If automated exception handling is implemented, then service downtime is minimized and maintenance costs decrease, but the system complexity increases with multiple inspection phases
Solution Approach 1:
The centralized inspection engine serves multiple functions: it receives inspection results from various inspector instances, stores and manages predefined rules, executes rules automatically, and coordinates with administrators. This multi-functionality consolidates complexity into a single manageable component rather than distributing it across multiple independent systems.
Solution Approach 2:
The inspection system performs self-service through automated execution of predefined rules. The centralized inspection engine automatically selects and executes appropriate rules based on the inspection results, eliminating the need for continuous human intervention in routine exception handling while maintaining systematic complexity management.
3Measurement precision
If multiple inspector instances monitor all services, then abnormal behavior is detected accurately, but the system requires well-trained administrators to manage complexity
Solution Approach 1:
The centralized inspection engine acts as an intermediary between the inspector instances and the administrators. It processes the raw inspection data, applies predefined rules, and presents only the necessary information to administrators, reducing their workload while maintaining accurate detection through multiple inspector instances.
Solution Approach 2:
The system extracts and automates the routine exception handling tasks from administrator workloads. By removing routine manual intervention and extracting it into automated rule execution, administrators are freed from routine monitoring and can focus only on complex issues requiring human judgment.
Data Source
AI summary
Described herein are systems and methods for three phase exception handling ensuring high availability of services. Via unified application program interface (API), one or more inspector instances are observing the behavior one or more services. The three phase exception handling is triggered upon observing an abnormal service behavior. Phase one comprises executing one or more commands from the unified API by the one or more inspector instances in attempt to resolve the abnormal service behavior and notifying a centralized inspector engine. Phase two comprises executing one or more predefined rules according to the current service state by the centralized inspector engine in attempt to resolve the abnormal service behavior and notifying an administrator. Phase three comprises abnormal service behavior analysis by the administrator and executing customized by the administrator rules to resolve the abnormal service behavior.