Runbook Scoring via Inverse Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automation systems for resolving IT infrastructure issues are inefficient as they often recommend runbooks or scripts that fail to resolve problems, requiring extensive human effort and time, and lack effective scoring mechanisms to assess their success or failure.
Innovation Solution
A system and method that utilize a neural network and Inverse Reinforcement Learning techniques to analyze ticket data, identify policies, determine functions, recommend runbooks or scripts, execute them, and generate scores based on success or failure, leveraging historical data and user feedback to improve recommendation accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automation systems recommend runbooks or scripts from repository, then time required for resolving issues is reduced, but the runbooks or scripts may not be able to resolve the issue
Solution Approach 1:
The system implements feedback mechanisms by tracking whether recommended runbooks successfully resolve issues, using this information to continuously improve future recommendations. The feedback loop allows the system to learn from past performance and adjust its recommendation strategy accordingly.
Solution Approach 2:
The automation system performs self-evaluation by automatically assessing the effectiveness of its own recommendations through monitoring issue resolution outcomes, enabling continuous self-improvement without requiring external intervention for each recommendation.
2Reliability
If SMEs manually resolve issues based on knowledge and severity, then resolution accuracy is maintained, but extensive human effort and time are required
Solution Approach 1:
The system introduces an intermediary intelligence layer that bridges human expertise and automated execution. This intermediary analyzes issues, selects appropriate runbooks, and coordinates resolution actions, combining the accuracy of human knowledge with the efficiency of automation.
Solution Approach 2:
The system performs preliminary analysis and preparation by pre-processing issue data, identifying relevant patterns, and selecting appropriate runbooks before actual resolution execution, thereby reducing the cognitive load and time required for manual intervention.
3Device complexity
If set of runbooks or scripts is recommended without scoring mechanism, then recommendation process is simple, but effectiveness of recommendations cannot be assessed
Solution Approach 1:
The system replaces complex manual evaluation mechanisms with automated scoring algorithms that objectively assess recommendation effectiveness based on tracked performance data, eliminating the need for complex human judgment processes.
Data Source
AI summary
The present disclosure relates to system(s) and method(s) for generating a score for a runbook or a script. The system receives a ticket and a ticket description. The system further identifies a set of policies based on an analysis of historical data, the ticket and the ticket description. Further, the system determines a set of functions based on an analysis of the set of policies using a neural network technique and an Inverse Reinforcement Learning technique. Furthermore, the system recommends a runbook or a script based on the set of functions, the ticket and the ticket description. The runbook or the script is further executed to resolve the ticket. Based on the execution, the system records a script success or a script failure. The system further generates a score for the runbook or the script based on the script success or the script failure.