Unsupervised ML Clustering for Interpretable Routing Rules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Incident management systems require manual, time-consuming, and error-prone creation and maintenance of complex rules and policies for alert triggers, escalation policies, and notification conditions, which are difficult to adapt to dynamically changing data.
Innovation Solution
A two-tiered machine learning approach involving unsupervised and supervised models to automatically create and maintain rules by clustering message data and generating specialized decision trees for each cluster, allowing for real-time alert creation and optimization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual rule creation and maintenance is used, then domain knowledge and service level agreements can be incorporated, but the process becomes time-consuming and error-prone
Solution Approach 1:
The system performs self-service by automatically generating routing rules through unsupervised machine learning clustering. The incident management system analyzes historical data and message patterns autonomously to create rules without requiring manual configuration, thereby reducing time consumption while maintaining reliability through data-driven insights
Solution Approach 2:
The manual mechanical process of rule creation is replaced with an automated machine learning system. The unsupervised learning model substitutes human analysts by automatically identifying patterns in incident data and generating routing rules, eliminating manual effort while preserving the essence of domain knowledge through learned patterns
2Adaptability or versatility
If complex rules with multiple conditions are created manually, then specific thresholds and triggers can be defined, but the complexity increases and requires expensive manual work
Solution Approach 1:
The system extracts essential routing patterns from complex historical data by using unsupervised learning to identify and isolate key features. Instead of manually managing complex rule sets, the model extracts underlying patterns and generates simplified yet effective routing rules that capture the essence of complex conditions without the manual complexity
Solution Approach 2:
The system dynamically adjusts routing parameters based on learned patterns from historical data. Instead of static manual configuration, the machine learning model continuously optimizes routing parameters such as thresholds and conditions based on observed incident patterns, enabling adaptability while reducing the apparent complexity through automated parameter tuning
3Adaptability or versatility
If manual rule maintenance is performed, then policies can be tailored to specific services, but real-time response to changing data is not possible
Solution Approach 1:
The system transitions from static manual rule maintenance to dynamic automated rule generation. The unsupervised learning model continuously adapts to changing incident patterns by reprocessing historical data and generating updated routing rules in real-time, enabling the system to respond dynamically to changing data while maintaining high productivity through automation
Solution Approach 2:
The system performs preliminary action by continuously analyzing historical incident data and pre-generating optimized routing rules before new incidents occur. This proactive approach allows the system to be prepared with updated routing policies in advance, enabling real-time response to changing patterns without requiring manual intervention when incidents actually occur
Data Source
AI summary
Embodiments of the disclosure relate to systems and methods for leveraging unsupervised machine learning to produce interpretable routing rules. In various embodiments, a training dataset comprising a plurality of data records is created. The plurality of data records includes message data comprising a plurality of messages and action data comprising a plurality of actions that correspond to the plurality of messages. A first machine learning model is trained using the training dataset. The first machine learning model as trained provides cluster data that indicates, for each data record of the plurality of data records of the training dataset, membership in a cluster of a plurality of clusters. An enhanced training dataset is created that comprises the message data from the training dataset, the action data from the training dataset, and the cluster data. A set of second machine learning models is trained using the enhanced training dataset, each respective second machine learning model of the set of second machine learning models providing a decision tree of a plurality of decision trees and corresponding to a distinct cluster of the plurality of clusters. Rules can be extracted from each decision tree of the plurality of decision trees and used as a basis for creating and transmitting alerts based on incoming messages.