Inline Event Categorization Using Keyword Vectors and Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex IT systems generate overwhelming numbers of disparate event messages, overwhelming IT teams and limiting scalability and evolution in managing events, especially in large, distributed systems.
Innovation Solution
Implementing a system that employs a hash function to generate key values from event messages, calculates similarity scores, and uses learner objects to categorize events into groups based on user, account, or organization associations, while removing non-semantic information and utilizing user feedback to refine categorization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual techniques and pre-programmed rules are used to manage events, then event management can be performed with simple systems, but it becomes labor intensive and expensive while limiting scalability
Solution Approach 1:
The system employs machine learning models that automatically categorize and manage events without human intervention. The models learn from historical event data and autonomously perform classification tasks, eliminating the need for manual event management while maintaining high accuracy and enabling scalability to millions of events.
Solution Approach 2:
Manual event management processes are replaced with automated machine learning systems. The mechanical/manual operations of categorizing and managing events are substituted with computational algorithms that process events automatically, dramatically improving productivity and scalability while reducing labor requirements.
2Loss of information
If monitoring systems are arrayed to provide visibility to operational metrics, then event detection capability is improved, but the sheer size and complexity result in a flooding of disparate event messages that overwhelm IT teams
Solution Approach 1:
The system merges disparate event messages from multiple monitoring systems into unified event groups. By clustering similar events together and representing them as group vectors, the system consolidates the flood of individual event messages into manageable groups, reducing complexity while maintaining complete visibility of all operational metrics.
Solution Approach 2:
Machine learning models serve as intermediaries between the raw event flood and IT teams. The models process and categorize events automatically, acting as a mediator that translates the complex flood of disparate event messages into organized, manageable information that IT teams can efficiently handle.
3Ease of manufacture
If traditional event management approaches are used, then implementation is straightforward with simple systems, but the ability to scale and evolve for future advances in IT systems capabilities is limited
Solution Approach 1:
The system uses dynamic machine learning models that continuously learn and adapt from new event data. Unlike static traditional approaches, the models evolve over time by incorporating new patterns and event types, enabling the system to scale and adapt to future IT system capabilities while maintaining ease of implementation through automated processes.
Solution Approach 2:
The machine learning-based event management system provides universal functionality that handles diverse event types from various monitoring systems. The same core infrastructure can manage different kinds of events and adapt to new IT system capabilities, making the system both easy to implement and highly scalable across different environments and future technologies.
Data Source
AI summary
A machine learning model is applied to extract keywords from an event associated with network operations. A keyword vector is constructed where each keyword extracted is represented as a unique dimension in the keyword vector. The keyword vector is compared against event category vectors associated with event categories to identify potential matches. A relevance score is calculated for each category based on the comparing. The event is classified into one or more categories based on the relevance scores exceeding a predetermined threshold. A user interface configured to visually display and obtain feedback regarding the classifying is generated. At least one of the event categories is updated based on a user feedback received via the user interface and machine learning retraining.