Inline Event Categorization Using Hash-Based Similarity Grouping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex IT systems generate overwhelming numbers of disparate event messages, overwhelming IT teams and limiting scalability and evolution in managing operations, with manual techniques being labor-intensive and ineffective.
Innovation Solution
Implementing a system that employs a hash function to generate key values from event messages, calculates similarity scores, and uses learner objects to categorize events into groups, enhancing automated event management and reducing manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If manual techniques and pre-programmed rules are used for event management, then event categorization can be performed, but labor intensity and cost increase significantly
Solution Approach 1:
The system enables self-service automated event categorization by having the event management system automatically generate group vectors from historical events and autonomously categorize new events based on similarity scores, eliminating the need for continuous manual rule creation and maintenance while adapting to new event patterns over time
Solution Approach 2:
The patent replaces manual mechanical categorization processes with an automated computational system that uses hash functions to generate message vectors, calculates similarity scores, and automatically assigns events to groups based on learned patterns from historical data, substituting human labor with algorithmic processing
2Loss of information
If traditional monitoring systems are deployed to provide visibility to operational metrics, then event detection is enabled, but event flooding overwhelms IT teams
Solution Approach 1:
The system merges similar events into unified group representations by calculating similarity between event message vectors and group vectors, consolidating thousands of individual events into manageable categories that maintain comprehensive visibility while reducing the volume of information presented to IT teams
Solution Approach 2:
The patent segments the overwhelming stream of individual events into organized groups based on similarity, dividing the monolithic event flood into structured categories that are easier to monitor and manage, with each group representing a cluster of related events
3Productivity
If manual techniques are used for event management in large centralized IT Operations, then event processing can be performed, but scalability is limited
Solution Approach 1:
The system dynamically adapts to changing event patterns by continuously learning from new events and updating group vectors in real-time, allowing the categorization model to evolve with the system rather than requiring static manual rule updates, enabling scalability as the IT environment grows and changes
4Extent of automation
If pre-programmed rules are used for event categorization, then event grouping can be achieved, but labor intensity and cost increase
Solution Approach 1:
The system performs preliminary learning by automatically generating group vectors from historical events during off-peak periods or system initialization, preparing the categorization model in advance so that new events can be rapidly categorized without requiring real-time manual rule creation or maintenance
Data Source
AI summary
A hash function is applied to word sequences extracted from an event message to generate hash key values. A message vector is constructed having components, where each component corresponding to a hash key value is set to a non-zero value. Similarity scores are computed between the message vector and group vectors, where each group vector represents previously grouped events. It is determined that a similarity score between the message vector and a first group vector exceeds a threshold. In response to determining that the similarity score between the message vector and the first group vector exceeds the threshold, the event message is associated with a first event group corresponding to the first group vector. The first group vector is then updated by adding the message vector to the first group vector.