Typicality Filter for IT Event Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing monitoring systems for information technology resources face challenges in minimizing false positives and false negatives while ensuring the generation of true positives, leading to operational disruptions and desensitization to critical alerts.
Innovation Solution
The introduction of typicality filters, which maintain a time-dependent history of event occurrences and compare current counts to historical data to determine if an event is atypical, triggering specific actions based on predetermined proportions, thereby reducing false alerts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If persistence filters are configured to be permissive to minimize false negatives, then false positives increase and operators become desensitized to critical alerts
Solution Approach 1:
The filtering system is segmented into multiple independent filters (persistence filter, typicality filter, and novelty filter) that work in sequence. Each filter handles a specific aspect of event analysis, allowing the system to reduce false positives without compromising critical event detection. The typicality filter specifically addresses false positives by comparing events against historical patterns, while the persistence filter ensures critical events are not missed.
Solution Approach 2:
The typicality filter acts as an intermediary between the persistence filter and the event console. It receives events from the persistence filter, analyzes them against historical data, and determines whether they represent true threats or false positives. This intermediary layer prevents false positives from reaching the event console while allowing true critical events to pass through.
2Object-generated harmful factors
If persistence filters are configured to be strict to minimize false positives, then false negatives increase and critical alerts may be missed
Solution Approach 1:
The system performs preliminary analysis using the typicality filter before final event determination. By comparing events against historical patterns in advance, the system can identify potentially critical events that warrant further investigation, ensuring that true positives are not missed even if the persistence filter is configured to be strict.
Solution Approach 2:
The system uses historical event data as feedback to improve future event analysis. The typicality filter continuously learns from past events, adjusting its understanding of normal versus abnormal patterns. This feedback mechanism ensures that the system becomes more accurate over time in distinguishing true critical events from false positives.
3Object-generated harmful factors
If multiple filtering mechanisms are added to reduce false positives, then system complexity increases
Solution Approach 1:
The filtering system is designed to be dynamically configurable, allowing operators to adjust the sensitivity and parameters of each filter based on specific operational requirements. The typicality filter can adapt its historical comparison window and thresholds dynamically, reducing the need for complex fixed configurations while maintaining effectiveness in reducing false positives.
Data Source
AI summary
A monitor for information technology resources improved by introducing typicality filters to analyze potential event indicators such as alerts. A typicality filter keeps a time-dependent history of the frequency of occurrence of an associated event, wherein time is segmented into monitoring periods. At the end of each monitoring period, a present count of occurrences of the event is determined, and compared with the numbers of occurrences of that event in a subset of monitoring periods read from the history. If the present count exceeds the number of occurrences of the event in a predetermined proportion of the subset of historical monitoring periods, a first action is invoked; otherwise, a second action is invoked.