Iterative Log Filtering for Unknown Security Issue Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing filters are ineffective in detecting previously unknown security issues in computing devices, and there is a shortage of skilled security personnel to address these issues, leading to a false sense of security and high costs for organizations.
Innovation Solution
An iterative process of applying user-selected filters from a library, including rules-based and model-based filters, to identify potential security issues, allowing less skilled users to winnow down log events for further analysis by skilled personnel.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing filters are applied to detect security issues, then known security issues can be identified, but previously unknown security issues cannot be detected
Solution Approach 1:
The system performs preliminary actions by collecting and storing diverse log events from multiple sources before security analysis is needed. This pre-collection of data enables flexible filtering and analysis when unknown security issues need to be detected, without requiring pre-configured filters for every possible threat
Solution Approach 2:
The system changes parameters by allowing dynamic modification of filter criteria and analysis parameters. Users can adjust filter sensitivity, select different log event types, and modify analysis parameters to adapt to emerging security threats that differ from known attack patterns
2Reliability
If more skilled security personnel are hired to analyze log events, then unknown security issues can be detected, but costs increase significantly
Solution Approach 1:
The system introduces an intermediary automated filtering and analysis layer between log event collection and skilled security personnel review. This intermediary process pre-processes logs, applies initial filtering, and prioritizes events for human review, enabling fewer skilled personnel to effectively manage larger numbers of systems
Solution Approach 2:
The system enables self-service security monitoring through automated log collection, filtering, and initial analysis capabilities. Less skilled users can perform basic security analysis using the platform's tools, reducing dependency on expensive skilled security personnel for routine monitoring while they focus on complex unknown threats
3Measurement precision
If comprehensive log event analysis is performed to detect unknown security issues, then detection accuracy improves, but processing time and computational resources increase
Solution Approach 1:
The system segments log event analysis into multiple stages: initial filtering by event type and source, priority-based sorting, and progressive analysis depth. This segmentation allows the system to quickly process high-volume routine logs with simple filters while applying more comprehensive analysis only to prioritized events requiring deeper inspection
Solution Approach 2:
The system applies partial analysis to most log events using standardized filters, and excessive (comprehensive) analysis only to a subset of high-priority or suspicious events. This selective approach maintains high detection accuracy for critical threats while avoiding unnecessary processing time for routine logs
Data Source
AI summary
Log events for a target system are received. In each of a number of iterations, selection of a filter from a library of preexisting filters is received from a user, the selected filter is applied to the log events to generate filtered log events, and the filtered log events are displayed to the user. In each iteration other than a first iteration, the selected filter is applied to the filtered log events that are generated in an immediately preceding iteration.