Network Event Detection via Compressed Log Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cyber security detection mechanisms often lag in identifying new cyber attacks, as they rely on real-time comparisons and may not detect previously undetected malware or threats that have infiltrated networks unnoticed, leading to delayed identification and prolonged presence of security threats.
Innovation Solution
A method that records network events over time in a network accounting log, compresses the data into a probabilistic structure for efficient querying, and uses threat intelligence to asynchronously detect and alert on newly identified security threats, allowing for rapid identification of existing threats and automatic containment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If real-time comparison methods are used to detect cyber attacks, then the detection speed is improved, but the detection precision deteriorates because previously undetected threats are not identified
Solution Approach 1:
The system performs preliminary actions by recording and compressing network event data into accounting logs before threats are detected. This allows historical data to be prepared and stored in advance, enabling rapid retrospective analysis when new threat intelligence becomes available, thus resolving the contradiction between detection speed and precision.
2Measurement precision
If comprehensive network event data is stored for analysis, then the detection precision is improved, but the device complexity increases due to data management requirements
Solution Approach 1:
The system extracts only essential metadata from comprehensive network event data and stores it in accounting logs. This extraction process removes unnecessary details while retaining critical information needed for threat detection, thereby maintaining detection precision while reducing data management complexity.
Solution Approach 2:
The system creates compressed copies of network event data in accounting logs that retain the essential characteristics needed for threat detection. These compressed representations use less storage space and are faster to query while preserving the ability to detect threats accurately.
3Measurement precision
If historical network data is scanned for new threats, then the detection precision is improved, but the loss of time increases due to the computational burden
Solution Approach 1:
The system creates compressed copies of historical network data in accounting logs that can be rapidly queried. This copying approach allows the system to scan historical data efficiently without the computational burden of analyzing raw network events, thus reducing analysis time while maintaining detection precision.
Solution Approach 2:
The system changes the parameters of data storage by compressing network events into condensed accounting log entries. This parameter transformation reduces the size and complexity of historical data, enabling faster scanning and analysis when new threat intelligence becomes available.
Data Source
AI summary
One variation of a method for detecting a cyber attack includes: recording representations of network events occurring on a network over a period of time to a network accounting log; writing metadata values of network events in the accounting log to a compressed log file; in response to receipt of a new threat intelligence representing a newly-identified security threat identified after the period of time, querying the compressed log file for a set of metadata values of a threat element defined in the new threat intelligence; in response to detecting the set of metadata values of the threat element in the compressed log file, querying the network accounting log for a set of threat elements defined in the new threat intelligence; and in response to detecting the set of threat elements in the network accounting log, issuing an alert to respond to the newly-identified security threat on the network.