Security Log Lineage Mapping for Cost-Effective Alert Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing complexity and volume of cyber security threats require efficient management of security logs and data to identify high-value alerts and reduce unnecessary data processing costs, as existing systems struggle to track data lineage and determine the cost-effectiveness of security data sources.
Innovation Solution
A security event management system that aggregates and processes network security data to determine data lineage from log data to alerts, using a graphing unit to map data fields and models, and a cost estimation unit to assess the cost of monitoring security data, enabling users to identify high-value and low-value data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If security logs from multiple sources are aggregated and processed, then comprehensive security monitoring capability is improved, but data processing cost and system complexity increase
Solution Approach 1:
The system segments security log processing by creating separate processing pipelines for different data sources (firewalls, IDS/IPS, antivirus, etc.) and organizing logs into categorized groups. Each log source is processed independently through standardized procedures, reducing overall system complexity while maintaining comprehensive monitoring capability.
Solution Approach 2:
The patent introduces intermediate components including log normalization layers, standardized data formats, and intermediary processing modules that mediate between diverse log sources and the analysis engine. These intermediaries standardize varying log formats into unified structures, simplifying downstream processing while preserving comprehensive security monitoring.
2Loss of information
If all security log data is processed and stored, then complete security audit trail is improved, but storage cost and processing time increase
Solution Approach 1:
The system extracts and prioritizes high-value security events from the comprehensive log stream using filtering rules and anomaly detection algorithms. Critical security incidents are extracted for immediate analysis while routine logs are processed in batches or archived, reducing processing time while maintaining complete audit trails through selective extraction of important events.
Solution Approach 2:
The patent implements periodic processing cycles where logs are analyzed at different frequencies based on their importance and type. Critical security events trigger immediate analysis, while routine logs are processed in scheduled batches. This periodic action reduces overall processing time while ensuring complete audit trail maintenance through systematic periodic review.
3Measurement precision
If advanced analytics and machine learning are applied to security logs, then threat detection accuracy is improved, but computational cost and resource requirements increase
Solution Approach 1:
The system applies advanced analytics and machine learning selectively to specific log types and security events rather than processing all logs uniformly. High-value targets such as authentication failures, intrusion detection alerts, and anomalous behavior patterns receive intensive analytical processing, while routine logs use simpler analysis methods, reducing overall computational cost while maintaining high threat detection accuracy for critical events.
Solution Approach 2:
The patent implements local quality by applying different levels of analytical processing to different log sources and event types based on their security value. Critical security logs from high-risk sources receive advanced machine learning analysis, while logs from lower-risk sources use rule-based filtering. This localized application of computational resources optimizes threat detection accuracy while controlling computational costs.
Data Source
AI summary
A network security monitoring system having a plurality of data sources for providing security data, an extraction unit for extracting the security data from the sources of the security data to form extracted security data, a graphing unit for mapping the extracted security data to at least one or more data fields and data models to form graph data, and a data lineage determination unit for determining a data lineage between the log data and one or more of the alerts based on the graph data.