Host-Based Log Integration for Network Traffic Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network-based logging systems like Bro (Zeek) face limitations in integrating host-based logging data from remote hosts with different operating systems, such as Windows, which are not compatible with its SQL-style queries, hindering incident response and security analysis.
Innovation Solution
A system comprising a transport module, event parser, and consumer modules that utilize lightweight applications like WinLogBeat and Logstash to forward Windows host-based logs in JSON format to a network traffic analysis system, where an event parser, potentially distributed, converts these logs into events consumable by Bro, using Python bindings to raise events and correlate network and host-based metadata.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If SQL-style queries are used for log collection, then network-based logging works well, but host-based logging from Windows systems is incompatible
Solution Approach 1:
The patent introduces an intermediary component (log collector agent) that runs on remote hosts and translates host-based logs into a standardized format compatible with the network-based logging system. This intermediary layer enables Windows hosts to integrate with the Bro logging system without requiring SQL-style queries or modifying the core Bro architecture, thus resolving the compatibility issue while maintaining system integrity.
2Loss of information
If link traffic analysis is performed, then intrusion detection is achieved, but incident response requires additional information beyond network traffic
Solution Approach 1:
The patent merges network-based logging (Bro) with host-based logging (Windows Event Logs, Sysmon) by collecting both types of logs through a unified architecture. The log collector agent gathers host-based logs and forwards them to the Bro system, where they are correlated with network traffic data. This combination provides comprehensive information for incident response, including both network observables and host process details, without creating separate siloed systems.
3Loss of information
If host-based logs are collected from multiple remote hosts, then comprehensive security analysis is enabled, but data normalization and correlation become complex
Solution Approach 1:
The patent transforms host-based logs from their native heterogeneous formats into a standardized parameter structure that Bro can process. The log collector agent extracts relevant fields from various log sources and normalizes them into consistent event types with standardized attributes. This parameter transformation enables Bro's existing correlation and analysis capabilities to work uniformly across different host types without requiring complex custom processing for each log format.
Data Source
AI summary
In an example, a network traffic analysis system including a network based logging host may include a transport module, an event parser, and one or more consumer modules. The transport module may collect one or more messages including one or more event logs from one or more remote hosts, respectively. The event parser may generate normalized events consumable by the network logging host from the collected messages. The consumer modules may host process metadata of the event out to file for analysis. Other embodiments may be disclosed and/or claimed.