Endpoint Log Filtering for Security Server Load Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional logging systems send excessive non-suspicious log events to security servers, overwhelming them and potentially leading to missed security threats, as they struggle to filter out irrelevant data effectively.
Innovation Solution
Implementing a system on endpoint computing devices to identify and filter out non-suspicious log events before forwarding them to security servers, using reputation analysis and behavioral scoring to determine trustworthiness of processes, thereby excluding unnecessary logs and sending only suspicious events for analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If traditional logging systems forward all log events to security servers, then complete log data is available for analysis, but the security servers become overwhelmed by excessive non-suspicious log events
Solution Approach 1:
The system performs preliminary filtering action at the endpoint computing device before logs reach the security server. By analyzing log events locally and excluding non-suspicious events in advance, the system reduces the volume of data transmitted to the security server, preventing server overload while maintaining security monitoring effectiveness
Solution Approach 2:
The invention extracts and removes non-suspicious log events from the data stream before transmission to the security server. By identifying and taking out irrelevant log data at the endpoint, the system enables security servers to focus only on suspicious events that require analysis, thereby resolving the contradiction between data completeness and server processing capacity
2Reliability
If security servers analyze all received log files, then thorough security analysis is performed, but the servers cannot effectively identify security threats due to overwhelming data volume
Solution Approach 1:
The endpoint computing device performs preliminary analysis of log events before transmission to the security server. By pre-filtering and preparing only suspicious events for transmission, the system reduces the complexity of data processing at the security server while maintaining thorough security analysis capability
Solution Approach 2:
The system extracts and transmits only suspicious log events to the security server, removing non-suspicious events from the data stream. This extraction approach simplifies the security server's processing tasks while ensuring that thorough security analysis is performed on the remaining critical data
3Productivity
If the system filters out non-suspicious log events at the endpoint, then the security server burden is reduced, but the complexity of log analysis at the endpoint increases
Solution Approach 1:
The system changes the parameter of log event selection by introducing trustworthiness scores and thresholds. By evaluating processes based on their trustworthiness scores and using this parameter to determine which logs to transmit, the system simplifies the filtering logic at the endpoint while significantly improving security server efficiency
Solution Approach 2:
The invention applies local quality assessment by evaluating the trustworthiness of individual processes and log events at the endpoint. By assigning quality metrics (trustworthiness scores) to local log sources, the system enables intelligent filtering that improves security server efficiency without requiring complex centralized control
Data Source
AI summary
The disclosed computer-implemented method for filtering log files may include (1) identifying, on the endpoint computing device, log files that recorded events performed by processes executing on the endpoint computing device, (2) prior to sending the log files from the endpoint computing device to a security server for analysis, filtering, based on an analysis of the events recorded by the log files, the log files by excluding log files that recorded non-suspicious events, and (3) forwarding the filtered log files from the endpoint computing device to the security server for analysis. Various other methods, systems, and computer-readable media are also disclosed.