Security Audit Log Prioritization via Dynamic Throttling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed systems face congestion due to the excessive transmission of logging information, which can lead to network bandwidth, storage, and processing limitations, causing servers to fail under heavy data loads.
Innovation Solution
A method is introduced that uses a rule-based algorithm to categorize log data, trains a classifier to prioritize records based on user selection, and dynamically throttles data transmission by assigning priority scores and adjusting transmission rates, ensuring only critical data is transmitted promptly, while non-essential data is sent during off-peak times or reduced in volume.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all log data is transmitted from distributed nodes to collectors, then complete security audit coverage is achieved, but network congestion and server overload occur
Solution Approach 1:
The patent segments log data into different priority levels (suspicious, normal, low-priority) based on security relevance. The segmentation is achieved through rule-based classification at the edge nodes, dividing the monolithic log transmission into categorized streams that are handled differently in terms of transmission timing and priority.
Solution Approach 2:
The patent extracts and prioritizes only the most critical log data (suspicious activity logs) for immediate transmission to collectors. Non-critical logs are either delayed or filtered out, effectively taking out only the essential security-relevant information from the complete log set for urgent processing.
2Loss of information
If log transmission rate is increased to ensure all data is captured, then audit completeness improves, but network bandwidth and storage resources are exhausted
Solution Approach 1:
The patent applies local quality by assigning different transmission qualities to different log categories. Suspicious logs receive high-priority treatment with immediate transmission, normal logs are transmitted at standard rates, and low-priority logs are either sampled or delayed. This creates a quality hierarchy in data transmission based on security relevance.
Solution Approach 2:
The patent implements periodic action by transmitting logs in batches rather than continuously. Edge nodes accumulate logs locally and transmit them in periodic intervals, with the timing and frequency adjusted based on log priority and current system load conditions.
3Speed
If all nodes transmit logs simultaneously, then real-time security monitoring is achieved, but network congestion causes server failures
Solution Approach 1:
The patent implements dynamics by making log transmission rates adaptive rather than static. Edge nodes dynamically adjust their transmission rates based on local log priority assessments and collector feedback signals. When collectors indicate high load, edge nodes reduce transmission rates for non-critical logs while maintaining rates for suspicious activity logs.
Solution Approach 2:
The patent establishes a feedback loop where collectors monitor their processing capacity and send feedback signals to edge nodes about current load conditions. Edge nodes use this feedback to dynamically adjust their log transmission rates, reducing transmission when collectors are overwhelmed and increasing transmission when capacity is available.
4Loss of time
If priority-based transmission is implemented to reduce congestion, then critical security events are detected faster, but non-critical log data may be lost or delayed
Solution Approach 1:
The patent applies partial action by transmitting only a subset of logs at any given time based on priority. Instead of transmitting all logs simultaneously, the system transmits suspicious logs immediately, normal logs at scheduled intervals, and low-priority logs only when capacity permits, accepting that some non-critical data may be delayed or filtered.
Data Source
AI summary
A method for controlling transmission of security audit logs based on a model, a method for controlling transmission of log data based on a model, and a logging system. One embodiment may comprise receiving transmitted log information from a plurality of nodes, applying a rule-based algorithm to the transmitted log information to categorize a first batch of data as included in a security analysis, a second batch of data as excluded from the security analysis, and a third batch of data as actually reviewed in the security analysis based on a user selection, training a classifier based on outcomes of the rule-based algorithm, converting the classifier to run as a trained model executable on the plurality of nodes, and transmitting the trained model executable to the plurality of nodes.