Log Event Message Type Identification for Distributed Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed computing systems face inefficiencies and complexities in managing and processing large volumes of log/event messages, leading to significant computational and networking overheads, which hinder effective administration and management.
Innovation Solution
Incorporating a pre-processing step to identify the type of log/event message and specifying initial processing criteria, allowing for efficient preprocessing by message-collector and message-ingestion-and-processing systems through the use of event_type() functions and message-restoration functions, reducing data volume and improving processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional log/event message processing is used without type identification, then all messages are processed uniformly, but computational overhead and processing time increase significantly
Solution Approach 1:
The patent segments the log processing workflow by introducing type identification as a preliminary step. Messages are divided into different processing paths based on their event types (e.g., application logs, system logs, security logs), allowing each segment to be handled by specialized processing logic optimized for its specific characteristics, thereby improving overall throughput while reducing latency for each message type
Solution Approach 2:
The patent applies preliminary action by identifying and classifying message types before the main processing occurs. The type identification step prepares messages by categorizing them into predefined event types, enabling subsequent processing steps to operate more efficiently on already-classified data rather than treating all messages uniformly, thus reducing redundant computational overhead
2Reliability
If comprehensive processing is applied to all log messages, then no message type is overlooked, but data volume and processing complexity increase
Solution Approach 1:
The patent applies local quality by assigning different processing characteristics to different message types. Each event type receives processing tailored to its specific requirements (e.g., security logs may require stricter validation, while application logs may need different formatting), ensuring each message type is handled with appropriate rigor while avoiding unnecessary processing complexity for other message types
Solution Approach 2:
The patent changes processing parameters based on message type classification. By identifying event types, the system can dynamically adjust processing parameters such as validation rules, storage formats, and analysis methods for each category, maintaining comprehensive coverage of all message types while optimizing processing complexity for each specific type rather than using a one-size-fits-all approach
Data Source
AI summary
The current document is directed to methods and systems that efficiently process log/event messages within and among distributed computer facilities. Various different types of initial processing steps may be applied to a stream of log/event messages received by a message-collector system or a message-ingestion-and-processing system. By including a pre-processing step two identify the type of a received log/event message, and by specifying initial-processing-step criteria with respect to log/event-message types, significant increases in the efficiency of log/event-message preprocessing by message-collector systems and message-ingestion-and-processing systems is achieved.