Real-time Log Analysis Service Integrating External Event Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Real-time log analysis systems typically fail to provide significant insight into the root cause of problems or failures in host computers, making it difficult and time-consuming to determine the cause of errors, especially after software updates.
Innovation Solution
An RTLA service that integrates log data with event data from external services, allowing for quicker identification of the root cause of issues by presenting both log and event data for analysis through a user interface that filters, annotates, and displays relevant information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If log data is collected and analyzed in real-time, then service monitoring and trend identification are improved, but the ability to identify root cause is not significantly enhanced
Solution Approach 1:
The patent combines log data from host computers with event data from external services into a unified analysis system. The RTLA service integrates multiple data sources (log data, event data, and optionally change data) to provide comprehensive root cause analysis capabilities that go beyond traditional single-source log monitoring.
Solution Approach 2:
The patent introduces an intermediary data integration layer that connects log data from host computers with event data from external services. This intermediary mechanism enables correlation between internal log events and external service events, facilitating root cause identification without requiring direct access to multiple data sources.
2Ease of manufacture
If only log data from host computers is analyzed, then collection and processing are straightforward, but root cause identification becomes difficult and time-consuming
Solution Approach 1:
The patent performs preliminary data collection and integration of event data from external services alongside log data. By pre-collecting and correlating event data with log data, the system prepares comprehensive analysis materials in advance, enabling rapid root cause identification when issues occur without requiring time-consuming data gathering during incident response.
Solution Approach 2:
The RTLA service is designed to handle multiple data types (log data, event data, change data) through a unified processing architecture. This multi-functional approach allows the system to maintain simple log data processing while simultaneously integrating and analyzing external event data, eliminating the need for separate analysis pipelines.
3Loss of information
If external event data is integrated with log data, then root cause analysis capability is improved, but system complexity increases
Solution Approach 1:
The patent segments the data integration process into distinct functional components: log data collection module, event data collection module, data correlation module, and analysis module. This segmentation allows each component to handle specific data types and processing tasks independently, reducing overall system complexity while enabling comprehensive root cause analysis.
Data Source
AI summary
A real-time log analyzer (“RTLA”) network service obtains log data from host computers. The RTLA network service also obtains event data from external network services. A user interface (“UI”) associated with the RTLA network service retrieves the log data and the event data from the RTLA network service for a time period, and presents the log data and the event data for the time period together for use in analyzing the root cause of failures indicated in the log data. The UI can also provide functionality for defining the time period, selecting the hosts for which log data and event data is to be presented, specifying the type of log data and event data to be displayed or otherwise filtering the log and event data, displaying data from a trouble ticket system, defining and displaying annotations relating to the event data, and/or displaying a human-friendly description of the event data.