Template-Based Log Event Generation for Missing Data Gaps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing logging analytics systems face challenges due to missing or incomplete event logs, leading to incomplete analysis, skewed results, and delayed troubleshooting, making it difficult to diagnose and fix issues effectively.
Innovation Solution
A system generates log events using templates and schemas, simulating realistic sequences of events by applying static and variable values, and aggregates these into event logs, which can be visualized and analyzed using logging analytics services.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If event logs are collected from real systems, then analysis accuracy is improved, but data availability deteriorates due to missing or incomplete logs
Solution Approach 1:
The patent creates synthetic copies of event logs by generating template instances that mimic the structure and characteristics of real logs. These synthetic logs are produced using schema definitions and template patterns, providing complete data where real logs are missing or incomplete, thereby maintaining analysis accuracy without relying solely on unavailable real data.
Solution Approach 2:
The system performs preliminary actions by pre-defining schemas and templates that capture the essential structure of event logs before actual logging occurs. These pre-established patterns enable the generation of synthetic logs that can fill gaps when real logs are unavailable, ensuring data availability is maintained proactively rather than reactively.
2Loss of information
If synthetic log events are generated using templates and schemas, then data completeness is improved, but system complexity increases
Solution Approach 1:
The patent segments the log generation system into distinct modular components: schema definitions, template patterns, instance generators, and composition rules. Each component handles a specific aspect of log synthesis, making the overall complex task manageable through division. This modular segmentation allows the system to achieve data completeness while keeping individual components relatively simple and maintainable.
Solution Approach 2:
The schema and template structures are designed to be universal and reusable across multiple log types and scenarios. A single schema definition can generate multiple template instances, and templates can be instantiated repeatedly to produce various log events. This universality reduces system complexity by avoiding the need to create separate generation logic for each log type, thereby achieving data completeness through reusable patterns rather than redundant code.
Data Source
AI summary
Techniques for generating simulated log events are disclosed herein. Simulated log events are generated using schemas for templates determined according to a variable parameter for the template. The templates correspond to different clients, request types associated with the simulated event, and/or servers within an organization for which events are logged. For a particular template corresponding to a client role and request type, the schemas provide rules and/or schedules for determining sequences of one or more events according to event type, timestamp, and/or other event details. The sequences of events are visualized by a logging analytics service. Schemas are manually defined or automatically determined. Organization event logs are ingested by the system to identify templates and/or schemas for the organization which are used to simulate log events for the organization.