Application Event Logging Augmentation via AI Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Software event logs are inconsistent in quality, with important events for debugging often not logged, and current solutions require access to application source code, which is not always available.
Innovation Solution
A method that extracts log sequences and interaction sequences from software applications, using a trained text generation model to generate log entries for missing events and anomalies, without modifying the application, by analyzing log data and interaction data to identify gaps in logging and fill them.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If log entries are manually added to capture important events, then log data quality improves, but device complexity and maintenance effort increase
Solution Approach 1:
The system automatically generates log entries by monitoring application execution and detecting events of interest, eliminating the need for manual log entry creation. The logging system serves itself by using AI/ML models to identify what should be logged based on patterns in the application behavior and historical log data.
Solution Approach 2:
The system dynamically adjusts logging parameters such as log level, detail depth, and event selection criteria based on application state, execution context, and detected anomalies. This allows the logging behavior to adapt automatically without manual configuration changes.
2Difficulty of detecting and measuring
If comprehensive logging is implemented to capture all important events, then debugging capability improves, but loss of time for log analysis increases
Solution Approach 1:
The system extracts and prioritizes only the most relevant log entries based on AI/ML analysis of application behavior and event importance. Rather than presenting all logs, it extracts and surfaces critical events that require attention, reducing analysis time while maintaining debugging effectiveness.
Solution Approach 2:
Manual log analysis is replaced with automated AI/ML-based analysis that can quickly process and interpret large volumes of log data, identifying patterns and anomalies without human intervention. This substitution dramatically reduces the time required to analyze logs while improving detection accuracy.
3Reliability
If logging is enhanced to capture distributed system events, then system observability improves, but device complexity and implementation difficulty increase
Solution Approach 1:
The logging system is designed to work across multiple distributed components and technologies with a unified approach. The AI/ML model and logging framework are technology-agnostic, allowing them to capture events from diverse sources (Java, .NET, Python, cloud services, on-premises systems) without requiring component-specific implementations.
Solution Approach 2:
An intermediary logging layer is introduced between distributed system components and the analysis tools. This intermediary standardizes event capture and normalization, making the system observable without requiring changes to individual components. The intermediary handles the complexity of distributed tracing and event correlation automatically.
Data Source
AI summary
Responsive to determining that a step in an interaction sequence has below a threshold similarity to any step in a set of log sequences, a first log entry is caused to be generated. The first log entry is emitted responsive to execution of the step in the interaction sequence by the software application. Responsive to determining that a starting key action lacks a corresponding ending key action, a second log entry is caused to be generated. The second log entry is emitted responsive to execution of code related to the corresponding ending key action by the software application. Responsive to determining, using a trained anomaly detection model, that a keyword related to an injected fault is not present in resulting fault log data, a third log entry is generated. The third log entry is emitted responsive to execution of code related to the keyword by the software application.