Transformer Log Anomaly Detection for Faster Database Diagnosis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in efficiently identifying the cause of performance anomalies due to diverse causes such as resource saturation, incorrect settings, and physical design, requiring significant time and effort from specialized administrators.
Innovation Solution
A method and device for detecting anomalies in log data using a transformer to reconstruct log data, calculate anomaly scores, and classify causes, incorporating principal component analysis and Mahalanobis distance to automate the detection process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual analysis of log data is performed by database administrators, then the accuracy of anomaly detection can be maintained through expert judgment, but the time and effort required increases significantly
Solution Approach 1:
The system performs self-diagnosis by automatically detecting anomalies in log data through machine learning models, eliminating the need for manual analysis by database administrators while maintaining high detection accuracy through automated pattern recognition
Solution Approach 2:
The patent replaces the mechanical process of manual log analysis with an automated electronic system using transformer-based machine learning models that process log data and detect anomalies algorithmically, significantly reducing analysis time while maintaining precision
2Reliability
If comprehensive log data analysis is performed to identify all possible causes, then the reliability of anomaly detection improves, but the device complexity increases
Solution Approach 1:
The system segments the complex log data analysis task into distinct functional modules: log data collection, transformer-based reconstruction, anomaly scoring, and cause classification. Each module handles a specific aspect of the analysis, improving reliability through specialized processing while managing complexity through modular design
Solution Approach 2:
The patent introduces an anomaly score as an intermediary metric that bridges the gap between raw log data and final anomaly detection. This intermediate representation simplifies the complex relationship between diverse log entries and anomaly causes, making the system more manageable while maintaining detection reliability
3Productivity
If automated anomaly detection is implemented, then productivity in log analysis improves, but the measurement precision may deteriorate due to lack of expert judgment
Solution Approach 1:
The patent replaces expert human judgment with transformer-based machine learning models that have been trained to recognize anomaly patterns in log data. This substitution maintains high detection accuracy through learned patterns while dramatically improving productivity through automated processing
Solution Approach 2:
The system performs preliminary training of the transformer model on labeled log data before deployment. This preliminary action enables the automated system to learn expert-level patterns during the training phase, allowing it to maintain high detection accuracy during automated operation without requiring expert intervention during actual analysis
Data Source
AI summary
The following disclosure relates to a method of detecting an anomaly in log data and includes obtaining first log data by collecting log data from a database and processing the log data in a time axis through a log data manager, reconstructing the first log data into second log data through a transformer, calculating an anomaly score of each time point through the transformer based on a difference between the first log data and the second log data, and detecting an anomaly for each time point through an anomaly detector based on the anomaly score.