Log Analysis Using AI Pattern Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional rule-based log file analysis mechanisms are inflexible and cannot exhaust all log records, often missing newly emerged logs and requiring excessive labor for rule construction and maintenance, while natural language processing technologies struggle with the specific code-based expression of log records.
Innovation Solution
The method determines patterns in log records, converts them into log identifiers, and selects a target reference log identifier with high similarity for analysis, using a comparison model that can include machine learning or deep learning algorithms to process log records efficiently and accurately.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rule-based matching process is used to detect critical logs, then the detection can be performed systematically, but it cannot exhaust all possibilities and requires excessive labor costs for rule construction and maintenance
Solution Approach 1:
The patent replaces the mechanical rule-based matching system with an AI model-based system. The AI model automatically learns patterns from historical log data and performs detection without requiring manual rule construction, thereby eliminating the complexity of rule maintenance while improving detection completeness through the model's ability to generalize from training data.
Solution Approach 2:
The patent transforms the detection approach by changing from fixed rule parameters to dynamic AI model parameters. The AI model's parameters are automatically adjusted through training on historical data, allowing the system to adapt to new log patterns without manual intervention, thus reducing the labor burden of rule maintenance while maintaining high reliability.
2Reliability
If rule-based matching process is used to detect critical logs, then the detection can be performed systematically, but it cannot exhaust all possibilities
Solution Approach 1:
The patent substitutes the mechanical rule-based matching process with an AI model that can process and analyze log data more comprehensively. The AI model's ability to understand contextual relationships and patterns allows it to exhaust more possibilities in log detection, improving both detection completeness and efficiency simultaneously.
Solution Approach 2:
The patent introduces dynamic capabilities to the detection system through the AI model. Unlike static rules, the AI model can dynamically adapt its detection criteria based on learned patterns from historical data, enabling it to exhaust more possibilities and improve detection completeness without sacrificing efficiency.
3Adaptability or versatility
If natural language processing technologies are used to process log records, then language understanding can be improved, but they struggle with the specific code-based expression of log records
Solution Approach 1:
The patent applies local quality by training the AI model specifically on log record data rather than using general natural language processing. This specialized training enables the model to understand the specific code-based expressions and patterns unique to log records, thereby improving analysis accuracy while maintaining the adaptability to handle various log formats.
Solution Approach 2:
The patent uses copying by training the AI model on numerous examples of log records with their corresponding meanings and contexts. Through this training process, the model learns to accurately interpret code-based expressions in log records, bridging the gap between natural language understanding capabilities and the specific syntax of log files.
Data Source
AI summary
Embodiments of the present disclosure relate to a method, an electronic device, and a computer program product for analyzing a log file. The method may include: determining, based on a plurality of reference patterns, corresponding patterns for a plurality of log records in the log file. The method may further include: respectively determining the plurality of log records as a plurality of log identifiers corresponding to the corresponding patterns. The method further includes: determining, from the plurality of log identifiers, a log identifier to be analyzed corresponding to a predetermined event. In addition, the method may further include: selecting a target reference log identifier from a plurality of reference log identifiers corresponding to the plurality of reference patterns, wherein a first similarity between the target reference log identifier and the log identifier to be analyzed is higher than a first threshold similarity.