Hybrid log analysis method based on large model and structure tree

Abstract

The invention discloses a mixed log analysis method based on a large model and a structure tree, and the method comprises the following steps: firstly intelligently discriminating an original log type and carrying out key value conversion through an LLM (Log Language Model), and then carrying out the precise analysis through a special structure tree classifier according to different log types. According to the scheme, effective log analysis in a mixed log scene is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of log analysis, in particular to a hybrid log analysis method based on a large model and a structure tree and a storage medium. BACKGROUND

[0002] With the wide application of software systems in various fields of society, their number and functional complexity are growing exponentially, and extremely large amounts of log files are generated every day. These log files play a crucial role in the development and maintenance of software, such as abnormality detection, fault prediction, and user profile construction, which are all based on log data for analysis. Therefore, logs have become an extremely important key tool for developers and system managers. When developers write applications, systems, or other software, they insert statements that generate logs into the code. The generated logs are usually saved in chronological order to trace and analyze events in a specific time period. Common log formats include plain text files, database records, and binary files in specific formats. As shown in Figure 1 The log usually has the following components: log level (e.g., "I", "D", "W", etc.), log content (e.g., "visible is system.time.showampm"), timestamp (e.g., "16:13:38.859"), and log component (e.g., "TextView").

[0003] In recent years, with the rapid growth of software and the continuous development of cloud technology, the amount of log data has also shown explosive growth, making it extremely difficult to manually identify valuable information from massive logs. For this reason, researchers at home and abroad have carried out extensive research on automated log analysis. Log parsing is the first and core step, aiming to extract log templates and convert semi-structured log messages into structured formats for subsequent analysis tasks. The core challenge lies in accurately identifying the constant part (i.e., fixed descriptive text, such as verbs or keywords indicating specific software operations) and the variable part (i.e., runtime dynamically changing values, such as timestamps, user IDs, or IP addresses) in the log. For this reason, many valuable methods have emerged.

[0004] However, there are still limitations in three aspects:

[0005] (1) Existing methods usually preset the scenario of processing single-line logs in log parsing, relying on extracting general patterns (e.g., Drain or Spell) line by line to generate templates, thereby achieving structured extraction. However, actual system logs often exhibit highly heterogeneous multi-line structured content, which significantly increases the complexity of parsing. For example, index monitoring tables (such as CPU usage, memory occupancy statistics) may span multiple lines in tabular form, containing numerical and label grid layouts; stack trace information (e.g., Java Exception or Python Traceback) is unfolded in a hierarchical indentation call chain, involving multiple levels of details such as exception types, file paths, and line numbers; in addition, key-value pair enumeration (e.g., Hadoop Counters or Kubernetes resource indicators) often appears in nested key-value lists, covering dynamically generated performance counters and configuration parameters. The presence of these multi-line structured elements not only challenges the robustness of traditional template matching, but also may lead to pattern generalization failure, semantic loss, or low parsing efficiency, thereby affecting the accuracy of downstream tasks such as anomaly detection and root cause analysis.

[0006] (2) The centralized log collection architecture widely adopted by modern software results in log streams that are essentially a mixture of single-line log messages and multi-line log messages. As shown in Figure 2 , an example of mixed log parsing is given, which contains a single-line log message, a table log message, and a text log message. Single-line log messages are usually concise event records (e.g., current IP records), with a relatively regular format, making them easy to parse directly; table log messages are presented in multi-line grid form (e.g., index monitoring data), containing numerical, label, and timestamp, with complex structure and cross-line association; text log messages (e.g., stack trace or key-value pair enumeration) involve multi-line nested content, which may include hierarchical indentation call chains, dynamically generated performance counters, or configuration files, with rich semantics but diverse patterns.

[0007] (3) The diversity and continuous iteration of modern software systems result in significant differences in log formats, increasing the complexity of log parsing

[11] . As shown in Figure 3As shown, log examples generated by different software clearly demonstrate this heterogeneity, including logs from Windows, macOS, and HiBench, which exhibit drastically different formats in terms of timestamps, log levels, and log content. Windows logs typically use an event-driven structured format, with timestamps following the ISO 8601 standard. Log levels are explicitly marked with terms such as "information," "warning," or "error," and the content mainly describes the status of system services or applications. macOS logs often use the system logging framework, with log levels represented by numbers or symbols, and the content covers kernel events, user operations, or application debugging information. HiBench logs are optimized for large data workloads, with log levels closely related to performance metrics, and the content includes dynamic data such as key-value pair enumerations, task execution duration, or resource utilization. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a hybrid log parsing method that integrates Large Language Model (LLM) and Structure Tree Classifier. This method fully utilizes the excellent adaptability of LLM in processing heterogeneous log data. Through advanced semantic modeling and contextual understanding capabilities, it can automatically identify and extract complex patterns in single-line logs, multi-line table logs, and nested text logs without requiring a large amount of manually labeled data, achieving highly convenient few-sample or zero-sample adaptation. At the same time, the Structure Tree Classifier, with its efficient decision tree structure and rule-making mechanism, quickly classifies and regularizes the hybrid log stream, enabling efficient clustering of log messages based on features in real-time scenarios, improving overall efficiency and robustness.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] The hybrid log parsing method based on large models and structure trees includes the following steps:

[0011] By using an LLM model combined with structured JSON hints, hybrid logs are intelligently categorized and key-value transformations are performed.

[0012] A dedicated structure tree classifier is designed to parse different log types based on the classification results.

[0013] Mixed logs are classified into three categories: single-line logs, table logs, and multi-line logs.

[0014] The method also includes: dynamically adapting prompt words to heterogeneous formats, outputting log types based on semantic features using LLM, and extracting timestamps using regular expressions to refine parameters into semantic wildcards, laying the foundation for structured parsing.

[0015] In the method, a differentiated grouping strategy is implemented for different types. For single-line logs, a feature index is constructed based on the number of tokens and the number of keys. For table and text logs, a structure tree classifier is constructed based on the number of log lines and the distribution of key types. TF-IDF vectorization and cosine similarity threshold are used to perform semantic filtering and dynamic fusion of candidate templates, so as to realize a high-precision generation and incremental update mechanism for the template library.

[0016] Intelligent classification of mixed logs includes:

[0017] 1) Log reading and splitting: Read log files from a specified path and extract independent log blocks using a splitting algorithm based on timestamps or empty lines;

[0018] 2) Prompt Term Design: Design structured JSON prompt terms, clearly define classification criteria, and dynamically adapt to heterogeneous log formats. The prompt terms consist of four parts: task description, classification criteria, output format, and requirements. LLM returns the results in JSON format. Prompt terms are dynamically inserted into log content using placeholders to adapt to different log formats.

[0019] 3) LLM Classification: Each log block is combined with a prompt word, converted into an input token by a tokenizer, and truncated to 1024 tokens. LLM utilizes semantic understanding to determine the log type based on the criteria in the prompt words. If the LLM output is invalid, classification is performed based on log features.

[0020] 6) Result saving: Save the classification results as single-line log, multi-line log and table log files, retain the original format, and generate classification statistics and debugging logs.

[0021] In the method, log parameters are refined into multiple types of key-value tokens with preset semantics by quantitative statistics of log keywords, and structured prompt words are designed to guide LLM to achieve accurate identification of variable types.

[0022] Intelligent classification of mixed logs also includes: when the LLM classification output is invalid, classification is based on log features, where single-line logs are single-line content, table logs match table header patterns or separators, and multi-line logs contain indentation, stack traces, or separators.

[0023] The date data after intelligent classification is preprocessed using LLM. By quantifying and statistically analyzing log keywords, log parameters are refined into eight categories of key-value tokens with preset semantics. Structured prompt words are designed to guide LLM in achieving accurate identification of variable types.

[0024] The preprocessed log messages are assigned a LogID with structural representation to each log message. The preprocessed log messages contain log type and semantic wildcards. The encoded messages are used to obtain grouped log clusters, which are used as input for template filtering, similarity calculation and template merging.

[0025] LogID generation rules are strictly adapted to log type characteristics: single-line logs use a complete token sequence as LogID, while table / text logs form a common sequence by connecting the first and last tokens;

[0026] The grouping process adopts a two-level tuple architecture: First, for single-line logs, the number of tokens and keys are calculated based on LogID to construct a tuple, and for table / text logs, the number of log lines and keys are counted to construct a tuple. The first log grouping is completed based on tuple consistency. Second, within each tuple group, key tuples are constructed by extracting key types, and secondary classification is performed based on key tuple consistency.

[0027] A template filtering strategy based on document similarity is used to merge and judge two log templates within a secondary group. The TF-IDF vectorization method is used to convert the text data into numerical features. Based on the numerical features, cosine similarity is used to measure the similarity between log entries. The similarity is evaluated by calculating the cosine value of the angle between two vectors. A similarity threshold is set. If the similarity between two entries is higher than the set threshold, the label of one entry is updated to the label of the other entry; otherwise, the labels of both entries are retained. The updated labels and the original log content are written to a new log file to obtain the parsing results.

[0028] The advantages of this invention are as follows: It fully leverages the superior adaptability of LLM in processing heterogeneous log data. Through advanced semantic modeling and contextual understanding capabilities, it can automatically identify and extract complex patterns from single-line logs, multi-line tabular logs, and nested text logs without requiring extensive manual data annotation, achieving highly convenient few-shot or zero-shot adaptation. Simultaneously, the tree structure classifier, with its efficient decision tree structure and rule-based mechanism, rapidly classifies and regularizes mixed log streams, enabling efficient clustering of log messages based on features in real-time scenarios, improving overall efficiency and robustness. This complementarity between LLM and the tree classifier not only overcomes the generalization shortcomings of traditional methods for multi-line structured content but also significantly improves parsing accuracy and speed, providing a reliable foundation for downstream tasks such as anomaly detection and root cause analysis. Attached Figure Description

[0029] The following is a brief explanation of the contents of each of the accompanying drawings and the markings in the drawings:

[0030] Figure 1 This is a schematic diagram illustrating the messages and their contents recorded in log files in the prior art.

[0031] Figure 2 This is a schematic diagram illustrating an example of hybrid log parsing in existing technologies;

[0032] Figure 3This is a schematic diagram illustrating different software logs in the prior art;

[0033] Figure 4 This is an architecture diagram of the log parsing method of the present invention;

[0034] Figure 5 This is a schematic diagram illustrating the principle of the prompt words in this invention;

[0035] Figure 6 This is a schematic diagram illustrating the log key-value conversion principle of the present invention. Detailed Implementation

[0036] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and the description of the preferred embodiments.

[0037] Logs, as crucial records generated during software and hardware operation, are of great significance for system diagnosis and maintenance. Log parsing is the first and critical step in analyzing log files, aiming to parse semi-structured raw logs into structured log templates. Existing log parsing methods have made significant progress in parsing single-line logs. However, with the increasing complexity of software and hardware, the generated log formats are becoming more diverse, and log types have gradually evolved into hybrid logs, including single-line logs, table logs, and text logs. Existing methods for parsing hybrid logs have limitations. To address this, a method for parsing hybrid logs, LTP (LLM-Tree Hybrid Log Parsing Method), is proposed. This method first intelligently identifies the original log type and performs key-value conversion using an LLM (Large Language Model), and then uses a dedicated tree structure classifier for accurate parsing based on different log types. Experimental results show that when processing hybrid logs, LTP achieves an average parsing accuracy of 0.921 and an F1-Score of 0.891, representing an improvement of over 60% compared to traditional single log processors. The results validate the superiority of LTP as an efficient log parsing method, especially demonstrating significant stability advantages in mixed log scenarios.

[0038] The parsing method in this embodiment consists of two steps: First, using LLM (e.g., loaded via Hugging Face's transformers library) combined with structured JSON hints, mixed logs are intelligently classified (single-line, table, multi-line logs) and key-value conversion is performed. Hints dynamically adapt to heterogeneous formats. LLM outputs the log type based on semantic features and uses regular expressions to remove timestamps, refining parameters into semantic wildcards (e.g., <*time>, <*int>), laying the foundation for structured parsing. Second, based on the classification results, LTP designs dedicated structure tree classifiers for different log types. Single-line logs generate LogIDs using token and key tuples, while table / text logs construct LogIDs based on row count and key distribution. A multi-way search tree is used for two groupings, combined with TF-IDF vectorization and cosine similarity filtering to generate a high-precision template library.

[0039] To efficiently parse hybrid logs, this paper proposes the LTP (LLM-Tree Hybrid Log Parsing Method), which achieves high-precision hybrid log parsing through a two-stage process. Figure 4 The demonstration showcases an example of the LTP framework. First, LLM is applied to intelligently categorize hybrid logs into three types: single-line logs, multi-line logs, and tabular logs. Regular expressions based on domain knowledge are used for log preprocessing, injecting semantic wildcards (such as <*int> and <*time>) to preserve the business meaning of parameters, laying a structured foundation. Second, differentiated grouping strategies are implemented for different types. Single-line logs are indexed based on the number of tokens and keys, while tabular and text logs are indexed based on the number of log lines and key type distribution to build a structure tree classifier. Finally, TF-IDF vectorization and cosine similarity thresholding are used to semantically filter and dynamically fuse candidate templates, achieving a high-precision generation and incremental update mechanism for the template library.

[0040] The specific steps of the parsing method in this solution include:

[0041] 1. LLM Classification

[0042] In modern information systems, log files serve as the core carriers for recording system operation status and events, and their accurate classification is crucial for fault diagnosis, security monitoring, and performance optimization. This study proposes a hybrid log classification method based on LLM (Log Modeling), aiming to efficiently identify single-line, multi-line, and table logs to support the prediction of potential problem patterns. The input to this step is the original log file, and the output is the classification result for each log block (log_type: single_line, multi_line, or table, along with the classification reason). These classification results will serve as input for key-value transformation and will ultimately be passed to the pattern extraction stage for grouping based on log type adaptation.

[0043] 1. The LLM classification steps are as follows:

[0044] (1) Log Reading and Segmentation: Log files are read from a specified path using a Python program. Independent log blocks are extracted using a segmentation algorithm based on timestamps (e.g., [HH:MM:SS]) or blank lines. (2) Prompt Design: Structured JSON prompts are designed with clear classification criteria to dynamically adapt to heterogeneous log formats. The prompts consist of four parts: task description, classification criteria, output format, and requirements. The LLM returns a JSON format result. The prompts are dynamically inserted into the log content using placeholders {log_content} to adapt to different log formats (e.g., single-line events, multi-line logs with indentation, or table logs with headers). The structured design ensures that the LLM quickly identifies the log type according to the classification criteria and outputs a standardized JSON result. Examples of prompts are shown below. Figure 5 As shown.

[0045] 3) Model Loading and Configuration: Load the local LLM model using Hugging Face's transformers library, configure 16-bit floating-point precision (torch_dtype=torch.float16), and enable device_map="auto" to optimize hardware performance (e.g., using a GPU). Load the tokenizer via AutoTokenizer to prepare for processing prompt words and log content.

[0046] (4) LLM Classification: Each log block is combined with a prompt word, converted into input tokens by a tokenizer, and truncated to 1024 tokens. The call_llm function is used, with the generation parameters set to low temperature (0.01), top-p sampling (0.5), and a maximum of 100 new tokens to ensure a stable JSON output, including log_type (single_line, multi_line, or table) and reason (classification rationale). LLM utilizes semantic understanding capabilities to determine the log type based on the criteria in the prompt words.

[0047] (5) Rule-based classification fallback: If the LLM output is invalid (e.g., non-JSON format, parsing error, or inference anomaly), the `rule_based_classification` function is called to classify logs based on log characteristics: single-line logs are single-line content, table logs match the table header pattern (e.g., PID USER PR NI) or separator lines (e.g., +--+), and multi-line logs contain indentation, stack traces (e.g., Traceback), or separators (e.g., ----SEP----). By default, multi-line logs do not have a table header. Rule classification ensures the robustness of the process.

[0048] (6) Result saving: Save the classification results as single-line log, multi-line log, and table log files, retaining the original format, and generate classification statistics and debugging logs.

[0049] The core of the above steps lies in designing prompt words for heterogeneous log formats. By flexibly adjusting the classification criteria portion of the prompt words (e.g., modifying the regular expression for timestamps, adding feature descriptions for new log types, or adding typical examples), log formats from different systems can be quickly adapted. This design fully leverages the powerful semantic understanding capabilities of LLM, enabling it to efficiently process diverse log data.

[0050] 2. LLM Preprocessing

[0051] In modern information systems, log files typically consist of a log header (such as a timestamp) and core content, and log templates need to be extracted from the content. Therefore, this study first uses regular expressions to strip the log header while retaining the core content. The input to this step is the log blocks classified by the LLM classification step, and the output is preprocessed log messages (variables replaced with semantic wildcards). These preprocessed results will serve as direct input to the pattern extraction module for generating log codes and groupings.

[0052] Based on domain knowledge analysis, it was found that while traditional methods widely use wildcards <*> to generalize and replace variables to support subsequent analysis, regular expressions have limitations in recognizing special variables such as non-standard timestamps and custom paths. To overcome this problem, this paper proposes a token partitioning method. By quantitatively analyzing log keywords, log parameters are refined into eight categories of key-value tokens with predefined semantics (<*time>, <*path>, <*int>, <*http>, <*url>, <*dr>, <*ip>, <*bl>). Structured prompt words are also designed to guide LLM in achieving accurate identification of variable types.

[0053] like Figure 6 As shown, the log entry [18:01:02] 25 / 02 / 20 18:01:02 INFOmapreduce.Job:map100 reduce 33, after transformation, becomes: <*time> <*time> <*time> INFOmapreduce.Job: map <*int> reduce <*int>. Compared to traditional preprocessing methods, LLM's key-value transformation mechanism significantly simplifies the process and improves the accuracy and generalization ability of identifying special variables.

[0054] 3. Pattern Extraction

[0055] To overcome the efficiency bottleneck faced by traditional log parsers that rely on clustering complete log messages, especially the redundant computation caused by the flattening of hybrid logs and the interference of differences in the number of log lines on clustering accuracy, this paper proposes a grouping mechanism based on log type identification. This method deeply integrates the classification results of LLM (Log Modeling), achieving efficient and accurate grouping by assigning a structurally representative "Log ID" to each log entry. The input to this step is the preprocessed log message (containing log type and semantic wildcards) output by the LLM preprocessing module, and the output is the grouped log clusters (i.e., log groups based on tuple consistency). These log clusters serve as input to the template filtering module for similarity calculation and template merging.

[0056] LogID generation rules strictly adapt to log type characteristics: single-line logs use a complete token sequence as LogID, while table / text logs form a common sequence by connecting the first and last tokens (effectively compressing redundancy in multi-line structures).

[0057] The grouping process employs a two-layer tuple architecture to improve parsing accuracy. First, for single-line logs, a tuple (T_t, T_k) is constructed based on the number of tokens (T_t, where T_t represents the total number of tokens in the log message) and the number of keys (T_k, where T_k represents the total number of key-value pairs in the log message) calculated using LogID. For table / text logs, a tuple (L_t, T_k) is constructed based on the number of log rows (L_t, where L_t represents the total number of rows in the log block) and the number of keys (T_k, where T_k represents the total number of key-value pairs in the log message). The initial log grouping is completed based on tuple consistency. Second, within each tuple group, a key tuple (Token Tuple) is constructed by extracting key types, and a secondary classification is performed based on key tuple consistency. (The Drain algorithm shows that the key distribution of logs with the same template is highly consistent. Algorithm source: HE P, ZHU J, ZHENG Z, et al. Drain: an online log parsing approach with fixed depth tree[C] / / 2017 IEEE International Conference on WebServices (ICWS). Honolulu: IEEE, 2017: 33-40.

[0058] The technology is implemented dynamically using a multi-way search tree. During initialization, the tree structure contains only the root node. When a new log is introduced, the system creates a corresponding branch node based on its tuple value. Subsequent logs are initially classified through tuple matching, and then reclassified through key-tuple matching. Finally, a unified template group is accurately located, significantly improving the accuracy and efficiency of mixed log parsing.

[0059] 4. Template filtering based on document similarity

[0060] To determine whether two log templates in a secondary group should be merged, the templates need to be evaluated to ensure accurate identification of similar log entries. The input to this step is the grouped log clusters output by the pattern extraction module, and the output is the final structured log template library. These template libraries are the final result of the LTP method and are used for downstream analysis tasks.

[0061] After data collection, the log content is first read, and each line of log is saved as an independent entry in a list for easy organization, processing, and storage later. Next, initial labels are assigned to each log entry, laying the foundation for subsequent similarity analysis. These initial labels not only help identify log entries but also provide a convenient reference for similarity judgment, enabling them to be quickly located and compared during the analysis process.

[0062] To calculate the similarity between log entries, the TF-IDF (Term Frequency-Inverse Document Frequency) vectorization method is employed. TF-IDF is a commonly used text feature extraction technique that effectively converts text data into numerical features. Specifically, it assesses the importance of a token in a log entry by analyzing the term frequency (TF) of token w in the current log entry d and its inverse document frequency (IDF) across the entire log set. Using this method, each log entry is transformed into a high-dimensional vector, where each element represents the TF-ID of the corresponding token.

[0063]

[0064] in This indicates the token currently being calculated. This represents the log message document currently being processed, and the function... Indicates token In the document The number of times it appears in This represents any token appearing in document d (used as a traversal variable during summation). This represents the sum of the occurrences of all tokens in document d.

[0065]

[0066] Where N represents the total number of log messages in the entire log set. It contains tokens The number of documents.

[0067] Finally, the TF-IDF value of token w in document d is:

[0068]

[0069] After obtaining the vector representations, cosine similarity is used to measure the similarity between log entries. The cosine similarity value ranges from -1 to 1, where 1 represents perfect similarity, 0 represents no similarity, and -1 represents complete opposites. It assesses similarity by calculating the cosine of the angle between two vectors; therefore, even if the details of the log entries differ, as long as the overall structure and theme are similar, the cosine similarity will be high. Given two log vectors A and B, the cosine similarity... Calculated from the dot product and vector length:

[0070]

[0071] Where A and B represent vectors obtained after TF-IDF processing, A · B represents the dot product of vectors A and B, |A| represents the magnitude of vector A, and |B| represents the magnitude of vector B.

[0072] After inputting the TF-IDF vectors into the cosine similarity calculation function, a symmetric similarity matrix is ​​obtained, where each element represents the similarity between corresponding log entry pairs. A similarity threshold is set to determine which entries are considered similar. The choice of this threshold is based on previous experimental results and aims to minimize false matches. A reasonable threshold is crucial for improving accuracy: too low a threshold may lead to the incorrect merging of irrelevant entries; too high a threshold may miss truly similar entries.

[0073] The similarity matrix is ​​iterated through, and log entries are compared pairwise. If the similarity between two entries is higher than a set threshold, the label of one entry is updated to the label of the other. This process groups similar log entries into the same category, thus simplifying subsequent analysis and processing.

[0074] Finally, the updated tags are written to a new log file along with the original log content to ensure the results are persistent and easy to analyze and reference later. Maintaining the integrity and accuracy of the original logs is crucial throughout this process, providing a reliable foundation for subsequent auditing and data mining. The generated new file also facilitates data sharing and reporting. These steps effectively manage and analyze log templates, improving the efficiency and accuracy of data processing.

[0075] Experiments verified the effectiveness of this scheme:

[0076] (1) Experimental equipment and dataset

[0077] All experiments were run on a Windows 11 system equipped with an Intel Core i9-13900HX processor, 32GB of RAM, and an NVIDIA RTX 4060 graphics card. Log parsing was implemented using Python 3.10, employing real open-source hybrid log datasets CTS and Hibench. To reduce the randomness of the experimental results, each experiment was repeated 10 times, and the average value was taken as the final result.

[0078] Table 1 Public Dataset Information

[0079]

[0080] (2) Baseline method

[0081] This paper selects six log parsing methods as benchmarks for comparative research, including: AEL, LenMa, Spell, IPLoM, Drain, and Hue.

[0082] AEL: A structured parsing-based technique that automatically extracts and formats key information from logs. This method introduces an automated mechanism to identify and define the potential semantics of parameter fields in logs, thereby improving parsing accuracy and efficiency. (JIANG ZM, HASSAN AE, FLORA P, et al. Abstracting execution logs to execution events for enterprise applications[C] / / The Eighth International Conference on Quality Software. Oxford: IEEE, 2008: 181-186.)

[0083] LenMa: A method based on minimum edit distance. It calculates the edit distance between log messages and known templates to achieve dynamic matching of similar templates, thereby automatically extracting structured log data. (SHIMA K. Lengthmatters: clustering system log messages using length of words[EB / OL]. [2016-11-10]. http: / / arxiv.org / abs / 1611.03213.)

[0084] Spell: Based on the Longest Common Subsequence (LCS) principle, this method supports dynamic extraction of log patterns and maintains identified message types in a streaming manner. It can also automatically discover the semantics of parameter fields (e.g., spelling recognition). (DU M, LI FF. Spell: streaming parsing of system event logs[C] / / 2016 IEEE 16th International Conference on Data Mining (ICDM). Barcelona: IEEE, 2016: 859-864.)

[0085] IPLoM: An automated log parsing solution. It extracts key log data through initial segmentation and pattern recognition, and continuously improves parsing accuracy and format adaptability through iterative optimization strategies. (MAKANJU AAO, ZINCIR-HEYWOOD AN, MILIOS E E. Clustering event logs using iterative partitioning[C] / / Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Paris: ACM, 2009: 1255-1264.)

[0086] Drain: This method possesses real-time processing capabilities and utilizes a fixed-depth parse tree to encode specific rules to accelerate the parsing process. It performs exceptionally well in anomaly detection and is also suitable for log analysis tasks. (DU M, LI FF. Spell: streaming parsing of system event logs[C] / / 2016 IEEE 16th International Conference on Data Mining (ICDM). Barcelona: IEEE, 2016: 859-864.)

[0087] Hue innovatively introduces a user-adaptive hybrid parsing mechanism, identifying log types through line aggregation and pattern extraction, and employing a novel merge rejection strategy to effectively utilize user feedback. (XU JJL, FU QA, ZHU Z RX, et al. Hue: a user-adaptive parser for hybrid logs[C] / / Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. San Francisco: ACM, 2023: 413-424.)

[0088] (3) Evaluation indicators

[0089] The core of this log parsing evaluation method is based on the concept of cluster analysis, viewing log parsing essentially as a process of pattern recognition and classification aggregation of raw log entries. To accurately measure the degree of matching between the parsing results and the actual situation, this method introduces parsing accuracy, a commonly used metric in log parsing, and F1 score of template accuracy, a widely adopted evaluation metric in clustering algorithms. .

[0090] Parsing accuracy is defined as the ratio of the number of correctly parsed log messages to the total number of log messages. A log message is considered "correctly parsed" if and only if all its constants and variables are accurately distinguished within the log template. Since parsing accuracy is independent of the number of templates and only relates to the total number of parsed log messages, we use parsing accuracy as a message-level evaluation metric to measure the parser's most basic parsing capabilities.

[0091] The F1 score of Template Accuracy is a template-level accuracy, calculated as the harmonic mean of precision and recall. Precision is defined as the proportion of message pairs that the parser determines belong to the same log event group, but which actually correspond to the same log event. Recall is defined as the proportion of all message pairs that actually correspond to the same log event, but which are correctly assigned to the same group by the parser. A template is considered correct if and only if it meets two conditions: the predicted log message belongs to the same group as the log messages in the reference standard, and all tags in the template are identical to those in the reference template.

[0092] (4) Performance evaluation

[0093] To comprehensively evaluate the effectiveness of the newly proposed LTP method, this section compares it with seven other methods—five representative log parsing methods provided by the LogPa benchmark platform, as well as the Hue and LTP methods, which have performed exceptionally well in recent years—on key performance indicators (accuracy and F1 score). The experimental results are presented in Tables 2 and 3, providing a solid data foundation for subsequent comparative analysis.

[0094] Table 2 Comparison of analytical accuracy between the two benchmark datasets

[0095]

[0096] Table 3. Comparison of F1 scores for the two benchmark datasets.

[0097]

[0098] Analysis of Tables 2 and 3 shows that LTP performs exceptionally well on mixed datasets (Accuracy: 0.926 / 0.916; F1-Score: 0.844 / 0.937), significantly outperforming other methods. Drain, the best-performing traditional parser (average Accuracy: 0.594, F1-Score: 0.769), performs far worse than LTP, highlighting the limitations of traditional methods due to the lack of analysis and classification mechanisms for mixed logs. While the recent excellent method Hue achieves a high Accuracy (0.932) on one dataset, its F1-Score (0.848) is lower than LTP's 0.937 on the same dataset. Furthermore, LTP's overall average performance (Accuracy: 0.921; F1-Score: 0.891) is significantly better than Hue and most of the comparison methods.

[0099] Table 4. Performance of LTP on each module of the HiBench dataset.

[0100]

[0101] Table 5. Performance of LTP in each module of the CTS dataset

[0102]

[0103] The performance of LTP across different types of log modules is shown in Tables 4 and 5. On the HiBench dataset, the event / table / text log metrics are 0.868 / 1.000 / 0.880, respectively, and on the CTS dataset, they are 0.931 / 1.000 / 0.906. Notably, table log parsing achieved a perfect score (1.000), thanks to the high compatibility between table logs (whose headers are typically single strings separated by spaces) and the LTP parsing mechanism. In conclusion, LTP is an efficient, accurate parsing method particularly suitable for mixed log scenarios. Its excellent mixed parsing capabilities and significant overall performance advantages provide users with a high-precision solution.

[0104] Obviously, the specific implementation of this invention is not limited to the above-described methods. Any non-substantial improvements made using the inventive concept and technical solution of this invention are within the protection scope of this invention.

Claims

1. A hybrid log parsing method based on large models and structure trees, characterized in that, Includes the following steps: By using an LLM model combined with structured JSON hints, hybrid logs are intelligently categorized and key-value transformations are performed. A dedicated structure tree classifier is designed to parse different log types based on the classification results.

2. The hybrid log parsing method based on large model and structure tree as described in claim 1, characterized in that, Mixed logs are classified into three categories: single-line logs, table logs, and multi-line logs.

3. The hybrid log parsing method based on large model and structure tree as described in claim 1 or 2, characterized in that, The method also includes: dynamically adapting prompt words to heterogeneous formats, outputting log types based on semantic features using LLM, and extracting timestamps using regular expressions to refine parameters into semantic wildcards, laying the foundation for structured parsing.

4. The hybrid log parsing method based on large model and structure tree as described in claim 1, characterized in that, In the method, a differentiated grouping strategy is implemented for different types. For single-line logs, a feature index is constructed based on the number of tokens and the number of keys, while for table and text logs, a structure tree classifier is constructed based on the number of log lines and the distribution of key types. TF-IDF vectorization and cosine similarity thresholding are used to perform semantic filtering and dynamic fusion of candidate templates, thereby realizing a high-precision generation and incremental update mechanism for the template library.

5. The hybrid log parsing method based on large model and structure tree as described in any one of claims 1-4, characterized in that, Intelligent classification of mixed logs includes: 1) Log reading and splitting: Read log files from a specified path and extract independent log blocks using a splitting algorithm based on timestamps or empty lines; 2) Prompt word design: Design structured JSON prompt words, clarify classification criteria, and dynamically adapt to heterogeneous log formats; the prompt words are divided into four parts: task description, classification criteria, output format, and requirements, and LLM returns JSON format results; the prompt words are dynamically inserted into the log content through placeholders to adapt to different log formats; 3) LLM Classification: Each log block is combined with a prompt word, converted into an input token by a tokenizer, and truncated to 1024 tokens. LLM utilizes semantic understanding to determine the log type based on the criteria in the prompt words. If the LLM output is invalid, classification is performed based on log features. 6) Result saving: Save the classification results as single-line log, multi-line log and table log files, retain the original format, and generate classification statistics and debugging logs.

6. The hybrid log parsing method based on large model and structure tree as described in claim 5, characterized in that, In the method, log parameters are refined into multiple types of key-value tokens with preset semantics by quantitative statistics of log keywords, and structured prompt words are designed to guide LLM to achieve accurate identification of variable types.

7. The hybrid log parsing method based on large model and structure tree as described in claim 5, characterized in that, Intelligent classification of mixed logs also includes: when the LLM classification output is invalid, classification is based on log features, where single-line logs are single-line content, table logs match table header patterns or separators, and multi-line logs contain indentation, stack traces, or separators.

8. The hybrid log parsing method based on large model and structure tree as described in any one of claims 1-7, characterized in that, The date data after intelligent classification is preprocessed using LLM. By quantifying and statistically analyzing log keywords, log parameters are refined into eight categories of key-value tokens with preset semantics. Structured prompt words are designed to guide LLM in achieving accurate identification of variable types.

9. The hybrid log parsing method based on large model and structure tree as described in claim 8, characterized in that, The preprocessed log messages are assigned a LogID with structural representation to each log message. The preprocessed log messages contain log type and semantic wildcards. The encoded messages are used to obtain grouped log clusters, which are used as input for template filtering, similarity calculation and template merging. LogID generation rules are strictly adapted to log type characteristics: single-line logs use a complete token sequence as LogID, while table / text logs form a common sequence by connecting the first and last tokens; The grouping process adopts a two-level tuple architecture: First, for single-line logs, the number of tokens and keys are calculated based on LogID to construct a tuple, and for table / text logs, the number of log lines and keys are counted to construct a tuple. The first log grouping is completed based on tuple consistency. Second, within each tuple group, key tuples are constructed by extracting key types, and secondary classification is performed based on key tuple consistency.

10. The hybrid log parsing method based on large model and structure tree as described in claim 9, characterized in that, A template filtering strategy based on document similarity is adopted to merge and judge two log templates within the secondary group. The TF-IDF vectorization method is used to convert text data into numerical features. Based on the numerical features, cosine similarity is used to measure the similarity between log entries. The similarity is evaluated by calculating the cosine value of the angle between two vectors. A similarity threshold is set. If the similarity between two entries is higher than the set threshold, the label of one entry is updated to the label of the other entry. Otherwise, retain the tags of both entries; write the updated tags and the original log content together into a new log file to obtain the parsing result.

Images