Multi-source log classification method based on BiGRU-LSTM and TFWAF
By combining BiGRU-LSTM with the TFWAF module, the problem of insufficient utilization of contextual information in multi-source log classification is solved, and high-accuracy and robust automatic log classification is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-17
AI Technical Summary
Existing technologies struggle to effectively parse multi-source heterogeneous log formats and fail to fully utilize the deep semantics and contextual information of log sequences during training, resulting in blurred classification boundaries and affecting the model's accuracy in distinguishing similar categories.
By combining BiGRU-LSTM with TFWAF modules, the model captures contextual information of log sequences through bidirectional gated recurrent units and long short-term memory networks, and utilizes an adaptive hard negative sampling strategy to enhance the model's ability to distinguish similar log sources, thus achieving high-accuracy automatic classification of multi-source logs.
It possesses powerful context modeling capabilities and deep semantic information fusion, which improves the accuracy and robustness of the model in fine-grained classification and can effectively distinguish similar log sources.
Smart Images
Figure CN121880284A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of network security and data processing technology, specifically to a log analysis technology, and in particular to an automated classification method for multi-source heterogeneous log data. Background Technology
[0002] With the rapid development of information technology, networks and information systems have become deeply integrated into critical sectors such as finance, healthcare, and transportation. However, the increased complexity of these systems also makes them more vulnerable to internal failures and external intrusions, potentially leading to severe service interruptions and data breaches. Logs, as the core carrier for recording system operating status and events, are crucial for effective anomaly detection and security protection.
[0003] Modern networks are typically composed of heterogeneous devices and subsystems from different vendors, running different services. The logs they generate (i.e., multi-source logs) vary significantly in format, semantics, and structure. Traditional log analysis methods are usually designed for specific, known log formats and are ill-suited to this dynamic, heterogeneous, multi-source environment. Especially in scenarios involving system aging, lack of maintenance, or loss of original source information after log aggregation, accurate source classification of mixed log streams becomes both a prerequisite and a challenge for subsequent in-depth analysis (such as anomaly detection and root cause analysis).
[0004] In existing technologies, rule-based or template-based log classification methods lack flexibility and struggle to adapt to emerging log formats. Traditional machine learning methods (such as SVM and decision trees) rely on complex manual feature engineering and have limited ability to model the contextual information of sequence data. In recent years, recurrent neural networks (RNNs) and their variants, such as Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs), have demonstrated advantages in processing sequence data; however, unidirectional networks struggle to fully utilize the bidirectional contextual information of sequences, and standard LSTMs still have limitations in capturing complex global dependencies in long sequences. Furthermore, existing methods typically employ random negative sampling during training, failing to effectively distinguish semantically similar log sources, resulting in blurred classification boundaries and affecting the model's accuracy in distinguishing similar categories.
[0005] Therefore, there is an urgent need for a classification method that can effectively parse multi-source log formats, fully mine the deep semantics and contextual information of log sequences, and improve the ability to distinguish similar source categories. Summary of the Invention
[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a multi-source log classification method that combines a bidirectional gated recurrent unit-long short-term memory network and a token-feature weaving attention fusion module. This method can automatically learn the semantic features and contextual relationships of logs, and utilizes an adaptive hard negative sampling strategy to enhance the model's ability to distinguish similar log sources, thereby achieving high-accuracy automatic classification of multi-source logs.
[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A multi-source log classification method based on BiGRU-LSTM and TFWAF includes the following steps: S1, Data Preprocessing: Preprocess the training set logs, including deleting log headers, retaining the message portion, using regular expressions to replace specific format information in the message with predefined placeholders, and cleaning and segmenting the log content. S2, Model building and training: The preprocessed logs are input into the BiGRU-LSTM model combined with the TFWAF module for training, and the trained model is obtained. S3, Classification Reasoning: Input the test set log data to be classified into the trained model and output the classification result.
[0008] Furthermore, step S2 specifically includes: S2.1, each preprocessed log sentence is represented as a word sequence, each word is converted into a word vector through an embedding matrix, and the positional encoding function is used to add its position information in the sequence to the word vector; S2.2, the word vector with positional information is input into the BiGRU bidirectional gated recurrent unit layer to obtain the hidden state representations of forward propagation and backward propagation respectively, and the two are concatenated at the corresponding positions to form a hidden representation that integrates bidirectional context; S2.3, the concatenated hidden representation is input into the Long Short-Term Memory (LSTM) network layer, which further captures the long-range dependencies in the sequence and outputs the hidden state sequence of the LSTM layer. S2.4, The hidden state sequence of the LSTM layer is input into the TFWAF module. This module captures the semantic association of sequence elements and the correlation between feature dimensions through parallel token-view self-attention and feature-view self-attention, respectively. Then, the information from the two perspectives is deeply fused through the weaving fusion mechanism to obtain a text-level fused feature representation. S2.5, perform nonlinear transformation and dimensionality reduction on the fused feature representation: First, the fused feature representation is mapped from low dimension to high dimension space through a fully connected layer, and the nonlinear expressive power is enhanced by the rectified linear unit activation function; then, dimensionality reduction is performed through pooling operation, and residual connections are introduced to alleviate the gradient vanishing problem, resulting in the dimensionality-reduced fused feature representation; S2.6 inputs the dimensionality-reduced fused feature representation into the classifier, calculates the loss function using an adaptive hard negative sampling strategy, and optimizes the model parameters.
[0009] Furthermore, the processing procedure of the TFWAF module is as follows: Let Z∈ Let L be the sequence of hidden states output by the LSTM layer, where L is the sequence length and d is the feature dimension. S2.4.1, will The token-view self-attention module (TVSA) processes the data, and the output is the processing result. ; Among them, Z tvsa express The result after TVSA processing; TVSA represents token-view self-attention processing; Att represents self-attention computation, and F represents feature output; , , For a given shape The learnable weight matrix , , , ; S2.4.2, then After processing by the Feature View Self-Attention (FVSA) module, the output result is shown. ; Among them, Z fvsa express Result after FVSA processing; FVSA represents feature-view self-attention processing; , , For a given shape The learnable weight matrix , , , ,in for The transpose of , where softmax is the activation function.
[0010] S2.4.3, will and Perform WAF processing using the Weave Attention Fusion Module and output the processing results. ; in, express and The symbol is processed by WAF. express LinearA, LinearB, and LinearC represent three different linear layers, and Flatten refers to the flatness function.
[0011] Furthermore, in S2.6, the specific steps for inputting the dimensionality-reduced fused feature representation into the classifier, calculating the loss function using an adaptive hard negative sampling strategy, and optimizing the model parameters are as follows: The dimensionality-reduced fused feature representation is input into the classifier. The adaptive hard negative sampling strategy selects "hard negative sample" labels with high prediction confidence but not true class based on the logits value predicted by the model. An additional penalty is imposed on them in the cross-entropy loss function, forcing the model to better distinguish easily confused classes. The hard negative labels are obtained based on the logits, and then combined with the softmax function to obtain the classification result.
[0012] Hard negative labels are sampled based on logits; hard negative labels with high logits are those that are not correct. The sampling formula is as follows: ; in, This indicates the hard negative label sampling result. Indicates category index, The minimum logits value is given.
[0013] In the adaptive hard negative sampling strategy, the loss function is: , The standard cross-entropy loss is calculated using the following formula: ; in, For the true label of the i-th log entry, The label inferred from the model for the i-th log, where n is the number of logs and m is the number of categories; ; Where p(q) i | X i ) for log X i Under the condition of log X i Belongs to category q i The probability of; and Let these represent the weight and bias of the i-th log file, respectively. The resulting feature representation; This is a hard negative penalty term, and the specific formula is as follows: ; in, It is a hyperparameter used to control the intensity of the penalty. It is the predicted probability of a hard negative label.
[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the steps of the above-mentioned multi-source log classification method based on BiGRU-LSTM and TFWAF.
[0015] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aforementioned multi-source log classification method based on BiGRU-LSTM and TFWAF. Compared to the prior art, the advantages of this invention are: 1. This invention has powerful context modeling capabilities: through the cascaded structure of BiGRU and LSTM, BiGRU is first used to effectively capture bidirectional short-term context information of log sequences, and then LSTM is used to further model long-range dependencies, forming a hierarchical and complementary feature extraction of log sequences.
[0016] 2. This invention features deep semantic information fusion: The innovative TFWAF module, through a dual-path self-attention mechanism using both token and feature perspectives, can simultaneously model the sequential semantic associations between log terms and the global implicit associations between feature dimensions, thereby uncovering deeper and more comprehensive semantic representations.
[0017] 3. The adaptive hard negative sampling strategy introduced in this invention can dynamically identify and enhance the discriminative learning of easily confused (similar) log source categories during the training process, effectively improving the accuracy and robustness of the model in fine-grained classification. Attached Figure Description
[0018] Figure 1 This is an overall flowchart of the multi-source log classification method provided in the embodiments of the present invention; Figure 2 This is a schematic diagram of the model structure of BiGRU-LSTM combined with TFWAF module in an embodiment of the present invention. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] like Figure 1-2 The present invention provides a multi-source log classification method based on BiGRU-LSTM and TFWAF, the overall process of which is as follows: S1, Data Preprocessing: Preprocess the training set logs, including deleting log headers, retaining the message portion, using regular expressions to replace specific format information in the message with predefined placeholders, and cleaning and segmenting the log content. Specifically, raw log data from different network devices or system components is collected as the training set logs. For each log entry, firstly, the log header (such as fixed fields like timestamps and hostnames) is removed, retaining only the variable message body. Then, a predefined set of regular expression rules is used to scan the message body to identify dynamic variables: for example, IPv4 / IPv6 addresses are replaced with placeholders [IP], URLs or file paths are replaced with [FILE], and consecutive numeric strings are replaced with [NUM]. This step aims to normalize the log format, directing the model's focus to semantic content rather than volatile parameter values. Finally, the processed text is cleaned and segmented, removing irrelevant symbols.
[0021] S2, Model building and training: The preprocessed logs are input into the BiGRU-LSTM model combined with the TFWAF module for training, and the trained model is obtained. Specifically, step S2 includes the following steps: S2.1, each preprocessed log sentence is represented as a word sequence, each word is converted into a word vector through an embedding matrix, and the positional encoding function is used to add its position information in the sequence to the word vector; Specifically, the set of preprocessed logs is denoted as... Each of them Assign a domain tag Represents its source domain, each The set of words is denoted as ;Through embedding matrix Each word in the log Convert to vector Where d is the dimension of the vector representation, and L is the maximum sequence length of the log. , where i represents the i-th word in the log sentence; Then, position information is added using a sine-cosine position encoding function. The information is sent to a positional encoding network to obtain the position information of the word in the input sequence. Assuming... express The result after position encoding.
[0022] The formula for the sine-cosine position coding function is as follows: ; Where pos represents the position of the word in the sequence, and PE represents the position code value. Set the dimension of the network, where i is the vector dimension.
[0023] S2.2, the word vector with positional information is input into the BiGRU bidirectional gated recurrent unit layer to obtain the hidden state representations of forward propagation and backward propagation respectively, and the two are concatenated at the corresponding positions to form a hidden representation that integrates bidirectional context; Specifically, will Inputting a forward GRU, we obtain the forward hidden representation. Meanwhile, the backward GRU imports vectors. Generate backward hidden representation By concatenating the forward and backward hidden representations, we obtain .
[0024] S2.3, the concatenated hidden representation is input into the Long Short-Term Memory (LSTM) network layer, which further captures the long-range dependencies in the sequence and outputs the hidden state sequence of the LSTM layer. Specifically, the concatenated GRU hidden representation is used as input and fed into the LSTM network; the LSTM further processes the concatenated GRU hidden representation and generates a new hidden representation. : .
[0025] S2.4, The hidden state sequence of the LSTM layer is input into the TFWAF module. This module captures the semantic association of sequence elements and the correlation between feature dimensions through parallel token-view self-attention and feature-view self-attention, respectively. Then, the information from the two perspectives is deeply fused through the weaving fusion mechanism to obtain a text-level fused feature representation. The Token-Feature Woven Attention Fusion (TFWAF) module consists of three main parts: a Token-View Self-Attention (TVSA) module, a Feature-View Self-Attention (FVSA) module, and a Woven Attention Fusion (WAF) module. The TVSA module focuses on semantic feature extraction from the perspective of the token, while the FVSA module, as a dual-perspective module, can model the semantic correlations between features. These two self-attention modules, operating from dual perspectives, complement each other and help capture deep semantic representation information.
[0026] The specific processing procedure of the TFWAF module is as follows: Let Z∈ Let L be the sequence of hidden states output by the LSTM layer, where L is the sequence length and d is the feature dimension. S2.4.1, will The token-view self-attention module (TVSA) processes the data, and the output is the processing result. ; Among them, Z tvsa express The result after TVSA processing; TVSA represents token-view self-attention processing; Att represents self-attention computation, and F represents feature output; , , For a given shape The learnable weight matrix , , , Softmax is the activation function.
[0027] S2.4.2, then After processing by the Feature View Self-Attention (FVSA) module, the output result is shown. ; Among them, Z fvsa express Result after FVSA processing; FVSA represents feature-view self-attention processing; , , For a given shape The learnable weight matrix , , , ,in for The transpose of .
[0028] S2.4.3, will and Perform WAF processing using the Weave Attention Fusion Module and output the processing results. ; in, express and The symbol is processed by WAF. express LinearA, LinearB, and LinearC represent three different linear layers, and Flatten refers to the flatness function.
[0029] S2.5, perform nonlinear transformation and dimensionality reduction on the fused feature representation: First, the fused feature representation is mapped from low dimension to high dimension space through a fully connected layer, and the nonlinear expressive power is enhanced by the rectified linear unit activation function; then, dimensionality reduction is performed through pooling operation, and residual connections are introduced to alleviate the gradient vanishing problem, resulting in the dimensionality-reduced fused feature representation; Specifically, the fused feature representation is mapped from low dimension to high dimension by applying a fully connected layer, and then mapped back to low dimension. The rectified linear unit (ReLU) activation function is used to strengthen the large part of the word representation while suppressing the small part to enhance the expressiveness of the word representation. Then, dimensionality reduction is performed by pooling, and residual connections are introduced to alleviate the gradient vanishing problem, resulting in the dimensionality-reduced fused feature representation.
[0030] S2.6 inputs the dimensionality-reduced fused feature representation into the classifier, calculates the loss function using an adaptive hard negative sampling strategy, and optimizes the model parameters.
[0031] Specifically, the steps for inputting the dimensionality-reduced fused feature representation into the classifier, calculating the loss function using an adaptive hard negative sampling strategy, and optimizing the model parameters are as follows: The dimensionality-reduced fused feature representation is input into the classifier. The adaptive hard negative sampling strategy selects "hard negative sample" labels with high prediction confidence but not true categories based on the logits value predicted by the model, and takes the category with the highest probability after softmax as the predicted source category of the log.
[0032] Specifically, hard negative labels are sampled based on logits, and labels with high logits but not necessarily correct labels are hard negatives; the sampling formula is: ; in, This indicates the hard negative label sampling result. Indicates category index, The minimum logits value is given.
[0033] In the adaptive hard negative sampling strategy, the loss function is: , The standard cross-entropy loss is calculated using the following formula: ; in, For the true label of the i-th log entry, The label inferred from the model for the i-th log, where n is the number of logs and m is the number of categories;
[0034] Where p(q) i | X i ) for log X i Under the condition of log X i Belongs to category q i The probability of; and Let these represent the weight and bias of the i-th log file, respectively. The resulting feature representation; This is a hard negative penalty term, and the specific formula is as follows: ; in, It is a hyperparameter used to control the intensity of the penalty. It is the predicted probability of a hard negative label.
[0035] S3. Classification reasoning: Input the test set log data to be classified into the trained model and output the classification result.
[0036] Specifically, the test set log data to be classified is input into the optimized and trained model, and the classification result is output.
[0037] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0038] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims
1. A multi-source log classification method based on BiGRU-LSTM and TFWAF, characterized in that, Includes the following steps: S1, Data Preprocessing: Preprocess the training set logs, including deleting log headers, retaining the message portion, using regular expressions to replace specific format information in the message with predefined placeholders, and cleaning and segmenting the log content. S2, Model building and training: The preprocessed logs are input into the BiGRU-LSTM model combined with the TFWAF module for training, and the trained model is obtained. S3, Classification Reasoning: Input the test set log data to be classified into the trained model and output the classification result.
2. The multi-source log classification method based on BiGRU-LSTM and TFWAF according to claim 1, characterized in that, Step S2 includes: S2.1, each preprocessed log sentence is represented as a word sequence, each word is converted into a word vector through an embedding matrix, and the positional encoding function is used to add the positional information of the word vector in the sequence; S2.2, input the word vector with position information into the BiGRU layer, obtain the hidden state representations of forward propagation and backward propagation respectively, and concatenate the two at the corresponding positions; S2.3, Input the concatenated hidden representation into the LSTM layer to obtain the hidden state sequence of the LSTM layer; S2.4, Input the hidden state sequence of the LSTM layer into the TFWAF module to obtain the text-level fused feature representation; S2.5, the fused feature representation is sequentially subjected to fully connected mapping, activation function processing and dimensionality reduction operation to obtain the dimensionality-reduced fused feature representation; S2.6 inputs the dimensionality-reduced fused feature representation into the classifier, calculates the loss function using an adaptive hard negative sampling strategy, and optimizes the model parameters.
3. The multi-source log classification method based on BiGRU-LSTM and TFWAF according to claim 2, characterized in that, The processing procedure of the TFWAF module includes: Specifically, the processing procedure of the TFWAF module is as follows: S2.4.1, the hidden state sequence output by the LSTM layer The token-view self-attention module (TVSA) processes the data, and the output is the processing result. ； Among them, Z tvsa express The result is after TVSA processing; TVSA represents token-view self-attention processing; Att represents self-attention computation, and F represents feature output; , , For a given shape The learnable weight matrix, , , , ; softmax is the activation function; S2.4.2, will After processing by the Feature View Self-Attention (FVSA) module, the output result is shown. ； Among them, Z fvsa express Result after FVSA processing; FVSA represents feature-view self-attention processing; , , For a given shape The learnable weight matrix, , , , ,in for Transpose of; Will and Perform WAF processing using the Weave Attention Fusion Module and output the processing results. ； in, express and The symbol is processed by WAF. express LinearA, LinearB, and LinearC represent three different linear layers, and Flatten refers to the flattening function.
4. The multi-source log classification method based on BiGRU-LSTM and TFWAF according to claim 2, characterized in that, The adaptive hard negative sampling strategy includes: Based on the model's predicted logits value for the current sample, the category with a high predicted probability but not the true label is selected as the hard negative label; when calculating the cross-entropy loss function, a penalty weight is added to the loss term of the hard negative label.
5. The multi-source log classification method based on BiGRU-LSTM and TFWAF according to claim 4, characterized in that, In the aforementioned adaptive hard negative sampling strategy, the loss function is calculated using the following formula: ; The standard cross-entropy loss is calculated using the following formula: ； in, For the true label of the i-th log entry, The label inferred from the model for the i-th log, where n is the number of logs and m is the number of categories; Where p(q) i | X i ) for log X i Under the condition of log X i Belongs to category q i The probability of; and Let these represent the weight and bias of the i-th log file, respectively. The resulting feature representation; This is a hard negative penalty term, and the specific formula is as follows: ； in, It is a hyperparameter. It is the predicted probability of a hard negative label.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the multi-source log classification method based on BiGRU-LSTM and TFWAF as described in any one of claims 1 to 5.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements steps such as those of a multi-source log classification method based on BiGRU-LSTM and TFWAF.