Log anomaly detection method based on pattern awareness and synthetic attention mechanism

Abstract

The invention relates to the technical field of artificial intelligence and computer system security, in particular to a log anomaly detection method based on pattern awareness and a synthetic attention mechanism, which comprises the following steps: generating initial event embedding through log analysis and vectorization; constructing a parallel mode perception path and a sequence context path, respectively utilizing a dynamic graph attention network to extract macroscopic service mode structure features, and capturing a microscopic time sequence dependency relationship based on a Transform encoder; designing a bidirectionally guided synthetic attention mechanism, realizing mode-to-sequence focusing guidance and sequence-to-mode anomaly backtracking, and generating a synthetic anomaly representation fused with a context abstract and a mode deviation; and finally, outputting an abnormal score and interpretable alarm information through an MLP classifier. According to the technical scheme, collaborative modeling of the macroscopic mode and the microscopic details in the log data can be achieved, and the accuracy, robustness and interpretability of anomaly detection in a complex scene are remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer system security technology, and in particular to a log anomaly detection method based on pattern perception and synthetic attention mechanism. Background Technology

[0002] With the rapid development of information technology, log anomaly detection is becoming increasingly important in ensuring system stability and security. Existing log anomaly detection methods mainly rely on traditional machine learning algorithms or deep learning models, identifying potential abnormal behavior by analyzing the time-series features, pattern features, and contextual information of log data. However, existing technologies still have shortcomings in terms of detection capabilities in complex scenarios, comprehensive representation of multi-dimensional features, and adaptability to dynamically changing environments, making it difficult to meet the high-precision, highly interpretable log anomaly detection requirements of modern systems. A search revealed a log anomaly detection system with publication number CN112363896B. This patent proposes a log anomaly detection method based on Prophet and bLSTM models. By training a time-series prediction model and a deep learning model, combined with the DTW (Dynamic Time Warping) algorithm to determine a similarity threshold, it achieves anomaly detection in log data. However, in this technical solution, the Prophet model and bLSTM model lack an effective feature fusion mechanism when processing time-series and contextual features respectively, resulting in limited comprehensive representation capabilities of multi-dimensional features. Furthermore, this method has weak interpretability for abnormal patterns, especially in complex scenarios where it is difficult to provide intuitive analysis of the causes of abnormalities, which affects the practical application value of the system.

[0003] On the other hand, a log anomaly detection method disclosed in patent CN110750412B is described. This patent constructs a log pattern library and a dependency relationship library, and uses pattern vectors and thresholds to detect anomalies in real-time log data, exhibiting high interpretability. However, in this technical solution, the construction of the log pattern library relies on static analysis of historical data, lacking adaptability to dynamically changing environments. Furthermore, when processing large-scale log data, the computational complexity of pattern matching is high, which may lead to a decrease in detection efficiency and make it difficult to meet the needs of applications with high real-time requirements.

[0004] The aforementioned problems indicate that existing log anomaly detection methods still have certain shortcomings in areas such as multi-dimensional feature fusion, adaptability to dynamic environments, and the balance between real-time performance and interpretability. Therefore, this invention provides a log anomaly detection method based on pattern awareness and synthetic attention mechanisms. By introducing a pattern awareness module and a synthetic attention mechanism, it aims to enhance the comprehensive representation capability of multi-dimensional features, improve adaptability to dynamically changing environments, and simultaneously balance detection accuracy and interpretability, thereby meeting the needs of modern systems for efficient and intelligent log anomaly detection. Summary of the Invention

[0005] The purpose of this invention is to overcome the technical contradictions faced by existing log anomaly detection methods in processing high-level semantic patterns and low-level sequence features. Specifically, these contradictions include the loss of dynamic contextual information due to high-level semantic abstraction and the dilution of key semantic information and lack of high-level pattern awareness due to low-level sequence modeling. To achieve this objective, this invention provides a log anomaly detection method based on pattern awareness and synthetic attention mechanisms. This method constructs parallel dual representation pathways of pattern awareness and sequence context, and designs a bidirectional guided synthetic attention mechanism. This enables collaborative modeling and deep fusion of macroscopic business patterns and microscopic temporal details in log data, thereby significantly improving the detection accuracy and robustness for complex, hidden, and unknown anomaly patterns.

[0006] To achieve the above-mentioned objectives, this invention provides a log anomaly detection method based on pattern awareness and synthetic attention mechanisms, which includes the following steps:

[0007] Step 1: Log Data Preprocessing and Vectorization. This step aims to transform the unstructured raw log stream into a structured numerical representation that the model can process. First, a log parsing module is defined, employing an online log parsing algorithm based on a fixed-depth tree structure, specifically an optimized "Drain" algorithm. This module receives the raw log message string as real-time input and parses it into two separate components: an invariant log event template and a dynamically changing parameter list. The log event template represents the core semantic category of the log entry, while the parameter list contains the specific instantiation variables for that event. Next, a log event vectorization unit is constructed. For each parsed log event, the vectorization unit performs the following operations to generate an initial event embedding vector: First, for the log event template, a global template embedding matrix is ​​maintained, with dimensions V×D. t Where V is the total number of unique log templates in the training corpus, and D... t The template embedding dimension is preset, for example, 256. Each template looks up its corresponding D in this matrix using its unique index. t First, for the parameter list, each parameter is type-identified and encoded. If the parameter is numerical, it is converted into a scalar using Z-score normalization. If the parameter is categorical, it is mapped to a D-value through a separate categorical parameter embedding layer. c Third, the obtained template embedding vector, the normalized scalars of all numerical parameters, and the embedding vectors of all categorical parameters are concatenated along the feature dimension to form an initial event embedding vector of fixed dimension.

[0008] Step two, parallel feature extraction, includes pattern structure awareness and sequence context modeling. This step extracts features from the log sequence at the macro and micro levels through two parallel and structurally heterogeneous neural network pathways, respectively.

[0009] First, it performs macroscopic pattern structure perception. This invention defines a pattern perception pathway, the core of which is a dynamic graph construction and graph attention network module, used to capture the structural features of business logic patterns or operational processes composed of multiple log events within a time window. Specifically, a fixed-length sliding time window is set, for example, containing N consecutive log events, where N is 100. For the N log events falling within the current window, an undirected weighted graph G = (V, E, W) is dynamically constructed, where the node set V corresponds to the set of unique log event templates appearing within the window. For any two nodes v... i and v j If the corresponding log template co-occurs at least once in the event sequence within the window, and the sequence distance between them is less than a preset threshold d (e.g., d = 10), then in v i and v j Create an edge e between them ij Add it to the edge set E. The weight w of this edge... ij Calculated from the Gaussian kernel function, Where dist(i,j) is the difference in index of the most recent occurrence of templates i and j in the sequence, and σ is a hyperparameter controlling the weight decay rate. The constructed dynamic graph G is input into a graph attention network (GAT). The graph attention network contains L graph attention layers, where L is 3. In each layer, each node v i Node v updates its feature representation by aggregating information from its neighboring nodes through a self-attention mechanism. Specifically, node v i With neighbors v j Attention coefficient α between ij α is calculated using the following formula: ij =softmax j (LeakyReLU(a T [Wh i ||Wh j ])), where h i and h j Let be the current feature vectors of nodes i and j, W be a learnable linear transformation weight matrix, a be a learnable attention weight vector, and || denote the vector concatenation operation. The updated feature vector h′ of node i is... i It is given by the weighted sum of the characteristics of all its neighbors: Where N iLet be the set of neighbors of node i, and σ be a non-linear activation function, here using the exponential linear unit (ELU). After information propagation and aggregation through L layers, a global readout operation is performed on the final feature vectors of all nodes in the graph. Specifically, average pooling is performed on the vectors of all nodes to generate a fixed-dimensional graph-level embedding vector, denoted as Pattern Structure Embedding (PSE), with dimension D. p For example, 512. This PSE vector contains topological information about the macroscopic behavioral patterns reflected by the combination of log events within the current time window.

[0010] Secondly, it performs micro-sequence context modeling. This invention defines a sequence context pathway, the core of which is a deep sequence model based on a Transformer encoder architecture, used to finely capture the local and long-range temporal dependencies of each event in a log event sequence. Specifically, the initial event embedding vector sequence of N log events within the same time window is used as input. The Transformer encoder consists of M identical encoder layers stacked together, where M is 6. Each encoder layer contains two core sub-layers: a multi-head self-attention sub-layer and a position-wise feed-forward network sub-layer. In the multi-head self-attention sub-layer, the embedding vector at each position of the input sequence simultaneously generates three vectors: query, key, and value. The self-attention mechanism calculates the dot product of the query vector and all key vectors to obtain the attention weight of each position to all other positions, and then performs a weighted summation of the value vectors accordingly, thereby capturing contextual information. The multi-head mechanism executes this process in parallel H times (H being the number of heads, e.g., 8), each time using a different, learnable linear projection matrix. The H results are then concatenated and subjected to another linear transformation, allowing the model to focus on information from different locations across different representation subspaces. Its mathematical expression is MultiHead(Q,K,V)=Concat(head1,…,head) H W O Among them, hea The location-based feedforward sublayer is a fully connected feedforward network applied independently to the output at each location. It contains two linear transformations and a ReLU activation function: FFN(x) = max(0, xW1+b1)W2+b2. The output of each sublayer employs residual connections and layer normalization to facilitate gradient propagation and stabilize the training process. After processing through M encoder layers, the model generates a deep context-aware embedding vector for each event in the input sequence. The set of these N output vectors is denoted as Contextual Event Embeddings (CEE), where each vector has a dimension of D. e For example, 512. The CEE collection finely encodes the dynamic semantics of each log event within its specific sequence context.

[0011] Step three involves executing the synthetic attention mechanism and feature fusion. This step is the core of the invention. It uses a bidirectional, asymmetric attention mechanism to deeply couple and focus macroscopic pattern structure information with microscopic sequence context information, generating a highly condensed and discriminative synthetic anomaly representation. This mechanism includes two interrelated attention calculation processes.

[0012] The first process is pattern-to-sequence attention guidance. This process aims to leverage macroscopic pattern structure embeddings (PSEs) as high-level knowledge priors to guide the model in locating the key log events most relevant to the current business pattern within a microscopic context event embedding (CEE) sequence. Specifically, the PSE is used as the query vector in the attention mechanism, and all vectors in the CEE set are used as keys and values. The PSE and each CEE vector c are then computed. i Attention scores between i : Among them W q and W k It is a learnable projection matrix, d k That is the dimension of the key vector. Then, for all scores s i The attention weights α are obtained by normalization using the softmax function. i Finally, these weights are applied to the value vectors in the CEE set (here, the CEE vectors themselves), through a value projection matrix W. v The transformation is used to perform a weighted summation, resulting in a pattern-guided sequence context summary vector v. p2s :v p2s =∑ i α i (c i ·W v This vector vp2s This represents the feature aggregation of the underlying log event sequences that are most worthy of attention under the current macro business model.

[0013] The second process is sequence-to-pattern attention backtracking. This process aims to utilize potential anomalous signals in the sequence to trace their correlation with known normal behavioral patterns, thereby quantifying the degree of deviation. First, a learnable normal pattern memory M is defined. p It is a K×D p A matrix of dimensions, where K is the number of predefined normal pattern prototypes, and D... p This is the dimension of pattern structure embedding. During the training phase, this memory is initialized by K-Means clustering of the PSEs of all normal log windows in the training data and is optimized end-to-end along with the model. During the detection phase, potential anomalous signal points in the CEE sequence are first identified. This identification is achieved by calculating the embedding c for each event. i This is achieved by mitigating local reconstruction or prediction errors. For example, a lightweight autoencoder reconstructs the CEE sequence, and events with reconstruction errors exceeding a preset threshold are marked as suspicious events. The set of embedding vectors for all suspicious events is used as the query in the attention mechanism, while the normal pattern memory M... p The row vectors serve as keys and values. For each suspicious event, embed c... j Calculate its relationship with all normal mode prototypes m k Attention score jk And normalize to obtain the weight β jk Then, for each suspicious event c j Calculate its correlation representation r with the normal pattern library. j =∑ k β jk m k This vector r j Indicates suspicious event c j A hybrid representation that is semantically closest to the normal pattern. Finally, the correlation representation r of all suspicious events. j Aggregation (e.g., averaging or max pooling) yields a sequence reversal pattern bias vector v. s2p .

[0014] Finally, the pattern-guided sequence context summary vector v p2s Pattern deviation vector v from sequence tracing s2p The features are concatenated to form an enhanced fused feature vector. This v fusedThe vector is a synthetic anomaly representation that simultaneously encodes two complementary anomaly discrimination information: "which underlying events are critical under the current business model" and "which known normal patterns have the suspicious underlying events deviated from?"

[0015] Step four, anomaly detection and output. This step involves performing the final anomaly score based on the synthesized anomaly representation. The fused feature vector v generated in step three is then used for this purpose. fused The input is fed into a Multi-Layer Perceptron (MLP) classifier. The MLP classifier consists of three fully connected layers, using ReLU activation between layers and a Sigmoid activation function in the output layer. The number of neurons in the first layer is (D... p +D e The second layer is The output layer consists of a single neuron. This classifier outputs a scalar value between 0 and 1, representing the anomaly score for the current log window. A global discrimination threshold θ is set, for example, 0.9. If the output anomaly score is greater than θ, the current log window is considered anomaly, and an alarm event is generated; otherwise, it is considered normal. The alarm event not only includes the anomaly score and time window information but also the attention weight distribution calculated by the synthetic attention mechanism, i.e., interpretability information. This includes the most critical log events guided by the pattern and the most relevant normal pattern prototypes from which suspicious events deviate, providing operations personnel with decision support for in-depth analysis of the root causes of anomalies.

[0016] The training process of this invention adopts a supervised learning paradigm, using log window data with normal / abnormal labels. The loss function employs a weighted binary cross-entropy loss to address the data imbalance problem in real-world scenarios where normal samples far outnumber abnormal samples. Its mathematical form is: L = -[w1·y·log(p) + w0·(1-y)·log(1-p)], where y is the true label (1 for anomalous, 0 for normal), p is the anomalous score output by the model, and w1 and w0 are the weights assigned to the anomalous and normal categories, respectively, with values ​​inversely proportional to the category frequency. During training, all parameters of the entire model, including the embedding matrix in the log event vectorization unit, the graph attention network parameters of the pattern-aware pathway, the Transformer encoder parameters of the sequence context pathway, the projection matrix in the synthetic attention mechanism, and the normal pattern memory M, are used. p The parameters of the final MLP classifier are jointly optimized end-to-end using the backpropagation algorithm.

[0017] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0018] 1. This invention, through parallel pattern-aware pathways and sequence context pathways, achieves for the first time synchronous, independent, and in-depth modeling of macroscopic structural patterns and microscopic temporal contexts in log data. The pattern-aware pathway captures the business logic of event combinations through dynamic graph networks, surpassing the limitations of traditional static template libraries; the sequence context pathway uses the Transformer model to finely characterize the long-range dependencies between events, avoiding the information forgetting problem of RNN-like models.

[0019] 2. The innovative synthetic attention mechanism of this invention enables bidirectional information flow guidance between high-level semantics and low-level features. Pattern-to-sequence attention utilizes macroscopic patterns to focus on key microscopic events, solving the problem of key information being overwhelmed by noise; sequence-to-pattern attention uses the deviation of microscopic anomaly signals from normal patterns to provide quantifiable and interpretable evidence for anomaly tracing. This mechanism makes fusion no longer a simple feature splicing, but a deeply coupled process of mutual focusing and verification.

[0020] 3. Because the method of this invention can simultaneously detect macroscopic anomalies consisting of "abnormal combinations of normal patterns" and microscopic anomalies consisting of "minor timing deviations of single or a few events," it has a wider coverage of anomaly patterns and higher detection accuracy. For large-scale systems with complex business logic and variable log behavior, this method exhibits stronger robustness and adaptability.

[0021] 4. The output of this invention is not only an anomaly score, but also clearly indicates, through the weight distribution of the synthetic attention mechanism, which specific log events contribute the most to the detected macro-pattern; and which known normal business patterns deviate from the suspicious events in the sequence. This inherent interpretability greatly reduces the cognitive burden on operations and maintenance personnel in analyzing and locating faults, and improves operational efficiency. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the method of the present invention;

[0023] Figure 2 This is a schematic diagram of the system architecture of the method of the present invention;

[0024] Figure 3 This is a schematic diagram illustrating the principle of the macroscopic pattern structure perception pathway in this invention;

[0025] Figure 4 This is a schematic diagram illustrating the principle of the micro-sequence context modeling pathway in this invention;

[0026] Figure 5 This is a schematic diagram illustrating the principle of the synthetic attention mechanism in this invention;

[0027] Figure 6 This is a schematic diagram of the sequence-to-pattern attention backtracking process in this invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, a log anomaly detection method based on pattern awareness and synthetic attention mechanism provided by this invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. Furthermore, the embodiments described herein are some preferred embodiments of this invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments herein without inventive effort are within the scope of protection of this invention.

[0029] Reference Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating a log anomaly detection method based on pattern awareness and synthetic attention mechanism according to the present invention. Figure 2 This is a schematic diagram of the system architecture of the method of the present invention. The core idea of ​​the technical solution proposed in this invention is to construct parallel pattern-aware pathways and sequence context pathways to deeply characterize the macro-level business patterns and micro-level temporal dependencies of log sequences, respectively, and to design a bidirectional guided synthetic attention mechanism to achieve deep fusion and collaborative discrimination of the two heterogeneous features. Finally, the abnormal state of the log window is determined through anomaly detection and output steps.

[0030] In one specific embodiment, the entire method execution flow begins with the step of log data preprocessing and vectorization. The goal of this step is to transform the massive, unstructured raw text log stream into a structured, fixed-length sequence of numerical vectors that the model can process. This step contains two core modules: a log parsing module and a log event vectorization unit.

[0031] Specifically, the log parsing module receives real-time raw log message strings and precisely separates them into two parts: an unchanging log event template representing the core semantics, and a dynamically changing parameter list containing specific instance information. In this embodiment, the log parsing module employs an optimized online log parsing algorithm based on a fixed-depth tree structure, a specific implementation of the "Drain" algorithm. This algorithm maintains a trie, used for fast matching and grouping based on the lexical sequence of the log message. In a specific configuration, the maximum depth of this parse tree is set to 4 to balance parsing granularity and computational overhead. Simultaneously, its internal similarity threshold (sim...) is... thThe threshold is set to 0.4, which is used to determine whether a new log should be assigned to an existing log group. For example, for a raw log message “Block anr_block_id of size65536received from / 10.250.15.195”, the log parsing module will parse it into a log event template “Block*of size*received from*”, and a parameter list [“anr_block_id”, “65536”, “ / 10.250.15.195”].

[0032] Next, the log event vectorization unit receives the event template and parameter list output by the parsing module and generates an initial event embedding vector for each log event. This process involves the fusion of multi-part information encoding. First, for the log event template, the system maintains a global template embedding matrix. This matrix is ​​either pre-trained on a large-scale log corpus or randomly initialized and learned end-to-end during model training. In a specific implementation, the dimension of this matrix is ​​set to V×D. t Where V represents the total number of all unique log templates in the training corpus, for example, V = 5000, D t In this embodiment, D is the preset template embedding dimension. t The value is set to 256. Each log template is lookuped in this matrix using its unique integer index to obtain a 256-dimensional dense vector representation. Next, for the parameter list, the vectorization unit performs type discrimination and encoding on each parameter. If the parameter is purely numerical, such as "65536" in the example above, Z-score normalization is used to convert its value into a standardized scalar. This method is achieved by subtracting the mean of the parameter in the training dataset and dividing by its standard deviation to eliminate the influence of differences in units and numerical ranges. If the parameter is categorical or an identifier, such as "anr_block_id" or the IP address " / 10.250.15.195", it is mapped to a D vector through a separate categorical parameter embedding layer. c A vector of dimension D, where D cIn this embodiment, the dimension is set to 32. Finally, the obtained 256-dimensional template embedding vector, the normalized scalars of all numerical parameters (if multiple exist, they are concatenated into one vector), and the 32-dimensional embedding vectors of all categorical parameters are concatenated along the feature dimensions to form an initial event embedding vector of fixed dimensions. If the total dimensions after concatenation are not uniform, they are processed to a preset uniform dimension, such as 300 dimensions, through padding or truncation. This series of operations ensures that each original log is transformed into an information-rich and dimensionally consistent numerical vector, laying the foundation for subsequent deep neural network processing.

[0033] After preprocessing and vectorization, the method proceeds to the parallel feature extraction step, which is a key aspect of the architecture design of this invention. Through two heterogeneous and complementary neural network pathways, namely the pattern-aware pathway and the sequence context pathway, the features of the log sequence are extracted in parallel from both macroscopic and microscopic levels.

[0034] First, refer to Figure 3 This paper details the macroscopic pattern structure perception pathway. The core objective of this pathway is to transcend the linear temporal relationship of a single log event and capture the structural features of a higher-level business logic pattern or operational process constituted by multiple log events within a time window. Its implementation relies on a dynamic graph construction module and a graph attention network. In a specific embodiment, the system sets a fixed-length sliding time window containing N consecutive log events, where N is set to 100. For each of the 100 log events falling within the current window, the dynamic graph construction module constructs an undirected weighted graph G = (V, E, W) in real time. In this graph, the node set V corresponds to the set of unique log event templates appearing in these 100 events. For example, if 15 different templates appear within the window, the graph contains 15 nodes. The edge construction rules are as follows: for any two nodes v... i and v j If the log templates they represent co-occur at least once in the event sequence of the window, and the absolute value of the difference between their most recent occurrence position indices in the sequence is less than a preset distance threshold d (in this embodiment, d = 10), then in v i and v j Create an edge e between them ij This is then added to the edge set E. This rule aims to capture log patterns that are temporally proximate and potentially related. The weight w of the edge... ij It is not a simple binary value, but rather its correlation strength is quantified by a Gaussian kernel function, the specific calculation formula of which is: Where dist(i,j) is the absolute value of the difference between the position indices of the most recent occurrences of templates i and j in the sequence, and σ is a hyperparameter that controls the rate at which the weights decay with distance, set to 1.5 here. This weighting method results in stronger connections between template pairs that are closer in distance.

[0035] The constructed dynamic graph G is then input into a Graph Attention Network (GAT) for processing. The GAT consists of L stacked graph attention layers; in this embodiment, L is 3. In each layer, each node aggregates information from its neighbors through a self-attention mechanism, thereby updating its own feature representation. Specifically, node v... i To its neighbor v j Attention coefficient α ij It is calculated using a single-layer feedforward network, processed by the LeakyReLU nonlinear activation function (with a negative slope set to 0.2), and finally normalized over all neighbors of node i using the softmax function. Its mathematical expression is: α ij =softmax j (LeakyReLU(a T [Wh i ||Wh j ])), where h i and h j Let be the feature vectors of nodes i and j in the current layer (for the first layer, this is the initial feature vector of the node, which can be its template embedding vector), W be a learnable linear transformation weight matrix, a be a learnable attention weight vector, and || denote the vector concatenation operation. The updated feature vector h′ of node i. i It is then given by the weighted sum of the characteristics of all its neighbors: Where N i Let be the set of neighbors of node i, and ELU (Exponential Linear Unit) is used as the non-linear activation function. To enhance the expressiveness and stability of the model, a multi-head attention mechanism is adopted in each layer, that is, the above process is executed in parallel 4 times (4 heads), and the results are concatenated and then linearly transformed. After 3 layers of such information propagation and aggregation process, the feature vector of each node in the graph has fused with the structural information of its third-order neighborhood. Finally, in order to obtain the global representation of the entire graph, a global readout operation is performed on the final feature vectors of all nodes in the graph, specifically, average pooling is performed on the vectors of all nodes to generate a fixed-dimensional graph-level embedding vector, which is denoted as Pattern Structure Embedding (PSE). Its dimension Dp In this embodiment, it is set to 512. This PSE vector highly condenses the topological structure information of the macroscopic behavioral patterns reflected by the combination of log events within the current time window, and is an abstract representation of the business process.

[0036] Meanwhile, the micro-sequence context modeling pathway processes log event sequences within the same time window in parallel. (See reference...) Figure 4 The core of this pathway is a deep sequence model based on a Transformer encoder architecture, aiming to precisely capture the local and long-range temporal dependencies of each event in a log event sequence. The input is an initial sequence of event embedding vectors (each vector has a dimension of 300) for N=100 log events within the time window. Before inputting into the Transformer encoder, a sinusoidal positional encoding is superimposed on the embedding vector at each position to introduce positional information into the sequence. The Transformer encoder consists of M identical encoder layers stacked together; in this embodiment, M is set to 6 to ensure sufficient depth for the model to learn complex sequence patterns. Each encoder layer contains two core sub-layers: a multi-head self-attention sub-layer and a position-wise feed-forward network sub-layer. In the multi-head self-attention sub-layer, the embedding vector at each position of the input sequence is independently linearly projected three times to generate three vectors: query, key, and value. The self-attention mechanism calculates the attention weight of each query vector to all other positions in the sequence by computing the dot product of each query vector and all key vectors. It then performs a weighted summation of the value vectors based on this weight, dynamically aggregating the contextual information of the entire sequence. The multi-head mechanism executes this process in parallel H times (H = 8 in this embodiment), each time using a different set of learnable linear projection matrices, allowing the model to focus on information from different positions across eight different representation subspaces. The output dimension of each head is... The outputs of these eight heads are concatenated and then subjected to a linear transformation to obtain the final output of this sublayer. Its mathematical expression is: MultiHead(Q,K,V)=Concat(head1,…,head) H W O Among them, hea The position-based feedforward sublayer is a fully connected feedforward network applied independently to each position in the sequence. This network contains two linear transformations and a ReLU activation function, and its hidden layer dimension is typically set to four times the model dimension, i.e., 4 × 512 = 2048. Its formula is: FFN(x) = max(0, xW1+b1)W2+b2. To facilitate deep network training, the output of each sublayer (self-attention and feedforward network) undergoes residual connections and layer normalization. After sequential processing through six such encoder layers, the model generates a deep context-aware embedding vector for each event in the input sequence. The set of these 100 output vectors is denoted as Contextual Event Embeddings (CEE), where each vector has a dimension of D. e =512. The CEE set finely encodes the semantic information of each log event within its specific, dynamically changing sequence context.

[0037] After the parallel feature extraction step is completed, the method flow proceeds to the synthetic attention mechanism and feature fusion step, which is the core technology of this invention. Through a designed, bidirectional asymmetric attention mechanism, macroscopic pattern structure information (PSE) and microscopic sequence context information (CEE) are deeply coupled and focused to generate a highly condensed and discriminative synthetic anomaly representation. (Refer to...) Figure 5 The mechanism involves two interconnected attention computation processes.

[0038] The first process is executed by the pattern-to-sequence attention guidance module. This process aims to leverage macroscopic, highly abstract pattern structure embeddings (PSEs) as a high-level knowledge prior to guide the model in locating the key log events most relevant to the current overall business pattern within microscopic, richly detailed context event embedding (CEE) sequences. Specifically, a 512-dimensional PSE vector is used as the query vector in the attention mechanism, and a CEE set containing 100 512-dimensional vectors is used as the key and value. First, the calculation of c for each vector in the PSE and CEE sequences is performed. i Attention scores between i This step is accomplished by scaling the dot product attention: Among them W q and W k It is a learnable 512×512-dimensional projection matrix used to map queries and keys to a representation space more suitable for matching, d k This is the dimension of the key vector (512 in this case). Then, for all 100 scores s...i Normalization is performed using the softmax function to obtain a set of attention weights α. i , where ∑ i α i =1. These weights reflect the importance of each event in the sequence within the macroscopic pattern represented by the current PSE. Finally, these weights are applied to the value vectors in the CEE set (here, the CEE vectors themselves, projected through a value projection matrix W). v The transformation is used to perform a weighted summation, resulting in a pattern-guided sequence context summary vector v. p2s :v p2s =∑ i α i (c i ·W v This 512-dimensional vector v p2s This represents the feature aggregation of the underlying log event sequence that is most worthy of attention from the perspective of the current macro business model, effectively filtering out noise information that is irrelevant to the current model.

[0039] The second process is performed by the sequence-to-pattern attention backtracking module. (See reference...) Figure 6 This process aims to utilize potential, localized anomalous signals in a sequence to trace their correlation with known normal behavioral patterns of the system, thereby quantifying the degree of deviation and providing another dimension of evidence for anomaly detection. First, this invention defines a learnable normal pattern memory M. p It is a K×D p A matrix of dimensions, where K is the number of predefined normal pattern prototypes, and D... p This is the dimension of the pattern structure embedding (512). In this embodiment, K is set to 50. Before the training phase begins, this memory is initialized by performing K-Means clustering on the PSE vectors generated from all log windows labeled as normal in the training data, resulting in 50 cluster centers as the initial normal pattern prototypes. During training, this memory M... p As part of the model parameters, end-to-end gradient optimization is performed along with the entire network to better capture the essence of normal behavioral patterns. During the detection phase, potential anomalous signal points need to be identified from the CEE sequence. This identification process is implemented using a lightweight, auxiliary autoencoder. During training, this autoencoder only learns to reconstruct the CEE sequence for normal windows. During detection, the current CEE sequence is input into the autoencoder, and the reconstruction error (e.g., mean squared error) at each event location is calculated. Events with reconstruction errors greater than a preset threshold (e.g., the 98th percentile of reconstruction errors for normal samples on the training set) are marked as suspicious events. Assuming m suspicious events are identified in this window, their embedding vector set {c jThe group |j=1,…,m} is used as the query in the attention mechanism. The normal pattern memory M p 50 row vectors {m k The set |k=1,…,50} is used as the key and value. For each suspicious event, embed c j Calculate its relationship with all 50 normal pattern prototypes m k Attention score jk The weights β are obtained by softmax normalization. jk Then, for each suspicious event c j Calculate its correlation representation r with the normal pattern library. j =∑ k β jk m k This vector r j This can be understood as suspicious event c j A hybrid representation that is semantically closest to the normal pattern. Finally, the correlation representation r of all m suspicious events. j In this embodiment, aggregation is performed using max pooling to obtain a sequence reversal mode bias vector v. s2p =max({r j |j=1,…,m}). This 512-dimensional vector v s2p It encodes the maximum semantic similarity between the abnormal signal in the sequence and the normal pattern library, and its norm or difference from other vectors can indirectly reflect the degree of deviation.

[0040] Finally, the pattern-guided sequence context summary vector v p2s Pattern deviation vector v from sequence tracing s2p By concatenating along the feature dimensions, an enhanced fused feature vector with a dimension of 1024 is formed. This v fused The vector is the synthetic anomaly representation constructed in this invention, which simultaneously encodes two highly complementary core pieces of information for anomaly detection: "which underlying events are critical under the current business model" and "which known normal patterns have the suspicious underlying events deviated from?"

[0041] The final step of the method is the anomaly detection and output step, which performs the final anomaly scoring and decision based on the synthetic anomaly representation generated in the previous step. This involves processing the 1024-dimensional fused feature vector v... fusedThe input is fed into a Multi-Layer Perceptron (MLP) classifier. In this embodiment, the MLP classifier consists of three fully connected layers, with ReLU activation functions used between layers to introduce non-linearity. The specific network structure is as follows: the input layer has 1024 neurons, the first hidden layer has 512 neurons, the second hidden layer has 256 neurons, and the output layer has only 1 neuron. To prevent overfitting, a dropout rate of 0.4 is applied after each hidden layer. The output layer uses a Sigmoid activation function to compress the output value to between 0 and 1; this value is considered the anomaly score of the current log window. The system sets a global discrimination threshold θ, for example, θ = 0.9. If the anomaly score output by the MLP classifier is greater than θ, the current log window is determined to be abnormal, and an alarm event is generated. Conversely, if the score is not greater than θ, it is determined to be normal. The generated alert events not only include anomaly scores, timestamps, and raw logs within the window, but also highly interpretable information computed by the synthetic attention mechanism. Specifically, this includes the attention weight α computed by the pattern-to-sequence attention guidance module. i The distribution of log events is highlighted, showing the key log events that contributed the most to this abnormal pattern; and the attention weight β calculated by the sequence-to-pattern attention backtracking module. jk The distribution of these events indicates which known normal business model prototypes the suspicious events in the sequence deviated significantly from. This information provides operations and maintenance personnel with decision support for in-depth analysis of the root causes of anomalies, greatly improving the efficiency of troubleshooting.

[0042] The entire model of this invention is trained using a supervised learning paradigm, requiring log window data with normal / abnormal labels. The loss function employs a weighted binary cross-entropy loss to effectively address the data imbalance problem in real-world operational scenarios where normal samples far outnumber abnormal samples. Its mathematical form is: L = -[w1·y·log(p) + w0·(1-y)·log(1-p)], where y is the true label (1 for abnormal, 0 for normal), p is the abnormal score output by the model, and w1 and w0 are the weights assigned to the abnormal and normal categories, respectively. Their values ​​are inversely proportional to the category frequency; for example, if the ratio of normal to abnormal samples is 99:1, then w1 = 99, w0 = 1. During training, all learnable parameters of the model, including the embedding matrix in the log event vectorization unit, the parameters of the graph attention network in the pattern-aware pathway, the parameters of the Transformer encoder in the sequence context pathway, all projection matrices in the synthetic attention mechanism, and the normal pattern memory M, are used. pThe parameters of the final MLP classifier, as well as the parameters of the classifier, were jointly optimized end-to-end using the backpropagation algorithm and the AdamW optimizer (learning rate set to 1e-4, weight decay set to 0.01). The training process was performed on a server containing NVIDIA Tesla V100 GPUs, using the PyTorch deep learning framework, with a batch size of 64, for a total of 50 training epochs.

[0043] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A log anomaly detection method based on pattern awareness and synthetic attention mechanism, characterized in that, Includes the following steps: Log data preprocessing and vectorization involves parsing the raw log messages and converting them into an initial event embedding vector sequence. Parallel feature extraction: For the time window formed by the initial event embedding vector sequence, feature extraction is performed through two parallel processing paths, which include: The pattern-aware pathway is used to perceive the macroscopic pattern structure of log events within the time window and generate a pattern structure embedding (PSE) that contains the macroscopic behavioral pattern topology information reflected by the combination of log events within the time window. And a sequence context path, used to perform micro-sequence context modeling on log events within the time window, generating a deep context-aware context event embedding (CEE) for each event in the sequence, forming a context event embedding set; The synthetic attention mechanism and feature fusion are achieved by deeply fusing the pattern structure embedding (PSE) and the context event embedding (CEE) set through a bidirectional guided synthetic attention mechanism to generate a fused feature vector of synthetic anomaly representation. Anomaly detection and output: Based on the fused feature vector, an anomaly score of the time window is calculated by a classifier, and whether the time window is an anomaly is determined according to a preset detection threshold.

2. The log anomaly detection method based on pattern awareness and synthetic attention mechanism according to claim 1, characterized in that, The log data preprocessing and vectorization steps specifically include: A log parsing module parses the input raw log message string into an unchanging log event template and a dynamically changing parameter list; the log parsing module adopts an online log parsing algorithm based on a fixed-depth tree structure. The initial event embedding vector is generated for each parsed log event using a log event vectorization unit, and the process is as follows: First, for the log event template, its corresponding template embedding vector is searched and obtained in a global template embedding matrix; Secondly, for the parameter list, each parameter in the list is identified and encoded. If the parameter is numerical, it is normalized to obtain a scalar. If the parameter is categorical, it is mapped to a categorical parameter embedding vector through a categorical parameter embedding layer. Third, the obtained template embedding vector, the normalized scalars of all numerical parameters, and the embedding vectors of all categorical parameters are concatenated along the feature dimension to form the initial event embedding vector.

3. The log anomaly detection method based on pattern awareness and synthetic attention mechanism according to claim 1, characterized in that, In the pattern-aware pathway, the process of macroscopic pattern structure perception specifically includes: A dynamic graph construction module is used to dynamically construct an undirected weighted graph G = (V, E, W) for N consecutive log events within the time window, where: The set of nodes V in the graph corresponds to the set of unique log event templates that appear within the time window; The set of edges E of the graph is constructed according to the following rule: for any two nodes v i and v j If the corresponding log template co-occurs at least once in the event sequence of the time window, and the sequence distance between them is less than a preset distance threshold d, then in v i and v j Create an edge e between them ij ; The weight w of each edge in the graph ij The value is calculated by the Gaussian kernel function based on the difference in the position index of the two templates in the sequence at their most recent occurrence.

4. The log anomaly detection method based on pattern awareness and synthetic attention mechanism according to claim 3, characterized in that, In the pattern-aware pathway, the process of generating the pattern structure embedding (PSE) is specifically as follows: The undirected weighted graph constructed by the dynamic graph construction module is input into a graph attention network, which is composed of L stacked graph attention layers; In each graph attention layer, each node aggregates information from its neighbors through a self-attention mechanism to update its own feature representation; The self-attention mechanism is implemented by calculating the attention coefficient between a node and the features of its neighboring nodes, and by performing a weighted summation of the features of the neighboring nodes. After information propagation and aggregation through L graph attention layers, a global readout operation is performed on the final feature vectors of all nodes in the graph to generate the fixed-dimensional graph-level embedding vector, which serves as the pattern structure embedding (PSE).

5. The log anomaly detection method based on pattern awareness and synthetic attention mechanism according to claim 1, characterized in that, In the aforementioned sequence context pathway, the process of micro-sequence context modeling specifically involves: The initial event embedding vector sequence of N log events within the time window is input into a deep sequence model based on a Transformer encoder architecture; The Transformer encoder consists of M identical encoder layers stacked together. Each encoder layer contains a multi-head self-attention sublayer and a position feedforward network sublayer. The output of each sublayer employs residual connection and layer normalization operations. The multi-head self-attention sublayer enables the model to focus on information from different positions in the sequence in different representation subspaces by performing H self-attention calculations in parallel. The position feedforward network sublayer is a fully connected feedforward network that is independently applied to the output at each position in the sequence; After processing through M encoder layers, the model generates a deep context-aware embedding vector for each event in the input sequence, thus forming the context event embedding (CEE) set.

6. The log anomaly detection method based on pattern awareness and synthetic attention mechanism according to claim 1, characterized in that, In the synthetic attention mechanism and feature fusion step, the bidirectional guided synthetic attention mechanism includes a pattern-to-sequence attention guidance process, which is executed by the pattern-to-sequence attention guidance module, specifically as follows: The pattern structure embedding (PSE) is used as the query vector in the attention mechanism, and all vectors in the context event embedding (CEE) set are used as key vectors and value vectors; Calculate the attention score between the query vector and each key vector, and normalize all scores to obtain a set of attention weights; By weighting and summing the value vector using the attention weights, a pattern-guided sequence context summary vector (v) is obtained. p2s This vector represents the feature aggregation of the most relevant underlying log event sequences under the current macro business model.

7. The log anomaly detection method based on pattern awareness and synthetic attention mechanism according to claim 1, characterized in that, In the synthetic attention mechanism and feature fusion step, the bidirectional guided synthetic attention mechanism further includes a sequence-to-pattern attention backtracking process, which is executed by the sequence-to-pattern attention backtracking module, specifically as follows: Identify potential suspicious events in the context event embedding (CEE) set, and construct a query vector set from the embedding vectors of all suspicious events; Attention is computed between the set of query vectors and a predefined learnable normal pattern memory, wherein the row vectors of the normal pattern memory serve as key vectors and value vectors. For each query vector of a suspicious event, calculate its attention weight with all normal pattern prototypes, and then perform a weighted summation on the normal pattern prototypes to obtain the association representation between the suspicious event and the normal pattern library. Aggregate the correlation representations of all suspicious events to obtain a sequence tracing pattern bias vector (v). s2p ).

8. The log anomaly detection method based on pattern awareness and synthetic attention mechanism according to claim 7, characterized in that, The normal mode memory is a K×D p A matrix of dimensions, where K is the number of predefined normal pattern prototypes, and D... p The normal pattern memory is initialized before model training by performing K-Means clustering on the pattern structure embeddings (PSEs) generated from all normal log windows in the training data, and is used as a learnable parameter for end-to-end joint optimization during model training.

9. The log anomaly detection method based on pattern awareness and synthetic attention mechanism according to claim 7, characterized in that, The process of generating the fused feature vector is as follows: The sequence context summary vector (v) guided by the aforementioned pattern p2s ) and the pattern deviation vector (v) of the sequence backtracking s2p The fused feature vector (v) is formed by concatenating the features along their respective dimensions. fused This fused feature vector simultaneously encodes key event information guided by the pattern and information on the degree to which the sequence deviates from the normal pattern.

10. The log anomaly detection method based on pattern awareness and synthetic attention mechanism according to claim 1, characterized in that, The anomaly detection and output step further includes: The classifier is a multilayer perceptron (MLP) consisting of at least three fully connected layers, with ReLU activation function used between layers and Sigmoid activation function used in the output layer to output the anomaly score between 0 and 1; When the time window is determined to be abnormal, an alarm event is generated. In addition to the abnormality score and time window information, the alarm event also includes the attention weight distribution calculated by the synthetic attention mechanism as interpretability information. The interpretability information includes the attention weight indicating key log events under pattern guidance, and the attention weight indicating the normal pattern prototype from which the suspicious event deviates.

Images