Log anomaly detection method, device and storage medium based on multi-feature fusion

Abstract

The present application relates to the technical field of log anomaly detection, and particularly relates to a log anomaly detection method and device based on multi-feature fusion and a storage medium.In the present application, log data is parsed and log templates and log levels are extracted through a Drain algorithm, semantic features are extracted in combination with a BERT model, and adjacent log time differences are calculated and time features are generated;after the semantic, time and log level features are fused, they are input into a time series convolution network (TCN) and a Transformer module, the TCN is responsible for capturing fine-grained local time sequence features to complement the global context features captured by the Transformer module, the two kinds of features are deeply fused through a cross-attention mechanism to form a more comprehensive log sequence representation, and the accuracy, robustness and real-time performance of log anomaly detection are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of log anomaly detection technology, specifically to a log anomaly detection method, device, and storage medium based on multi-feature fusion. Background Technology

[0002] Traditional log anomaly detection relies on rule matching or statistical analysis, which is suitable for anomalies with fixed patterns, but due to manually set rules and static characteristics, it is difficult to cope with dynamic changes in log patterns. With the development of deep learning technology, its ability to mine features from large-scale log data has significantly improved detection performance. However, existing methods have the following problems: First, most models only focus on single features (such as time series features or semantic features), lacking the ability to comprehensively analyze multiple features; second, models such as LSTM and GRU experience performance degradation when handling long-distance dependencies.

[0003] To address these challenges, researchers have begun exploring methods that combine multi-feature fusion with advanced deep learning. For example, Temporal Convolutional Networks (TCNs) efficiently extract local features through causal convolution, while Transformers utilize self-attention mechanisms to capture global contextual information; both have demonstrated advantages in log anomaly detection. However, designing an efficient detection method that can fully integrate log semantic features, temporal features, and structured information remains a pressing challenge, crucial for improving detection performance and system stability. Summary of the Invention

[0004] Based on this, the present invention provides a log anomaly detection method, device and storage medium based on multi-feature fusion, which solves at least one problem in the prior art.

[0005] In a first aspect, the present invention provides a log anomaly detection method, which includes the following steps:

[0006] In response to receiving a log anomaly detection instruction, feature extraction is performed on the original log sequence indicated by the log anomaly detection instruction to obtain a semantic feature vector sequence, a latency feature vector sequence, and a level feature vector sequence; wherein, the original log sequence includes at least two original logs;

[0007] The semantic feature vector sequence, the time delay feature vector sequence, and the level feature vector sequence are fused to obtain the fused feature vector sequence.

[0008] The above-mentioned fused feature vector sequence is used to extract features through a temporal convolutional network (TCN) to obtain a local temporal feature sequence;

[0009] The global context feature sequence is obtained by extracting features from the above fused feature vector sequence using the Transformer module.

[0010] The aforementioned local temporal feature sequence and global contextual feature sequence are weighted and fused using a cross-attention mechanism to obtain a comprehensive feature vector sequence.

[0011] The above-mentioned comprehensive feature vector sequence is input into the classifier, and the classifier outputs the log anomaly detection result.

[0012] In some optional embodiments, the above-mentioned feature extraction of the original log sequence indicated by the log anomaly detection instruction to obtain a semantic feature vector sequence, a latency feature vector sequence, and a level feature vector sequence includes:

[0013] The Drain algorithm is used to parse the raw log sequence and extract the log template sequence and log level sequence.

[0014] The above log template sequence is semantically encoded using a pre-trained BERT model to obtain a sequence of semantic feature vectors;

[0015] Calculate the time difference between adjacent raw logs to obtain a sequence of delay feature vectors;

[0016] The above log level sequence is mapped to a numerical code to obtain a level feature vector sequence.

[0017] In some optional embodiments, the above-mentioned feature extraction from the fused feature vector sequence using a temporal convolutional network (TCN) to obtain a local temporal feature sequence includes:

[0018] The fused feature vector sequence is input into a temporal convolutional network (TCN);

[0019] Perform causal dilated convolution on the fused feature vector sequence;

[0020] Weight normalization is performed on the output feature sequence of causal dilated convolution;

[0021] Activate using the ReLU activation function;

[0022] Perform a Dropout operation;

[0023] The fused feature vector sequence is residually connected to the input to obtain a local temporal feature sequence.

[0024] In some alternative embodiments, the Transformer module described above includes a multi-layer Transformer encoder, each of which includes a multi-head attention sublayer and a feedforward neural network sublayer.

[0025] In some optional embodiments, the weighted fusion of the local temporal feature sequence and the global contextual feature sequence through the cross-attention mechanism to obtain the comprehensive feature vector sequence includes:

[0026] The query vector sequence, key vector sequence, and value vector sequence of the local temporal feature sequence and the global context feature sequence are obtained by linear transformation.

[0027] The attention weight matrix is ​​calculated based on the query vector sequence of the local temporal feature sequence, the key vector sequence of the global context feature sequence, and the value vector sequence of the global context feature sequence to obtain the first feature representation;

[0028] The attention weight matrix is ​​calculated based on the query vector sequence of the global context feature sequence, the key vector sequence of the local temporal feature sequence, and the value vector sequence of the local temporal feature sequence to obtain the second feature representation;

[0029] The first and second feature representations are concatenated and subjected to a nonlinear transformation to obtain a sequence of comprehensive feature vectors.

[0030] In some alternative embodiments, the classifier includes a fully connected neural network whose output layer uses a Softmax function to generate anomaly probabilities.

[0031] In some optional embodiments, the classifier obtains the log anomaly detection result based on the relationship between the preset threshold and the anomaly probability. If the anomaly probability is greater than the preset threshold, the log anomaly detection result is considered abnormal.

[0032] In some optional embodiments, the preset threshold is 0.5.

[0033] In a second aspect, the present invention provides an electronic device comprising:

[0034] At least one processor;

[0035] and memory that is communicatively connected to at least one processor;

[0036] The memory stores instructions that, when executed by at least one processor, implement the aforementioned log anomaly detection method.

[0037] Thirdly, the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the above-described log anomaly detection method.

[0038] Due to the adoption of the above technical solutions, the embodiments of the present invention have at least the following beneficial effects:

[0039] Log data is parsed using the Drain algorithm to extract log templates and log levels. Semantic features are extracted using the BERT model, and the time difference between adjacent logs is calculated to generate temporal features. The semantic, temporal, and log level features are then fused and input into a Temporal Convolutional Network (TCN) and a Transformer module. The TCN is responsible for capturing fine-grained local temporal features to complement the global context features captured by the Transformer module. The two types of features are deeply fused through a cross-attention mechanism to form a more comprehensive log sequence representation, which effectively improves the accuracy, robustness, and real-time performance of log anomaly detection. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the log anomaly detection method in an embodiment of the present invention.

[0041] Figure 2 This is a schematic diagram of the TCN structure used in the embodiments of the present invention.

[0042] Figure 3 This is a schematic diagram of the causal dilated convolution used in the embodiments of the present invention.

[0043] Figure 4 This is a schematic diagram of the causal convolution structure used in the embodiments of the present invention.

[0044] Figure 5 This is a schematic diagram of the dilated convolution structure used in the embodiments of the present invention.

[0045] Figure 6 This is a schematic diagram of the Transformer encoder structure used in the embodiments of the present invention.

[0046] Figure 7 This is a schematic diagram of the cross-attention fusion mechanism used in the embodiments of the present invention. Detailed Implementation

[0047] The following will provide a clear and complete description of the concept and technical effects of the present invention, so as to fully explain the purpose, solution and effects of the present invention.

[0048] Figure 1 A flowchart illustrating a log anomaly detection method according to an embodiment of the present invention is shown, the log anomaly detection method including steps S100 to S600:

[0049] S100. In response to receiving a log anomaly detection instruction, feature extraction is performed on the original log sequence indicated by the log anomaly detection instruction to obtain a semantic feature vector sequence, a delay feature vector sequence, and a level feature vector sequence; wherein, the original log sequence includes at least two original logs;

[0050] S200. The semantic feature vector sequence, the time delay feature vector sequence, and the level feature vector sequence are fused to obtain the fused feature vector sequence.

[0051] S300. The fused feature vector sequence is extracted using a temporal convolutional network (TCN) to obtain a local temporal feature sequence.

[0052] S400. The global context feature sequence is obtained by extracting features from the fused feature vector sequence using the Transformer module.

[0053] S500: The local temporal feature sequence and the global contextual feature sequence are weighted and fused through a cross-attention mechanism to obtain a comprehensive feature vector sequence;

[0054] S600. Input the comprehensive feature vector sequence into the classifier, and the classifier outputs the log anomaly detection result.

[0055] According to an embodiment of the present invention, in response to receiving a log anomaly detection instruction, an original log sequence can be obtained from a data source (e.g., a local database, a cloud database, or a network resource) based on the log anomaly detection instruction. The original log sequence includes at least two original logs. The at least two original logs have a temporal relationship with each other.

[0056] In step S100, feature extraction is performed on the original log sequence indicated by the log anomaly detection instruction to obtain a semantic feature vector sequence, a latency feature vector sequence, and a level feature vector sequence, which may include:

[0057] S110. Parse the original log sequence using the Drain algorithm to extract the log template sequence and log level sequence;

[0058] S120. Use a pre-trained BERT model to perform semantic encoding on the log template sequence to obtain a semantic feature vector sequence;

[0059] S130. Calculate the time difference between adjacent original logs to obtain a sequence of time delay feature vectors;

[0060] S140. Map the log level sequence to a numerical code to obtain the level feature vector sequence.

[0061] According to embodiments of the present invention, after obtaining the original log sequence, the original log sequence can be preprocessed. Preprocessing may include grouping and sorting the original logs. Preprocessing ensures data consistency and integrity, thereby providing a reliable input source. The Drain algorithm is an efficient log parsing algorithm that can automatically extract log templates and log levels, and identify the fixed and variable parts of the logs. The Drain algorithm categorizes similar log messages into the same template through clustering and extracts dynamic parameters from the logs. The log levels may include DEBUG, INFO, WARN, and ERROR. Both the log template and log levels can be converted into numerical vector forms through one-hot encoding or numerical mapping for computation.

[0062] By semantically encoding the log template sequence using a pre-trained BERT model, semantic features can be extracted, resulting in a 768-dimensional semantic feature vector sequence. After sorting the timestamps of the original logs, the time difference Δt between adjacent original logs is calculated. i And apply logarithmic transformation Δt i ′=log(1+Δt i This allows us to obtain a sequence of time-delay feature vectors. Where Δt... i This represents the time difference between the i-th original log (where i is a positive integer) and the previous original log. The time difference for the first log is set to 0 to facilitate unified processing of latency characteristics. Δt i ' represents the value after logarithmic transformation. Logarithmic transformation avoids the problem of excessively large differences in time intervals. We can use L={l1,l2,…,l... k Let} represent the set of all log levels. One-hot encoding is performed to map each log level to a binary level feature vector sequence of length k, where k represents the number of all different levels appearing in the system log.

[0063] In step S200, the semantic feature vector sequence, the time delay feature vector sequence, and the level feature vector sequence are concatenated to form a fused feature vector sequence.

[0064] In step S300, a temporal convolutional network (TCN) is used to efficiently capture local temporal features at different scales in the log sequence. For example... Figure 2 As shown, step S300 may include:

[0065] S310. Input the fused feature vector sequence into the TCN ( Figure 2 The black nodes in the middle represent the starting point of the fused feature vector sequence, marking the beginning of data processing;

[0066] S320. Perform causal dilated convolution on the fused feature vector sequence;

[0067] S330. Normalize the weights of the output feature sequence of the causal dilated convolution (by normalizing the network parameters, the convergence speed of the model is significantly accelerated and the stability of the training process is improved).

[0068] S340, Activate using the ReLU activation function;

[0069] S350. Perform Dropout operation (apply Dropout random deactivation technology to randomly discard a certain proportion of neurons to prevent overfitting and improve generalization ability).

[0070] S360. Residual connection is performed with the input fused feature vector sequence to obtain the local temporal feature sequence.

[0071] According to an embodiment of the present invention, steps S320 to S350 can be repeated. Figure 2 The display is repeated once. The processed feature sequence is residually connected to the original fused feature vector sequence to obtain a local temporal feature sequence. The residual connection can alleviate the gradient vanishing problem in deep networks.

[0072] The causal dilatation convolution process combines causal convolution and dilatation convolution, such as... Figure 3 As shown, the top and bottom rows of squares represent the output and input layers, respectively, implementing causal dilated convolution from input to output. Causal convolution is as follows: Figure 4 As shown, each output node connects only to the current and previous input nodes, forming a progressive connection structure that ensures unidirectional information flow and prevents the leakage of future information. Dilated convolution is as follows... Figure 5 As shown, by introducing skip connections, each node can be directly connected to nodes at greater distances, thereby expanding the receptive field and capturing a wider range of dependencies without increasing the number of parameters.

[0073] Before performing feature extraction on the fused feature vector sequence, the fused feature vector can be transposed. The specific calculation process can be expressed as follows:

[0074] X′=X.permute(0,2,1)∈R B×D×L

[0075] Where X represents the fused feature vector sequence, permute(·) represents the transpose operation, and X′ represents the transposed feature tensor sequence; B represents the batch size, D represents the feature dimension, and L represents the sequence length.

[0076] The transposed feature tensor sequence is input into the TCN for processing. The TCN utilizes a causal convolution mechanism and relies only on information from the current time step and previous steps, strictly adhering to temporal causality and avoiding information leakage. Simultaneously, dilated convolution inserts holes during the convolution operation, increasing the dilation rate of each convolutional layer by a power of 2, effectively expanding the network's receptive field without significantly increasing the number of parameters. The specific calculation process of causal dilated convolution can be represented as follows:

[0077]

[0078] Where x represents the input sequence (i.e., the transposed feature tensor sequence), f represents the convolution kernel of size k, i represents the position of the current convolution kernel weights involved in the calculation (i.e., the local index variable of the convolution kernel, numbered starting from 0), d represents the dilation rate, and s represents the current position.

[0079] To enhance nonlinear expressiveness and improve feature extraction performance, a ReLU activation function is introduced after each causal dilation convolution operation. This combination not only enhances the ability to capture complex patterns but also helps alleviate the gradient vanishing problem. The specific computation process of each layer can be represented as follows:

[0080] H i =ReLU(Conv1D(H i-1 ,K i ,d i ))

[0081] Where ReLU(·) represents the ReLU activation function, Conv1D(·) represents the causal dilation convolution calculation process, and H i-1 This represents the output of the previous layer (initially H0 = X). ′ ), K i This represents a convolution kernel of size k, d i This represents the expansion rate of the i-th layer.

[0082] During causal dilation convolution, padding is required to maintain the input and output sequence lengths and avoid information loss. Under the constraints of causal dilation convolution, padding is only applied to the beginning (left side) of the sequence. The calculation process for the padding size of each causal dilation convolution layer can be represented as follows:

[0083] padding=(k-1)×d i

[0084] Where k represents the kernel size, d i This indicates the expansion rate of the current layer.

[0085] After n layers of convolution (n represents a positive integer, e.g., n equals 2), the feature tensor of the TCN branch needs to be transposed back to the dimensional arrangement corresponding to the original tensor, so that it can be fused with the features extracted by the Transformer module later. The transposition process can be represented as follows:

[0086] H TCN =H n .permute(0,2,1)∈R B×L×D

[0087] Among them, H n H represents the output of the nth layer. TCN This represents the output of TCN (i.e., the local temporal feature sequence).

[0088] In step S400, the Transformer module may include a multi-layer Transformer encoder, and the number of layers in the Transformer encoder can be three. For example... Figure 6 As shown, each Transformer encoder layer includes a multi-head attention sublayer and a feedforward neural network sublayer, where the number of attention heads can be up to four. For the fused feature vector sequence X∈R B×L×D First, position encoding needs to be added to provide position information within the sequence. The classic sine-cosine position encoding is used, and the specific calculation process can be represented as follows:

[0089]

[0090] Where pos represents the position index in the sequence (starting from 0), j represents the index of the feature dimension, and d model This indicates the feature dimension. This positional encoding provides unique coding information for each sequence position while maintaining the continuity and periodicity between positions.

[0091] Next, the positional encoding is added to the input features and fed into a multi-layer Transformer encoder for processing. Multiple attention sets are computed in parallel to capture sequence information from different representation subspaces. Specifically, the input sequence undergoes a linear transformation to generate a query matrix Q, a key matrix K, and a value matrix V. Then, attention weights and weighted representations are calculated. The specific calculation process can be represented as follows:

[0092]

[0093] in, represents the scaling factor, which prevents the dot product from becoming too large, leading to gradient instability or vanishing gradients; softmax(·) represents the softmax function. After processing by m layers (m represents a positive integer, e.g., m equals 3) of Transformer encoders, the final output sequence features can be represented as:

[0094] H Trans =X m ∈R B×L×D

[0095] Among them, X m H represents the output of the m-th layer. Trans This represents the global context feature sequence.

[0096] The global contextual information captured by the Transformer module is crucial for identifying anomalous behaviors that require long-range dependencies, complementing the local anomalous pattern capture capabilities that TCN excels at. Furthermore, the Transformer module's self-attention mechanism offers good interpretability.

[0097] In step S500, the local temporal feature sequence and the global contextual feature sequence are weighted and fused using a cross-attention mechanism to obtain a comprehensive feature vector sequence, which may include:

[0098] S510. Obtain the query vector sequence, key vector sequence, and value vector sequence of the local temporal feature sequence and the global context feature sequence respectively through linear transformation;

[0099] S520. Calculate the attention weight matrix based on the query vector sequence Q1 of the local temporal feature sequence, the key vector sequence K1 of the global context feature sequence, and the value vector sequence V1 of the global context feature sequence to obtain the first feature representation H. T→G ;

[0100] S530. Calculate the attention weight matrix based on the query vector sequence Q2 of the global context feature sequence, the key vector sequence K2 of the local temporal feature sequence, and the value vector sequence V2 of the local temporal feature sequence to obtain the second feature representation H. G→T ;

[0101] S540, Represent the first feature H T→G Second feature representation H G→T The sequences are concatenated and subjected to nonlinear transformation to obtain a comprehensive feature vector sequence.

[0102] According to embodiments of the present invention, the cross-attention fusion mechanism allows the TCN and Transformer modules to dynamically "question" each other for key information. For example... Figure 7 As shown, the local temporal feature sequence H is obtained from TCN.TCN ∈R B×L×D The global context feature sequence H is obtained from the Transformer module. Trans ∈R B×L×D Then, cross-attention in two directions is calculated: the attention mechanism from TCN to Transformer enables local features to query and integrate global contextual information, enhancing the perception of the overall sequence structure; the attention mechanism from Transformer to TCN endows global features with the ability to accurately locate and integrate key local temporal patterns, thereby improving the ability to identify fine-grained anomalies. The specific calculation process of cross-attention can be represented as follows:

[0103] H T→G =MH-Attention(Q=H) TCN K = H Trans V=H Trans )

[0104] H G→T =MH-Attention(Q=H) Trans K = H TCn V=H TCN )

[0105] Among them, H TCN H represents the local temporal feature sequence output by TCN. Trans H represents the global context feature sequence output by the Transformer module, and MH-Attention(·) represents the cross-attention fusion mechanism. After the above attention calculation, two complementary and enhanced feature representations H can be obtained. T→G and H G→T , representing the local features of global information enhancement and the global features of local detail enhancement, respectively. The two enhanced features are concatenated and further fused through a nonlinear transformation to obtain the final comprehensive feature vector sequence H. fused The specific calculation process can be expressed as follows:

[0106] H fused =ReLU(W f [H T→G H G→T ]+b f )

[0107] Among them, W f ∈R 2D×D and b f ∈R D represents the learnable parameters, which can adaptively adjust the importance of different enhanced features to achieve optimal feature fusion; ReLU(·) represents the ReLU activation function.

[0108] This cross-attention mechanism dynamically integrates key information from local and global features, automatically adjusting their importance based on the feature distribution of the actual input data. It is suitable for complex scenarios requiring simultaneous consideration of local temporal patterns and global contextual information. This deep feature fusion design fully preserves TCN's sensitivity to local temporal patterns while effectively leveraging the Transformer module's advantage in capturing global long-distance dependencies, resulting in a more comprehensive and in-depth understanding of log sequences. The fused feature representation provides a more solid foundation for subsequent anomaly classification tasks, enabling more accurate identification of complex system anomaly patterns, including local event anomalies, global state anomalies, and combinations of both.

[0109] In step S600, the classifier may include a fully connected neural network, the output layer of which uses a Softmax function to generate anomaly probabilities. The classifier obtains the log anomaly detection result based on the relationship between a preset threshold and the anomaly probability. If the anomaly probability is greater than the preset threshold, the log anomaly detection result is considered anomaly. The default preset threshold can be 0.5.

[0110] To verify the effectiveness of the log anomaly detection method of this invention, log anomaly detection experiments were conducted on the HDFS dataset and compared with other state-of-the-art methods (PCA, DeepLog, LogRobust, LogAnomaly, LogBert). As shown in Table 1, the log anomaly detection method of this invention shows significant improvements in precision, recall, and F1 score. For example, the F1 score of the log anomaly detection method of this invention reaches 0.972, which is superior to methods such as LogAnomaly (F1 score 0.945) and LogBert (F1 score 0.823).

[0111] Table 1 Comparison of experimental results of different methods on the HDFS dataset.

[0112]

[0113] Furthermore, ablation experiments verified the contribution of the cross-attention mechanism. As shown in Table 2, the experimental results show that the F1 value is 0.954 when using TCN alone and 0.965 when using Transformer alone; however, when the two are combined and the cross-attention mechanism is introduced, the F1 value is further improved to 0.972, indicating the key role of this mechanism in feature fusion.

[0114] Table 2 shows the ablation experiment results on the HDFS dataset.

[0115]

[0116] In Table 2, TCN indicates that feature extraction of the fused feature vector sequence is performed solely through TCN; Transformer indicates that feature extraction of the fused feature vector sequence is performed solely through the Transformer module; TCN+Transformer indicates that feature extraction of the fused feature vector sequence is performed through both TCN and the Transformer module; TCN+Transformer+Cross-Attention indicates that feature extraction of the fused feature vector sequence is performed through both TCN and the Transformer module, and a weighted fusion of the local temporal feature sequence and the global contextual feature sequence is also performed through a cross-attention mechanism.

[0117] To verify the advantages of the cross-attention mechanism fusion, it was replaced with additive fusion, concatenation fusion, and weighted sum fusion for comparison. As shown in Table 3, the cross-attention mechanism fusion performed excellently in precision, recall, and F1 score, especially in precision, reaching 0.993, which is significantly higher than other fusion methods. Cross-attention, by establishing effective relationships and weight allocation among different features, can dynamically focus on the most valuable information, thus achieving a balance between improving precision and maintaining high recall.

[0118] Table 3 shows the experimental results of different fusion methods on the HDFS dataset.

[0119]

[0120] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any embodiment that achieves the technical effects of the present invention by the same or equivalent means should fall within the protection scope of the present invention. Within the protection scope of the present invention, various modifications and variations can be made to the technical solutions and / or implementation methods.

Claims

1. A method for detecting log anomalies, characterized in that, Includes the following steps: In response to receiving a log anomaly detection instruction, feature extraction is performed on the original log sequence indicated by the log anomaly detection instruction to obtain a semantic feature vector sequence, a latency feature vector sequence, and a level feature vector sequence; wherein, the original log sequence includes at least two original logs; The semantic feature vector sequence, the time delay feature vector sequence, and the level feature vector sequence are fused to obtain a fused feature vector sequence. The fused feature vector sequence is extracted using a temporal convolutional network to obtain a local temporal feature sequence. The global context feature sequence is obtained by extracting features from the fused feature vector sequence using the Transformer module. The local temporal feature sequence and the global contextual feature sequence are weighted and fused using a cross-attention mechanism to obtain a comprehensive feature vector sequence. The comprehensive feature vector sequence is input into the classifier, and the classifier outputs the log anomaly detection result. The step of extracting features from the original log sequence indicated by the log anomaly detection instruction to obtain a semantic feature vector sequence, a latency feature vector sequence, and a level feature vector sequence includes: The Drain algorithm is used to parse the raw log sequence and extract the log template sequence and log level sequence. The log template sequence is semantically encoded using a pre-trained BERT model to obtain a sequence of semantic feature vectors; Calculate the time difference between adjacent raw logs to obtain a sequence of delay feature vectors; The log level sequence is mapped to a numerical code to obtain a level feature vector sequence; The weighted fusion of the local temporal feature sequence and the global contextual feature sequence through a cross-attention mechanism to obtain the comprehensive feature vector sequence includes: The query vector sequence, key vector sequence, and value vector sequence of the local temporal feature sequence and the global context feature sequence are obtained by linear transformation. The attention weight matrix is ​​calculated based on the query vector sequence of the local temporal feature sequence, the key vector sequence of the global context feature sequence, and the value vector sequence of the global context feature sequence to obtain the first feature representation; The attention weight matrix is ​​calculated based on the query vector sequence of the global context feature sequence, the key vector sequence of the local temporal feature sequence, and the value vector sequence of the local temporal feature sequence to obtain the second feature representation; The first and second feature representations are concatenated and subjected to a nonlinear transformation to obtain a sequence of comprehensive feature vectors.

2. The method according to claim 1, characterized in that, The step of extracting features from the fused feature vector sequence using a temporal convolutional network to obtain a local temporal feature sequence includes: The fused feature vector sequence is input into a temporal convolutional network; Perform causal dilated convolution on the fused feature vector sequence; Weight normalization is performed on the output feature sequence of causal dilated convolution; Activate using the ReLU activation function; Perform a Dropout operation; The fused feature vector sequence is residually connected to the input to obtain a local temporal feature sequence.

3. The method according to claim 1, characterized in that, The Transformer module includes a multi-layer Transformer encoder, and each layer of the Transformer encoder includes a multi-head attention sublayer and a feedforward neural network sublayer.

4. The method according to claim 1, characterized in that, The classifier includes a fully connected neural network, the output layer of which uses a Softmax function to generate anomaly probabilities.

5. The method according to claim 4, characterized in that, The classifier obtains the log anomaly detection result based on the relationship between a preset threshold and the anomaly probability. If the anomaly probability is greater than the preset threshold, the log anomaly detection result is anomaly.

6. The method according to claim 5, characterized in that, The preset threshold is 0.

5.

7. An electronic device, characterized in that, include: At least one processor; and memory that is communicatively connected to at least one processor; The memory stores instructions that, when executed by at least one processor, implement the log anomaly detection method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The system stores instructions that, when executed by a processor, implement the log anomaly detection method as described in any one of claims 1 to 6.

Images