A log anomaly detection method based on spatio-temporal feature fusion

Abstract

The application provides a log anomaly detection method based on space-time feature fusion, and relates to the technical field of intelligent operation and maintenance of computer software systems. The method first constructs a log original sample data sequence set and forms a corresponding log template data sequence; then extracts features in the time dimension and the space dimension from the log template sequence data, and fuses the two kinds of features to obtain the final feature representation of the log template data sequence; finally, a Deep SVDD algorithm is used to train an anomaly detection model based on log space-time feature fusion, to learn and optimize a hypersphere associated with the log template sequence representation vector, and to realize log anomaly detection. The method of the application mines the time and space correlation in the log original data sequence, and further gives the system good anomaly detection accuracy, which helps to accurately capture anomalies in the software system operation mode and improve the stability of the system.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance technology for computer software systems, and in particular to a log anomaly detection method based on spatiotemporal feature fusion. Background Technology

[0002] With the rapid increase in the complexity and scale of computer systems, ensuring the reliability and stability of software systems has become more challenging. Anomaly detection, which can promptly identify and report information about program malfunctions or anomalies, is a crucial way to maintain software system security and reduce potential risks and losses. Computer software systems generate a large amount of state and event data during runtime. State data descriptions of applications or software systems are mainly expressed as specific metrics and parameters, such as network connection status, CPU utilization, memory usage, and the number of interface calls, which quantitatively describe the current state of the system. Event data descriptions mainly represent the mechanisms of system operation logic, such as module calls, database access, interface completion records, and human-computer interaction, which qualitatively describe the execution phase of system business. System runtime generates massive amounts of log files, clearly recording and describing the state and events of program runtime. Log streams support fine-grained fault diagnosis and localization, enabling intuitive and thorough discovery of the causes of system anomalies. Almost all software systems have logging functionality, and the strong versatility ensures the stability of the original data source. This massive amount of data becomes a crucial factor in log detection, and log-based system anomaly detection is a widely adopted solution in intelligent operation and maintenance within the industry.

[0003] In recent years, with the development of semantic vector embedding methods, many log anomaly detection methods have applied language models or semantic vectors to encode log statements. For example, the LogAnomaly method uses the Word2Vec vector encoding model and employs word vector averaging to embed log statements into vectors, thereby calculating the temporal features of the log stream. LogRobust and PLELog use the Globe vector encoding model, calculating the weight of word frequency occurrences through TF-IDF to clarify the importance of certain words in the log statement, and then using a weighted average method to calculate the vector value of the log statement. Methods such as LogBert use conventional methods such as BERT pre-trained models to encode the vector information of log statements, learning contextual relationships by masking key statements, and then deriving the vector information of log templates, or directly calculating the feature information of commonly used log templates through sentence vector BERT pre-trained models, which are then directly applied to downstream anomaly detection model learning tasks.

[0004] Currently, unsupervised deep learning solutions that learn the characteristics of normal log stream patterns occupy an important position in the field. The general framework involves parsing log events, extracting features from them using word embedding or sentence embedding methods, inputting the log event sequence into a corresponding neural network model for further information extraction, and finally updating the parameters of the overall network model based on the appropriate loss function such as SVDD or TOP K. Mainstream models include GNN, RNN, and Transformer, with representative solutions such as DeepLog, LogAnomaly, and DeepTraLog. The drawback of this mainstream approach is that its feature extraction methods for log sequence events are still relatively limited, mostly considering only temporal or spatial patterns, failing to fully integrate the two.

[0005] The development of log anomaly detection algorithms has gone through three main stages. The initial stage focused on traditional log anomaly detection methods based on manual analysis or keyword matching. This gradually evolved into machine learning-based log anomaly detection methods, and in recent years, multi-feature fusion log anomaly detection methods based on deep learning have been introduced. However, as the scale and complexity of computer software systems have gradually increased, and the content of operation and maintenance work has also been continuously updated, the problems faced by methods at each stage of practical application have become increasingly complex.

[0006] Traditional log anomaly detection methods typically involve system administrators manually analyzing logs, using timestamps or keywords to pinpoint the exact location of abnormal log events, and then relaying this information to relevant developers or administrators for diagnosis and repair. This approach requires personnel with strong domain knowledge and a deep understanding of the entire software system to quickly locate and analyze the root causes. Manual analysis or relying on key feedback often results in a poor customer experience and can lead to significant losses.

[0007] Traditional supervised machine learning methods typically use a log event counting template matrix as the basic data, employing methods such as SVM and PCA to learn the inherent quantitative relationships. In the implementation of a closed system, this approach can detect abnormal log event flows (behaviors). The main drawback of this approach is that the quantitative relationships of log events still cannot fully represent the inherent patterns of system operation, failing to capture the temporal, spatial, and semantic relationships of the log stream. Furthermore, the data features used are relatively limited, making it unsuitable for the needs of software system updates. Moreover, supervised methods are difficult to adapt to the mainstream concepts of intelligent operation and maintenance.

[0008] Supervised deep learning methods can effectively learn the temporal and semantic features of log streams. By designing corresponding loss functions based on ground truth or probabilistic labels and continuously updating the neural network parameters, good anomaly detection results are achieved. Representative solutions include LogRobust and PLELog. Most methods of this type improve upon the semantic feature extraction of log events and employ recurrent neural network models such as LSTM or GRU to deeply extract the temporal features of the log stream. The main drawback of this type of approach is its failure to fully explore the spatial patterns of the log stream. Supervised methods require a significant amount of manual annotation, and probabilistic label-based supervision has a relatively coarse granularity in learning log sequence patterns.

[0009] Feature learning of spatiotemporal patterns is not simply about piecing together neural network models. Instead, it involves transforming the log event sequence into a graph structure, making it a true graph, in order to better learn the normal patterns of the log stream and enhance the effectiveness of anomaly detection. Summary of the Invention

[0010] The technical problem to be solved by the present invention is to provide a log anomaly detection method based on spatiotemporal feature fusion to address the shortcomings of the prior art and realize log anomaly detection.

[0011] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a log anomaly detection method based on spatiotemporal feature fusion, comprising the following steps:

[0012] Step 1: Construct a set of raw sample data sequences of logs under the normal operating mode of the computer software system;

[0013] Extract several raw log stream data samples from the historical operation log database of the computer software system, and divide the stream data into sequence segments using a sliding window or session window to construct a set of raw log sample data sequences under the normal operation mode of the system.

[0014] Step 2: Generate corresponding log template data sequences from the original log sample data sequences through parsing operations;

[0015] Each log entry in the original log sample data sequence is parsed to extract relevant parameters and keywords, resulting in a log template for each original log entry. Thus, the original log sample data sequence is transformed into a corresponding log template data sequence, Log. T_seq =(Log T_1 Log T_2 ,…,Log T_n ), where Log T_1 This represents the first log template in the log template data sequence.

[0016] Step 3: Use a semantic pre-trained model as a sentence vector converter to embed semantic vectors into the original log template data sequence to obtain a log template vector sequence that covers semantic digital information;

[0017] Step 3-1: Treat each log template in the log template data sequence as a statement composed of concatenated words, denoted as (token). k,1 token k,2 ,…,token k,m ), token k,i This represents the i-th word in the k-th log template in the log template data sequence, where i = 1, 2, ..., m, and m is the total number of words in the log template;

[0018] Step 3-2: Add special markers to the word sequences of the log template to facilitate feature reading by the model, denoted as tokens = [[CLS], token k,1 ,…,token k,m [SEP]], input the statement of the log template into the Siamese network encoder layer of the semantic pre-trained model, and output the vector representation N of each token. Input the vector representation of all words or tokens in the log template into the pooling layer network to obtain the output representation u,v of the Siamese network respectively.

[0019] Step 3-3: Combining the output representations u and v of the semantic pre-trained Siamese network, according to formula W... t *(u,v,|uv|) calculates the vector representation V of the k-th log template in the log template sequence. k This leads to the vector representation V of the log template sequence. E =(V1,V2,…,V n ), W in the formula t The weight parameter matrix represents the Siamese network in the semantic pre-training model. It is used to adjust the connection strength between different neurons in the model network to adapt to and learn the log template sequence data, and then calculate the vector representation of the entire log template sequence.

[0020] Step 4: Construct a time anomaly detection model using an LSTM network model to extract time dimension features from log template sequence data; use the log template vector sequence representation containing semantic digital information obtained in Step 3 as input to the time anomaly detection model, and calculate the time dimension feature vector representation T of the log template vector sequence containing semantic digital information through the time anomaly detection model to help iterate the parameters of the LSTM network model;

[0021] Step 5: Obtain the spatial node vector representation of each log template in the log template data sequence based on the shared attention mechanism;

[0022] Different types of log templates in the log template data sequence are extracted separately according to their positions in the sequence, and position vectors are added. The different types of log templates are then input into the shared attention mechanism to obtain the spatial node vector representation of each log template in the log template data sequence.

[0023] Step 6: Calculate the attention similarity matrix of spatial nodes for each log template to determine the edge set relationship of spatial nodes in the log template data sequence;

[0024] A multi-head attention network is used to calculate the attention similarity of spatial node vectors of all log templates in the log sequence, and the attention similarity matrix of spatial nodes is obtained. Based on the matrix, a dynamic threshold is set to calculate the spatial association between log templates, that is, to obtain the spatial node edge set relationship of the log template data sequence.

[0025] Step 7: Construct the graph structure corresponding to the log template sequence data based on the spatial node vector representation of the log template data sequence generated in Step 5 and the spatial node edge set relationship of the log template data sequence generated in Step 6;

[0026] Step 8: Use the GAT graph neural network model to construct a spatial anomaly detection model to extract spatial dimension features from the log template data sequence;

[0027] The graph structure corresponding to the log template data sequence obtained in step 7 is input into the GAT graph neural network. The feature representation of the spatial nodes is continuously updated by fusing the long-distance information of nodes and edges, thereby updating the parameter information of the spatial anomaly detection model. A pooling network is used to post-process the output of the GAT graph neural network to further fuse the node information and structural information of the entire graph, thereby obtaining the spatial feature vector S of the graph corresponding to the log template data sequence.

[0028] Step 9: Fuse the time dimension features and spatial dimension features of the log template data sequence to obtain the final feature representation of the log template data sequence;

[0029] The feature vector T obtained in step 4 and the feature vector S obtained in step 8 are fused by lateral feature concatenation to obtain the spatiotemporal feature vector ST. The fused spatiotemporal feature vector is then passed through a fully connected neural network to obtain the final feature representation STF of the log template data sequence.

[0030] Step 10: Train an anomaly detection model based on log spatiotemporal feature fusion using the deep support vector data description algorithm, and learn and optimize a hypersphere of associated log template sequence representation vector; the anomaly detection model based on log spatiotemporal feature fusion includes a temporal anomaly detection model and a spatial anomaly detection model;

[0031] Step 11: Train the anomaly detection model based on log spatiotemporal feature fusion and deploy it to the computer software system for log anomaly detection;

[0032] Normal log stream data is trained using temporal and spatial anomaly detection models. The parameters of the relevant network models are updated, and the trained models are deployed to the computer software system. Whenever a new log stream is generated, features are extracted from the log stream data according to steps 2-9 to obtain a vector representation of the log template sequence. This vector is then mapped to a coordinate point in a multidimensional computational space, and the distance from this coordinate point to the hypersphere is calculated as the log anomaly score. The specific calculation method is as follows:

[0033] ans(STF) = ||STF - c|| 2 -R 2

[0034] Where c represents the center of the hypersphere, R represents the radius of the sphere, indicating the size of the spherical space the model attempts to learn, and ans(STF) represents the target value of the loss function used to train the anomaly detection model. This value indicates whether a sample is inside the spherical space. By minimizing this loss function, the log anomaly detection model based on spatiotemporal features can learn the optimal values ​​of c and R to better include normal samples. If the coordinate point is inside the hypersphere, the corresponding log template sequence data is determined to be a normal log stream; if the coordinate point is outside the hypersphere, the corresponding log template sequence data is determined to be an abnormal log stream.

[0035] The beneficial effects of adopting the above technical solution are as follows: The present invention provides a log anomaly detection method based on spatiotemporal feature fusion. (1) The present invention proposes a graph structure construction method based on log stream. First, the features of each type of log template are extracted as the point set of the graph according to the shared attention mechanism technology. Then, the relationship between the templates is calculated according to the attention network to form an attention similarity matrix. Then, the edge set of the graph is calculated according to the threshold. The graph structure of the log stream is formed according to the point set and the edge set.

[0036] (2) This invention proposes a novel method for fusion of spatiotemporal features of log streams. In order to learn the working mode of log streams, this invention uses a graph attention network to extract features from the formed graph structure based on log streams, mining the spatial information of log streams, and uses a recurrent neural network to extract features from the temporal relationships of log streams, mining the temporal information of log streams. By fusing spatiotemporal information, the spatiotemporal features of log streams are obtained.

[0037] (3) This invention proposes a fine-tuning method based on deep support vector data description. By using Deep SVDD to calculate and minimize the spherical loss, the normal logs and abnormal logs of the log stream are separated. At the same time, the boundary is continuously adjusted through network learning to maximize the separation of the relationship between the normal and abnormal log working modes, while ensuring that the normal log data is aggregated in the same sphere, thus ensuring the effectiveness and stability of log anomaly detection.

[0038] (4) The method of this invention not only utilizes the temporal feature information of the log stream, but also the semantic vector information of the log statements and the spatial pattern information of the log stream. A shared attention mechanism is used to construct the point set of the log stream graph, while the edge set of the log stream graph is constructed using methods such as calculating the attention similarity matrix and setting thresholds. This allows for better acquisition and updating of the graph structure information of the log stream. Based on this, a spatial and temporal feature fusion strategy for the log stream is proposed to enhance the learning of the log stream's working pattern features, thereby improving the recognition effect of the log anomaly detection algorithm. Fusion of the spatiotemporal features of the log stream can increase the understanding of the complex relationships between logs and the learning of working patterns. By using Deep SVDD to minimize the spherical loss method, the detection results of log anomaly detection can be fine-tuned. Compared with the TOP K mode, it can update the internal parameters of the algorithm with finer granularity, which has a positive impact on the practicality of the log anomaly detection method.

[0039] For the first time, log template sequences are converted into graph structures. By utilizing a shared attention mechanism and an attention similarity matrix, the vertex and edge sets of the graph structure are extracted, enhancing the integrity and stability of spatial feature representation.

[0040] This invention extracts spatial and temporal features of log sequences using graph neural networks and attention encoders, respectively, and fuses the feature representations of the two attributes. This attribute fusion mechanism can enhance the vector representation of log sequences, uncover the temporal and spatial correlations in the original log data sequence, thereby giving the system good anomaly detection accuracy, helping to accurately capture anomalies in the operating mode of the software system, and improving the stability of the system.

[0041] This invention introduces a loss calculation method for Deep SVDD, which uses the minimization of spherical loss to perform hyperspherical fusion of the spatiotemporal features of normal log sequences, ensuring the determinism and robustness of the log anomaly detection system in identifying anomalies, and enabling fine-grained adjustment of the detection effect within the system. Attached Figure Description

[0042] Figure 1 A schematic diagram of the framework of a log anomaly detection method based on spatiotemporal feature fusion provided in an embodiment of the present invention;

[0043] Figure 2This is a schematic diagram illustrating the process of the log anomaly detection method based on spatiotemporal feature fusion provided in an embodiment of the present invention. Detailed Implementation

[0044] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0045] This embodiment takes a computer software system as an example and uses the log anomaly detection method based on spatiotemporal feature fusion of the present invention to detect anomalies in the logs of the software system.

[0046] In this embodiment, a log anomaly detection method based on spatiotemporal feature fusion is described, such as... Figure 1 , 2 As shown, it includes the following steps:

[0047] Step 1: Construct a set of raw sample data sequences of logs under the normal operating mode of the computer software system;

[0048] Extract several raw log stream data samples from the historical operation log database of the computer software system, and divide the stream data into sequence segments using a sliding window or session window to construct a set of raw log sample data sequences under the normal operation mode of the system.

[0049] In this embodiment of the invention, the original log stream data consists of 11,175,629 log messages with corresponding IDs from a certain computer software system. Experts evaluate each log message or its ID. When creating the actual log sequence dataset, relevant segments are extracted based on the ID information of the log messages, and the segments are divided into sequences using a sliding window to form a log sequence for practical application learning.

[0050] Step 2: Generate corresponding log template data sequences from the original log sample data sequences through parsing operations;

[0051] Each log entry in the original log sample data sequence is parsed to extract relevant parameters and keywords, resulting in a log template for each original log entry. Thus, the original log sample data sequence is transformed into a corresponding log template data sequence, Log. T_seq =(Log T_1 Log T_2 ,…,Log T_n ), where Log T_1 This represents the first log template in the log template data sequence.

[0052] In this embodiment of the invention, a parser is used to parse log messages. Template extraction of log messages is achieved by relying on regular expressions, log message length, token design, and updating the parse tree, allowing the sequential acquisition of the log template sequence Log. T_seq =(Log T_1 Log T_2 ,…,Log T_n );

[0053] Step 3: Use the semantic pre-trained model Sentence-Bert as a sentence vector converter to embed semantic vectors into the original log template data sequence to obtain a log template vector sequence that covers semantic digital information;

[0054] Step 3-1: Treat each log template in the log template data sequence as a statement composed of concatenated words, denoted as (token). k,1 token k,2 ,…,token k,m ), token k,i This represents the i-th word in the k-th log template in the log template data sequence, where i = 1, 2, ..., m, and m is the total number of words in the log template;

[0055] Step 3-2: Add special markers to the word sequences of the log template to facilitate feature reading by the model, denoted as tokens = [[CLS], token k,1 ,…,token k,m [SEP]], The log template statement is input into the Siamese network encoder layer of the pre-trained Sentence-Bert model, and the output is the vector representation N of each token. In the Siamese network encoder layer, each word or token is mapped to a multi-dimensional vector N. This vector captures the representation of the word or token in the semantic space, enabling the model to better understand semantic relationships. The vector representations of all words or tokens in the log template are input into the pooling layer network to obtain the output representations u and v of the Siamese network respectively.

[0056] Step 3-3: Combining the output representations u,v of the pre-trained Sentence-Bert Siamese network, according to formula W... t *(u,v,|uv|) calculates the vector representation V of the k-th log template in the log template sequence. k This leads to the vector representation V of the log template sequence. E =(V1,V2,…,V n ), W in the formula tThe weight parameter matrix represents the Siamese network in the Sentence-Bert model. It is used to adjust the connection strength between different neurons in the model network to adapt to and learn the log template sequence data, and then calculate the vector representation of the entire log template sequence.

[0057] Step 4: Construct a time anomaly detection model using an LSTM network model to extract time dimension features from log template sequence data; use the log template vector sequence representation containing semantic digital information obtained in Step 3 as input to the time anomaly detection model, and calculate the time dimension feature vector representation T of the log template vector sequence containing semantic digital information through the time anomaly detection model to help iterate the parameters of the LSTM network model;

[0058] The feature extraction derivation formula for the LSTM network model is as follows:

[0059]

[0060] Among them, W xi W xf W xo and W hi W hf W ho All are weight parameters of the LSTM network model, b i b f b o All are bias parameters; X t This represents the input data of the LSTM network model at time step t. Its actual input is a vector representation V of the log template sequence. k H t-1 This represents the hidden state at the previous time step (t-1), which contains information observed by the LSTM network model in past time steps and serves as the network input at the current time step (t); t F t O t These are the weights of the input gate, forget gate, and output gate of the LSTM network model, respectively, which control the input, forgetting, and output of information.

[0061] The parameters b are learned based on the bias terms of the LSTM network model. c The weight matrix parameters W of the input gate xc and W hc The weight value I of the input gate t The weight value F of the forget gate t The weight value O of the output gate t Calculate the candidate memory units of the LSTM network model respectively and memory unit C t The eigenvalues ​​are shown in the following formula:

[0062]

[0063] in, C is a candidate memory unit at time step t, representing new information that can be added to the memory unit. t For the characteristic value of the memory cell, the formula essentially uses the previous memory cell state information C. t-1 and candidate memory cell state information Combined, by the forget gate weight value F t and input gate weights I t Controls are in place: the forget gate determines how much of the previous memory state is retained, and the input gate determines how much new information is added, according to C. t The value can be used to further calculate the hidden state output features of the LSTM network model, as shown in the following formula:

[0064] H t =O t ⊙tanh(C t )

[0065] Among them, H t It is the hidden state output generated when the log template sequence is input into the LSTM network. This output condenses the temporal and textual semantics of the log stream. Then, it is passed through a fully connected network to obtain the final temporal dimension feature T of the log template sequence.

[0066] Step 5: Obtain the spatial node vector representation of each log template in the log template data sequence based on the shared attention mechanism;

[0067] Different types of log templates in the log template data sequence are extracted separately according to their positions in the sequence, and position vectors are added. The different types of log templates are then input into the shared attention mechanism (TransformerEncoder) to obtain the spatial node vector representation of each log template in the log template data sequence.

[0068] In this embodiment, the log template sequence is set as X = [x1, x2, ..., x n ], where x i Let H represent the i-th template in the sequence. Based on different template types, we perform classification and extraction to obtain the position information and textual semantic information H of each template in the log sequence. i The formula for combining the semantic information of the two is as follows:

[0069] Position_Semantic(H i )=[h1+p1,h2+p2,…,h n +p n ]

[0070] Where, p i For template x i The relevant position vector, h i For template x i The corresponding textual semantic information is processed by the Transformer Encoder with a shared attention mechanism to obtain a spatial node vector representation of the sequence of each log template.

[0071] Z=Transformer_Encoder(Position_Semantic(H i ))

[0072] Here, Z represents the spatial node vector representation of a log template, and there are multiple log templates in the log template sequence, so the Z node vector of each template needs to be generated in parallel.

[0073] Step 6: Calculate the attention similarity matrix of spatial nodes for each log template to determine the edge set relationship of spatial nodes in the log template data sequence;

[0074] A multi-head attention network is used to calculate the attention similarity of spatial node vectors of all log templates in the log sequence, and the attention similarity matrix of spatial nodes is obtained. Based on the matrix, a dynamic threshold is set to calculate the spatial association between log templates, that is, to obtain the spatial node edge set relationship of the log template data sequence.

[0075] In this embodiment, when the multi-head attention network calculates the spatial node vector of the log template, it uses a relevant formula to calculate the similarity between nodes:

[0076]

[0077] The weights are calculated by performing similarity calculations on Q and each K, i.e., f(Q,K). i The available similarity functions generally include dot product and concatenation. The softmax function is used to normalize this similarity weight matrix, and the calculation method is as follows:

[0078]

[0079] Set the threshold for α i By constraining the similarity matrix, the connection relationships between nodes are calculated, thereby obtaining the spatial node edge set of the log template sequence.

[0080] Step 7: Construct the graph structure corresponding to the log template sequence data based on the spatial node vector representation of the log template data sequence generated in Step 5 and the spatial node edge set relationship of the log template data sequence generated in Step 6;

[0081] In this embodiment, the graph structure corresponding to the constructed log template sequence data is shown in the following formula:

[0082] G i =f(α) i ,V)

[0083] Where V represents the node feature information of the log template sequence graph, G i The graph data features representing the log template sequence can be used to mine spatial information through graph neural networks;

[0084] Step 8: Use the GAT graph neural network model to construct a spatial anomaly detection model to extract spatial dimension features from the log template data sequence;

[0085] The graph structure corresponding to the log template data sequence obtained in step 7 is input into the GAT graph neural network. The feature representation of the spatial nodes is continuously updated by fusing the long-distance information of nodes and edges, thereby updating the parameter information of the spatial anomaly detection model. A pooling network is used to post-process the output of the GAT graph neural network to further fuse the node information and structural information of the entire graph, thereby obtaining the spatial feature vector S of the graph corresponding to the log template data sequence.

[0086] The GAT network allocates attention to the set of neighboring nodes of node i, and uses first-order neighboring nodes for fusion to update the node's feature vector. The calculation formula is as follows:

[0087]

[0088] After attention update, the features of node i are calculated as follows:

[0089]

[0090] Based on the characteristics of all nodes in the global mean pooling fusion graph, spatial characteristics of the log template sequence are formed.

[0091] Step 9: Fuse the time dimension features and spatial dimension features of the log template data sequence to obtain the final feature representation of the log template data sequence;

[0092] The feature vector T obtained in step 4 and the feature vector S obtained in step 8 are fused by lateral feature concatenation to obtain the spatiotemporal feature vector ST. The fused spatiotemporal feature vector is then passed through a fully connected neural network to obtain the final feature representation STF of the log template data sequence.

[0093] Step 10: Train an anomaly detection model based on log spatiotemporal feature fusion using the Deep Support Vector Data Description algorithm (Deep SVDD) to learn and optimize a hypersphere of associated log template sequence representation vectors; the anomaly detection model based on log spatiotemporal feature fusion includes a temporal anomaly detection model and a spatial anomaly detection model.

[0094] This study aims to enable an anomaly detection model based on log spatiotemporal feature fusion to learn an effective log sequence representation vector. A hypersphere, a convex spherical surface, is learned and optimized using deep support vector data description (DSV) algorithms to enclose normal samples, forming a compact region within the vector embedding space. The center of the hypersphere represents the concentration tendency of normal samples in the vector embedding space, while the radius represents the typical distance of normal samples within this space. During training, an optimal hypersphere is learned through DSV to enclose as many normal samples as possible while repelling anomalies. By confining normal samples within this hypersphere, the model's ability to capture normal system behavior is enhanced, ensuring that most normal log sample data clusters near the center of the hypersphere.

[0095] Step 11: Train the anomaly detection model based on log spatiotemporal feature fusion and deploy it to the computer software system for log anomaly detection;

[0096] Normal log stream data is trained using temporal and spatial anomaly detection models. The parameters of the relevant network models are then updated, and the trained models are deployed to the computer software system. Whenever a new log stream is generated, features are extracted from the log stream data according to steps 2-9 to obtain a vector representation of the log template sequence. This vector is then mapped to a coordinate point in a multidimensional computational space. This coordinate point is a feature vector output by the anomaly detection model from the log template sequence data, which can be represented as a coordinate point in the multidimensional computational space. The distance from this coordinate point to the hypersphere is then calculated as the log anomaly score. The specific calculation method is as follows:

[0097] ans(STF) = ||STF - c|| 2 -R 2

[0098] Where c represents the center of the hypersphere. In the spherical loss calculation, the model attempts to learn a spherical space such that most samples are inside the sphere. The learning process of c is to find the optimal center position of the sphere. R represents the radius of the sphere, indicating the size of the spherical space that the model is trying to learn. ans(STF) represents the target value of the loss function, used to train the anomaly detection model. This value indicates whether a sample is inside the spherical space. By minimizing this loss function, the log anomaly detection model based on spatiotemporal features can learn the optimal values ​​of c and R to better include normal samples. If the coordinate point is inside the hypersphere, the corresponding log template sequence data is judged to be a normal log stream. If the coordinate point is outside the hypersphere, the corresponding log template sequence data is judged to be an abnormal log stream, and the computer software system will issue an alarm to remind the operation and maintenance personnel.

[0099] This embodiment also conducts tests and comparison experiments on multiple publicly available log datasets to verify the effectiveness of the method and the improvement in log anomaly detection. This embodiment selects the most advanced methods in the field of log anomaly detection, both domestically and internationally, for performance comparison, mainly including the time-series-based DeepLog method, the time-series and semantic-based LogAnomaly method, and the spatial feature-based DeepTraLog method. The DeepLog method primarily uses the sequence information of log templates for unsupervised time-series prediction. It learns existing sequence patterns to predict the type of the next log template, and then uses a top-K method to determine whether the actual data matches the predicted data. If they do not match, it indicates that the actual log stream data violates the normal working pattern. The LogAnomaly method integrates the semantic information of the logs into the time series and also adds a template counting matrix to assist in learning log patterns. The prediction method also primarily uses a top-K approach. The DeepTraLog method constructs a graph structure for the log sequence, learns its spatial features through a GGRU network, and uses the Deep SVDD method for prediction training. The comparative experiments in this embodiment selected these three methods, which are log anomaly detection methods based on a combination of temporal, semantic, or spatial features. These methods are widely representative, and comparison with them can fully demonstrate the effectiveness of the present invention in this regard.

[0100] In this embodiment, the evaluation metrics used in the experimental evaluation process are based on the traditional classification method metrics, mainly precision, recall, and F1 score. Precision evaluates the number of true anomalies among the log stream anomalies detected by the model; recall measures the ratio of anomaly samples detected by the model to actual anomaly samples; and the F1 score combines the geometric mean of precision and recall to comprehensively measure the model's detection performance. The formula for calculating the F1 score is as follows:

[0101]

[0102] Table 1 shows the comparison of the log anomaly detection performance of the method of this invention on datasets such as HDFS and BGL with other methods, illustrating the interference and impact of different semantic embedding modules on the log anomaly detection performance. Compared with the spatial information feature learning of DeepTraLog and the temporal information feature learning of LogAnomaly, the log anomaly detection model combining spatiotemporal information feature extraction has better feature performance to achieve good results. Contextual statement vector embedding not only improves the ability to resist noise, but also helps to enhance the frame detection effect. The Deep SVDD calculation method, which minimizes the spherical loss, can improve the stability of the frame in a fine-grained manner. On public datasets, the method of this invention has a significant performance improvement over existing unsupervised methods. Compared with traditional supervised or semi-supervised methods, it can greatly reduce the workload of manual labeling and improve the efficiency of log anomaly detection.

[0103] Table 1 Comparison results of different detection methods on different datasets

[0104]

[0105]

[0106] The method of this invention shows better improvement in accuracy, recall and F1 score on HDFS and BGL datasets compared with other time-, space- or semantic-based methods. This further illustrates that the STF-Log method, which minimizes the spherical loss based on spatiotemporal features, can not only mine the features and patterns of log streams, but also has good advantages in fine-tuning the effect.

[0107] This invention proposes a log anomaly detection framework for extracting spatiotemporal information from log sequences, addressing the problem of poor learning performance for normal patterns in log sequences. It proposes a shared attention mechanism to effectively extract spatial features of log template events, solving the problem of feature fusion for identical events in current log sequences. A method for calculating the log event attention similarity matrix is ​​designed, solving the problem of extracting effective edge sets for log event spatial nodes. Minimizing spherical loss calculation is introduced to address the issues of coarse-grainedness and large fluctuations in traditional top-K prediction methods. The effectiveness of this invention in log anomaly detection is verified through comparison with publicly available datasets.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A log anomaly detection method based on spatiotemporal feature fusion, characterized in that: Includes the following steps: Step 1: Construct a set of raw sample data sequences of logs under the normal operating mode of the computer software system; Step 2: Generate corresponding log template data sequences from the original log sample data sequences through parsing operations; Step 3: Use a semantic pre-trained model as a sentence vector converter to embed semantic vectors into the original log template data sequence to obtain a log template vector sequence that covers semantic digital information; Step 4: Use an LSTM network model to construct a time anomaly detection model to extract time dimension features from the log template sequence data; Step 5: Obtain the spatial node vector representation of each log template in the log template data sequence based on the shared attention mechanism; Step 6: Calculate the attention similarity matrix of spatial nodes for each log template to determine the edge set relationship of spatial nodes in the log template data sequence; Step 7: Construct the graph structure corresponding to the log template sequence data based on the spatial node vector representation of the log template data sequence generated in Step 5 and the spatial node edge set relationship of the log template data sequence generated in Step 6; Step 8: Use the GAT graph neural network model to construct a spatial anomaly detection model to extract spatial dimension features from the log template data sequence; Step 9: Fuse the time dimension features and spatial dimension features of the log template data sequence to obtain the final feature representation of the log template data sequence; Step 10: Train an anomaly detection model based on log spatiotemporal feature fusion using the Deep Support Vector Data Description algorithm (Deep SVDD) to learn and optimize a hypersphere of associated log template sequence representation vectors; the anomaly detection model based on log spatiotemporal feature fusion includes a temporal anomaly detection model and a spatial anomaly detection model. Step 11: Train the anomaly detection model based on log spatiotemporal feature fusion and deploy it to the computer software system for log anomaly detection.

2. The log anomaly detection method based on spatiotemporal feature fusion according to claim 1, characterized in that: Step 1 extracts several original log stream data as samples from the historical operation log database of the computer software system, and divides the stream data into sequence segments using a sliding window or session window to construct a set of original log sample data sequences under the normal operation mode of the system.

3. The log anomaly detection method based on spatiotemporal feature fusion according to claim 2, characterized in that: The specific method for step 2 is as follows: Each log entry in the original log sample data sequence is parsed to extract relevant parameters and keywords, resulting in a log template for each original log entry. Thus, the original log sample data sequence is transformed into a corresponding log template data sequence, Log. T_seq =(Log T_1 Log T_2 ,…,Log T_n ), where Log T_1 This represents the first log template in the log template data sequence.

4. The log anomaly detection method based on spatiotemporal feature fusion according to claim 3, characterized in that: The specific method for step 3 is as follows: Step 3-1: Treat each log template in the log template data sequence as a statement composed of concatenated words, denoted as (token). k，1 token k，2 , ..., token k，m ), token k，i This represents the i-th word in the k-th log template in the log template data sequence, where i = 1, 2, ..., m, and m is the total number of words in the log template; Step 3-2: Add special markers to the word sequences of the log template to facilitate feature reading by the model. This is denoted as tokens = [[CLS], token k，1 , ..., token k，m [SEP]], The statements in the log template are input into the Siamese network encoder layer of the semantic pre-trained model, and the vector representation N of each token is output. The vector representations of all words or tokens in the log template are input into the pooling layer network to obtain the output representations u and v of the Siamese network respectively. Step 3-3: Combining the output representations u and v of the semantic pre-trained Siamese network, according to formula W... t *(u, v, |uv|) calculates the vector representation V of the k-th log template in the log template sequence. k This leads to the vector representation V of the log template sequence. E = (V1, V2, ..., V n ), W in the formula t The weight parameter matrix represents the Siamese network in the semantic pre-trained model. It is used to adjust the connection strength between different neurons in the model network to adapt to and learn the log template sequence data, and then calculate the vector representation of the entire log template sequence.

5. The log anomaly detection method based on spatiotemporal feature fusion according to claim 4, characterized in that: Step 4 takes the log template vector sequence representation containing semantic digital information obtained in step 3 as the input of the time anomaly detection model, and calculates the time dimension feature vector representation T of the log template vector sequence containing semantic digital information through the time anomaly detection model to help iterate the parameters of the LSTM network model.

6. The log anomaly detection method based on spatiotemporal feature fusion according to claim 5, characterized in that: The specific method for step 5 is as follows: Different types of log templates in the log template data sequence are extracted separately according to their positions in the sequence, and position vectors are added. The different types of log templates are then input into the shared attention mechanism to obtain the spatial node vector representation of each log template in the log template data sequence.

7. The log anomaly detection method based on spatiotemporal feature fusion according to claim 6, characterized in that: The specific method for step 6 is as follows: A multi-head attention network is used to calculate the attention similarity of spatial node vectors of all log templates in the log sequence, and the attention similarity matrix of spatial nodes is obtained. Based on the matrix, a dynamic threshold is set to calculate the spatial association between log templates, that is, to obtain the spatial node edge set relationship of the log template data sequence.

8. The log anomaly detection method based on spatiotemporal feature fusion according to claim 7, characterized in that: The specific method for step 8 is as follows: The graph structure corresponding to the log template data sequence obtained in step 7 is input into the GAT graph neural network. The feature representation of the spatial nodes is continuously updated by fusing the long-distance information of nodes and edges, thereby updating the parameter information of the spatial anomaly detection model. A pooling network is used to post-process the output of the GAT graph neural network, further fusing the node information and structural information of the entire graph, so as to obtain the spatial feature vector S of the graph corresponding to the log template data sequence.

9. The log anomaly detection method based on spatiotemporal feature fusion according to claim 8, characterized in that: In step 9, the feature vector T of the time dimension obtained in step 4 and the feature vector S of the spatial dimension obtained in step 8 are fused by the feature lateral concatenation method to obtain the spatiotemporal feature vector ST. The fused spatiotemporal feature vector is then passed through a fully connected neural network to obtain the final feature representation STF of the log template data sequence.

10. The log anomaly detection method based on spatiotemporal feature fusion according to claim 9, characterized in that: The specific method for step 11 is as follows: Normal log stream data is trained using temporal and spatial anomaly detection models. The parameters of the relevant network models are updated, and the trained models are deployed to the computer software system. Whenever a new log stream is generated, features are extracted from the log stream data according to steps 2-9 to obtain a vector representation of the log template sequence. This vector is then mapped to a coordinate point in a multidimensional computational space, and the distance from this coordinate point to the hypersphere is calculated as the log anomaly score. The specific calculation method is as follows: ans(STF)=||STF-c|| 2 -R 2 Where c represents the center of the hypersphere, R represents the radius of the sphere, indicating the size of the spherical space the model attempts to learn, and ans(STF) represents the target value of the loss function used to train the anomaly detection model. This value indicates whether a sample is inside the spherical space. By minimizing this loss function, the log anomaly detection model based on spatiotemporal features can learn the optimal values ​​of c and R to better include normal samples. If the coordinate point is inside the hypersphere, the corresponding log template sequence data is determined to be a normal log stream; if the coordinate point is outside the hypersphere, the corresponding log template sequence data is determined to be an abnormal log stream.

Images