A log anomaly detection device based on component subsequence correlation perception

Abstract

The present invention discloses a log anomaly detection device based on component subsequence correlation perception, which belongs to the field of intelligent operation and maintenance and deep learning technology. The device comprises the following steps: dividing a portion of a log sequence S as a sample, obtaining a time feature embedding representation t and a semantic feature embedding representation V, and obtaining a time feature embedding representation t. c and semantic feature embedding representation V c Based on the proposed method, the system undergoes feature extraction, sequence dependency capture module, latent relation encoding module, graph convolution module, subsequence fusion module, embedding representation concatenation module, and log template prediction module to obtain the probability distribution of the log template in the next log message. All log templates are sorted and the prediction result is the top-k ranked log template set. The prediction result of the next real log template and sample is used to determine whether the log sequence has an anomaly. This effectively captures anomaly information and improves detection efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the field of intelligent operation and maintenance and deep learning technology, and in particular to a log anomaly detection device based on component subsequence correlation perception. Background Art

[0002] Today, numerous software systems provide online services to users worldwide. System failures not only impact user experience but can also cause significant financial losses. Therefore, timely and accurate anomaly detection is crucial to ensuring system operation. To facilitate the identification and diagnosis of potential anomalies, software systems generate logs. Log data, consisting of a chronological series of unstructured or semi-structured data, describes the system's operating status and key events. This helps operations personnel quickly determine system anomalies and identify the causes of failures, providing rich data support for anomaly detection research. As system scale grows, the number of logs generated by systems increases. Furthermore, the development and application of distributed software systems has increased the complexity of system logs, making manual inspection and location of anomalies using methods such as keywords or regularization more difficult. Consequently, automated log anomaly detection methods have become increasingly necessary.

[0003] Currently, automated log anomaly detection methods fall into two main categories. The first category is traditional methods, which can be further divided into those based on supervised learning and those based on unsupervised or self-supervised learning. Supervised learning methods classify log sequences by extracting statistical features such as the number of templates or keyword distribution in log data. However, these methods require labeled data, making them difficult to apply in practice. While unsupervised or self-supervised learning methods do not require labels, they cannot capture long-term dependencies in log sequences.

[0004] The second category of methods is based on deep neural networks. These methods extract the number, semantic, or temporal features of log templates in log sequences and combine them with sequence dependencies to capture log patterns that reflect normal or abnormal system states. To capture more complex log patterns, some methods incorporate component sequences, auxiliary datasets, or multi-scale designs, incorporating sequence dependencies to enhance the model's ability to perceive system operating conditions. In addition to recurrent neural networks, other deep neural networks have also been used to address various challenges in anomaly detection. For example, introducing generative adversarial networks to obtain new log data can enhance the model's ability to capture anomaly information. To model the temporal correlation of log sequences, log templates are arranged into matrices and convolutional neural networks and their variants are introduced to capture temporal dependencies in log sequences. Furthermore, log sequences are converted into graphs, where nodes represent log templates and edges represent the order in which log templates are executed in the sequence. Graph neural networks are then introduced to model information interactions within the graph, enabling the capture of complex log patterns. Due to their ability to automatically learn features, log anomaly detection methods based on deep neural networks often achieve better performance than traditional methods. Leveraging information such as thread, component, and log level in log messages can further improve the performance of these methods. However, these methods ignore the interactions between component subsequences, making it difficult to effectively capture component anomalies by relying solely on sequence dependencies. Summary of the Invention

[0005] In view of the above, the purpose of the present invention is to provide a log anomaly detection device with component subsequence correlation awareness, which combines the captured sequence dependency and performs anomaly detection by predicting the log template in the next log message, thereby improving detection efficiency and accuracy.

[0006] To achieve the above-mentioned object, the present invention provides a component subsequence correlation-aware log anomaly detection device, comprising a memory and a processor, wherein the memory is used to store a computer executable program for performing component subsequence correlation-aware log anomaly detection, and the processor is communicatively connected to the memory and configured to execute the computer executable program stored in the memory. When the processor executes the computer executable program, the following steps are implemented:

[0007] A part of the log sequence S is selected as a sample, and the timestamp and log template are encoded to obtain the temporal feature embedding representation t and the semantic feature embedding representation V. Subsequences are extracted from the log sequence according to the component, and the timestamp and log template are encoded to obtain the temporal feature embedding representation t of each subsequence. c and semantic feature embedding representation V c ;

[0008] Construct a log anomaly detection model based on component subsequence correlation perception, including feature extraction module, sequence dependency capture module, latent relationship encoding module, graph convolution module, subsequence fusion module, embedding representation splicing module, log template prediction module and log anomaly detection module. The feature extraction module is used to obtain the embedding representation X of log sequence features based on the time feature embedding representation t and the semantic feature embedding representation V. The feature extraction module is used to obtain the embedding representation X of log sequence features based on the subsequence time feature embedding representation t. c and semantic feature embedding representation V c Get the embedding representation of the i-th subsequence feature The sequence dependency capture module is used to obtain the embedding representation x of the log sequence S feature based on the embedding representation X of the log sequence feature, and the embedding representation x of the i-th subsequence feature Get the embedding representation of the subsequence feature The implicit relation encoding module is used to concatenate the embedding representations of the subsequences to obtain the two concatenated subsequences. The correlation embedding representation X edge , the graph convolution module is used to obtain the embedding representation X based on the subsequence through sequence dependency modeling c , the correlation weight of the subsequence x rel Get the updated embedding representation X of the subsequence m The subsequence fusion module is used to represent the embedding of the i-th subsequence after GCN update. The attention weight β with the global vector i Get the embedded representation x after all subsequences are fused att , the splicing module is used to embed the fused subsequences into x att Concatenate with the embedding representation of the log sequence S to obtain the final embedding representation x cat , the log template prediction module is used to represent x according to the final embedding cat Get the probability distribution of the log template in the next log message The log anomaly detection module is used to determine whether the log sequence S is abnormal based on the prediction results of the next real log template and sample;

[0009] All samples are fed into the component's subsequence correlation-aware log anomaly detection model for training, and the model parameters are continuously updated for optimization.

[0010] Log anomaly detection is performed using a parameter-optimized log anomaly detection model.

[0011] Preferably, the feature extraction module obtains the embedding representation X of the log sequence and the embedding representation of the i-th subsequence through the feature extractor

[0012] For log sequences, the feature extractor first uses two fully connected layers to perform nonlinear transformations on the embedded representations of temporal features and semantic features, and then concatenates the two to form the embedded representation X of the log sequence features. The specific calculation process is:

[0013] V′=f v (Linear(V,β v )),

[0014] T′=f t (Linear(t,β t )),

[0015] X=Concat(V′,T′),

[0016] Among them, Linear is the calculation process of the fully connected layer, β v and β t is a learnable parameter, f v and f t is the Relu activation function, and Concat is the concatenation operation;

[0017] For the i-th subsequence, the feature extractor first uses two fully connected layers to embed the temporal feature of the i-th subsequence into representation t c and the embedding representation V of semantic features c Perform nonlinear transformation and then concatenate the two to form the embedding representation of the i-th subsequence feature The specific calculation process is:

[0018]

[0019]

[0020]

[0021] Among them, Linear is the calculation process of the fully connected layer. and is a learnable parameter, f cv and f ct is the Relu activation function, and Concat is the concatenation operation.

[0022] Preferably, the sequence dependency capture module uses a long short-term memory network LSTM to capture the sequence dependency based on the embedding representation X of the log sequence feature to obtain the embedding representation x of the log sequence S feature, and the embedding representation x of the i-th subsequence feature Get the embedding representation of the i-th subsequence feature The log sequence S and the i-th subsequence do not share LSTM network parameters. Different subsequences use LSTM networks with shared parameters. The specific calculation process is as follows:

[0023] x=LSTM1(X),

[0024]

[0025] Among them, LSTM1 and LSTM2 represent the calculation process of two LSTM networks that do not share parameters, X represents the embedded representation of log sequence features, Represents the embedding representation of the i-th subsequence feature.

[0026] Preferably, the implicit relation encoding module is used to splice the embedded representations of the subsequences in pairs, and adopt the multi-perception mechanism MLP to obtain the correlation embedded representation of the two spliced ​​subsequences. The specific calculation process is:

[0027]

[0028] in, and is the embedding representation of the i-th and j-th subsequences obtained through sequence dependency modeling, Concat is the splicing operation, is the calculation process of MLP, f is the Relu activation function, and the implicit relation encoder uses convolution operation to obtain the correlation weight x of the subsequence rel , the calculation process is as follows:

[0029]

[0030] in, is the correlation embedding representation between the i-th and j-th subsequences, Conv is the calculation process of the one-dimensional convolutional neural network 1D-CNN, and σ is the Sigmoid activation function.

[0031] Preferably, the graph convolution module uses a graph convolutional neural network (GCN) to implement information interaction between subsequences to obtain the updated embedding representation X of the subsequences. m , different subsequences are regarded as nodes in the graph, and the correlation weight x of the subsequences rel Treated as edge weights to construct a hidden relationship graph, the updated embedding representation X of the subsequence is obtained through GCN calculation m , the specific calculation process is:

[0032] X m =σ(GCN(X c ,x rel )),

[0033] Among them, x c is the embedding representation of the subsequence obtained through sequence dependency modeling, x relis the correlation weight of the subsequence, GCN is the calculation process of message passing, and σ is the ReLU activation function.

[0034] Preferably, the subsequence fusion module is used to introduce the global vector u att , use the attention mechanism MHA to obtain the embedded representation x after all subsequences are fused att ,

[0035] Introducing the global vector u att , the attention mechanism is used to calculate the embedding representation of the i-th subsequence after GCN update The attention weight β with the global vector i , the specific calculation process is:

[0036]

[0037] Among them, N c is the number of subsequences, AttScore is the calculation process of attention score. By introducing the attention weight β i The dynamic limit of β ensures the stability of training and avoids i There is a large jump between two adjacent iterations, specifically:

[0038]

[0039] in, is the attention weight of the i-th subsequence corresponding to the j-th iteration, γ is the dynamic threshold, and the value range of γ is all real numbers between 0 and 1.

[0040] According to the attention weights corresponding to different subsequences, the updated embedded representations of the subsequences are weighted aggregated to obtain the embedded representation x after the fusion of all subsequences. att , specifically:

[0041]

[0042] Among them, β i is the attention weight of the i-th subsequence, is the embedding representation of the i-th subsequence after GCN update, N c is the number of subsequences.

[0043] Preferably, the concatenation module is used to represent the embedded representation x after all subsequences are fused. att Concatenate with the embedding representation x of the log sequence to obtain the final embedding representation x after all subsequences are fused cat , specifically:

[0044] x cat =Concat(x,xatt ),

[0045] Among them, Concat is a concatenation operation, x is the embedded representation of the log sequence obtained through sequence dependency modeling, and x att is the embedded representation after fusion of all subsequences.

[0046] Preferably, the log template prediction module converts the final embedding representation x cat Send it to the classifier to calculate the probability distribution of the log template in the next log message Specifically:

[0047]

[0048] Among them, x cat is the final embedding representation after concatenation, is the calculation process of MLP, σ out is the Softmax activation function.

[0049] Preferably, the log anomaly detection module is used to determine whether the log sequence S is abnormal based on the prediction results of the next real log template and sample.

[0050] According to the probability distribution of the log template in the next log message All log templates are sorted, and the prediction results are the top-k log template sets. The prediction results of the next real log template and sample are used to determine whether the log sequence is abnormal. Specifically:

[0051]

[0052] Among them, e true To detect the real log template in the next log message corresponding to the sample, When it is 1, it means the sample prediction is wrong, and the number of wrong samples is increased by one. Otherwise, it means the sample prediction is correct. When the number of wrong samples reaches the set threshold α anom When , it is detected that the log sequence S is abnormal.

[0053] To achieve the above-mentioned object of the invention, the present invention also provides a log anomaly detection device based on component subsequence correlation perception, which is characterized by comprising a data acquisition unit, a model building unit, a training unit, and an application unit.

[0054] The data acquisition unit is used to divide a part of the log sequence S as a sample, encode the timestamp and log template to obtain the time feature embedding representation t and the semantic feature embedding representation V, extract subsequences from the log sequence according to the component, encode the timestamp and log template to obtain the time feature embedding representation t of each subsequence cand semantic feature embedding representation V c ;

[0055] The model construction unit is used to build a log anomaly detection model based on component subsequence correlation perception, including a feature extraction module, a sequence dependency capture module, a latent relationship encoding module, a graph convolution module, a subsequence fusion module, an embedding representation splicing module, a log template prediction module and a log anomaly detection module. The feature extraction module is used to obtain the embedding representation X of the log sequence feature based on the time feature embedding representation t and the semantic feature embedding representation V, and the embedding representation t of the subsequence time feature is obtained based on the subsequence time feature embedding representation t. c and semantic feature embedding representation V c Get the embedding representation of the i-th subsequence feature The sequence dependency capture module is used to obtain the embedding representation x of the log sequence feature according to the embedding representation X of the log sequence feature, and the embedding representation x of the i-th subsequence feature Get the embedding representation of the subsequence feature The implicit relation encoding module is used to concatenate the embedding representations of the subsequences to obtain the two concatenated subsequences. The correlation embedding representation X edge , the graph convolution module is used to obtain the embedding representation X based on the subsequence through sequence dependency modeling c , the correlation weight of the subsequence x rel Get the updated embedding representation X of the subsequence m The subsequence fusion module is used to represent the embedding of the i-th subsequence after GCN update. The attention weight β with the global vector i Get the embedded representation x after all subsequences are fused att , the splicing module is used to embed the fused subsequences into x att Concatenate with the embedding representation of the log sequence S to obtain the final embedding representation x cat , the log template prediction module is used to represent x according to the final embedding cat Get the probability distribution of the log template in the next log message The log anomaly detection module is used to determine whether the log sequence S is abnormal based on the prediction results of the next real log template and sample;

[0056] All samples are fed into the log anomaly detection model for training, and the model parameters are continuously updated for optimization.

[0057] The application unit is used to perform log anomaly detection using a log anomaly detection model with optimized parameters.

[0058] Compared with the prior art, the technical effects of the present invention include at least:

[0059] The present invention proposes a log anomaly detection method based on component subsequence correlation perception. By extracting component subsequences and capturing sequence dependencies, combined with implicit relation encoders to adaptively model implicit correlations between subsequences, the log anomaly detection model's ability to perceive abnormal information in components is enhanced, and the mutual influence between component subsequences can be effectively captured, thereby improving the accuracy and robustness of anomaly detection. At the same time, the present invention also introduces an attention mechanism and GCN for information fusion and interaction, further improving the performance and effect of anomaly detection. The attention mechanism is used to fuse the updated embedding representations of all subsequences, and the embedded representation of the log sequence is combined to predict the log template in the next log message. The present invention can effectively detect and diagnose system anomalies in the actual application of log anomaly detection, thereby improving the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0061] Figure 1 This is a flow chart of a log anomaly detection method based on component subsequence correlation awareness provided by an embodiment of the present invention;

[0062] Figure 2 This is a framework diagram of a log anomaly detection model based on component subsequence correlation awareness provided by an embodiment of the present invention;

[0063] Figure 3 1 is a structural diagram of an implicit relation encoder in a component subsequence correlation-aware log anomaly detection model provided by an embodiment of the present invention;

[0064] Figure 4 This is a structural diagram of a log anomaly detection device based on component subsequence correlation awareness provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0066] In order to solve the problem of how to fully model the mutual influence between component subsequences, the embodiment provides a component subsequence correlation-aware log anomaly detection method and apparatus.

[0067] like Figure 1As shown, the component subsequence correlation-aware log anomaly detection method provided in the embodiment includes the following steps:

[0068] S110: A portion of the log sequence S is selected as a sample, and the timestamp and log template are encoded to obtain the temporal feature embedding representation t and the semantic feature embedding representation V. Subsequences are extracted from the log sequence according to the component, and the timestamp and log template are encoded to obtain the temporal feature embedding representation t of each subsequence. c and semantic feature embedding representation V c ;

[0069] In the embodiment, log data is collected and parsed, and the timestamp, component, and log template in each log message are obtained and pre-processed.

[0070] Collect log data and remove non-text lines, system configuration lines, and other irrelevant content.

[0071] The processed log data is parsed using a log parsing tool. Structured information such as the timestamp t and component p in the log message m is retained, and unstructured information is formalized into a log template e and corresponding log parameters.

[0072] The timestamp is preprocessed to remove meaningless contents such as spaces, characters, and punctuation marks, and then the timestamp is encoded into a unified Unix timestamp format.

[0073] Log templates are preprocessed, treating all different types of log templates as a collection of natural language text. First, non-character tokens such as delimiters, operators, punctuation, and numbers are removed from each log template. Prepositions and pause words such as "a," "the," and "for" are then removed from each log template. Finally, all words in the processed log template text are extracted to form a vocabulary.

[0074] In the embodiment, Figure 2 As shown in the figure, log sequences are divided from the log data according to a fixed time window, the timestamp and log template are encoded, and the embedded representation t of its time feature and the embedded representation V of its semantic feature are obtained.

[0075] Use a fixed time window to divide the log data in chronological order. If the current log message m i If the difference between the timestamp and the initial timestamp is less than the window length, it is divided into the log sequence S.

[0076] The timestamp is encoded, and the timestamp t1 of the log message m1 in the log sequence is used as the starting time. The difference between the timestamps of the remaining log messages and the starting time is extracted as the time feature to obtain the embedded representation t of the time feature.

[0077] To encode the log template, first use the trained word vector model (such as Bert) to extract the embedding representation v corresponding to each word from the vocabulary i Then use the TF-IDF algorithm to calculate the TF-IDF weight w of each word in the log template i Then perform weighted aggregation to obtain the embedded representation v' of the semantic features of the log template, specifically:

[0078]

[0079] Among them, w i and v i are the TF-IDF weight and corresponding embedding representation of the i-th word in the log template, N e is the number of words contained in the log template. Finally, the above embedding representation is extracted for each log message in the log sequence to obtain the embedding representation V of the semantic features of the log sequence.

[0080] In the embodiment, subsequences are extracted from the log sequence according to the component, the timestamp and log template are encoded, and the embedded representation of the time feature of each subsequence is obtained. c and the embedding representation V of semantic features c .

[0081] According to the log message m i Component p in i Extract subsequences from the log sequence. Log messages containing the same components are extracted into the same subsequence. Then, query the components not included in the log sequence based on the component set and create an empty sequence for them. Use the same method to obtain the embedded representation of the time feature of each subsequence t c and the embedding representation V of semantic features c .

[0082] S120 builds a component subsequence correlation-aware log anomaly detection model, including a feature extraction module, a sequence dependency capture module, a latent relationship encoding module, a graph convolution module, a subsequence fusion module, an embedding representation splicing module, a log template prediction module, and a log anomaly detection module.

[0083] In the embodiment, the overall model framework is as follows Figure 2 As shown in Figure 2, for log sequences, the feature extractor first uses two fully connected layers to perform nonlinear transformations on the embedded representations of temporal features and semantic features, and then concatenates the two to form the embedded representation X of the log sequence features. The specific calculation process is:

[0084] V′=f v (Linear(V,β v )),

[0085] T′=f t (Linear(t,β t )),

[0086] X=Concat(V′, T′),

[0087] Among them, Linear is the calculation process of the fully connected layer, β v and β t is a learnable parameter, f v and f t is the Relu activation function, and Concat is the concatenation operation;

[0088] For the i-th subsequence, the feature extractor first uses two fully connected layers to embed the temporal feature of the i-th subsequence into representation t c and the embedding representation V of semantic features c Perform nonlinear transformation and then concatenate the two to form the embedding representation of the i-th subsequence feature The specific calculation process is:

[0089]

[0090]

[0091]

[0092] Among them, Linear is the calculation process of the fully connected layer. and is a learnable parameter, f cv and f ct is the Relu activation function, and Concat is the concatenation operation.

[0093] In the embodiment, the sequence dependency capture module uses the long short-term memory network LSTM to capture the sequence dependency based on the embedding representation X of the log sequence feature to obtain the embedding representation x of the log sequence S feature, and the embedding representation x of the i-th subsequence feature Get the embedding representation of the i-th subsequence feature The log sequence S and the i-th subsequence do not share LSTM network parameters. Different subsequences use LSTM networks with shared parameters. The specific calculation process is as follows:

[0094] x=LSTM1(X),

[0095]

[0096] Among them, LSTM1 and LSTM2 represent the calculation process of two LSTM networks that do not share parameters, X represents the embedded representation of log sequence features, Represents the embedding representation of the i-th subsequence feature.

[0097] In the embodiment, Figure 3 As shown in the figure, the implicit relation encoding module is used to splice the embedding representations of the subsequences in pairs, and the multi-perception mechanism MLP is used to obtain the correlation embedding representation of the two spliced ​​subsequences. The specific calculation process is:

[0098]

[0099] in, and is the embedding representation of the i-th and j-th subsequences obtained through sequence dependency modeling, Concat is the splicing operation, is the calculation process of MLP, f is the Relu activation function, and the implicit relation encoder uses convolution operation to obtain the correlation weight x of the subsequence rel , the calculation process is as follows:

[0100]

[0101] in, is the correlation embedding representation between the i-th and j-th subsequences, Conv is the calculation process of the one-dimensional convolutional neural network 1D-CNN, and σ is the Sigmoid activation function.

[0102] In this embodiment, the graph convolution module uses the convolutional neural network GCN to realize information interaction between subsequences and obtain the updated embedding representation X of the subsequence m .

[0103] GCN is used to realize information interaction between subsequences. Different subsequences are regarded as nodes in the graph, and the correlation weights of subsequences are regarded as edge weights to construct a hidden relationship graph. The updated embedding representation X of the subsequence is obtained through GCN calculation. m , specifically:

[0104] X m =σ(GCN(X c , x rel )),

[0105] Among them, X c is the embedding representation of the subsequence obtained through sequence dependency modeling, x rel is the correlation weight of the subsequence, GCN is the calculation process of message passing, and σ is the ReLU activation function.

[0106] In the embodiment, the subsequence fusion module is used to introduce the global vector u att , use the attention mechanism MHA to obtain the embedded representation x after all subsequences are fused att ,

[0107] Introducing the global vector u att , the attention mechanism is used to calculate the embedding representation of the i-th subsequence after GCN update The attention weight β with the global vector i , specifically:

[0108]

[0109] Among them, N c is the number of subsequences, AttScore is the calculation process of attention score. By introducing the attention weight β i The dynamic limit of β ensures the stability of training and avoids i There is a large jump between two adjacent iterations, specifically:

[0110]

[0111] in, is the attention weight of the i-th subsequence corresponding to the j-th iteration, γ is the dynamic threshold, and the value range of γ is all real numbers between 0 and 1.

[0112] According to the attention weights corresponding to different subsequences, the updated embedded representations of the subsequences are weighted aggregated to obtain the embedded representation x after the fusion of all subsequences. att , specifically:

[0113]

[0114] Among them, β i is the attention weight of the i-th subsequence, is the embedding representation of the i-th subsequence after GCN update, N c is the number of subsequences.

[0115] In this embodiment, the embedding representation concatenation module is used to combine the embedding representation x after all subsequences are fused. att Concatenate with the embedding representation x of the log sequence to obtain the final embedding representation x after all subsequences are fused cat , specifically:

[0116] x cat =Concat(x, x att ),

[0117] Among them, Concat is a concatenation operation, x is the embedded representation of the log sequence obtained through sequence dependency modeling, and x att is the embedded representation after fusion of all subsequences.

[0118] In this embodiment, the log template prediction module converts the final embedding representation xcat Send it to the classifier to calculate the probability distribution of the log template in the next log message Specifically:

[0119]

[0120] Among them, x cat is the final embedding representation after concatenation, is the calculation process of MLP, σ out is the Softmax activation function.

[0121] In the embodiment, Figure 2 As shown, samples are divided from the newly arrived log sequence S according to the sliding window, and the samples are fed into the model to obtain the probability distribution of the log template in the next log message

[0122] Determine whether the log sequence S is abnormal based on the prediction results of the next real log template and sample.

[0123] According to the probability distribution of the log template in the next log message All log templates are sorted, and the prediction results are the top-k log template sets. The prediction results of the next real log template and sample are used to determine whether the log sequence is abnormal. The calculation formula is as follows:

[0124]

[0125] Among them, e true This is the actual log template for the next log message corresponding to the sample. When it is 1, it means the sample prediction is wrong, and the number of wrong samples is increased by one. Otherwise, it means the sample prediction is correct. When the number of wrong samples reaches the set threshold α anom When , it is detected that the log sequence S is abnormal.

[0126] S130: Send all samples to the component's subsequence correlation-aware log anomaly detection model for training, and continuously optimize by updating model parameters.

[0127] Take an input sample as an example, obtain its time features and semantic features, and output the probability distribution of the log template in the next log message after model calculation. In order to make the category with the highest probability in the output representation of the model consistent with the actual log template, the cross entropy loss function is selected, and the calculation formula is as follows:

[0128]

[0129] Among them, N event is the number of log templates, The probability value of the i-th log template output by the model, y i is the true value corresponding to the i-th log template, is the output value of the loss function. i The calculation formula is as follows:

[0130]

[0131] The model optimizes the output value of the above loss function through the back propagation algorithm. The smaller the value, the stronger the model's fitting ability. The algorithm's update calculation formula is as follows:

[0132]

[0133] Where θ represents the learnable parameters in the model and η represents the learning rate. After training is complete, the model with updated parameters is output.

[0134] S140: Perform log anomaly detection using the parameter-optimized log anomaly detection model.

[0135] In the embodiment, a log anomaly detection model with optimized parameters is used to perform log anomaly detection and output the log anomaly detection result.

[0136] To fully model the interactions between component subsequences, a method is proposed to extract subsequences from log sequences based on components, combining semantic and temporal features in log messages to capture sequence dependencies. A latent relational encoder is also introduced to adaptively model the implicit dependencies between subsequences, effectively improving the log anomaly detection model's ability to detect anomalies in components. The method collects and parses log data to obtain timestamps, components, and log templates in log messages, and performs preprocessing. Subsequences are then extracted from log sequences based on components, and feature embeddings are obtained based on their inherent feature information. An LSTM is then used to capture the sequence dependencies of log sequences and subsequences, respectively, and obtain their embeddings. A latent relational encoder is then introduced to adaptively model the implicit dependencies between subsequences, combined with GCN to enable information interaction between subsequences and update the embeddings, further improving the performance and effectiveness of anomaly detection. The updated embeddings of all subsequences are then fused using an attention mechanism, and combined with the embeddings of the log sequence to predict the log template in the next log message. The log sequence is then tested for anomalies based on the actual log template.

[0137] Based on the same inventive concept, an embodiment further provides a component subsequence correlation-aware log anomaly detection device, comprising a memory and a processor. The memory is used to store a computer program. When the processor executes the computer program, the steps of the component subsequence correlation-aware log anomaly detection method provided in the above embodiment are implemented, including the following steps:

[0138] S110: A portion of the log sequence S is selected as a sample, and the timestamp and log template are encoded to obtain the temporal feature embedding representation t and the semantic feature embedding representation V. Subsequences are extracted from the log sequence according to the component, and the timestamp and log template are encoded to obtain the temporal feature embedding representation t of each subsequence. c and semantic feature embedding representation V c ;

[0139] S120: Build a component subsequence correlation-aware log anomaly detection model, including a feature extraction module, a sequence dependency capture module, a latent relationship encoding module, a graph convolution module, a subsequence fusion module, an embedding representation concatenation module, a log template prediction module, and a log anomaly detection module.

[0140] S130: Send all samples to the log anomaly detection model for training and continuously optimize by updating model parameters;

[0141] S140: Perform log anomaly detection using the parameter-optimized log anomaly detection model.

[0142] In an embodiment, the memory can be a local volatile memory such as RAM, or a non-volatile memory such as ROM, FLASH, a floppy disk, a mechanical hard disk, or a remote storage cloud. The processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA). These processors can be used to implement the steps of the component subsequence correlation-aware log anomaly detection method.

[0143] Based on the same inventive concept, the embodiment further provides a component subsequence correlation-aware log anomaly detection device 400, comprising a data acquisition unit 410, a model building unit 420, a training unit 430, and an application unit 440.

[0144] The data acquisition unit 410 is used to divide a portion of the log sequence S as a sample, encode the timestamp and log template to obtain the time feature embedding representation t and the semantic feature embedding representation V, extract subsequences from the log sequence according to the component, encode the timestamp and log template to obtain the time feature embedding representation t of each subsequence c and semantic feature embedding representation V c .

[0145] The model construction unit 420 is used to build a log anomaly detection model with component subsequence correlation perception, including a feature extraction module, a sequence dependency capture module, a latent relationship encoding module, a graph convolution module, a subsequence fusion module, an embedding representation splicing module, a log template prediction module and a log anomaly detection module. The feature extraction module is used to obtain the embedding representation X of the log sequence feature based on the time feature embedding representation t and the semantic feature embedding representation V, and the embedding representation t of the subsequence time feature is obtained based on the subsequence time feature embedding representation t. c and semantic feature embedding representation V c Get the embedding representation of the i-th subsequence feature The sequence dependency capture module is used to obtain the embedding representation x of the log sequence feature according to the embedding representation X of the log sequence feature, and the embedding representation x of the i-th subsequence feature Get the embedding representation of the subsequence feature The implicit relation encoding module is used to concatenate the embedding representations of the subsequences to obtain the two concatenated subsequences. The correlation embedding representation X edg e. The graph convolution module is used to obtain the embedding representation X based on the subsequence through sequence dependency modeling c , the correlation weight of the subsequence x rel Get the updated embedding representation X of the subsequence m The subsequence fusion module is used to represent the embedding of the i-th subsequence after GCN update. The attention weight β with the global vector i Get the embedded representation x after all subsequences are fused att , the splicing module is used to embed the fused subsequences into x att Concatenate with the embedding representation of the log sequence S to obtain the final embedding representation x cat , the log template prediction module is used to represent x according to the final embedding cat Get the probability distribution of the log template in the next log message The log anomaly detection module is used to determine whether the log sequence S is abnormal based on the prediction results of the next real log template and sample.

[0146] The training unit 430 is used to send all samples to the log anomaly detection model for training and continuously optimize by updating model parameters.

[0147] The application unit 440 is configured to perform log anomaly detection using the log anomaly detection model with optimized parameters.

[0148] It should be noted that the component subsequence correlation-aware log anomaly detection device provided in the above embodiment should be illustrated with the division of the above-mentioned functional units when performing log anomaly detection. The above-mentioned functions can be assigned to different functional units as needed, that is, the internal structure of the terminal or server is divided into different functional units to complete all or part of the functions described above. In addition, the component subsequence correlation-aware log anomaly detection device provided in the above embodiment belongs to the same concept as the component subsequence correlation-aware log anomaly detection method embodiment. The specific implementation process is detailed in the component subsequence correlation-aware log anomaly detection method embodiment, which will not be repeated here. The above-mentioned specific implementation methods have detailed descriptions of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A component subsequence correlation-aware log anomaly detection device, comprising a memory and a processor, wherein the memory is configured to store a computer executable program for performing component subsequence correlation-aware log anomaly detection, and the processor is communicatively connected to the memory and configured to execute the computer executable program stored in the memory, characterized in that: When the processor executes the computer executable program, the following steps are implemented: A part of the log sequence S is selected as a sample, and the timestamp and log template are encoded to obtain the time feature embedding representation t and semantic feature embedding representation V of the log sequence. Subsequences are extracted from the log sequence according to the component, and the timestamp and log template are encoded to obtain the time feature embedding representation t of each subsequence. c and semantic feature embedding representation V c ; Construct a log anomaly detection model based on component subsequence correlation perception, including feature extraction module, sequence dependency capture module, latent relationship encoding module, graph convolution module, subsequence fusion module, embedding representation splicing module, log template prediction module and log anomaly detection module. The feature extraction module is used to obtain the embedding representation X of log sequence features based on the time feature embedding representation t and the semantic feature embedding representation V. The feature extraction module is used to obtain the embedding representation X of log sequence features based on the subsequence time feature embedding representation t. c and semantic feature embedding representation V c Get the embedding representation of the i-th subsequence feature The sequence dependency capture module is used to obtain the embedding representation x of the log sequence feature according to the embedding representation X of the log sequence feature, and the embedding representation x of the i-th subsequence feature Get the embedding representation of the subsequence feature The implicit relation encoding module is used to embed the subsequence features Splice two by two to obtain the two subsequences after splicing The correlation embedding representation The graph convolution module is used to obtain the embedding representation X based on the subsequence through the sequence dependency capture module c , the correlation weight x of the subsequence rel Get the updated embedding representation X of the subsequence m The subsequence fusion module is used to represent the embedding of the i-th subsequence after GCN update. The attention weight β with the global vector i Get the embedded representation x after all subsequences are fused att , the splicing module is used to embed the fused subsequences into x att Concatenate with the embedding representation of the log sequence S to obtain the final embedding representation x cat , the log template prediction module is used to represent x according to the final embedding cat Get the probability distribution of the log template in the next log message The log anomaly detection module is used to determine whether the log sequence S is abnormal based on the prediction results of the next real log template and sample; All samples are fed into the log anomaly detection model for training, and the model parameters are continuously updated for optimization. Log anomaly detection is performed using a parameter-optimized log anomaly detection model.

2. The component subsequence correlation-aware log anomaly detection device according to claim 1, characterized in that: The feature extraction module obtains the embedding representation X of the log sequence and the embedding representation of the i-th subsequence feature through the feature extractor For log sequences, the feature extractor first uses two fully connected layers to perform nonlinear transformations on the temporal feature embedding representation t and the semantic feature embedding representation V, and then concatenates the two to form the log sequence feature embedding representation X. The specific calculation process is: V′=f v (Linear(V,β v )), T′=f t (Linear(t,β t )), X=Concat(V′,T′), Among them, Linear is the calculation process of the fully connected layer, β v and β t is a learnable parameter, f v and f t is the Relu activation function, and Concat is the concatenation operation; For the i-th subsequence, the feature extractor first uses two fully connected layers to embed the temporal feature of the i-th subsequence into representation t c and the embedding representation V of semantic features c Perform nonlinear transformation and then concatenate the two to form the embedding representation of the i-th subsequence feature The specific calculation process is: Among them, Linear is the calculation process of the fully connected layer. and is a learnable parameter, f cv and f ct is the Relu activation function, and Concat is the concatenation operation.

3. The component subsequence correlation-aware log anomaly detection device according to claim 1, characterized in that: The sequence dependency capture module uses the long short-term memory network LSTM to capture the sequence dependency based on the embedding representation X of the log sequence feature to obtain the embedding representation x of the log sequence S feature, and the embedding representation x of the i-th subsequence feature Get the embedding representation of the i-th subsequence feature The log sequence S and the i-th subsequence do not share LSTM network parameters. Different subsequences use LSTM networks with shared parameters. The specific calculation process is as follows: x=LSTM1(X), Among them, LSTM1 and LSTM2 represent the calculation process of two LSTM networks that do not share parameters, X represents the embedded representation of log sequence features, Represents the embedding representation of the i-th subsequence feature.

4. The component subsequence correlation-aware log anomaly detection device according to claim 1, characterized in that: The implicit relation encoding module is used to splice the embedding representations of subsequence features in pairs, and adopt the multi-perception mechanism MLP to obtain the correlation embedding representation of the two spliced ​​subsequences. The specific calculation process is: in, and is the embedding representation of the i-th and j-th subsequences obtained by the sequence dependency capture module, Concat is the splicing operation, is the calculation process of MLP, f is the Relu activation function, and the implicit relation encoder uses convolution operation to obtain the correlation weight x of the subsequence rel , the calculation process is as follows: in, is the correlation embedding representation between the i-th and j-th subsequences, Conv is the calculation process of the one-dimensional convolutional neural network 1D-CNN, and σ is the Sigmoid activation function.

5. The component subsequence correlation-aware log anomaly detection device according to claim 1, characterized in that: The graph convolution module uses the graph convolutional neural network GCN to realize the information interaction between subsequences to obtain the updated embedding representation X of the subsequence m , different subsequences are regarded as nodes in the graph, and the correlation weight x of the subsequences rel Treated as edge weights to construct a hidden relationship graph, the updated embedding representation X of the subsequence is obtained through GCN calculation m , the specific calculation process is: X m =σ(GCN(X c ,x rel )), Among them, X c is the embedding representation of the subsequence obtained by the sequence dependency capture module, x rel is the correlation weight of the subsequence, GCN is the calculation process of message passing, and σ is the ReLU activation function.

6. The component subsequence correlation-aware log anomaly detection device according to claim 1, characterized in that: The subsequence fusion module is used to introduce the global vector u att , use the attention mechanism MHA to obtain the embedded representation x after all subsequences are fused att , Introducing the global vector u att , the attention mechanism is used to calculate the embedding representation of the i-th subsequence after GCN update The attention weight β with the global vector i , the specific calculation process is: Among them, N c is the number of subsequences, AttScore is the calculation process of attention score, by introducing the attention weight β i The dynamic limit of β ensures the stability of training and avoids i There is a large jump between two adjacent iterations, specifically: in, is the attention weight of the i-th subsequence corresponding to the j-th iteration, γ is the dynamic threshold, and the value range of γ is all real numbers between 0 and 1. According to the attention weights corresponding to different subsequences, the updated embedded representations of the subsequences are weighted aggregated to obtain the embedded representation x after the fusion of all subsequences. att , specifically:

7. The component subsequence correlation-aware log anomaly detection device according to claim 1, characterized in that: The concatenation module is used to represent the embedded representation x after all subsequences are fused. att Concatenate with the embedding representation x of the log sequence to obtain the final embedding representation x after all subsequences are fused cat , specifically: x cat =Concat(x,x att ), Among them, Concat is the concatenation operation, x is the embedded representation of the log sequence obtained by the sequence dependency capture module, and x att is the embedded representation after fusion of all subsequences.

8. The component subsequence correlation-aware log anomaly detection device according to claim 1, characterized in that: The log template prediction module converts the final embedding representation x cat Send it to the classifier to calculate the probability distribution of the log template in the next log message Specifically: Among them, x cat is the final embedding representation after concatenation, is the calculation process of MLP, σ out is the Softmax activation function.

9. The component subsequence correlation-aware log anomaly detection device according to claim 1, characterized in that: The log anomaly detection module is used to determine whether the log sequence S is abnormal based on the prediction results of the next real log template and sample. According to the probability distribution of the log template in the next log message All log templates are sorted, and the prediction results are the top-k log template sets. The prediction results of the next real log template and sample are used to determine whether the log sequence is abnormal. Specifically: Among them, e true To detect the real log template in the next log message corresponding to the sample, When it is 1, it means the sample prediction is wrong. At this time, the number of wrong samples increases by one. Otherwise, it means the sample prediction is correct. When the number of wrong samples reaches the set threshold α anom When , it is detected that the log sequence S is abnormal.

10. A log anomaly detection device based on component subsequence correlation perception, characterized in that: Including data acquisition unit, model building unit, training unit, application unit, The data acquisition unit is used to divide a part of the log sequence S as a sample, encode the timestamp and log template to obtain the time feature embedding representation t and the semantic feature embedding representation V, extract subsequences from the log sequence according to the component, encode the timestamp and log template to obtain the time feature embedding representation t of each subsequence c and semantic feature embedding representation V c ; The model construction unit is used to construct a log anomaly detection model based on component subsequence correlation perception, including a feature extraction module, a sequence dependency capture module, a latent relationship encoding module, a graph convolution module, a subsequence fusion module, an embedding representation splicing module, a log template prediction module and a log anomaly detection module. The feature extraction module is used to obtain the embedding representation X of the log sequence feature based on the time feature embedding representation t and the semantic feature embedding representation V, and the embedding representation t of the subsequence time feature is obtained based on the subsequence time feature embedding representation t. c and semantic feature embedding representation V c Get the embedding representation of the i-th subsequence feature The sequence dependency capture module is used to obtain the embedding representation x of the log sequence feature according to the embedding representation X of the log sequence feature, and the embedding representation x of the i-th subsequence feature Get the embedding representation of the subsequence feature The implicit relation encoding module is used to concatenate the embedding representations of subsequence features to obtain the two concatenated subsequences. The correlation embedding representation X edge , the graph convolution module is used to obtain the embedding representation X according to the subsequence through the sequence dependency capture module c , the correlation weight of the subsequence x rel Get the updated embedding representation X of the subsequence m The subsequence fusion module is used to represent the embedding of the i-th subsequence after GCN update. The attention weight β with the global vector i Get the embedded representation x after all subsequences are fused att , the splicing module is used to embed the fused subsequences into x att Concatenate with the embedding representation of the log sequence S to obtain the final embedding representation x cat , the log template prediction module is used to represent x according to the final embedding cat Get the probability distribution of the log template in the next log message The log anomaly detection module is used to determine whether the log sequence S is abnormal based on the prediction results of the next real log template and sample; All samples are fed into the log anomaly detection model for training, and the model parameters are continuously updated for optimization. The application unit is used to perform log anomaly detection using a log anomaly detection model with optimized parameters.

Images