A log anomaly detection method based on semantic embedding and hybrid neural network

Abstract

The invention discloses a log anomaly detection method based on semantic embedding and hybrid neural network, comprising: acquiring real-time log data and parsing the real-time log data by using a fixed parse tree structure to extract a log event template; performing serialization processing on the log event template to form a log template sequence; extracting semantic features of the log event template from the log template sequence based on a pre-trained RoBERTa model and an improved smooth inverse frequency algorithm; constructing the hybrid neural network model by using a multi-scale convolutional neural network, an efficient channel attention mechanism and a deformation gated recurrent unit; inputting the semantic features of the log event template into the trained hybrid neural network model to obtain a log detection result; the invention can better parse variable-length logs, extract semantic features, and improve the accuracy and robustness of anomaly detection through the hybrid neural network model.

Description

Technical Field

[0001] The present invention belongs to the technical field of log anomaly detection, and in particular relates to a log anomaly detection method based on semantic embedding and hybrid neural network. Background Art

[0002] With the development of the Internet and computer technology, the scale and complexity of cloud computing, big data, the Internet of Things and other systems are constantly expanding. Modern systems often generate many log files during operation. The logs reflect the operating status of the system and record the activity information of specific events in the system. Therefore, system logs are an important data source for performance monitoring and anomaly detection.

[0003] Given the rapid growth in the size of log messages and the high professional requirements for reviewers, manual review of log messages has become almost impractical. Although the traditional automated anomaly detection method based on keyword regular matching can identify obvious single anomalies to a certain extent, it has significant limitations. It is only effective when the log contains clear identification, and it is powerless for anomalies that cannot be located by keywords. Although the unsupervised log detection scheme based on cluster analysis has made some progress, it is still difficult to cope with complex situations such as log template updates and various types of anomalies. With the vigorous development of artificial intelligence technology, many automatic and semi-automatic log anomaly detection optimization methods based on various neural networks have emerged. These methods have made innovations in multiple aspects such as log parsing, model optimization and feature processing. They extract semantic information of logs through natural language processing, improve traditional detection models, and mine more features to detect anomalies that conventional methods fail to cover, thereby improving the detection effect.

[0004] Since most log data is unstructured text, which contains a large number of interference words that are irrelevant to the log semantic information, researchers generally use log template extraction technology to optimize the data processing process. Through this method, irrelevant interference items in the log can be effectively removed, and the log templates and parameters generated by the software system can be clearly distinguished. Furthermore, using log template analysis, researchers can accurately extract the semantic information of the log. For example, with the help of tree structure algorithms such as heuristic search trees (such as Drain and Spell), complex log data can be parsed into multiple templates, thereby achieving effective extraction and utilization of log semantic information.

[0005] The existing popular methods mainly use the template-based method extracted by log parsing to build a deep learning model for anomaly detection, which mainly consists of three steps: log parsing, feature extraction and anomaly detection. Although the existing anomaly detection methods are continuously being improved and optimized, they still have the following problems:

[0006] 1) Most anomaly detection methods are relatively effective in experiments, but are still not stable enough in practice. This is because real logs are unstable, which means that there are often variable-length log sequences, which causes logs of different lengths but with the same events to be separated from each other, resulting in log parsing errors, thus affecting the accuracy of anomaly detection.

[0007] 2) Most log anomaly detection models represent log sequences based on the order and quantity characteristics of log sequences, ignoring the inherent semantic information in the logs and not fully considering the impact of the importance of words on log statements, resulting in poor robustness to new log sequences.

[0008] 3) Current research on log anomaly detection mainly uses a single deep learning network model, such as RNN and LSTM. However, the detection of anomalies in log sequences depends heavily on the global and local dependencies between logs. Due to the sequential nature of the long short-term memory network, it is insufficient in effectively capturing global dependencies. In addition, due to the lack of interactive relationships, its ability to capture local dependencies is also limited. Summary of the invention

[0009] The present invention provides a log anomaly detection method based on semantic embedding and hybrid neural network, which can better parse variable-length logs, extract semantic features, and improve the accuracy and robustness of anomaly detection through a hybrid neural network model.

[0010] In order to achieve the above object, the technical solution adopted by the present invention is:

[0011] A first aspect of the present invention provides a log anomaly detection method based on semantic embedding and hybrid neural network, comprising:

[0012] Obtain real-time log data and parse the real-time log data using a fixed parsing tree structure to extract log event templates;

[0013] The log event template is serialized to form a log template sequence; the semantic features of the log event template are extracted from the log template sequence based on the pre-trained RoBERTa model and the smoothed inverse frequency algorithm;

[0014] Input the semantic features of the log event template into the trained hybrid neural network model to obtain the log detection results;

[0015] The construction process of the hybrid neural network model is:

[0016] The hybrid neural network model is constructed using a multi-scale convolutional neural network, an efficient channel attention mechanism and a deformation gated recurrent unit; a log data set is obtained and the hybrid neural network model is trained using the log data set, and the trained hybrid neural network model is output.

[0017] Furthermore, the process of parsing the real-time log data using the fixed parsing tree structure and extracting the log event template includes:

[0018] After replacing special symbols and variable features in real-time log data with wildcards, multiple consecutive wildcards are merged to obtain a real-time simplified log;

[0019] Constructing a fixed parse tree structure, each node of the fixed parse tree structure represents a real-time simplified log set;

[0020] In the first level of the fixed parse tree structure, a preliminary search is performed based on the constant token length in the real-time simplified log to perform primary grouping of the real-time simplified log;

[0021] In the second layer of the fixed parsing tree structure, the real-time simplified logs are precisely matched according to the constant tags of the real-time simplified logs, and the real-time simplified logs are grouped into two levels;

[0022] In the third layer of the fixed parse tree structure, an initial log template is selected from the real-time simplified logs of the second-level grouping; the literal similarity between the real-time simplified logs and the initial log template is calculated;

[0023] If the similarity between the real-time simplified log and the initial log template is greater than or equal to the set similarity threshold Y s , the real-time simplified log is classified as an instance of the log initial template; the log initial template is updated based on the real-time simplified log;

[0024] If the similarity between the real-time simplified log and the initial log template is lower than the set similarity threshold Y s , then the new log initial template is added to the fixed parsing tree structure according to the real-time simplified log;

[0025] Calculate the mixed similarity of each log initial template in the fixed parse tree structure. If the mixed similarity between the two log initial templates is higher than or equal to the set similarity threshold Y h , merge the two log initial templates to generate a new log initial template;

[0026] Repeat the iteration until all real-time log data is parsed, and output the final log initial template as the log event template.

[0027] Furthermore, the literal similarity between the real-time simplified log and the initial log template is calculated, and the expression formula is:

[0028]

[0029] In the formula, |LCS(M,T)| represents the length of the longest common subsequence between the real-time simplified log and the log initial template; M represents the real-time simplified log; T is the log initial template, m and n represent the lengths of the real-time simplified log and the log initial template respectively, and λ is the adjustment weight for controlling the degree of penalty for log length difference.

[0030] Furthermore, the mixed similarity is calculated for each log initial template in the fixed parse tree structure, and the expression formula is:

[0031]

[0032] In the formula, κ is the weight coefficient, A and B are log initial templates, a is the length of log initial template A; b is the length of log initial template B; α 1 Adjust parameters for similarity.

[0033] Furthermore, the log event template is serialized to form a log template sequence, including:

[0034] Sort the log event templates by timestamp to form a log event template queue;

[0035] The window size is set according to the number of occurrences of the log event template. The expression formula is:

[0036]

[0037] Among them, F max is the number of times the most log event template appears in the log event template queue, F min is the minimum number of occurrences of the log event template, size w Expressed as window size;

[0038] The step size is set according to the frequency and persistence of the log event template; the node ID of the log event template is obtained, and a sliding window is composed of the step size, the node ID and the window size; and the log template sequence is divided by using the sliding window to obtain the log template sequence.

[0039] Furthermore, based on the pre-trained RoBERTa model and the smoothed inverse frequency algorithm, the semantic features of the log event template are extracted from the log template sequence, including:

[0040] Replace the i-th word x in the log event template i Input into the pre-trained RoBERTa model to obtain word vector;

[0041] According to word x i The inverse document frequency is obtained by the number of occurrences in the log data set, and the smoothed inverse frequency weight is obtained based on the inverse document frequency calculation; the expression formula is:

[0042]

[0043] In the formula, p(x i ) is represented by word x i The probability of occurrence in the log data set; α 2 It is expressed as a hyperparameter to be adjusted; IDF(x i ) is represented by word x i The inverse document frequency of ; K represents the smoothing constant;

[0044] Introduce the part-of-speech weight factor c to the smooth inverse frequency weight to obtain word x i The weight w(x i ), by weighting w(x i ) and the word vector are weighted and summed to generate the sentence vector of the log event template. The expression formula is:

[0045]

[0046] V S =V′ S -μμ T V′ S

[0047] Among them, V xi Represented as word vector, X represents the set of words in the sentence, n represents the number of word vectors in the sentence vector, V′ S To remove the sentence vector before the largest principal component, μ is represented by all V′ S The constructed matrix is ​​the characteristic matrix through singular value decomposition; μ T is the transpose of μ; μμ T V′ s V′ s The largest principal component vector of s Sentence vectors represented as log event templates;

[0048] The generated sentence vector is mapped to the corresponding log template sequence to obtain the feature vector of the log template sequence; after the feature vector of the log template sequence is whitened, it is organized into a matrix form to obtain the semantic features of the log event template.

[0049] Furthermore, the semantic features of the log event template are input into the trained hybrid neural network model to obtain log detection results, including:

[0050] Inputting the semantic features of the log event template into the multi-scale convolutional neural network, performing multi-scale feature extraction on the semantic features to obtain key semantic vectors;

[0051] The key semantic vector is input into the efficient channel attention mechanism, and the local features of the log are captured from the key semantic vector to output the intermediate features; the intermediate features are input into the deformation gated recurrent unit, and the global features of the log are captured from the intermediate features to output the log detection results.

[0052] Furthermore, the semantic features of the log event template are input into the multi-scale convolutional neural network, and multi-scale feature extraction is performed on the semantic features to obtain key semantic vectors, including:

[0053] The multi-scale convolutional neural network includes a one-dimensional convolutional layer, a batch normalization layer, an ELU activation function and a pooling layer; the one-dimensional convolutional layer is provided with convolution kernels of multiple scales;

[0054] The local feature vector is extracted from the semantic features of the log event template through convolution kernels of various scales; the local feature vector is input into the batch normalization layer to obtain the key fitting feature, and the expression formula is:

[0055]

[0056] In the formula, γ is the scaling parameter; β is the offset parameter; μ is the local eigenvector q j The mean of 2 is the local eigenvector q j Variance; ∈ represents the adjustment parameter; BN(q j ) is represented as the key fitting feature;

[0057] The first intermediate feature is mapped and transformed through the ELU activation function to obtain the key mapping feature, which is expressed as follows:

[0058]

[0059] In the formula, τ is expressed as a tuning parameter, ELU(BN(q j )) is represented as the key mapping feature;

[0060] The key mapping features are input into the pooling layer to obtain the key semantic vector.

[0061] Furthermore, the key semantic vector is input into the efficient channel attention mechanism, and the local features of the log are captured from the key semantic vector to output intermediate features, including:

[0062] The efficient channel attention mechanism includes a global average pooling layer, an adaptive convolution layer and a sigmoid function;

[0063] The key semantic vector is input into the global average pooling layer to obtain the global relationship feature; the convolution kernel size K value in the adaptive convolution layer is calculated based on the current total number of channels, and the expression formula is:

[0064]

[0065] Among them, |·| odd It means selecting the nearest odd number; c represents the current total number of channels; and b are expressed as set constants;

[0066] Input the global relationship features into the adaptive convolution layer to obtain the feature weight U h , the expression formula is;

[0067]

[0068] in, Represents and y h The output feature set of the adjacent k channels, Represents and y h The output feature of the adjacent k-th channel; y h Represented as a global relationship feature; ε k Represents the weight parameter shared by all channels; sigmoid(·) is the sigmoid activation function;

[0069] The feature weight U h Multiply it with the key semantic vector to obtain the intermediate features.

[0070] Furthermore, the intermediate features are input into the deformation gated recurrent unit, and the global features of the logs are captured from the intermediate features to output the log detection results, including:

[0071] The intermediate features are interacted with the preset hidden state for multiple rounds. When the number of information interaction rounds is an odd number, the intermediate features are updated based on the hidden state. The expression formula is:

[0072]

[0073] In the formula, d l-2 It is represented as the intermediate feature after the l-2 rounds of information interaction; dl is represented as the intermediate feature after the l-2 rounds of information interaction; Represented as the hidden state after the l-1th round of information interaction; Q l is represented as a learnable parameter matrix; ⊙ is the Hadamard product;

[0074] When the number of information interaction rounds is even, the hidden state is updated based on the intermediate features, and the expression formula is:

[0075]

[0076] In the formula, Represented as the hidden state after the l-2th round of information interaction; Represents the hidden state after the l-2th round of information interaction; R l Represented as a learnable parameter matrix; x l-1 It is represented as the intermediate feature after the l-1th round of information interaction; sigmoid(·) is the sigmoid activation function;

[0077] The intermediate features and hidden states after information interaction update are input into the basic neural unit cell of the deformation gated recurrent unit to obtain the final hidden state. The calculation formula is:

[0078]

[0079]

[0080]

[0081]

[0082] In the formula, Z t Represented as the output value of the update gate at time step t; R t Represented as the output value of the reset gate at time step t, d L,t Represented as the intermediate features of the input basic neural unit cell at time step t; Represented as the hidden state of the input basic neural unit cell at time step t; Represented as the candidate hidden layer state at time step t; H t W is represented as the final hidden state at time step t; z , W r and W q Represented as a weight matrix;

[0083] The log detection result is output based on the comparison between the occurrence probability of all final hidden states and the next log item in the real-time log data.

[0084] Compared with the prior art, the present invention has the following beneficial effects:

[0085] The present invention obtains real-time log data and parses the real-time log data using a fixed parsing tree structure to extract and obtain a log event template; it can better parse variable-length logs;

[0086] The log event templates are serialized to form log template sequences. The semantic features of the log event templates are extracted from the log template sequences based on the pre-trained RoBERTa model and the smoothed inverse frequency algorithm. The semantic differences between different log events can be represented, so that the overall model can handle unstable log data.

[0087] The present invention uses a multi-scale convolutional neural network, an efficient channel attention mechanism and a deformation gated recurrent unit to construct the hybrid neural network model; the semantic features of the log event template are input into the trained hybrid neural network model to obtain the log detection result; the multi-scale convolutional neural network is combined with the efficient channel attention mechanism to capture the local features of the log sequence, and the deformation gated recurrent unit is used to capture the global features and long-distance dependencies of the log sequence, thereby improving the detection effect and having excellent accuracy and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 A flow chart of a log anomaly detection method provided by an embodiment of the present invention;

[0089] Figure 2 A schematic diagram of a fixed parse tree structure provided by an embodiment of the present invention;

[0090] Figure 3 A flow chart for serializing a log event template provided by an embodiment of the present invention;

[0091] Figure 4 A flowchart of extracting semantic features provided by an embodiment of the present invention;

[0092] Figure 5 A structural diagram of a multi-scale convolutional neural network provided by an embodiment of the present invention;

[0093] Figure 6 A structural diagram of an efficient channel attention mechanism provided by an embodiment of the present invention;

[0094] Figure 7 A structural diagram of a deformation gated recurrent unit provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0095] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0096] like Figure 1 As shown, this implementation provides a log anomaly detection method based on semantic embedding and hybrid neural network, including:

[0097] like Figure 2 As shown, the process of parsing real-time log data using a fixed parsing tree structure and extracting a log event template includes:

[0098] As the number of log formats increases, variable parameters may appear. For example, the following content may appear in the log: "Node 1 connected" and "Node 1 2 connected" belong to the same log template; replace the special symbols and variable features in the real-time log data with wildcards, thereby effectively improving the accuracy of the log template.

[0099] Multiple consecutive wildcard symbols are merged to obtain a real-time simplified log, which alleviates the problem of variable-length logs being incorrectly grouped, resulting in a decrease in parsing accuracy; a fixed parsing tree structure is constructed, and each node of the fixed parsing tree structure represents a real-time simplified log set; at the first layer of the fixed parsing tree structure, a preliminary search is performed according to the constant tag length in the real-time simplified log, and the real-time simplified log is primary grouped; if a real-time simplified log does not have a group, a new primary group is created.

[0100] In the second layer of the fixed parse tree structure, the real-time simplified log is accurately matched according to the constant tag, and the real-time simplified log is grouped into two levels; if the corresponding node is matched, the next layer of nodes is searched, otherwise a new node is created.

[0101] At the third level of the fixed parse tree structure, the initial log template is selected from the real-time simplified log of the secondary grouping; the literal similarity between the real-time simplified log and the initial log template is calculated, and the expression formula is:

[0102]

[0103] In the formula, |LCS(M,T)| represents the length of the longest common subsequence between the real-time simplified log and the log initial template; M represents the real-time simplified log; T is the log initial template, m and n represent the lengths of the real-time simplified log and the log initial template respectively, and λ is the adjustment weight for controlling the degree of penalty for log length difference.

[0104] If the similarity between the real-time simplified log and the initial log template is greater than or equal to the set similarity threshold Y s , the real-time simplified log is classified as an instance of the log initial template, and the log initial template is updated based on the real-time simplified log. The updating method is to regard the longest common sequence of all real-time simplified logs in the node as the new log initial template; if the similarity between the real-time simplified log and the log initial template is lower than the set similarity threshold Y s , then a new log initial template is created based on the real-time simplified log and added to the fixed parsing tree structure.

[0105] The mixed similarity is calculated for each log initial template in the fixed parse tree structure, and the expression formula is:

[0106]

[0107] In the formula, κ is the weight coefficient. In this implementation, κ is set to 0.5. A and B are log initial templates. a is the length of log initial template A; b is the length of log initial template B; α 1 Adjust parameters for similarity.

[0108] If the mixed similarity between the two log initial templates is greater than or equal to the set similarity threshold Y h , merge the two log initial templates to generate a new log initial template; repeat the iteration until all real-time log data are parsed, and output the final log initial template as the log event template.

[0109] like Figure 3 As shown, the log event template is serialized to form a log template sequence, including:

[0110] Sort the log event templates by timestamp to form a log event template queue; set the window size according to the number of times the log event template appears, and the expression formula is:

[0111]

[0112] Among them, F max is the number of times the most log event template appears in the log event template queue, F min is the minimum number of occurrences of the log event template, size w Expressed as the window size.

[0113] The step size is set according to the frequency and continuity of the log event template; the node ID of the log event template is obtained, and a sliding window is composed of the step size, the node ID and the window size; the log template sequence is divided by the sliding window to obtain the log template sequence. This implementation can balance the number of samples of the majority class and the minority class, as well as the relationship between the sequence length and the sequence integrity, and more effectively process and analyze log data.

[0114] like Figure 4 As shown in the figure, based on the pre-trained RoBERTa model and the smoothed inverse frequency algorithm, the semantic features of the log event template are extracted from the log template sequence, including:

[0115] Replace the i-th word x in the log event template i Input into the pre-trained RoBERTa model to obtain word vector; according to the word x i The inverse document frequency is obtained by the number of occurrences in the log data set, and the smoothed inverse frequency weight is obtained based on the inverse document frequency calculation; the expression formula is:

[0116]

[0117] In the formula, p(x i ) is represented by word x i The probability of occurrence in the log data set; α 2 It is expressed as a hyperparameter to be adjusted; IDF(x i ) is represented by word x i The inverse document frequency of ; K is represented as a smoothing constant.

[0118] Introduce the part-of-speech weight factor c to the smooth inverse frequency weight to obtain word x i The weight w(x i ), as shown in Table 1, the size of the part-of-speech weight factor c depends on the part-of-speech of the word, and the size is set to noun>verb>adjective>other;

[0119] Table 1, value table of part-of-speech weight factor c;

[0120]

[0121] By weighting w(x i ) and the word vector are weighted and summed to generate the sentence vector of the log event template. The expression formula is:

[0122]

[0123] V S =V′ S -μμ T V′ s

[0124] Among them, V xi Represented as word vector, X represents the set of words in the sentence, n represents the number of word vectors in the sentence vector, V′ s To remove the sentence vector before the largest principal component, μ is represented by all V′ S The constructed matrix is ​​the characteristic matrix through singular value decomposition; μ T is the transpose of μ; μμ T V′ S V′ S The largest principal component vector of S Sentence vectors represented as log event templates.

[0125] The generated sentence vector is mapped to the corresponding log template sequence to obtain the feature vector of the log template sequence; after the feature vector of the log template sequence is whitened, it is organized into a matrix form to obtain the semantic features of the log event template, and the expression formula is:

[0126] G=E·D -1 / 2 ·E T

[0127] X white =X·G

[0128] In the formula, E represents the eigenvector covariance matrix of the log template sequence; D represents the eigenvalue diagonal matrix of the eigenvector; G represents the whitening matrix; X represents the eigenvector matrix of the log event template; X white It is represented as a whitened feature vector matrix, which is the semantic feature of the log event template.

[0129] The process of constructing the hybrid neural network model is as follows: constructing the hybrid neural network model using a multi-scale convolutional neural network, an efficient channel attention mechanism, and a deformation gated recurrent unit; obtaining a log data set and using the log data set to train the hybrid neural network model, and outputting the trained hybrid neural network model.

[0130] The semantic features of the log event template are input into the trained hybrid neural network model to obtain the log detection results, including:

[0131] 1) Inputting the semantic features of the log event template into the multi-scale convolutional neural network, performing multi-scale feature extraction on the semantic features to obtain key semantic vectors, the specific process is as follows:

[0132] like Figure 5 As shown, the multi-scale convolutional neural network includes a one-dimensional convolutional layer, a batch normalization layer, an ELU activation function and a pooling layer; the one-dimensional convolutional layer is provided with convolution kernels of multiple scales;

[0133] The local feature vector is extracted from the semantic features of the log event template through convolution kernels of various scales; the local feature vector is input into the batch normalization layer to obtain the key fitting feature, and the expression formula is:

[0134]

[0135] In the formula, γ is the scaling parameter; β is the offset parameter; μ is the local eigenvector q j The mean of 2 is the local eigenvector q j Variance; ∈ represents the adjustment parameter; BN(q j ) is represented as the key fitting feature;

[0136] The first intermediate feature is mapped and transformed through the ELU activation function to obtain the key mapping feature, which is expressed as follows:

[0137]

[0138] In the formula, τ is expressed as a tuning parameter, ELU(BN(q j )) is represented as the key mapping feature;

[0139] The key mapping features are input into the pooling layer to obtain the key semantic vector.

[0140] 2) Input the key semantic vector into the efficient channel attention mechanism, capture the local features of the log from the key semantic vector and output the intermediate features. The specific process is as follows:

[0141] like Figure 6 As shown, the efficient channel attention mechanism includes a global average pooling layer, an adaptive convolution layer and a sigmoid function;

[0142] The key semantic vector is input into the global average pooling layer to obtain the global relationship feature; the convolution kernel size K value in the adaptive convolution layer is calculated based on the current total number of channels, and the expression formula is:

[0143]

[0144] Among them, |·| odd It means selecting the nearest odd number; c represents the current total number of channels; and b are expressed as set constants;

[0145] Input the global relationship features into the adaptive convolution layer to obtain the feature weight U h , the expression formula is;

[0146]

[0147] in, Represents and y h The output feature set of the adjacent k channels, Represents and y h The output feature of the adjacent k-th channel; y h Represented as a global relationship feature; ε k Represents the weight parameter shared by all channels; sigmoid(·) is the sigmoid activation function;

[0148] The feature weight U h Multiply it with the key semantic vector to obtain the intermediate features.

[0149] 3) Input the intermediate features into the deformation gated recurrent unit, capture the global features of the log from the intermediate features, and output the log detection results. The specific process is as follows:

[0150] like Figure 7 As shown, the intermediate features are subjected to multiple rounds of information interaction with the preset hidden state. When the number of information interaction rounds is an odd number, the intermediate features are updated based on the hidden state. The expression formula is:

[0151]

[0152] In the formula, d l-2 Represented as the intermediate features after the l-2 rounds of information interaction; d l Represented as the intermediate features after the lth round of information interaction; Represented as the hidden state after the l-1th round of information interaction; Q l is represented as a learnable parameter matrix; ⊙ is the Hadamard product;

[0153] When the number of information interaction rounds is even, the hidden state is updated based on the intermediate features, and the expression formula is:

[0154]

[0155] In the formula, Represented as the hidden state after the l-2th round of information interaction; Represents the hidden state after the l-2th round of information interaction; R l Represented as a learnable parameter matrix; x l-1 It is represented as the intermediate feature after the l-1th round of information interaction; sigmoid(·) is the sigmoid activation function;

[0156] The intermediate features and hidden states after information interaction update are input into the basic neural unit cell of the deformation gated recurrent unit to obtain the final hidden state. The calculation formula is:

[0157]

[0158]

[0159]

[0160]

[0161] In the formula, Z t Represented as the output value of the update gate at time step t; R t Represented as the output value of the reset gate at time step t, d L,t Represented as the intermediate features of the input basic neural unit cell at time step t; Represented as the hidden state of the input basic neural unit cell at time step t; Represented as the candidate hidden layer state at time step t; H t W is represented as the final hidden state at time step t; z , W r and W q Represented as a weight matrix;

[0162] The probability of occurrence of all final hidden states is ranked and a probability ranking threshold is set. If the probability of occurrence of the next log item in the log data corresponding to the final hidden state is within the top K items, it is judged to be a normal log; only when all log items in the log sequence are judged to be normal, the log sequence is judged to be normal; conversely, if the probability of a log item is ranked after the K items, it is judged to be an abnormal log item, and thus the log sequence is judged to be an abnormal log sequence.

[0163] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0164] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0165] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0167] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A log anomaly detection method based on semantic embedding and hybrid neural network, characterized in that: include: Obtain real-time log data and parse the real-time log data using a fixed parsing tree structure to extract log event templates; Serialize the log event template to form a log template sequence; The semantic features of log event templates are extracted from log template sequences based on the pre-trained RoBERTa model and the smoothed inverse frequency algorithm, including: The i-th word in the log event template Input into the pre-trained RoBERTa model to obtain word vector; According to the word The inverse document frequency is obtained by the number of occurrences in the log data set, and the smoothed inverse frequency weight is obtained based on the inverse document frequency calculation; the expression formula is: ； In the formula, is the smoothed inverse frequency weight, Represented as words The probability of occurrence in the log data set; denoted as a hyperparameter to be tuned; Represented as words The inverse document frequency of ; K represents the smoothing constant; Introduce the part-of-speech weight factor c to the smooth inverse frequency weight to obtain the word Weight , by weighting The sentence vector of the log event template is generated by weighted summation of the word vector. The expression formula is: ； ； in, It is represented as a word vector, X represents the set of words in the sentence, and n represents the number of word vectors in the sentence vector. To remove the sentence vector before the largest principal component, Represents all The constructed matrix is ​​the characteristic matrix through singular value decomposition; for The transpose of for The largest principal component vector of Sentence vectors represented as log event templates; The generated sentence vector is mapped to the corresponding log template sequence to obtain the feature vector of the log template sequence; the feature vector of the log template sequence is whitened and organized into a matrix form to obtain the semantic features of the log event template; The construction process of the hybrid neural network model is as follows: construct the hybrid neural network model using a multi-scale convolutional neural network, an efficient channel attention mechanism, and a deformation gated recurrent unit; obtain a log data set and use the log data set to train the hybrid neural network model, and output the trained hybrid neural network model; The semantic features of the log event template are input into the trained hybrid neural network model to obtain the log detection results, including: The semantic features of the log event template are input into the multi-scale convolutional neural network, and multi-scale feature extraction is performed on the semantic features to obtain key semantic vectors. The process includes: The multi-scale convolutional neural network includes a one-dimensional convolutional layer, a batch normalization layer, an ELU activation function and a pooling layer; the one-dimensional convolutional layer is provided with convolution kernels of multiple scales; The local feature vector is extracted from the semantic features of the log event template through convolution kernels of various scales; the local feature vector is input into the batch normalization layer to obtain the key fitting feature, and the expression formula is: ； In the formula, is the scaling parameter; is the offset parameter; is the local eigenvector The mean of; is the local eigenvector The variance of It is expressed as a tuning parameter; denoted as key fitting features; The first intermediate feature is mapped and transformed through the ELU activation function to obtain the key mapping feature, which is expressed as follows: ； In the formula, Expressed as a tuning parameter, It is represented as the key mapping feature; e is the natural base; Input the key mapping features into the pooling layer to obtain the key semantic vector; The key semantic vector is input into the efficient channel attention mechanism, and the local features of the log are captured from the key semantic vector to output the intermediate features. The process includes: The efficient channel attention mechanism includes a global average pooling layer, an adaptive convolution layer and a sigmoid function; The key semantic vector is input into the global average pooling layer to obtain the global relationship feature; the convolution kernel size K value in the adaptive convolution layer is calculated based on the current total number of channels, and the expression formula is: ； in, It means selecting the nearest odd number; c represents the current total number of channels; and Expressed as a set constant; Input the global relationship features into the adaptive convolution layer to obtain feature weights , the expression formula is; ； in, Representatives and The output feature set of the adjacent k channels, Representatives and The output features of the adjacent k-th channel; Represented as global relation features; Represents the weight parameters shared by all channels; is the sigmoid activation function; Weight the features Multiply with the key semantic vector to obtain intermediate features; The intermediate features are input into the deformation gated recurrent unit, and the log global features are captured from the intermediate features to output the log detection results. The process includes: The intermediate features are interacted with the preset hidden state for multiple rounds. When the number of information interaction rounds is an odd number, the intermediate features are updated based on the hidden state. The expression formula is: ； In the formula, Expressed as Intermediate features after rounds of information interaction; Expressed as Intermediate features after rounds of information interaction; Expressed as Hidden state after round information interaction; is represented as a learnable parameter matrix; ⊙ is the Hadamard product; When the number of information interaction rounds is even, the hidden state is updated based on the intermediate features, and the expression formula is: ； In the formula, Expressed as Hidden state after round information interaction; Expressed as Hidden state after round information interaction; Represented as a learnable parameter matrix; Expressed as Intermediate features after rounds of information interaction; is the sigmoid activation function; The intermediate features and hidden states after information interaction update are input into the basic neural unit cell of the deformation gated recurrent unit to obtain the final hidden state. The calculation formula is: ； ； ； ； In the formula, Represented as the output value of the update gate at time step t; Represented as the output value of the reset gate at time step t, Represented as the intermediate features of the input basic neural unit cell at time step t; Represented as the hidden state of the input basic neural unit cell at time step t; Represented as the candidate hidden layer state at time step t; Represented as the final hidden state at time step t; , and Represented as a weight matrix; The log detection result is output based on the comparison between the occurrence probability of all final hidden states and the next log item in the real-time log data.

2. The log anomaly detection method according to claim 1, characterized in that: The process of parsing real-time log data using a fixed parsing tree structure and extracting log event templates includes: After replacing special symbols and variable features in real-time log data with wildcards, multiple consecutive wildcard symbols are merged to obtain a real-time simplified log; Constructing a fixed parse tree structure, each node of the fixed parse tree structure represents a real-time simplified log set; In the first level of the fixed parse tree structure, a preliminary search is performed based on the constant token length in the real-time simplified log to perform primary grouping of the real-time simplified log; In the second layer of the fixed parsing tree structure, the real-time simplified logs are precisely matched according to the constant tags of the real-time simplified logs, and the real-time simplified logs are grouped into two levels; In the third layer of the fixed parse tree structure, an initial log template is selected from the real-time simplified logs of the second-level grouping; the literal similarity between the real-time simplified logs and the initial log template is calculated; If the similarity between the real-time simplified log and the initial log template is greater than or equal to the set similarity threshold , the real-time simplified log is classified as an instance of the log initial template; the log initial template is updated based on the real-time simplified log; If the similarity between the real-time simplified log and the initial log template is lower than the set similarity threshold , then the new log initial template is added to the fixed parsing tree structure according to the real-time simplified log; Calculate the mixed similarity of each log initial template in the fixed parse tree structure. If the mixed similarity between the two log initial templates is higher than or equal to the set similarity threshold , merge the two log initial templates to generate a new log initial template; Repeat the iteration until all real-time log data is parsed, and output the final log initial template as the log event template.

3. The log anomaly detection method according to claim 2, characterized in that: Calculate the literal similarity between the real-time simplified log and the initial log template. The expression formula is: ； In the formula, To simplify the literal similarity between the log and the log initial template in real time, represents the length of the longest common subsequence between the real-time simplified log and the log initial template; M represents the real-time simplified log; T is the log initial template, m and n represent the lengths of the real-time simplified log and the log initial template respectively, and λ is the adjustment weight for controlling the degree of penalty for log length difference.

4. The log anomaly detection method according to claim 2, characterized in that: The mixed similarity is calculated for each log initial template in the fixed parse tree structure, and the expression formula is: ； In the formula, is the mixed similarity between the log initial template A and the log initial template B; e is the natural base; is the weight coefficient, A and B are the log initial templates, a is the length of the log initial template A; b is the length of the log initial template B; Adjust parameters for similarity.

5. The log anomaly detection method according to claim 1, characterized in that: The log event template is serialized to form a log template sequence, including: Sort the log event templates by timestamp to form a log event template queue; The window size is set according to the number of occurrences of the log event template. The expression formula is: ； in, is the number of times the most log event template appears in the log event template queue, is the minimum number of occurrences of the log event template, Expressed as window size; The step size is set according to the frequency and persistence of the log event template; the node ID of the log event template is obtained, and a sliding window is composed of the step size, the node ID and the window size; and the log template sequence is divided by using the sliding window to obtain the log template sequence.

Images