Log anomaly detection method, system and equipment and medium

Abstract

The invention discloses a log anomaly detection method, system and device and a medium, and relates to the field of security technology early warning, and the method comprises the steps: obtaining to-be-detected log sequence data of an electric power mobile micro-service system, inputting the to-be-detected log data into a trained anomaly detection model, and obtaining an anomaly detection result; the anomaly detection model is constructed based on a log data set and comprises a data processing module and a local attention processing module; the data processing module is used for converting input log sequence data into semantic feature vectors; and the local attention processing module is used for performing semantic feature representation according to the semantic feature vector, performing anomaly detection on each semantic feature to obtain an anomaly detection result, and the method can accurately detect the log anomaly condition of the micro-service system.

Description

Technical Field

[0001] The invention discloses a log anomaly detection method, system, equipment and medium, and relates to the field of security technology early warning. Background Art

[0002] In power systems, anomaly detection of log data is an important means to ensure safe and stable operation of the system. The operation of power systems includes multiple links such as power generation, transmission, distribution and consumption, and each link generates a large number of log records of operating status and event information. These log records are essential for fault detection, fault location and performance monitoring. However, existing log anomaly detection methods based on recurrent neural networks (RNNs) face the following key problems when applied in power systems: System logs are widely used for anomaly detection in software systems, where each log message is usually a semi-structured text string customized by developers. Traditional methods explicitly use keywords (such as "ERROR") or regular expressions to detect abnormal log entries. Although such rule-based methods can identify abnormal events with high accuracy, they can only identify predefined abnormal scenarios and require a lot of domain knowledge and manual processing. In addition, such methods cannot detect malicious attacks based on a series of operations, where each log appears normal, but the entire sequence is abnormal. Once undefined abnormal situations occur, rule-based methods cannot achieve good results.

[0003] The current problem of anomaly detection in log sequences is that the granularity of detection is the anomaly of the log sequence rather than the anomaly of a single log. A log sequence is a sequence formed by sorting log information according to the time of generation. Log sequences that deviate from the normal operation of the system are judged as anomalies because, even if the abnormal log sequence may not contain any single log with obvious output abnormal information, the combination of them shows an execution order that deviates from the normal behavior pattern, which is also a log sequence anomaly. This shows that logs are context-related, so the granularity of log anomaly detection should be log sequences.

[0004] Currently, recurrent neural networks in deep learning are widely used in log anomaly detection because they can learn the temporal information in sequence data. However, some existing methods still have some problems.

[0005] First, when processing log template data, existing methods either assign an index number to the template and ignore the semantic information in the log template; or use language models to understand the semantic information of the log from the token level, but do not rise to the level of fully understanding the log from the sentence level.

[0006] Traditional recurrent neural networks can only encode contextual information in one direction, from forward to backward. However, when detecting log information, it is important to use the complete contextual information rather than just the information from the previous time step. Although bidirectional RNNs can be used to learn contextual information and transmit information from both the forward and backward directions, it still faces the problem of difficulty in obtaining long-term dependencies when processing long sequences. Log sequences usually include multiple log information, so it is particularly important to learn long-term dependencies.

[0007] The current training process of the RNN-based anomaly detection model predicts the next log message based on the previous time step in order to learn normal sequence patterns. The main purpose of such a training process is to learn the correlation of log messages in normal sequences. When a violation of the correlation occurs, that is, when the model predicts the next message incorrectly, the sequence is marked as an abnormal sequence. However, it is not enough to simply set the objective function of model training to predict the next element of the sequence. It is also hoped that the model can learn the patterns common to all normal sequences.

[0008] As the number of services in a microservice system increases, the complexity of the system also increases. Each service generates a large amount of logs to record its runtime status and event information. However, due to the independence and distributed nature of services, there is a lack of strong correlation between services, and logs within services are relatively more relevant. If all logs are analyzed uniformly, this situation is ignored. Summary of the invention

[0009] The technical problem to be solved by the present invention is to provide a log anomaly detection method, system, device and medium in view of the deficiencies in the above-mentioned prior art, with the aim of solving at least one of the problems of insufficient semantic information processing of electric mobile microservice system logs, limitations of one-way information encoding, limitations of training objective functions and complexity of microservice system log analysis.

[0010] The objective of the present invention is achieved by the following technical solutions:

[0011] In a first aspect, the present invention provides a log anomaly detection method, comprising:

[0012] Obtaining the log sequence data of the electric mobile microservice system to be detected, inputting the log sequence data to be detected into a pre-trained anomaly detection model to obtain an anomaly detection result;

[0013] The anomaly detection model is trained based on a log data set, and includes a data processing module and an attention processing module; the data processing module is used to convert the input log sequence data into a semantic feature vector; the attention processing module is used to perform semantic feature representation according to the semantic feature vector, perform anomaly detection based on the semantic feature, and obtain anomaly detection results; the log data set adopts a power mobile microservice system log template and anomaly log sequence.

[0014] As a further improvement of the present invention, the data processing module is obtained based on the BERT algorithm model or the SentenceBERT algorithm model, and specifically includes:

[0015] Generate a log template sliding window sequence according to the input log sequence data;

[0016] The BERT algorithm model or the SentenceBERT algorithm model is used to extract semantic features from each sliding window sequence, and each log template in the sliding window sequence is input into the algorithm model to obtain a semantic feature vector sequence.

[0017] As a further improvement of the present invention, the attention processing module includes a local attention submodule and a global attention submodule, specifically including:

[0018] The semantic feature vector sequence is divided into multiple sequences, and the multiple subsequences are respectively input into the local attention submodule, the semantics in the semantic feature vector sequence is positionally encoded, and after passing through the multi-head attention sublayer and the feedforward network sublayer, the semantic feature vector sequence is locally processed to obtain multiple subsequence feature vectors; each subsequence feature vector is input into the global attention submodule, and after passing through the second multi-head attention sublayer and the feedforward network sublayer, it is integrated, and the total semantic feature sequence is obtained after position decoding, and the total semantic feature sequence is parsed to obtain the anomaly detection result; the Transformer algorithm is:

[0019]

[0020] In the formula, K local ,V local are the key and value matrices containing only local neighbors, K is the key vector, V is the value vector, and Q is the query vector.

[0021] As a further improvement of the present invention, the log data set uses the normal log sequence and the abnormal log sequence of the electric mobile microservice system as preprocessed data, and the preprocessing step includes:

[0022] Use the log template parsing algorithm to parse the log sequence data in the data set and decompose it into a fixed text data part and a variable parameter data part;

[0023] The parsed data portion is converted into a microservice log tuple, where the microservice log tuple refers to a data structure corresponding to the parsed data.

[0024] As a further improvement of the present invention, the anomaly detection model is trained based on the log data set, and the specific training steps include:

[0025] Use the mask mark to replace a certain proportion of log templates in the sequence as the abnormal log sequence, and input the abnormal log sequence and the normal log sequence into the anomaly detection model.

[0026] Input the feature vector of the mask in the abnormal log sequence into the softmax function to obtain the probability distribution of the log template set;

[0027] Use the cross entropy loss function as the loss function for mask log template learning;

[0028] Use the cluster center distance minimization loss function to constrain the distribution of the learned normal log sequence and find the smallest hypersphere that contains all normal data;

[0029] Based on the smallest hypersphere containing all normal data, the average operation is used to calculate the center of the normal log sequence cluster, and the data optimization is performed by minimizing the loss function using the cluster center distance.

[0030] As a further improvement of the present invention, the cross entropy loss function is:

[0031]

[0032] In the formula, is the actual log template at the i-th mask mark, M is the number of all masks in the j-th sequence, and N is the total number of sequences.

[0033] As a further improvement of the present invention, the cluster center distance minimization loss function is:

[0034]

[0035] In the formula, Loss CCDM The loss function is minimized for the cluster center distance, N is the total number of sequences, and μ is the average operation to calculate the normal log sequence cluster center.

[0036] In a second aspect, the present invention further provides a log anomaly detection system, comprising:

[0037] A data detection module is used to obtain the log sequence data of the electric mobile microservice system to be detected, input the log sequence data to be detected into the trained anomaly detection model, and obtain the anomaly detection result;

[0038] The anomaly detection model module includes the anomaly detection model obtained by training based on a log data set, including a data processing module and an attention processing module; the data processing module is used to convert the input log sequence data into a semantic feature vector; the attention processing module is used to perform semantic feature representation according to the semantic feature vector, perform anomaly detection based on the semantic feature, and obtain anomaly detection results; the log data set adopts a power mobile microservice system log template and an abnormal log sequence.

[0039] In a third aspect, the present invention further provides a computing device, comprising:

[0040] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for executing the above-mentioned log anomaly detection method.

[0041] In a fourth aspect, the present invention also provides a computer-readable storage medium storing one or more programs, characterized in that the one or more programs include instructions, which, when executed by a computing device, enable the computing device to execute the above-mentioned log anomaly detection method.

[0042] The beneficial effects of the present invention are as follows: the log anomaly detection method of the present invention is based on the analysis of the poor log anomaly detection effect in the power microservice system environment and the shortcomings of the existing methods, and the log data processing and detection methods are designed to overcome the shortcomings of the existing methods, including insufficient understanding of log semantic features, difficulty in learning long-term dependencies of log sequences, and the lack of prominence of local features within the sequence. The present invention processes data through two modules, the data processing module and the attention processing module, to improve the accuracy of log anomaly detection.

[0043] Furthermore, the SentenceBERT algorithm model, a derivative model of BERT, understands the semantic feature information of logs from the sentence level, overcoming the limitations of existing methods that only assign index numbers or only understand semantics from the token level, and improving the depth and accuracy of understanding of log content.

[0044] Furthermore, the Transformer model is used to replace the traditional recurrent neural network (RNN) model, which enhances the model's ability to encode global information of log sequences. The Transformer's self-attention mechanism can cover the entire sequence, thereby better capturing the correlation between the previous and next logs, solving the difficulty of RNN in obtaining long-term dependencies when processing long sequences. ;

[0045] Furthermore, a cluster center distance minimization loss function is designed to constrain the distribution of normal sequence patterns learned by the network. By minimizing the distance between the normal sequence feature vector and the cluster center, not only can the next log information be predicted, but also the overall pattern of all normal sequences can be learned, thus improving the comprehensiveness and accuracy of anomaly detection;

[0046] Furthermore, considering that the system behavior represented by the complete log sequence is composed of multiple sub-processes from a macro perspective, each sub-process has its own semantics, the global attention mechanism of the classic Transformer is replaced by a local attention mechanism, which pays more attention to the correlation of local range logs. At the same time, the complete log sequence is divided into multiple local sequences, the semantics of each sub-process is first extracted locally, and then the sub-process semantics of the local sequence are integrated through the classic Transformer to form a complete log sequence semantic feature representation.

[0047] In summary, the present invention fully understands the semantic features of logs at the sentence level, divides log sequences into local sequences using a deep learning model, learns the features of each local sequence and then integrates the features of each local sequence, thereby fully solving the problems of insufficient understanding of log semantic features and lack of prominence of local features of log sequences. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 is a flow chart of a log anomaly detection method according to an embodiment of the present invention;

[0050] Figure 2 : is a diagram of the overall structure of the SentenceBERT and Local Attention Transformer models in an embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of the local attention Transformer structure in an embodiment of the present invention;

[0052] Figure 4 is a schematic diagram of the structure of an electronic device in an embodiment of the present invention;

[0053] Figure 5 It is a schematic diagram of the storage medium structure in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose and technical solution of the present invention clearer and easier to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] Terminology explanation:

[0056] SentenceBERT: is a pre-trained language representation method based on the BERT (Bidirectional Encoder Representations from Transformers) model, which is specifically used for sentence-level text. SentenceBERT learns the embedding representation of sentences by pre-training on a large amount of text data, so that semantically similar sentences are closer in the embedding space.

[0057] The core of the BERT model is the bidirectional encoder, which can simultaneously consider the contextual information of words in a sentence, thereby capturing the semantics of the sentence more accurately.

[0058] Transformer: Attention mechanism.

[0059] Word2Vec: A word embedding technology widely used in the field of natural language processing (NLP), which can map vocabulary into a vector space so that semantically similar words are closer in the vector space.

[0060] BOS Embedding (Beginning of Sentence Embedding) is a special embedding vector used to represent the beginning of a sentence in natural language processing. In some language models, BOS Embedding, together with the embeddings of other words in the sentence, helps the model understand the context of the entire sentence.

[0061] The present invention provides a log anomaly detection method, system, device and medium. The method mainly includes:

[0062] Obtaining the log sequence data of the electric mobile microservice system to be detected, inputting the log sequence data to be detected into a pre-trained anomaly detection model to obtain an anomaly detection result;

[0063] The anomaly detection model is trained based on a log data set, and includes a data processing module and an attention processing module; the data processing module is used to convert the input log sequence data into a semantic feature vector; the attention processing module is used to perform semantic feature representation according to the semantic feature vector, perform anomaly detection based on the semantic feature, and obtain anomaly detection results; the log data set adopts a power mobile microservice system log template and anomaly log sequence.

[0064] Specifically, the data processing module is obtained based on the BERT algorithm model or the SentenceBERT algorithm model, and specifically includes:

[0065] Generate a log template sliding window sequence according to the input log sequence data;

[0066] The BERT algorithm model or the SentenceBERT algorithm model is used to extract semantic features from each sliding window sequence, and each log template in the sliding window sequence is input into the algorithm model to obtain a semantic feature vector sequence.

[0067] The attention processing module includes a local attention submodule and a global attention submodule, specifically including:

[0068] The semantic feature vector sequence is divided into multiple sequences, and the multiple subsequences are respectively input into the local attention submodule, the semantics in the semantic feature vector sequence is positionally encoded, and after passing through the multi-head attention sublayer and the feedforward network sublayer, the semantic feature vector sequence is locally processed to obtain multiple subsequence feature vectors; each subsequence feature vector is input into the global attention submodule, and after passing through the second multi-head attention sublayer and the feedforward network sublayer, it is integrated, and the total semantic feature sequence is obtained after position decoding, and the total semantic feature sequence is parsed to obtain the anomaly detection result; the Transformer algorithm is:

[0069]

[0070] In the formula, K local ,V local are the key and value matrices containing only local neighbors, K is the key vector, V is the value vector, and Q is the query vector.

[0071] The log data set uses the normal log sequence and abnormal log sequence of the electric mobile microservice system as preprocessed data. The preprocessing steps include:

[0072] Use the log template parsing algorithm to parse the log sequence data in the data set and decompose it into a fixed text data part and a variable parameter data part;

[0073] The parsed data portion is converted into a microservice log tuple, where the microservice log tuple refers to a data structure corresponding to the parsed data.

[0074] The anomaly detection model is trained based on the log data set. The specific training steps include:

[0075] Use the mask mark to replace a certain proportion of log templates in the sequence as the abnormal log sequence, and input the abnormal log sequence and the normal log sequence into the anomaly detection model.

[0076] Input the feature vector of the mask in the abnormal log sequence into the softmax function to obtain the probability distribution of the log template set;

[0077] Use the cross entropy loss function as the loss function for mask log template learning;

[0078] Use the cluster center distance minimization loss function to constrain the distribution of the learned normal log sequence and find the smallest hypersphere that contains all normal data;

[0079] Based on the smallest hypersphere containing all normal data, the average operation is used to calculate the center of the normal log sequence cluster, and the data optimization is performed by minimizing the loss function using the cluster center distance.

[0080] Among them, the cross entropy loss function is:

[0081]

[0082] In the formula, is the actual log template at the i-th mask mark, M is the number of all masks in the j-th sequence, and N is the total number of sequences.

[0083] The cluster center distance minimization loss function is:

[0084]

[0085] In the formula, Loss CCDM The loss function is minimized for the cluster center distance, N is the total number of sequences, and μ is the average operation to calculate the normal log sequence cluster center.

[0086] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments, wherein the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0087] Example 1

[0088] As shown in the figure, Figure 2 As shown, a log anomaly detection method in this embodiment specifically includes the following implementation methods.

[0089] First, the power mobile microservice system logs L are statistically collected. The collected microservice system logs L are preprocessed to obtain the log data set D L .

[0090] Firstly, the log template parsing algorithm is used to parse the original log. The log template is a structured representation of the log message, which decomposes the log message into a fixed text part and a variable parameter part.

[0091] The fixed text part is usually the unchanging part of the log message, including the log level, event description, etc.; while the variable parameter part is the variable part of the log message, such as timestamp, user name, IP address, etc.

[0092] The log template parsing algorithm is a technology used to automatically identify and extract log templates from system logs. The purpose is to extract the common structure of log messages to facilitate the analysis and processing of log data. This technology is crucial for tasks such as log analysis, anomaly detection, and system monitoring because it can convert unstructured log data into a structured form, which is convenient for further processing and analysis. For example, the original log text is "[2021-03-15 10:00:00]INFO User JohnDoe logged in from IP 192.168.1.1" and the log template is "[DATETIME]INFO User <user>logged in from IP <ip>In this template, <timestamp> 、 <user>and <ip>are placeholders for variable parameters, which represent timestamp, username, and IP address respectively. The other parts are fixed text, representing the general structure of the log message.

[0093] Convert the original log into a microservice log tuple. The microservice log tuple is the data structure (trace_id, service_id, span_id, t, ts) corresponding to the original log after being parsed by the log template. Among them, trace_id identifies in which call chain this log is generated, service_id identifies which service the log is generated by, span_id identifies in which call in the call chain it is generated, t is the log template, and ts is the timestamp of log generation.

[0094] Construct an anomaly detection model including a data processing module and an attention processing module. The data processing module and the attention processing module are both constructed by neural networks. The data processing module is obtained based on the BERT algorithm model or the SentenceBERT algorithm model, and is used to convert the input text-based log sequence data into a semantic feature vector; the attention processing module is obtained based on the Transformer algorithm model, and includes a local attention submodule and a global attention submodule, which are used to represent semantic features according to the semantic feature vector, perform anomaly detection based on the semantic features, and obtain anomaly detection results.

[0095] Specifically, first generate a log template sliding window sequence. Let the sliding window width be w, and call the chain trace i The log template sequence is sorted in the order of generation timestamps to be S i = <t1,t2,...,t n >, for t j ∈S i , Represents the log template t j The semantic feature vector of the log template output after SentenceBERT. j The corresponding log template sliding window sequence SlidingWindow(S i ,t j ) is defined as follows:

[0096] If j+w-1≤n,

[0097] otherwise,

[0098] Traditional deep learning methods mostly use the index of log templates as input data for anomaly detection, which cannot make good use of the semantic feature information of logs. Therefore, the SentenceBERT model in the field of natural language processing is introduced to convert the textual log template information into semantic feature vectors, in order to better measure the correlation between different logs from the semantic features.

[0099] Traditional methods for extracting semantic features usually use Word2Vec to train the word vector matrix, and look up the word in the matrix by the index of the word to obtain the embedding representation of the corresponding word. Obviously, the Word2Vec method generates the same vector representation for each word. However, in many cases, the meaning of a word may change depending on its position and context in the sentence, so different vector representations should be generated for the word based on the context. In response to this demand, the BERT model came into being, which is able to generate different vector representations for a word based on its context.

[0100] SentenceBERT is a derivative model of BERT that can extract the semantic information of sentences. In the context of log anomaly detection, the log template is usually in the form of an English sentence. The common method used by the BERT model to extract the semantic features of a sentence is: first, the sentence is segmented. Then each word is input into the BERT model to obtain a word embedding vector, and then all the word embedding vectors are pooled (such as average or max-min pooling) to obtain the vector as the representation of the entire sentence. SentenceBERT is an improved model based on BERT, which is specially optimized for sentence-level representation. It is improved on the basis of BERT and can directly output the embedding representation of the entire sentence. SentenceBERT can capture the complex relationship and order information between words in a sentence, and the generated embedding vector has a strong semantic representation ability. In the problem scenario of logs, the overall semantics of the log is more concerned, that is, a log entry explains what happened, so the SentenceBERT model is used to extract the semantic features of the log.

[0101] Use the local attention Transformer to learn the feature representation of each local sequence, that is, the BOS Embedding output vector at the first token of each local sequence. After the SentenceBERT model of s302, a log template can obtain the corresponding semantic feature vector, and then a log sequence can be represented by a sequence of semantic feature vectors. The behavior of the system usually contains multiple sub-operations, each of which has its own semantics. When these sub-operations are arranged together, different patterns will produce new, system-level semantic features. Therefore, the semantic features of the sub-operations are first extracted through the local attention Transformer.

[0102] The classic Transformer model uses a global self-attention mechanism, where each element needs to calculate the attention score with all other elements in the sequence. Although this can capture global dependencies, it ignores the important local relationships between adjacent elements in the sequence, because the correlation between adjacent logs in a log sequence is usually higher than the correlation between logs that are farther apart. The introduction of the local attention mechanism limits each element to only consider a portion of its neighboring elements when calculating attention. The advantage of this is that it can focus more on capturing local information in the sequence, thereby improving the model's ability to understand local structures and details. The local attention Transformer model uses a design that is basically consistent with the classic Transformer in structure. Next, each part of the figure is introduced in detail.

[0103] like Figure 3 The bottom part of the input log template sequence is a length w, and a special token BOS (Begin Of Sequence) is added at the beginning. That is, S = {BOS, t1, t2, ..., t w },t i ∈T. T is the log template set, w is the sliding window width. After SentenceBERT, the corresponding log template semantic feature vector sequence {v BOS ,v t1 ,v t2 ,...v tw },v i ∈R d , d is the dimension of the log template semantic feature vector.

[0104] In order to model the position relationship in the sequence, Transformer uses position encoding to express the position information of the elements in the sequence. Specifically, the encoding PE of each position is generated in the same way as in the Transformer literature. i ∈R d , the dimension of the encoding vector is the same as the dimension of the log template semantic feature vector. The position encoding vector of each position is added to the log template semantic feature vector to obtain the final input representation:

[0105] x i =v i +PE i

[0106] After that, the input enters the encoder structure. The encoder consists of a multi-head attention sublayer and a feedforward network sublayer, each of which uses residual connections and layer normalization. The difference between the local attention Transformer model proposed in this section and the classic Transformer model lies in the calculation process of attention in the multi-head attention sublayer.

[0107] In the classic Transformer attention, each element is calculated with all other elements in the sequence. For the local attention mechanism, the calculation formula can be modified as

[0108]

[0109] Among them, K local ,V local is a key and value matrix containing only local neighbors.

[0110] Finally, the encoder output for the input sequence is Use h BOS As the sequence feature vector learned by the local attention Transformer.

[0111] The features of each local sequence are globally integrated through the second layer of the classic Transformer containing multi-head attention and feedforward network. After passing through the local attention Transformer of s302, the BOSEmbedding vector of each subsequence can be obtained. These vectors are regarded as a new sequence. Assuming that there are M subsequences and the dimension of the BOS vector is d, the dimension of the new sequence is M×d. The new sequence with position encoding added is input into the classic Transformer. This layer of Transformer will calculate the self-attention between subsequences and learn the correlation between them. The output dimension of this layer is still M×d.

[0112] Since the output of this layer of Transformer contains the global information of the entire sequence, the overall representation of the entire sequence can be obtained by pooling the output. The attention pooling method is adopted. First, for each element in the sequence, a learnable query vector q∈R is used. d To calculate its attention score. Then the score is normalized by softmax. Next, the calculated attention score is used to perform a weighted summation of all elements in the sequence. The final pooled representation is obtained by the Relu activation function. The result is a d-dimensional vector that integrates the information of the entire sequence and highlights the more important elements according to the attention score.

[0113] The anomaly detection model is trained based on the log data set. The specific training steps include:

[0114] Use the mask mark to replace a certain proportion of log templates in the sequence as the abnormal log sequence, and input the abnormal log sequence and the normal log sequence into the anomaly detection model.

[0115] Input the feature vector of the mask in the abnormal log sequence into the softmax function to obtain the probability distribution of the log template set;

[0116] Use the cross entropy loss function as the loss function for mask log template learning;

[0117] Use the cluster center distance minimization loss function to constrain the distribution of the learned normal log sequence and find the smallest hypersphere that contains all normal data;

[0118] Based on the smallest hypersphere containing all normal data, the average operation is used to calculate the center of the normal log sequence cluster, and the data optimization is performed by minimizing the loss function using the cluster center distance.

[0119] Among them, the first loss function specifically includes: masked log template learning. In order to learn the contextual information of the log sequence, the model is trained to learn the log templates masked by the mask in the log sequence. A certain proportion of the log templates in the sequence are randomly replaced with mask markers, and the model uses these randomly masked log sequences as input. The training goal is to accurately predict the randomly masked log templates. Its purpose is to enable the model to learn the correlation between log templates in normal log sequences. In order to achieve prediction, the feature vector of the i-th mask marker in the j-th sequence is replaced by Input to the softmax function, which will output the probability distribution over the entire log template set T:

[0120]

[0121] Where W k ,b k is a learnable parameter. Then, the cross entropy loss function is used as the loss function for mask log template learning, which is defined as:

[0122]

[0123] in is the actual log template at the i-th mask mark, M is the number of all masks in the j-th sequence, and N is the total number of sequences.

[0124] The second loss function specifically includes: minimizing the cluster center distance. Specifically: use the cluster center distance minimization loss function to constrain the distribution of the learned normal log sequence and find the smallest hypersphere containing all normal data. The main idea of ​​this method is that the distribution of normal log sequences in the feature space should be concentrated in one place and close to each other, while abnormal log sequences should be far away from the center of the normal sequence cluster. The output result is used as the feature representation of the entire sequence. This vector encodes the information of the entire sequence. In order to make the normal log sequences close to each other in the feature space, the center μ of the normal log sequence cluster is calculated by averaging:

[0125]

[0126] Then, the goal is to get the normal log sequence close to the cluster center, and the loss is calculated using the Euclidean distance:

[0127]

[0128] In the formula, Loss CCDM is the loss function that minimizes the distance between cluster centers, N is the total number of sequences, μ is the average operation to calculate the cluster center of the normal log sequence, It is the abnormal log sequence.

[0129] By minimizing the above formula, we hope that the center μ can represent the common pattern learned from all normal sequences. That is, after training, all normal log sequences in the training set should be close to the center, while abnormal sequences are farther from the center than normal sequences. At the same time, another advantage of defining the objective function in this way is that by making the normal sequence close to the center, the Transformer encoder can also utilize information from other normal log sequences through the center μ, because μ encodes the common pattern of normal log sequences.

[0130] The final loss function is defined as:

[0131] Loss=Loss MLTL +αLoss CCDM

[0132] Where α is a hyperparameter used to balance the two losses.

[0133] Given a test log sequence, similar to the training process, first randomly replace a certain proportion of the log template with a mask, and then use the masked log sequence as input to the model M.

[0134] For each mask, model M calculates the probability distribution, which represents the possibility of all log templates appearing at the mask position. Construct a candidate set consisting of c log templates, which correspond to the top c highest probabilities in the probability distribution. If the actual log template is in the candidate set, the location is considered normal. Then, since abnormal log sequences have different patterns from normal sequences, when using the model trained on normal sequences for prediction, the probability of the actual log template masked by the mask in the abnormal sequence being correct should be low. Therefore, if the actual log template is not in the top c candidate sets predicted by the model, it is considered that there is an anomaly there. Then, when a log sequence contains more than r abnormal log templates, the log sequence is considered to be an abnormal sequence.

[0135] Example 2

[0136] This embodiment also provides a log anomaly detection system, including:

[0137] A data detection module is used to obtain the log sequence data of the electric mobile microservice system to be detected, input the log sequence data to be detected into the trained anomaly detection model, and obtain the anomaly detection result;

[0138] The anomaly detection model module includes the anomaly detection model obtained by training based on a log data set, including a data processing module and an attention processing module; the data processing module is used to convert the input log sequence data into a semantic feature vector; the attention processing module is used to perform semantic feature representation according to the semantic feature vector, perform anomaly detection based on the semantic feature, and obtain anomaly detection results; the log data set adopts a power mobile microservice system log template and an abnormal log sequence.

[0139] The specific implementation principles of each module have been described in detail in the log anomaly detection method in Example 1, and will not be repeated here.

[0140] Example 3

[0141] Figure 4 A schematic diagram of a computer device provided by an embodiment of the present invention.

[0142] See also Figure 4 , the terminal device is a computer device. The computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the log anomaly detection method in the embodiment is implemented. To avoid repetition, it is not described one by one here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the fluid composition calculation system in the reservoir transformation wellbore of the embodiment are implemented. To avoid repetition, it is not described one by one here.

[0143] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Figure 4 This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0144] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, central processing units, graphics processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, data processing logic based on quantum computing, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0145] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0146] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0147] Any reference to memory, database or other media used in the embodiments provided in the present application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0148] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0149] Figure 5 A block diagram of a computer-readable storage medium of an electronic device provided according to an embodiment of the present invention.

[0150] See also Figure 5 The terminal device 600 is an electronic device, and the electronic device is in the form of a general computing device. The components of the electronic device may include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0151] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0152] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0153] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0154] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0155] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0156] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.< / ip> < / user> < / timestamp> < / ip> < / user>

Claims

1. A log anomaly detection method, characterized in that: include: Obtaining the log sequence data of the electric mobile microservice system to be detected, inputting the log sequence data to be detected into a pre-trained anomaly detection model to obtain an anomaly detection result; The anomaly detection model is trained based on a log data set, and includes a data processing module and an attention processing module; the data processing module is used to convert the input log sequence data into a semantic feature vector; the attention processing module is used to perform semantic feature representation according to the semantic feature vector, perform anomaly detection based on the semantic feature, and obtain anomaly detection results; the log data set adopts a power mobile microservice system log template and anomaly log sequence.

2. The log anomaly detection method according to claim 1, characterized in that: The data processing module is obtained based on the BERT algorithm model or the SentenceBERT algorithm model, and specifically includes: Generate a log template sliding window sequence according to the input log sequence data; The BERT algorithm model or SentenceBERT algorithm model is used to extract semantic features from each sliding window sequence. Each log template in the sliding window sequence is input into the algorithm model to obtain a semantic feature vector sequence.

3. The log anomaly detection method according to claim 1, characterized in that: The attention processing module includes a local attention submodule and a global attention submodule, specifically including: The semantic feature vector sequence is divided into multiple sequences, and the multiple subsequences are respectively input into the local attention submodule, the semantics in the semantic feature vector sequence is positionally encoded, and after passing through the multi-head attention sublayer and the feedforward network sublayer, the semantic feature vector sequence is locally processed to obtain multiple subsequence feature vectors; each subsequence feature vector is input into the global attention submodule, and after passing through the second multi-head attention sublayer and the feedforward network sublayer, it is integrated, and the total semantic feature sequence is obtained after position decoding, and the total semantic feature sequence is parsed to obtain the anomaly detection result; the Transformer algorithm is: In the formula, K local ,V local are the key and value matrices containing only local neighbors, K is the key vector, V is the value vector, and Q is the query vector.

4. The log anomaly detection method according to claim 1, characterized in that: The log data set uses normal log sequences and abnormal log sequences of the electric mobile microservice system as preprocessed data, and the preprocessing steps include: Use the log template parsing algorithm to parse the log sequence data in the data set and decompose it into a fixed text data part and a variable parameter data part; The parsed data portion is converted into a microservice log tuple, where the microservice log tuple refers to a data structure corresponding to the parsed data.

5. The log anomaly detection method according to claim 4, characterized in that: The anomaly detection model is trained based on the log data set, and the specific training steps include: Use the mask mark to replace a certain proportion of log templates in the sequence as the abnormal log sequence, and input the abnormal log sequence and the normal log sequence into the anomaly detection model. Input the feature vector of the mask in the abnormal log sequence into the softmax function to obtain the probability distribution of the log template set; Use the cross entropy loss function as the loss function for mask log template learning; Use the cluster center distance minimization loss function to constrain the distribution of the learned normal log sequence and find the smallest hypersphere that contains all normal data; Based on the smallest hypersphere containing all normal data, the average operation is used to calculate the center of the normal log sequence cluster, and the data optimization is performed by minimizing the loss function using the cluster center distance.

6. The log anomaly detection method according to claim 5, characterized in that: The cross entropy loss function is: In the formula, is the actual log template at the i-th mask mark, M is the number of all masks in the j-th sequence, and N is the total number of sequences.

7. The log anomaly detection method according to claim 5, characterized in that: The cluster center distance minimization loss function is: In the formula, Loss CCDM The loss function is minimized for the cluster center distance, N is the total number of sequences, and μ is the average operation to calculate the normal log sequence cluster center.

8. A log anomaly detection system, characterized in that: include: A data detection module is used to obtain the log sequence data of the electric mobile microservice system to be detected, input the log sequence data to be detected into the trained anomaly detection model, and obtain the anomaly detection result; The anomaly detection model module includes the anomaly detection model obtained by training based on a log data set, including a data processing module and an attention processing module; the data processing module is used to convert the input log sequence data into a semantic feature vector; the attention processing module is used to perform semantic feature representation according to the semantic feature vector, perform anomaly detection based on the semantic feature, and obtain anomaly detection results; the log data set adopts a power mobile microservice system log template and an abnormal log sequence.

9. A computing device, characterized in that include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for executing the log anomaly detection method described in any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to execute the log anomaly detection method according to any one of claims 1 to 7.

Images