Unsupervised log anomaly detection method and system based on pseudo anomaly generation

Abstract

The invention discloses an unsupervised log anomaly detection method and system based on pseudo anomaly generation, and the method comprises the steps: obtaining an original normal log sequence, and carrying out the preprocessing of the original normal log sequence, and obtaining an analysis log sequence; obtaining an initial log anomaly detection model, and training the initial log anomaly detection model according to the analyzed log sequence to obtain a target log anomaly detection model; and obtaining a target log sequence, inputting the target log sequence into the target log anomaly detection model, and outputting a target log state result. According to the invention, the normal log sequence is masked to generate the pseudo-abnormal log sequence, and the pseudo-abnormal log sequence is trained to obtain the target log anomaly detection model, so that the training efficiency of the log anomaly detection model is effectively improved, and meanwhile, the output accuracy of the target log state result is also improved.

Description

Technical Field

[0001] The present invention relates to the field of log detection technology, and in particular to an unsupervised log anomaly detection method, system, terminal and computer-readable storage medium based on pseudo-anomaly generation. Background Art

[0002] In the prior art, log anomaly detection is generally based on supervised and unsupervised anomaly detection methods of traditional machine learning and statistics. These methods have been applied to public log data sets. Unsupervised methods include principal component analysis and log aggregation, while supervised methods include support vector machines, logistic regression, and decision trees.

[0003] However, the log anomaly detection model training used for log anomaly detection in the prior art is based on the abnormal data in the data set as the benchmark, and the potential abnormal information is discovered by training the model. The abnormal data in the data set is required to cover all abnormal phenomena, which places high requirements on the data set. It can be seen that the log anomaly detection model training is completely dependent on the quality of the data set. If the abnormal information in the data set cannot fully cover all possible abnormal situations, then the detection model may have omissions or false alarms. On the other hand, most servers generate normal log information when operating, and may only generate one or two errors, which leads to low efficiency in log anomaly detection model training and low accuracy in log status monitoring results.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0005] The main purpose of the present invention is to provide an unsupervised log anomaly detection method, system, terminal and computer-readable storage medium based on pseudo-anomaly generation, aiming to solve the problem that the detection model training for detecting log status in the prior art is highly dependent on the abnormal data in the data set, resulting in low efficiency of detection model training and inaccurate log status detection results.

[0006] To achieve the above object, the present invention provides an unsupervised log anomaly detection method based on pseudo anomaly generation, and the unsupervised log anomaly detection method based on pseudo anomaly generation comprises the following steps:

[0007] Acquire an original normal log sequence, and preprocess the original normal log sequence to obtain a parsed log sequence;

[0008] Acquire an initial log anomaly detection model, and train the initial log anomaly detection model according to the parsed log sequence to obtain a target log anomaly detection model;

[0009] Obtain a target log sequence, input the target log sequence into the target log anomaly detection model, and output a target log status result.

[0010] Optionally, the unsupervised log anomaly detection method based on pseudo anomaly generation, wherein the step of obtaining an original normal log sequence and preprocessing the original normal log sequence to obtain a parsed log sequence, specifically includes:

[0011] Acquire an original normal log sequence, and perform log parsing processing on the original normal log sequence to obtain a first normal log sequence;

[0012] Performing log grouping processing on the first normal log sequence to obtain a second normal log sequence;

[0013] An abnormal record identification process is performed on the log events in the second normal log sequence to obtain a parsed log sequence.

[0014] Optionally, the unsupervised log anomaly detection method based on pseudo anomaly generation, wherein the performing log grouping processing on the first normal log sequence to obtain the second normal log sequence specifically includes:

[0015] Calculate the similarity between each log event in the first normal log sequence and the existing log events;

[0016] When the similarity is less than a preset threshold, the log events corresponding to the similarity less than the preset threshold are classified as non-log clusters;

[0017] When the similarity is greater than or equal to the preset threshold, the log events corresponding to the similarity greater than or equal to the preset threshold are classified as existing log clusters;

[0018] The non-log cluster and the existing log cluster are jointly constructed to obtain the second normal log sequence.

[0019] Optionally, the unsupervised log anomaly detection method based on pseudo anomaly generation, wherein the step of obtaining an initial log anomaly detection model and training the initial log anomaly detection model according to the parsed log sequence to obtain a target log anomaly detection model, specifically includes:

[0020] Acquire an initial log anomaly detection model, input the parsed log sequence into the initial log anomaly detection model, and perform mask processing on the parsed log sequence through the initial log anomaly detection model to obtain a pseudo anomaly log sequence;

[0021] Calculating an embedding vector norm score corresponding to the pseudo-abnormal log sequence, and when the embedding vector norm score is greater than a preset embedding vector norm score threshold, determining that the pseudo-abnormal log sequence is an abnormal log sequence;

[0022] When the initial log anomaly detection model training is completed, the target log anomaly detection model is obtained.

[0023] Optionally, the unsupervised log anomaly detection method based on pseudo anomaly generation, wherein the obtaining of the initial log anomaly detection model, inputting the parsed log sequence into the initial log anomaly detection model, and performing masking processing on the parsed log sequence through the initial log anomaly detection model to obtain the pseudo anomaly log sequence, specifically includes:

[0024] Acquire the initial log anomaly detection model, input the parsed log sequence into the generator of the initial log anomaly detection model, and perform word segmentation processing on the parsed log sequence through the generator to obtain a word segmentation log sequence;

[0025] Performing mask processing on the word segmentation log sequence to obtain context information of each word in the word segmentation log sequence;

[0026] Obtain a scalar corresponding to the context information, and use a sigmoid activation function to calculate a probability value corresponding to the scalar;

[0027] The vocabulary corresponding to the mask position in the word segmentation log sequence is predicted according to the probability value, and a pseudo abnormal log sequence is constructed according to the vocabulary.

[0028] Optionally, the unsupervised log anomaly detection method based on pseudo anomaly generation, wherein the initial log anomaly detection model is obtained, the parsed log sequence is input into the initial log anomaly detection model, and the parsed log sequence is masked by the initial log anomaly detection model to obtain a pseudo anomaly log sequence, and then further comprises:

[0029] A gated bilinear neural network is added after the generator, and the pseudo-abnormal log sequence is input into the gated bilinear neural network to optimize the initial log abnormality detection model.

[0030] Optionally, the unsupervised log anomaly detection method based on pseudo anomaly generation, wherein the step of calculating the embedding vector norm score corresponding to the pseudo anomaly log sequence, and determining that the pseudo anomaly log sequence is an anomaly log sequence when the embedding vector norm score is greater than a preset embedding vector norm score threshold, specifically includes:

[0031] Inputting the parsed log sequence into the discriminator of the initial log anomaly detection model, and performing substitution mark detection processing and hypersphere separation training processing on the parsed log sequence through the discriminator to obtain the preset embedding vector norm score threshold;

[0032] Inputting the pseudo-anomaly log sequence into the discriminator, and performing a replacement mark detection process and a hypersphere separation training process on the pseudo-anomaly log sequence through the discriminator to obtain the embedding vector norm score;

[0033] When the embedding vector norm score is greater than the preset embedding vector norm score threshold, the pseudo abnormal log sequence is determined to be an abnormal log sequence.

[0034] In addition, to achieve the above-mentioned purpose, the present invention also provides an unsupervised log anomaly detection system based on pseudo-anomaly generation, wherein the unsupervised log anomaly detection system based on pseudo-anomaly generation includes:

[0035] A preprocessing module, used to obtain an original normal log sequence, and preprocess the original normal log sequence to obtain a parsed log sequence;

[0036] A model training module, used to obtain an initial log anomaly detection model, and train the initial log anomaly detection model according to the parsed log sequence to obtain a target log anomaly detection model;

[0037] The log status output module is used to obtain a target log sequence, input the target log sequence into the target log anomaly detection model, and output a target log status result.

[0038] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and an unsupervised log anomaly detection program based on pseudo-anomaly generation stored in the memory and run on the processor, and when the unsupervised log anomaly detection program based on pseudo-anomaly generation is executed by the processor, the steps of the unsupervised log anomaly detection method based on pseudo-anomaly generation as described above are implemented.

[0039] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an unsupervised log anomaly detection program based on pseudo-anomaly generation, and when the unsupervised log anomaly detection program based on pseudo-anomaly generation is executed by a processor, the steps of the unsupervised log anomaly detection method based on pseudo-anomaly generation as described above are implemented.

[0040] In the present invention, an original normal log sequence is obtained, and the original normal log sequence is preprocessed to obtain a parsed log sequence; an initial log anomaly detection model is obtained, and the initial log anomaly detection model is trained according to the parsed log sequence to obtain a target log anomaly detection model; a target log sequence is obtained, and the target log sequence is input into the target log anomaly detection model to output a target log state result. The present invention combines the Electra-DP model with a gated bilinear network to construct a pseudo-abnormal log sequence, and the training of the log anomaly detection model can be completed only by relying on a normal log sequence, which not only effectively improves the training efficiency of the log anomaly detection model, but also greatly improves the accuracy of the output of the target log state result. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of a preferred embodiment of the unsupervised log anomaly detection method based on pseudo anomaly generation of the present invention;

[0042] Figure 2 It is a schematic diagram of log parsing of a preferred embodiment of the unsupervised log anomaly detection method based on pseudo anomaly generation of the present invention;

[0043] Figure 3 It is a schematic diagram of the overall structure of the Electra-DP model of a preferred embodiment of the unsupervised log anomaly detection method based on pseudo anomaly generation of the present invention;

[0044] Figure 4 It is a schematic diagram of a gated bilinear neural network structure of a preferred embodiment of an unsupervised log anomaly detection method based on pseudo anomaly generation of the present invention;

[0045] Figure 5 It is a schematic diagram of confusion matrices of a model of a preferred embodiment of the unsupervised log anomaly detection method based on pseudo anomaly generation of the present invention under three data sets: BGL, HDFS, and Thunderbird;

[0046] Figure 6 It is a schematic diagram of the AUC-PR curve and AUC-ROC curve of the model of the preferred embodiment of the unsupervised log anomaly detection method based on pseudo anomaly generation of the present invention under the BGL data set;

[0047] Figure 7 It is a schematic diagram of the AUC-PR curve and AUC-ROC curve of the model of the preferred embodiment of the unsupervised log anomaly detection method based on pseudo anomaly generation of the present invention under the HDFS data set;

[0048] Figure 8It is a schematic diagram of the AUC-PR curve and AUC-ROC curve of the model of the preferred embodiment of the unsupervised log anomaly detection method based on pseudo anomaly generation of the present invention under the Thunderbird data set;

[0049] Fig. 9 It is a schematic diagram of the recognition rates of normal samples and abnormal samples of a model of a preferred embodiment of the unsupervised log anomaly detection method based on pseudo anomaly generation of the present invention under different embedding norm scores;

[0050] Fig.10 It is a structural diagram of a preferred embodiment of the unsupervised log anomaly detection system based on pseudo anomaly generation of the present invention;

[0051] Fig.11 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] In the prior art, log anomaly detection is generally based on supervised and unsupervised anomaly detection methods of traditional machine learning and statistics. These methods have been applied to public log data sets. Unsupervised methods include principal component analysis and log aggregation, while supervised methods include support vector machines, logistic regression, and decision trees.

[0054] BERT (Bidirectional Encoder Representations from Transformer, also known as a pre-trained representation model) is built on the basis of Transformer (self-attention mechanism model), has powerful feature extraction and language representation capabilities, and can learn the bidirectional relationship in the sequence and the position relationship of the entire log message. Compared with previous models, BERT can better learn the contextual features of the text, accurately extract the semantic sentiment information in the text, and has achieved high-level results in 11 natural language processing tasks.

[0055] It can be seen that the existing technology is generally based on the abnormal data in the data set as a benchmark, and the potential abnormal information is discovered by training the model. The abnormal data of the data set is required to cover all abnormal phenomena, which has high requirements for the data set. On the other hand, a fully trained generator will have a high MLM (Masked Language Model) accuracy, so most of the replacement words are the original input words, which makes the sampling efficiency relatively low.

[0056] Considering that most of the log anomaly detection model training completely depends on the quality of the data set, if the abnormal information in the data set cannot fully cover all possible abnormal situations, then the detection model may have omissions or false alarms, etc. On the other hand, most servers generate normal log information when operating, and may rarely generate one or two errors. The present invention provides a LogAnomEX model (i.e., the target log anomaly detection model in the present invention), which can overcome the advantage that the model depends on the abnormal data in the data set, and only needs to input normal log data to automatically generate pseudo-abnormal data for model training. On the other hand, the LogAnomEX model provided in the present invention is an unsupervised model, which does not require a large amount of manpower to annotate the data set.

[0057] In order to overcome the defect that the traditional log anomaly detection model is highly dependent on abnormal data in the data set, the present invention proposes an unsupervised log anomaly detection method based on Electra-DP (a model generated by combining the Electra model and the difficulty prediction module DP) and a gated bilinear network. It makes full use of the idea of ​​the Electra model to generate pseudo-anomaly sequences and combines with the difficulty prediction module DP, which can efficiently and accurately detect log anomaly data when only normal log data is available.

[0058] The unsupervised log anomaly detection method based on pseudo anomaly generation described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the unsupervised log anomaly detection method based on pseudo anomaly generation includes the following steps:

[0059] Step S10: Acquire an original normal log sequence, and preprocess the original normal log sequence to obtain a parsed log sequence.

[0060] Logs are usually generated in unstructured or semi-structured formats. Before data input, they need to be converted into an organized structured format. This process is called log parsing. The parsed log contains a constant part and a variable part. Logs belonging to the same template are considered to be the same type of log events.

[0061] After log parsing, log events need to be grouped. Existing grouping strategies include fixed window, sliding window, and session window methods. Depending on the format of the data set, each log will choose a different grouping method based on its own specific situation. On the other hand, in order to perform quantitative analysis on log data, the parsed logs need to be tokenized. This involves breaking down structured log events into smaller segments or tags and assigning a unique numeric identifier to each tag.

[0062] The present invention uses the Drain parser to process the original log, first generates a regular expression to split the log information, then converts the log file into structured information, and on this basis builds a vocabulary from the training data, and uses this vocabulary to mark all parsed logs.

[0063] Specifically, an original normal log sequence is obtained, and log parsing processing is performed on the original normal log sequence to obtain a first normal log sequence; the similarity between each log event in the first normal log sequence and the existing log event is calculated; when the similarity is less than a preset threshold, the log events corresponding to the similarity less than the preset threshold are classified as non-log clusters; when the similarity is greater than or equal to the preset threshold, the log events corresponding to the similarity greater than or equal to the preset threshold are classified as existing log clusters; the non-log cluster and the existing log cluster are jointly constructed to obtain the second normal log sequence; abnormal record identification processing is performed on the log events in the second normal log sequence to obtain a parsed log sequence.

[0064] Data preprocessing process: The present invention uses the Drain parser to process log messages. The Drain parser uses a fixed-depth parse tree to evaluate the similarity of each log message with existing log events, and then decides whether to classify the message as an existing event or create a new template. The similarity calculation is mainly based on two formulas: Among them, simSeq is the similarity calculation, seq1(i) and seq2(i) are the elements at the i-th position in the two log messages, and n is the number of logs for similarity calculation. equ(t1, t2) is a function used to compare whether two elements (i.e. t1 and t2, two log messages) are equal. The meaning of the above formula is that for each position i, use equ(seq1(i), seq2(i)) to determine whether the elements of the two sequences at that position are the same. Assume that there are two log messages:

[0065] Sequence 1: ["Send", "message", "to", "server"]

[0066] Sequence 2: ["Send", "data", "to", "server"]

[0067] The comparison process is as follows:

[0068] equ("Send", "Send")=1

[0069] equ("message", "data") = 0

[0070] equ("to", "to")=1

[0071] equ("server", "server") = 1

[0072] Then the similarity is:

[0073] If the similarity is less than the set threshold, it is considered that the new log event does not belong to any existing log cluster and a new template needs to be created. In addition, the present invention will select whether to use a session or a sliding window strategy to group logs according to whether the log data contains a block identifier specific to the target domain (mainly depending on whether the target data contains blk_id, some data sets do not contain blk_id, and when it is impossible to distinguish whether it is a log message under the same event, a sliding window is used) and on this basis, a vocabulary is constructed from the training data, and this vocabulary is used to mark all parsed logs (most log records now come with warning signs).

[0074] like Figure 2 As shown, suppose a log sequence s = {s1, s2, ..., s d} consists of d log events parsed from the original log message. The steps for processing these log events are as follows Figure 2 Specifically, the present invention defines a training set sequence (reading the log sequence according to the format of the data set, and then the preprocessing part is uniformly processed by the expression) as Dtrain = {(s1, y1 = 0), (s2, y2 = 0), ..., (s N ,y N =0)}, where each y i Marks whether the i-th record in sequence s is an abnormal record.

[0075] Assume y i =0 means normal recording, while y i =1 indicates abnormal records. These labels appear in the data set in the form of labels. In the detection phase, the test set data is D test ={(s N+1 ,y N+1 ),…,(s N+M ,y N+M)} The test data contains a mixture of normal and abnormal logs. Even if some log events are N It may be normal, but the whole D test May contain abnormal information, the model needs to be in D test According to D train The knowledge learned can judge the whole D test Whether the log sequence is an abnormal sequence.

[0076] Step S20: obtaining an initial log anomaly detection model, and training the initial log anomaly detection model according to the parsed log sequence to obtain a target log anomaly detection model.

[0077] The LogAnomEX model (LogAnom corresponds to Log Anomaly, and EX stands for Extension) uses Electra's generator-discriminator architecture to implement log anomaly detection (ELECTRA model is a pre-trained model, and its full name is Efficiently Learning an Encoder that Classifies Token Replacements Accurately. The ELECTRA model uses a discriminative rather than generative pre-trained text encoder, borrowing the idea of ​​adversarial networks, and trains two neural network models: the generator and the discriminator. The task of the generator is to randomly mask words in the original text and perform predictive learning, while the task of the discriminator is to determine whether the word is consistent with the original text. This joint training method allows the parameters of the ELECTRA model to not be back-propagated in the generator and discriminator during training, and only the embedding is shared, thereby improving the efficiency and effectiveness of the model). The Electra model introduces a generative adversarial pre-training task called RTD (Replace Token Detection), which is a binary classification task that aims to identify replaced tokens in the input sequence.

[0078] The recognition process is as follows: 1. Process the input sequence through BERT to obtain the context representation of each token (the token here is the word or term in a single log message); 2. Convert the context representation of each token into a scalar (logits) through weighted summation of several fully connected layers; 3. Use the sigmoid activation function to convert logits into probability values, indicating the probability of each token being replaced. By counting the probabilities of all the tokens, the one with the largest probability is the word to be recognized and replaced.

[0079] During the pre-training process of the ELECTRA model, the generator cannot directly obtain information feedback from the discriminator, which makes the sampling process of the generator inefficient. Moreover, as the prediction accuracy of the generator continues to increase, the generator will have the problem of oversampling, and will use those correct words as replacement words, making the discriminator's learning inefficient. To this end, the difficulty prediction module HP and sampling smoothing can be used to improve the sampling efficiency of the generator and improve the performance of the model.

[0080] The present invention combines the ELECTRA model and the difficulty prediction module as the basis, and is applied to the field of anomaly detection to realize anomaly detection of server logs. A pseudo-anomaly log sequence can be generated under the condition of only normal logs, and the model is trained through the pseudo-anomaly log sequence. At the same time, an unsupervised training method is implemented to save manpower and material resources. The model is tested under the public data sets BGL, HDFS and Thunderbird respectively, and still maintains a good result.

[0081] During the training process, the generator uses a masked language model (MLM) to first segment each input log sequence, then randomly select some tokens and replace them with MASK tags in a certain ratio. The generator then tries to predict based on the contextual information of the sequence. The prediction process is as follows: the masked sequence is as follows: ['The', '[MASK]', 'brown', 'fox', 'jumps', 'over', 'the', '[MASK]', 'dog'], and these tokens process the input vector through the attention mechanism to capture the contextual information of the sequence. For example, the vector representation of the [MASK] position at position 2 can be updated by the information at position 1 ('The') and position 3 ('brown').

[0082] The generator outputs a probability distribution that represents the probability of each possible token in the vocabulary appearing at that position. Select the largest vocabulary replacement mask, and then create a pseudo-abnormal sequence An based on the appropriate token corresponding to the MASK position. At the same time, the task of the discriminator is to identify which tokens in the sequence are replaced by the generator. The two discriminators calculate the norm of the embedding vector corresponding to the CLS (CLS is a symbol used to represent the aggregate information of multiple input sequences) of the generated normal log sequence and pseudo-log sequence respectively. This method makes the discriminator very sensitive to inconsistent information in the sequence and can effectively distinguish normal from abnormal logs. Among them, CLS is not the input of the gated linear network, it is an independent part and is input into the HST module for training.

[0083] In simple terms, multiple log sequences are aggregated together and represented by a CLS. The CLS is parsed by BERT. The underlying embedding vector is assumed to be VCLS = [0.5, 0.2, -0.3, 0.1]. The norm is to calculate the Euclidean distance of this vector in space, that is, the square root of the sum of the squares of each vector. If there is a problem with the log sequence, then the Euclidean distance calculated by its corresponding CLS must be different from that of the normal log sequence. After multiple trainings, try to find such a threshold that can be distinguished. The threshold standard used in the present invention is the 99th percentile of the CLS scores corresponding to all normal log sequences sorted from small to large. If the CLS score of the log sequence in the test set is higher than this, it means that it is an abnormal sequence.

[0084] It is understandable that the present invention divides the structured log data obtained after parsing (i.e., the parsed log sequence in the present invention) into two batches, one batch is normal data, and the other batch is the data to be processed that will be used to generate the pseudo-abnormal log. The normal data is input into the discriminator, and the embedding vector norm score of the CLS corresponding log sequence is calculated (i.e., the preset embedding vector norm score threshold in the present invention). The data to be processed is generated into pseudo-abnormal data by the generator of the Electra model, and the discriminator is responsible for detecting which sequences in the pseudo-abnormal sequence are forged log rules after being modified by the generator, and calculating the norm score of the CLS embedding vector of this pseudo-abnormal log sequence.

[0085] Specifically, the initial log anomaly detection model is obtained, the parsed log sequence is input into the generator of the initial log anomaly detection model, and the parsed log sequence is segmented by the generator to obtain a segmented log sequence; the segmented log sequence is masked to obtain context information of each word in the segmented log sequence; a scalar corresponding to the context information is obtained, and a probability value corresponding to the scalar is calculated using a sigmoid activation function; the word corresponding to the mask position in the segmented log sequence is predicted according to the probability value, and a pseudo-anomaly log sequence is constructed according to the word.

[0086] Generator: The generator in Electra uses MLM to output the probability distribution PG on the vocabulary Vocab. The generator is trained using MLM loss: Among them, L MLM is the mask loss, E is the expected value operator, which is used to calculate the average loss of all mask patterns. d is the number of tokens in the sequence, mask token, binary variable. m i = 1 means the i-th token is masked. Generator model P G Predict the i-th tag s i The logarithmic probability of . s is the input sequence, θ Gare the parameters of the generator. Training with this loss ensures that the generator assigns higher probabilities to tokens that are likely to appear in the original normal sequence.

[0087] However, during the pre-training process, since there is no direct information feedback loop between the generator and the discriminator, the two parts of the model training process are completely independent, resulting in very inefficient sampling of the generator. On the other hand, after multiple rounds of training, the accuracy of the generator in the masked language model (MLM) task will become very high. Because the generator has learned the pattern of normal logs very well, when predicting masked words, the generator usually outputs the same words as the original input. This results in the replacement words generated by the generator not actually changing and still being the original words, thereby reducing the efficiency of sampling, because the generator does not provide enough diversity to create effective pseudo-anomaly data, so the sampling efficiency is very low. In response to the above problems, the present invention adds a difficulty prediction module (Difficulty Prediction, DP) to the generator to improve the performance of the model by improving the sampling efficiency of the generator.

[0088] The core of DP is to allow the generator to accept feedback from the discriminator and then sample more replacement words that are more difficult for the discriminator. Figure 3 The main structure of the Electra-DP model is that in addition to the original MLM head, the present invention adds a sampling head DP for sampling replacement words, aiming to let the generator learn the probability that the discriminator predicts a replacement word as the original word. Therefore, the sampling distribution is changed from the original masked language distribution to the following formula: Where, is the updated masked language distribution, c is the sentence obtained after masking operations are performed on some positions in the input sentence x, x' is a specific replacement word generated by the generator based on the perturbed sentence c, x* represents all possible replacement words, and P G (x'|c) represents the probability that the MLM generator generates word xi' at mask position i under the condition of a perturbation sentence c, L D (x', c) represents the loss of the discriminator corresponding to the generator when the replacement word is x', which can be expressed as: in, is the expected discriminator score, and D(x′,c) is the actual discriminator score for each replacement word x′.

[0089] Among them, regarding the generator tampering with the words in the log sequence, the vocabulary is the vocabulary obtained from the training set during the training phase. Here, the words tampered by the generator are found in the vocabulary. In addition, the generator replaces words according to a probability distribution P GHowever, as the training time increases, the discriminator will gradually adapt to the rule of replacing words by the generator, and the probability of successful prediction will become higher and higher, and the loss of the discriminator L D Will become smaller and smaller, so a change is made here to multiply the probability of generator word replacement by the loss L of the discriminator D , so that as the number of training times increases, the loss of the discriminator becomes smaller, then the numerator will become smaller, and the probability of the generator selecting a specific word x in the vocabulary will become smaller, thereby driving the generator to find other words for replacement.

[0090] The loss function of the sampling head is as follows: Among them, L S (x′, c) is the loss function of the sampling head. Z is defined as the weighted average of the discriminator loss under all possible replacement words generated by the generator, that is, the average discriminator loss of all possible replacement words. Then Z in the original Electra model can be expressed as: In short, in the original Electra model distribution P G The estimated variance of the discriminator loss is: Among them, Vocab is the output vocabulary, which is distributed in P G Next, for each possible replacement x, the loss L D The variance between its expectation Z. Here we introduce a new sampling distribution P S , by using P S Instead of the original distribution P G , implement importance sampling, and recalculate the expected value Z under this new distribution.

[0091] If the sampling distribution P S Enable If P is kept constant, the variance can be minimized. S satisfy The estimated variance can be made zero, which is the optimal solution in theory. It can minimize the variance of the estimated discriminator loss, thereby improving training efficiency and model robustness.

[0092] Here the numerator uses L D (x'|c) instead of L S (x'|c) is because when constructing the sampling distribution P S (x'|c), the goal of the model is to select replacement tags that are more challenging for the discriminator, and it is necessary to use a loss that directly reflects the discriminator on these replacement tags, so the discriminator loss L D (x'|c) is multiplied by the output probability P of the MLM head G, we can get the initial sampling distribution P S (x'|c). In addition, during the pre-training process, the MLM head of the generator usually achieves a high accuracy. This causes the generator to oversample the correct words as replacement words, which reduces the learning efficiency of the discriminator. To solve this problem, the focal loss is used to replace the traditional cross entropy loss. Compared with the cross entropy loss, the focal loss adds a regulation factor: L fc G (x′,c)=-(1-p G (x′|c)) γ logp G (x′|c), where L fc is the focal loss function, logP G (x'|c) is a base loss term, which is part of the standard cross entropy loss and is used to measure the gap between the generator's predicted words and the true words. 1-PG(x'|c) γ The role is to predict the probability P of the generator G (x'|c) adjusts the weight of the loss. G When (x'|c) is close to 1, the generator's prediction of word x is very accurate, 1-P G (x'|c) γ will be very small, thus reducing the contribution of this sample to the total loss, which means that the generator will not overly rely on these easily predicted words when sampling. On the contrary, when a position is difficult to predict, that is, P G (x'|c) is very low, and the adjustment factor of the loss function will make the loss weight at this position close to the standard cross entropy loss, thereby ensuring that these unpredictable words contribute more to the sampling. γ is a hyperparameter used to control the degree of influence. When γ = 0, the focal loss degenerates to the standard cross entropy loss because the adjustment factor becomes 1. As γ increases, the influence of the adjustment factor increases, the loss contribution to easy-to-predict samples decreases, and the loss contribution to difficult-to-predict samples increases.

[0093] Through such a mechanism, focal loss can effectively smooth the sampling distribution of the generator, reduce the problem of the generator always sampling the correct replacement words in the later stage of training, and improve the collaborative training effect of the generator and the discriminator.

[0094] The output of the Electra-DP model is mainly a binary classification result, which is used to generate pseudo-abnormal log sequences and identify which sequences in the log sequences have been replaced. The LogAnomEX model outputs the anomaly score of the log sequence and gives a judgment on whether it is an abnormality based on the threshold. Electra-DP and the gating network together constitute the LogAnomEX model.

[0095] A gated bilinear neural network is added after the generator, and the pseudo-abnormal log sequence is input into the gated bilinear neural network to optimize the initial log abnormality detection model.

[0096] like Figure 4 As shown, Figure 4 Gated bilinear neural network: Since the generator uses the pre-trained BERT model, the returned word vector already contains enough semantic information, but BERT uses absolute position encoding by default, which limits its ability to capture long distances and complex dependencies between tokens. In log data, relative position information is critical to understanding the order and context of log events. Therefore, the present invention adds a Gated-Bilinear neural network (i.e., a gated bilinear neural network) after BERT output embedding, and uses the features of adjacent tokens to enhance the model's ability to detect log anomaly information. At the same time, a gating mechanism and layer normalization are used to enhance the performance and stability of the model.

[0097] The Gated-Bilinear feature fusion layer receives the output of the BERT feature extraction layer as input, where x is the word vector corresponding to the i-th token, with a dimension of 256. First, the input tensor passes through the feature fusion layer, where the Bilinear feature fusion layer takes x and xi+1 as input and outputs a new 512-dimensional tensor xi′. For the entire sequence tensor x, the input of the Bilinear feature fusion layer is x itself and the result of shifting one step right in the sequence dimension. The model adds a special marker at the beginning of the sequence, but the effect of this marker on label prediction is small and can be ignored. x′=x T W roll(x,1)+b, where x is the word vector, x T is a transpose, used for mathematical calculations such as matrix multiplication. The roll function shifts the vector xi′ right by 1 step in the sequence dimension. W and b are the weight and bias of the bilinear neural network respectively. The gated linear unit can use the gate mechanism for feature selection. First, the tensor x′ is cut in half in the last dimension into tensors A and B with dimensions of 256 each, and then the gated linear unit is used to calculate the new vector x″. Sigmoid is an activation function that maps the input to between 0 and 1. Here, it means mapping the value of B to between 0 and 1. It is point-by-point multiplication.

[0098] The Dropout layer randomly sets the elements in the vector x″ to 0 with a probability of 15% to enhance the generalization ability of the model. After x″ passes through the Dropout layer, it passes through the LayerNorm layer. The formula of LayerNorm is as follows: Where E[x″] is the mean of x″, Variance[x″] is the variance of x″, δ is a small value used to avoid division by 0, and γ and β are learnable parameters. The LayerNorm layer performs normalization on the dimensions of each input sentence to avoid the gradient vanishing and gradient exploding problems and make the transfer of hidden states more stable.

[0099] The parsed log sequence is input into the discriminator of the initial log anomaly detection model, and the discriminator performs alternative label detection processing and hypersphere separation training processing on the parsed log sequence to obtain the preset embedding vector norm score threshold; the pseudo-anomaly log sequence is input into the discriminator, and the discriminator performs alternative label detection processing and hypersphere separation training processing on the pseudo-anomaly log sequence to obtain the embedding vector norm score; when the embedding vector norm score is greater than the preset embedding vector norm score threshold, the pseudo-anomaly log sequence is determined to be an abnormal log sequence; when the training of the initial log anomaly detection model is completed, the target log anomaly detection model is obtained.

[0100] The norms of the CLS embedding vectors of normal log sequences and pseudo-abnormal log sequences are input into the hypersphere separation training module HST. HST sorts the CLS norm scores of normal sequences from small to large through the hypersphere function, and selects the score at the 99% position as a threshold. After the training is completed, whether the log is an abnormal log is determined by judging whether the CLS embedding norm score of the log sequence exceeds the threshold.

[0101] Discriminator: Before the log sequence S is input into the discriminator, each sequence is added with a [CLS] tag, for example, the sequence S = {CLS, S1, S2, ..., Sn}. Here, S1, S2, ..., Sn may be a normal sequence marked as 0, or a pseudo-abnormal sequence marked as 1. The discriminator network implements anomaly detection through two independent training tasks: alternative label detection (RTD) and hypersphere separation training (HST).

[0102] Replacement Token Detection (RTD): This task involves binary classification of each token in the input sequence (except [CLS]) to distinguish which tokens are original and which are replaced by the generator. RTD, as a preliminary training step for the discriminator, not only helps the model identify potential anomalies at the token level, but also lays the foundation for subsequent overall sequence anomaly detection. Therefore, the goal of RTD is to minimize the binary cross entropy (BCE) loss on the training samples: Where P D is the probability distribution function of the discriminator, m i is a binary variable indicating whether the i-th token is replaced, PD (m i |s;θ D ) means that in a single sequence s and parameter θ D The probability that the i-th position is replaced under the condition .

[0103] Hypersphere Separation Training (HST): In anomaly detection models, since normal data and abnormal data are difficult to distinguish in the representation space, the discriminator has poor accuracy. Hypersphere Separation Training (HST) adopts an idea similar to contrastive learning, which aims to increase the discrimination by spatially separating the distance between normal and abnormal features, so that the features of normal logs are tightly clustered in the feature space, and the abnormal features are pushed to the periphery of this clustering area, thus forming a clear boundary on the hypersphere. The goal of HST is to minimize a specific loss function that measures the spatial distance between normal and abnormal feature vectors, so as to enhance the model's ability to identify anomalies.

[0104] The present invention introduces the hypersphere function and uses this geometric property to calculate the score corresponding to each log sequence [CLS] label to distinguish normal sequences from pseudo-abnormal sequences. In simple terms, binary classification problems usually use the standard binary cross entropy loss function, where the goal is to distinguish normal logs from abnormal logs in auxiliary data. The formula is as follows:

[0105] Among them, BCE is the abbreviation of binary cross entropy loss, and its full name is: Binary Cross Entropy Loss, S i is the i-th log sequence, y i is the label of the i-th sample, normal samples are 0, and abnormal samples are 1. φ(S i ; θ) is the CLS feature vector corresponding to the i-th sequence S output by the network. l(·) is a function that maps the output to anomaly scores, and in order to ensure that the log sequence is compact, the Gaussian Radial Basis Function (GRBF) is used as l(·). l(s) = exp(-‖s‖2), so here l(s) is the log sequence vector representation, || 2 Replacing the Gaussian radial basis function with the cross entropy loss function represents the Euclidean distance, and we get the loss function for hypersphere training: Here n represents the total number of log sequences. In summary, the anomaly score is defined as the distance from the log vector to the center. i ) represents the anomaly score represented by the feature vector of the i-th log sequence. Anomaly(S i )=‖φ(S i ;θ)‖2.

[0106] In Hypersphere Separation Training (HST), the loss function LHST consists of two parts. The first part: (1-y i )‖φ(S i ;θ)‖2, minimize the feature embedding norm of the normal sample to make it as close to the origin as possible, the second part: y i log(1-exp(-‖φ(S i ; θ)‖2)), ensuring that the feature embedding norm of the abnormal sample is large enough and far away from the origin, which helps prevent the model from converging to a trivial solution. The generator is responsible for generating pseudo-anomaly sequences during the training phase. Each log sequence is directly input to the discriminator after being combined with the [CLS] tag. The discriminator i )The final score evaluates the abnormality of each sequence. When the score of the log sequence exceeds the Anomaly(x i ) score is determined by the 99th percentile of the score, that is, Anomaly (S i )≥ε, the sequence is marked as abnormal.

[0107] Step S30: Obtain a target log sequence, input the target log sequence into the target log anomaly detection model, and output a target log status result.

[0108] After the target log anomaly detection model is trained, the present invention can perform anomaly detection analysis on the target log sequence input therein, and output the corresponding log status result, including a normal sequence or an abnormal sequence.

[0109] The present invention uses the electra model combined with the difficulty prediction module to generate pseudo-anomaly logs to train the anomaly detection model. Different from the traditional log detection model, it only needs a normal log sequence and does not rely on the error information in the log sequence.

[0110] Experimental verification process: This paper selects three most widely disseminated public data sets, BGL, HDFS, and Thunderbird, and conducts tests. The following gives the experimental results of the model under different data sets. Figure 5 As shown, Figure 5 This is the confusion matrix of the model under the three datasets of BGL, HDFS, and Thunderbird. Figure 6 , Figure 7 as well as Figure 8 The AUC-PR curve and AUC-ROC curve of the model under the three data sets of BGL, HDFS, and Thunderbird; Fig. 9is the recognition rate of normal samples and abnormal samples under different embedding norm scores. It can be seen that the trained target log anomaly detection model of the present invention can realize accurate recognition of the state of the data set.

[0111] Furthermore, if Fig.10 As shown, based on the above-mentioned unsupervised log anomaly detection method based on pseudo-anomaly generation, the present invention also provides an unsupervised log anomaly detection system based on pseudo-anomaly generation, wherein the unsupervised log anomaly detection system based on pseudo-anomaly generation includes:

[0112] A preprocessing module 51 is used to obtain an original normal log sequence and preprocess the original normal log sequence to obtain a parsed log sequence;

[0113] A model training module 52 is used to obtain an initial log anomaly detection model, and train the initial log anomaly detection model according to the parsed log sequence to obtain a target log anomaly detection model;

[0114] The log status output module 53 is used to obtain a target log sequence, input the target log sequence into the target log anomaly detection model, and output a target log status result.

[0115] Furthermore, if Fig.11 As shown, based on the above-mentioned unsupervised log anomaly detection method and system based on pseudo-anomaly generation, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Fig.11 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0116] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, an unsupervised log anomaly detection program 40 based on pseudo-anomaly generation is stored on the memory 20, and the unsupervised log anomaly detection program 40 based on pseudo-anomaly generation can be executed by the processor 10, thereby realizing the unsupervised log anomaly detection method based on pseudo-anomaly generation in the present application.

[0117] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the unsupervised log anomaly detection method based on pseudo-anomaly generation.

[0118] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0119] In one embodiment, when the processor 10 executes the unsupervised log anomaly detection program 40 generated based on pseudo anomalies in the memory 20, the following steps are implemented:

[0120] Acquire an original normal log sequence, and preprocess the original normal log sequence to obtain a parsed log sequence;

[0121] Acquire an initial log anomaly detection model, and train the initial log anomaly detection model according to the parsed log sequence to obtain a target log anomaly detection model;

[0122] Obtain a target log sequence, input the target log sequence into the target log anomaly detection model, and output a target log status result.

[0123] The obtaining of the original normal log sequence and preprocessing of the original normal log sequence to obtain the parsed log sequence specifically includes:

[0124] Acquire an original normal log sequence, and perform log parsing processing on the original normal log sequence to obtain a first normal log sequence;

[0125] Performing log grouping processing on the first normal log sequence to obtain a second normal log sequence;

[0126] An abnormal record identification process is performed on the log events in the second normal log sequence to obtain a parsed log sequence.

[0127] The performing log grouping processing on the first normal log sequence to obtain a second normal log sequence specifically includes:

[0128] Calculate the similarity between each log event in the first normal log sequence and the existing log events;

[0129] When the similarity is less than a preset threshold, the log events corresponding to the similarity less than the preset threshold are classified as non-log clusters;

[0130] When the similarity is greater than or equal to the preset threshold, the log events corresponding to the similarity greater than or equal to the preset threshold are classified as existing log clusters;

[0131] The non-log cluster and the existing log cluster are jointly constructed to obtain the second normal log sequence.

[0132] The step of obtaining an initial log anomaly detection model and training the initial log anomaly detection model according to the parsed log sequence to obtain a target log anomaly detection model specifically includes:

[0133] Acquire an initial log anomaly detection model, input the parsed log sequence into the initial log anomaly detection model, and perform mask processing on the parsed log sequence through the initial log anomaly detection model to obtain a pseudo anomaly log sequence;

[0134] Calculating an embedding vector norm score corresponding to the pseudo-abnormal log sequence, and when the embedding vector norm score is greater than a preset embedding vector norm score threshold, determining that the pseudo-abnormal log sequence is an abnormal log sequence;

[0135] When the initial log anomaly detection model training is completed, the target log anomaly detection model is obtained.

[0136] The step of obtaining an initial log anomaly detection model, inputting the parsed log sequence into the initial log anomaly detection model, and performing masking on the parsed log sequence through the initial log anomaly detection model to obtain a pseudo anomaly log sequence specifically includes:

[0137] Acquire the initial log anomaly detection model, input the parsed log sequence into the generator of the initial log anomaly detection model, and perform word segmentation processing on the parsed log sequence through the generator to obtain a word segmentation log sequence;

[0138] Performing mask processing on the word segmentation log sequence to obtain context information of each word in the word segmentation log sequence;

[0139] Obtain a scalar corresponding to the context information, and use a sigmoid activation function to calculate a probability value corresponding to the scalar;

[0140] The vocabulary corresponding to the mask position in the word segmentation log sequence is predicted according to the probability value, and a pseudo abnormal log sequence is constructed according to the vocabulary.

[0141] The method further includes: obtaining an initial log anomaly detection model, inputting the parsed log sequence into the initial log anomaly detection model, and performing masking on the parsed log sequence through the initial log anomaly detection model to obtain a pseudo anomaly log sequence, and then:

[0142] A gated bilinear neural network is added after the generator, and the pseudo-abnormal log sequence is input into the gated bilinear neural network to optimize the initial log abnormality detection model.

[0143] The step of calculating the embedding vector norm score corresponding to the pseudo-abnormal log sequence, and determining that the pseudo-abnormal log sequence is an abnormal log sequence when the embedding vector norm score is greater than a preset embedding vector norm score threshold, specifically includes:

[0144] Inputting the parsed log sequence into the discriminator of the initial log anomaly detection model, and performing substitution mark detection processing and hypersphere separation training processing on the parsed log sequence through the discriminator to obtain the preset embedding vector norm score threshold;

[0145] Inputting the pseudo-anomaly log sequence into the discriminator, and performing a replacement mark detection process and a hypersphere separation training process on the pseudo-anomaly log sequence through the discriminator to obtain the embedding vector norm score;

[0146] When the embedding vector norm score is greater than the preset embedding vector norm score threshold, the pseudo abnormal log sequence is determined to be an abnormal log sequence.

[0147] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an unsupervised log anomaly detection program based on pseudo-anomaly generation, and when the unsupervised log anomaly detection program based on pseudo-anomaly generation is executed by a processor, the steps of the unsupervised log anomaly detection method based on pseudo-anomaly generation as described above are implemented.

[0148] In summary, the present invention provides an unsupervised log anomaly detection method based on pseudo-anomaly generation and related equipment, the method comprising: obtaining an original normal log sequence, and preprocessing the original normal log sequence to obtain a parsed log sequence; obtaining an initial log anomaly detection model, and training the initial log anomaly detection model according to the parsed log sequence to obtain a target log anomaly detection model; obtaining a target log sequence, and inputting the target log sequence into the target log anomaly detection model to output a target log state result. The present invention combines the Electra-DP model with a gated bilinear network to construct a pseudo-anomaly log sequence, and the training of the log anomaly detection model can be completed only by relying on a normal log sequence, which not only effectively improves the training efficiency of the log anomaly detection model, but also greatly improves the accuracy of the output of the target log state result.

[0149] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0150] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0151] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. An unsupervised log anomaly detection method based on pseudo anomaly generation, characterized in that: The unsupervised log anomaly detection method based on pseudo anomaly generation includes: Acquire an original normal log sequence, and preprocess the original normal log sequence to obtain a parsed log sequence; Acquire an initial log anomaly detection model, and train the initial log anomaly detection model according to the parsed log sequence to obtain a target log anomaly detection model; Obtain a target log sequence, input the target log sequence into the target log anomaly detection model, and output a target log status result.

2. The unsupervised log anomaly detection method based on pseudo anomaly generation according to claim 1, characterized in that: The obtaining of the original normal log sequence and preprocessing the original normal log sequence to obtain the parsed log sequence specifically includes: Acquire an original normal log sequence, and perform log parsing processing on the original normal log sequence to obtain a first normal log sequence; Performing log grouping processing on the first normal log sequence to obtain a second normal log sequence; An abnormal record identification process is performed on the log events in the second normal log sequence to obtain a parsed log sequence.

3. The unsupervised log anomaly detection method based on pseudo anomaly generation according to claim 2, characterized in that: The performing log grouping processing on the first normal log sequence to obtain a second normal log sequence specifically includes: Calculate the similarity between each log event in the first normal log sequence and the existing log events; When the similarity is less than a preset threshold, the log events corresponding to the similarity less than the preset threshold are classified as non-log clusters; When the similarity is greater than or equal to the preset threshold, the log events corresponding to the similarity greater than or equal to the preset threshold are classified as existing log clusters; The non-log cluster and the existing log cluster are jointly constructed to obtain the second normal log sequence.

4. The unsupervised log anomaly detection method based on pseudo anomaly generation according to claim 1, characterized in that: The obtaining of the initial log anomaly detection model and training the initial log anomaly detection model according to the parsed log sequence to obtain the target log anomaly detection model specifically includes: Acquire an initial log anomaly detection model, input the parsed log sequence into the initial log anomaly detection model, and perform mask processing on the parsed log sequence through the initial log anomaly detection model to obtain a pseudo anomaly log sequence; Calculating an embedding vector norm score corresponding to the pseudo-abnormal log sequence, and when the embedding vector norm score is greater than a preset embedding vector norm score threshold, determining that the pseudo-abnormal log sequence is an abnormal log sequence; When the initial log anomaly detection model training is completed, the target log anomaly detection model is obtained.

5. The unsupervised log anomaly detection method based on pseudo anomaly generation according to claim 4, characterized in that: The obtaining of the initial log anomaly detection model, inputting the parsed log sequence into the initial log anomaly detection model, and performing masking on the parsed log sequence by the initial log anomaly detection model to obtain a pseudo anomaly log sequence specifically includes: Acquire the initial log anomaly detection model, input the parsed log sequence into the generator of the initial log anomaly detection model, and perform word segmentation processing on the parsed log sequence through the generator to obtain a word segmentation log sequence; Performing mask processing on the word segmentation log sequence to obtain context information of each word in the word segmentation log sequence; Obtain a scalar corresponding to the context information, and use a sigmoid activation function to calculate a probability value corresponding to the scalar; The vocabulary corresponding to the mask position in the word segmentation log sequence is predicted according to the probability value, and a pseudo abnormal log sequence is constructed according to the vocabulary.

6. The unsupervised log anomaly detection method based on pseudo anomaly generation according to claim 5, characterized in that: The initial log anomaly detection model is obtained, the parsed log sequence is input into the initial log anomaly detection model, and the parsed log sequence is masked by the initial log anomaly detection model to obtain a pseudo anomaly log sequence, and then the following steps are further included: A gated bilinear neural network is added after the generator, and the pseudo-abnormal log sequence is input into the gated bilinear neural network to optimize the initial log abnormality detection model.

7. The unsupervised log anomaly detection method based on pseudo anomaly generation according to claim 4, characterized in that: The calculating the embedding vector norm score corresponding to the pseudo-abnormal log sequence, and determining that the pseudo-abnormal log sequence is an abnormal log sequence when the embedding vector norm score is greater than a preset embedding vector norm score threshold, specifically includes: Inputting the parsed log sequence into the discriminator of the initial log anomaly detection model, and performing substitution mark detection processing and hypersphere separation training processing on the parsed log sequence through the discriminator to obtain the preset embedding vector norm score threshold; Inputting the pseudo-anomaly log sequence into the discriminator, and performing a replacement mark detection process and a hypersphere separation training process on the pseudo-anomaly log sequence through the discriminator to obtain the embedding vector norm score; When the embedding vector norm score is greater than the preset embedding vector norm score threshold, the pseudo abnormal log sequence is determined to be an abnormal log sequence.

8. An unsupervised log anomaly detection system based on pseudo anomaly generation, characterized in that: The unsupervised log anomaly detection system based on pseudo anomaly generation includes: A preprocessing module, used to obtain an original normal log sequence, and preprocess the original normal log sequence to obtain a parsed log sequence; A model training module, used to obtain an initial log anomaly detection model, and train the initial log anomaly detection model according to the parsed log sequence to obtain a target log anomaly detection model; The log status output module is used to obtain a target log sequence, input the target log sequence into the target log anomaly detection model, and output a target log status result.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and an unsupervised log anomaly detection program based on pseudo-anomaly generation stored in the memory and executable on the processor. When the unsupervised log anomaly detection program based on pseudo-anomaly generation is executed by the processor, the steps of the unsupervised log anomaly detection method based on pseudo-anomaly generation as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an unsupervised log anomaly detection program based on pseudo-anomaly generation, and when the unsupervised log anomaly detection program based on pseudo-anomaly generation is executed by a processor, the steps of the unsupervised log anomaly detection method based on pseudo-anomaly generation according to any one of claims 1 to 7 are implemented.

Images