Anomaly detection system and method for evolution logs

Abstract

The present invention provides an anomaly detection system and method for evolution logs. The system is mainly divided into two stages. In the first stage, semantic vectors are obtained by semantic analysis of the original log and the evolution log, and a preliminary log mapping table is obtained by determining the similarity of the semantic vectors, and then the accuracy of the log mapping table is determined by workflow verification. In the second stage, the evolution log is used as data input, and with the assistance of the final log mapping table, the original log is obtained through log mapping, and the training model (LSTM) of the original log is used to formally implement anomaly detection on the original log to obtain the anomaly detection result of the evolution log. The present invention is based on fine-grained analysis of the differences between the original log and the evolution log, adopts semantic similarity analysis and workflow verification, establishes the log event connection between the original log and the evolution log, makes full use of the training results of the original log detection algorithm, and can effectively reduce the cost of real-time log anomaly detection.

Description

Technical Field

[0001] The present invention relates to the technical field of network data processing, and in particular to an anomaly detection system and method oriented to evolution logs. Background Art

[0002] The software will be continuously upgraded, and the logs it outputs are also continuously upgraded. There are certain differences between the logs output by the old version of the software and the logs output by the upgraded software, but the differences are not large. Therefore, the results of the anomaly detection training of the logs output by the old version of the software can be applied to the logs output by the upgraded software to detect whether there are anomalies in the upgraded software, thereby reducing the training cost of the anomaly detection training of the logs output by the upgraded software.

[0003] Similar work to the present invention is Multi-CAD [Xie X, Jin Z, Wang J, et al. Confidence Guided Anomaly Detection Model for Anti-concept Drift in Dynamic Logs [J]. Journal of Network and Computer Applications, 2020, 162: 102659.]. Multi-CAD uses a multi-algorithm collaborative working mode to address the problem of concept drift, and examines the concept drift in the log through a comprehensive analysis of the results of multiple algorithms. In essence, concept drift is the difference in logs of the same type due to reasons such as upgrades. Multi-CAD first proposes a statistical value p_value to measure the inconsistency between logs, and establishes a link between the new log and the previous log; in addition, multiple suitable algorithms are selected as inconsistency measures to calculate the combined detection score for judging the consistency between logs. Then, a confidence-guided parameter adjustment method is designed for anti-concept drift of dynamic logs, and a score set with corresponding labels is updated from the trusted results containing labels, inconsistency scores and confidence through a feedback mechanism as preliminary experience for subsequent detection. Multi-algorithm collaboration uses confidence to analyze and determine the detection results of multiple algorithms and select the best one for anomaly detection. This method can be regarded as a coarse-grained method, because Multi-CAD does not have a fine-grained analysis and determination of the difference between the original log and the evolved log, and the multi-algorithm collaboration used cannot fully utilize the training results of the original log.

[0004] In summary, the existing method of using the old version log anomaly detection training results to detect the logs output by the upgraded software has the problem of not being able to analyze and determine the differences between the old version logs and the upgraded software logs, resulting in the inability to fully utilize the training results of the old version logs. Summary of the invention

[0005] In order to solve the problem in the prior art that there is no analysis and differences between original (old version) logs and evolved (upgraded software) logs, resulting in the inability to fully utilize the training results of the original logs, the present invention provides an anomaly detection system for evolved logs, which uses semantic similarity analysis and workflow verification to establish log event connections between original logs and evolved logs, and can fully utilize the training results of the original log detection algorithm.

[0006] In a first aspect, the present invention provides an anomaly detection system for evolving logs, comprising a log template word vector generation module, a similarity determination module, a workflow verification module, a log mapping module, and a trained LSTM module;

[0007] The log template word vector generation module is used to generate an original log template word vector and an evolved log template word vector by semantic analysis of the original log and the evolved log respectively;

[0008] The similarity determination module is used to generate a set of similar log template pairs by performing similarity analysis on the original log template word vector and the evolved log template word vector;

[0009] The workflow verification module is used to verify the set of similar log template pairs by using a workflow method to obtain a log mapping table;

[0010] The log mapping module is used to convert the evolution log into a log similar to the original log based on the log mapping table;

[0011] The trained LSTM module is used to detect the original log of the class through the LSTM model trained by the original log to obtain the detection result of the evolution log.

[0012] Further, the log template word vector generation module includes a log template mining unit, a word vector generation unit, a word weight generation unit and a log template word vector generation unit;

[0013] The log template mining unit is used to perform log template mining on the original log and the evolved log to obtain an original log template set and an evolved log template set, wherein the original log template set includes the core content of the original log text, and the evolved log template set includes the core content of the evolved log text;

[0014] The word vector generating unit is used to respectively extract the word vectors of the original log template set and the evolved log template set by using the Glove algorithm;

[0015] The word weight generating unit is used to extract the word weights of the original log template set and the evolved log template set respectively by using the TFIDF algorithm;

[0016] The log template word vector generating unit is used to generate the original log template word vector and the evolved log template word vector based on the outputs of the word vector generating unit and the word weight generating unit.

[0017] Further, the log template mining unit includes a log preprocessing subunit, a log template mining subunit and a log module output subunit;

[0018] Wherein, the log preprocessing subunit is used to preprocess the original log and the evolution log based on a regular expression;

[0019] The log template mining subunit is used to mine the preprocessed original log and the evolved log using three log template algorithms to obtain three initial original log template sets and three initial evolved log template sets;

[0020] The log module output subunit is used to merge the three initial original log template sets and the three initial evolved log template sets respectively based on big tree voting to obtain the original log template set and the evolved log template set.

[0021] Furthermore, the three log template algorithms are Drain algorithm, LogMine algorithm and Spell algorithm.

[0022] Furthermore, the similarity determination module is specifically used for:

[0023] Calculating the word vector set values ​​of the original log template and the evolved log template respectively, and the original log template and the evolved log template whose difference of the word vector set values ​​is less than the word vector threshold are judged as a preliminary similar log template pair;

[0024] The sum of square differences of word vectors of corresponding words in the similar log template pair is calculated, and the original log template and the evolved log template whose sum of square differences of word vectors of corresponding words is less than the log template pair threshold are determined to be similar log template pairs.

[0025] Furthermore, the workflow verification module is specifically used to:

[0026] The workflow verification module is specifically used for:

[0027] Select any mapping node pair in the similar log template pair set as the center to construct the original event association graph and the evolved event association graph;

[0028] Cutting the original event correlation graph and the evolved event correlation graph;

[0029] Based on the random walk method, the pruned original event correlation graph and the evolved event correlation graph are converted into the original event low-dimensional vector and the evolved event low-dimensional vector;

[0030] The similarity between the original event low-dimensional vector and the evolved event low-dimensional vector is calculated to generate the log mapping table.

[0031] Furthermore, the system also includes a trial inspection module and an abnormality inspection module; wherein the trial inspection module is used to inspect the validity of the log mapping table; and the abnormality inspection module is used to determine whether to update the trained LSTM module based on the output of the trial inspection module.

[0032] Further, the test inspection module is used to inspect the validity of the log mapping table, and specifically includes a first round inspection module and a second round inspection module;

[0033] The first round inspection module is used to use the sample class original log mapped by the log mapping table as the input data of the first round inspection module, use the LSTM model in the trained LSTM module as the detection model, implement anomaly detection, and calculate the accuracy, recall rate and F1 score. If the difference between the F1 score of the sample class original log detection and the F1 score of the original log detection is less than the preset first round threshold, it is considered that the log mapping table is feasible to replace and passes the first round of trial inspection;

[0034] The second-round inspection module is used to delete the independent content in the original log of the sample class that has not passed the first-round trial inspection, and use it as input data for the second-round inspection module. The LSTM model in the trained LSTM module is used as the detection model to implement anomaly detection, and to count the accuracy, recall rate and F1 score. If the difference between the F1 score of the input data of the second-round inspection module and the F1 score of the original log detection is less than a preset second-round threshold, it is considered that the final log mapping table is accurately mapped and passes the second-round trial inspection; wherein the independent content is the part of the evolution log that is inconsistent with the original log and cannot be mapped by the mapping table.

[0035] Furthermore, the anomaly detection module is specifically used for:

[0036] When the log mapping table used is replaceable, the trained LSTM module does not need to be updated;

[0037] When the used log mapping table is accurately mapped, supplementary training is performed on the LSTM model in the trained LSTM module;

[0038] When the used log mapping table fails the second round of test, the LSTM model in the trained LSTM module is retrained.

[0039] In a second aspect, the present invention provides an anomaly detection method for evolving logs, comprising:

[0040] Step 1: Obtain the original log and the evolution log, and generate the original log template word vector and the evolution log template word vector respectively through semantic analysis of the original log and the evolution log; wherein the original log is the log output by the old version of the software; and the evolution log is the log output by the upgraded software;

[0041] Step 2: Performing similarity analysis on the original log template word vector and the evolved log template word vector to generate a set of similar log template pairs;

[0042] Step 3: Using a workflow method, verify the set of similar log templates to obtain a log mapping table;

[0043] Step 4: Convert the evolution log into a log similar to the original log based on the log mapping table;

[0044] Step 5: Detect the class original log using the LSTM model trained with the original log to obtain the detection result of the evolution log.

[0045] Beneficial effects of the present invention:

[0046] (1) The present invention implements anomaly detection based on fine-grained analysis of the differences between the original log and the evolved log, fully utilizes the original detection algorithm, adopts semantic similarity analysis and workflow verification, establishes log event connections between the original log and the evolved log, provides support for subsequent anomaly detection of the same type of logs, fully utilizes the training results of the original log detection algorithm, and reduces the cost of real-time log anomaly detection.

[0047] (2) After the workflow is verified, the present invention performs a test test using sample data, and determines whether the log mapping table needs to be maintained and upgraded based on the test test results, so as to improve the accuracy of the evolution log detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a system flow provided by an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of the log template word vector generation process provided by an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of a log template mining process provided by an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of an event association graph generation process provided by an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of graph embedding provided by an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of the composition of an evolution log provided by an embodiment of the present invention;

[0054] Figure 7 A flowchart of an anomaly detection method for evolving logs provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] For ease of understanding, some of the terms that appear in the specific embodiments of the present invention are explained as follows:

[0056] Original log and evolution log: Software will be continuously upgraded, and its output log will also be continuously upgraded. For ease of understanding, the present invention refers to the log output by the old version of software as the original log, and the log output by the upgraded software as the evolution log.

[0057] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] like Figure 1 As shown, an embodiment of the present invention provides an anomaly detection system for evolving logs, including: a log template word vector generation module, a similarity determination module, a workflow verification module, a log mapping module and a trained LSTM module.

[0059] A log template word vector generation module is used to generate an original log template word vector and an evolved log template word vector by semantic analysis of the original log and the evolved log respectively;

[0060] A similarity determination module is used to generate a set of similar log template pairs by performing similarity analysis on the original log template word vector and the evolved log template word vector;

[0061] The workflow verification module is used to verify the set of similar log templates by using the workflow method to obtain a log mapping table;

[0062] A log mapping module, used to convert the evolution log into a log similar to the original log based on the log mapping table;

[0063] The trained LSTM module is used to detect the original logs using the LSTM model trained on the original logs to obtain the detection results of the evolved logs.

[0064] The system proposed in the present invention is based on a fine-grained analysis of the differences between the original log and the evolved log, implements anomaly detection on the basis of making full use of the original detection algorithm, adopts semantic similarity analysis and workflow verification, establishes log event connections between the original log and the evolved log, provides support for subsequent anomaly detection of the same type of logs, makes full use of the training results of the original log detection algorithm, and reduces the cost of real-time log anomaly detection.

[0065] The following is a detailed description of the system:

[0066] The system is mainly divided into two stages. In the first stage, semantic vectors are obtained from the original logs and the evolution logs through semantic analysis, and a preliminary log mapping table is obtained through semantic vector similarity determination. Then, the accuracy of the log mapping table is determined through workflow verification. In the second stage, the evolution log is used as data input. With the assistance of the final log mapping table, the original log is obtained through log mapping. The training model (LSTM) of the original log is used to formally implement anomaly detection on the original log to obtain the anomaly detection results of the evolution log.

[0067] It should be noted that log anomaly detection can be a model used by other anomaly detection. Considering the characteristics of LSTM, it is used as a detection model by many log anomaly detection schemes. In view of this, the present invention takes LSTM as an example to describe the workflow of the present invention.

[0068] 1. Log template word vector generation module

[0069] The main components of the log template word vector generation module are as follows Figure 2 As shown in the figure, the original log and the evolved log are respectively obtained by semantic analysis to obtain the original log template word vector and the evolved log template word vector, which are divided into a log template mining unit, a word vector generation unit, a word weight generation unit and a log template word vector generation unit. The log template mining unit filters the unimportant parts of the original log text, such as time, level and other information, and obtains the log template. The word vector generation unit takes the log template as input and uses the Glove algorithm to generate the word vector of each word. The word weight generation unit takes the log template as input and uses the TFIDF algorithm to generate the word weight of each word. The log template word vector generation unit generates each log template word vector based on the word vector and word weight.

[0070] 1.1 Log Template Mining Unit

[0071] The present invention uses log templates as data for similarity determination of new and old logs. The original text of the new and old logs in the evolution log is not used as the determination data, because there are a lot of obviously different contents in the original log text, such as the time of log recording. If these parts are included in the data for similarity determination, the new and old logs will be obviously different. Therefore, the log template is used as the data source for determination, because the log template retains the most core content in the original log text, and these contents are the most critical and relatively unchanged components.

[0072] like Figure 3 As shown in the figure, the log template mining unit is divided into three subunits, including the log preprocessing subunit, the log template mining subunit and the log module output subunit. The log preprocessing subunit preprocesses the original log and the evolution log. Specifically, it removes the unimportant content of the log text based on the regular expression method to avoid interfering with the consistency analysis of the new and old logs. In order to ensure the accuracy of the log template mined by the log template mining subunit, the log template mining subunit uses three log template algorithms for mining. Finally, the log template output subunit obtains accurate log templates based on the majority voting method.

[0073] The log template output subunit outputs the more likely accurate log template from the outputs of the three log template mining algorithms based on majority voting. Since no log template algorithm can parse the log template completely and accurately, three log template mining algorithms, including Drain, LogMine and Spell, are used to mine log templates respectively. Only log templates output by more than two algorithms are determined as log templates.

[0074]

[0075]

[0076] The present invention does not limit the log template mining algorithm used by the log template mining subunit, but different types of log template mining algorithms need to be used respectively, such as: log template mining algorithm based on frequent item mining (such as Spell), log template mining algorithm based on heuristic rules (such as Drain). Based on different types of log template mining algorithms, it is more likely to avoid log template mining errors caused by the same feature of the log. Only when different types of log template mining algorithms cooperate can the log template be obtained more accurately.

[0077] Based on the above description, the log template mining subunit processes information as follows:

[0078]

[0079] 1.2 Word vector generation unit

[0080] The embodiment of the present invention is based on the Glove algorithm, which takes the log template set T as input and generates the word vector of each word in the log template. For the evolution log, the semantics of several log items in the new and old logs are basically the same, and the difference is often reflected in the difference in word position and order, which is often due to the software developer's adjustment to the log output format, such as adding modifiers to further clarify the identity of the subject of the log activity. If the log items are understood strictly in order, then the semantics of the log items in the new and old logs will be greatly different. The Glove algorithm does not pay too much attention to the order of words, but emphasizes the frequency of common occurrence of words, so the embodiment of the present invention uses the Glove word vector generation algorithm to calculate the word vector of each word in the log template.

[0081]

[0082] Based on the above process, the word vector generation unit takes the original log template set generated by the original log and the evolved log template set generated by the evolved log as input, and obtains the corresponding word vectors of the original log template set and the word vectors of the evolved log template set respectively.

[0083] 1.3 Word Weight Generation Unit

[0084] In order to make the semantics in the log template clearer, the embodiment of the present invention further determines the weight of each word, thereby providing a basis for determining the semantic similarity of the log template as a whole. The embodiment of the present invention adopts the TFIDF algorithm to determine the importance of words in the log template. The TFIDF algorithm essentially analyzes the importance of a word in determining a certain document, that is, if a word only appears in a certain document and rarely appears in other documents, then the word is very important. The core of the TFIDF algorithm includes two steps: obtaining the word frequency (i.e. TF), and the inverse document frequency (i.e. IDF). Multiplying the two values ​​of each word is the TFIDF value of the word.

[0085]

[0086] Based on the above process, the word weight generation unit takes the original log template set generated by the original log and the evolved log template set generated by the evolved log as input to obtain the corresponding word weights of the original log template set and the evolved log template set.

[0087] 1.4 Log Template Word Vector Generation Unit

[0088] The log template word vector is generated based on several key words and word vectors in the log template. Log templates generally have multiple words, which are not always useful for determining the consistency of the log template and may even cause misjudgment. Considering that the number of words in a log template generally does not exceed 10, the embodiment of the present invention only retains 3 words in the log template, and the remaining words do not participate in the generation of log template word vectors.

[0089] The criterion for selecting words is based on word weight, that is, selecting the three words with the highest word weight in the log template.

[0090]

[0091]

[0092] 2 Similarity determination module

[0093] The similarity determination module determines the similarity between the original log template and the evolved log template based on the original log template word vector and the evolved template word vector. After the original log and the evolved log are processed by the log template word vector generation module, each log template in the corresponding generated set contains at most 3 most representative words, thereby avoiding the influence of irrelevant words on the similarity determination. The evolved log template set T obtained by the evolved log n Mainly, check the original log template set T obtained by the original log o Is there a log template similar to this?

[0094] In order to reduce the complexity of the judgment, a two-round judgment method is adopted. In the first round of judgment, the first round of similarity judgment is performed according to the word vector set values ​​of the original log template and the evolved log template. Two log templates whose word vector set values ​​differ by less than the word vector threshold fa_h are judged as preliminary similar log template pairs and enter the second round of similarity judgment. In the second round of judgment, similar log template pairs are used as input, and the square differences of the corresponding word vectors are summed. Only the log templates with the sum of square differences less than the log template pair threshold fa_x are considered similar and are judged as similar log template pairs. It can be understood that fa_h is relatively loose to avoid missing similar log template pairs, while fa_x is relatively strict to avoid mistakenly assuming that log template pairs are similar.

[0095] The method for calculating the word vector set value of the log template is: Assume that the log template p i The word vector is represented as p i ={s 1 *c 1 ,…,s i *c i}, then the word vector set value sum_p of the log template i =s 1*c 1 +…+s i *c i The log templates that are judged similar in the first round enter the second round of similarity judgment.

[0096] The similarity method used in the second round of judgment is: evolution log template p x ={x 1 ,…,x i}, where x i Represents a word, the original log template p y ={y 1 ,…,y j}, where y j Represents a word. If i is not equal to j, then p x and p y If i is equal to j, the similarity calculation formula is as follows:

[0097]

[0098] The final output of the similarity determination module is a set of similar log template pairs, denoted as U. The log template pair set contains zero or more elements, each of which is a tuple {to, tn}, where to represents a log template of the original log, and tn represents a log template of the evolved log, that is, U records the similarity of the log templates between the original log and the evolved log.

[0099]

[0100] 3. Workflow Verification Module

[0101] After the similarity determination module, a mapping relationship between the evolved log template and the original log template is established, namely U. The similarity determination module determines the similarity between the new and old log templates from a semantic perspective, but there is still a certain risk of error. Although the log templates are similar, it may be due to the software designer's casualness that the log output statements with the same syntax are used, but they describe different events.

[0102] The workflow verification module verifies the correctness of the log template mapping from the perspective of the software workflow. The log is the output of the software work process. If, from the perspective of the workflow, the old and new log event templates (that is, the evolved log template and the original log template still match), then the mapping can be verified to be correct. For example, the existing log template mapping E→e, in the original log, there is an event template sequence {A, B, C, D, E, H, I, J} that reflects the workflow. After scanning, it is found that in the evolved log, there is also an event template sequence {A, B, C, D, e, H, I, J} that reflects the workflow. Then the correctness of the log template mapping E→e is verified, that is: E in the original log becomes e in the evolved log after software upgrades and other reasons.

[0103] Workflow verification is approximately implemented from the perspective of graph embedding. Assuming that the log template to be verified is mapped as: E→e, then take E as the central node, scan the original log, and construct the event association graph G related to E. 1 ; Take e as the central node, scan the evolution log, and build the event association graph G related to E 2 ; Use the graph embedding method to obtain G 1 and G 2 The low-dimensional vector of the two is obtained and the similarity between them is determined accordingly; if they are similar, it is considered that the mapping relationship “E→e” has passed the workflow verification.

[0104] The method for constructing an event association graph with a log template x as the center is as follows. Determine the log message msg corresponding to the log template x; scan the first 5 and last 5 log messages of the log message msg, and obtain the event templates corresponding to these log messages; use these event templates as nodes and construct directed edges in the order in the original log (for example: if the node is the log template corresponding to the message before the log message msg, then this edge should point to x, if it is after msg, then this edge should point from x to the node); if the node already exists, then the weight of the node is +1 (the default weight of the new node is 1); scan the entire log and establish the event association graph of the log template x.

[0105] Further explanation of the establishment of the event association graph of the log template x. In the initial case, the graph has only one node x and no edges. When mining the log template, the correspondence between the log template and the log message is recorded, so the corresponding log message msg (may have multiple messages) can be found in the log according to x. The log messages adjacent to msg are queried (the query range is limited to 5 log messages before and after), and the log message sequence {m 1 ,m 2 ,m 3 ,m 4 ,m 5 ,msg,m 6 ,m 7,m 8 ,m 9 ,m 10}; According to the mapping relationship between log templates and log messages, the log message sequence can be converted into a log template sequence {t 1 ,t 2 ,t 3 ,t 4 ,t 5 ,x,t 6 ,t 7 ,t 8 ,t 9 ,t 10}; The log template sequence can get 10 graphs (t 1 →t 2 ),(t 2 →t 3 ),…,(t 9 →t 10 ); Check whether the above 10 graphs exist in the event association graph. If they exist, add 1 to the weight of the corresponding node. If they do not exist, add them to the event association graph. The default weight of the newly added node is 1; process all log messages in the log and obtain the event association graph of log template x.

[0106] like Figure 4 As shown in Figure 1, the event association graph generation process is schematically illustrated. In the initial case, the event association graph of the log template x has only one node x, and the log message corresponding to the log template is msg; when scanning the log, if the log message sequence {msg1, msg} is found, node 1 is added to the event association graph, with a default weight of 1 and an edge {1→x}; when scanning the log, if the log message sequence {msg, msg2} is found, node 2 is added to the event association graph, with a default weight of 1 and an edge {x→2}; when scanning the log, if the log message sequence {msg1, msg} is found, the event association graph now has node 1 and an edge {1→x}, so only node 1 is added with a weight of 2.

[0107]

[0108] The embodiment of the present invention verifies the correctness of log template mapping from the workflow based on the similarity judgment of event association graph. The event association graph of the event template reflects the event sequence related to the event template, which represents the workflow information related to the event template. If the event association graphs corresponding to two log templates are similar, it means that they have the same status in the software workflow. This verifies the establishment of the log template mapping relationship from another perspective.

[0109] It can be understood that by adopting the graph embedding method, the event association graph is converted into a low-dimensional vector, thereby facilitating similarity determination. Graph similarity determination often requires repeated comparisons and a large number of operations, so the present invention can adopt the graph embedding method to convert the high-level graph vector into a low-dimensional word vector based on the semantics of the log template, thereby reducing the complexity of similarity determination.

[0110] Furthermore, we will filter out some noise based on the weight. Due to the inevitable existence of noise, the expected strict event sequence often does not exist. For example, the event sequence reflecting the workflow should be {a, b, c}, but because multiple processes concurrently output to the same log, the actual event sequence may be {a, b, e, c} or {a, f, b, d, c}. We believe that noise always occurs with a low probability, and the node weight reflects the number of times an event occurs, so we will filter out events with a low probability based on the weight.

[0111]

[0112] The modified random walk method is used to implement graph embedding based on the pruned event association graph. The basic idea is to start from the initial node x of the event association graph and select node_x_1, the node whose neighbor node weight is the largest and has not been visited; starting from node_x_1, select node_xx_1, the node whose neighbor node weight is the largest and has not been visited; similarly, traverse the subtree with node_x_1 as the root according to the depth-first principle; then select node_x_2, the node whose neighbor node weight is the largest and has not been visited, and visit the subtree with node_x_2 as the root according to the depth-first principle; similarly, visit all nodes in G.

[0113] Combination Figure 5 The diagram embedding process is described. First, the event association graph is pruned. According to process 8, node 2 needs to be pruned. Because the weight of node 1 is 7, it has the largest weight among all neighbor nodes of x, so node 1 is visited first. Because the subtree with node 1 as the root has only one node, node x is returned. Starting from node x again, the node with the largest weight and not visited is visited, so node 3 is visited according to the rule. According to the depth-first rule, the next node to be visited is node 4. And so on, until all nodes are visited.

[0114] The graph embedding vector is obtained based on the word vector and weight of the subtree node. Assuming that the event association graph G of x has 5 subtrees (or node x has 5 neighboring nodes), the low-dimensional vector value PG of the event association graph of x can be expressed as PG = {pg_1, pg_2, pg_3, pg_4, pg_5}, where pg_i is the vector value of each subtree. The vector value pg_i of the subtree is calculated as follows: Assuming that the word vectors of all nodes in the subtree are {c 1 ,c 2 ,c 3 ,…,c n}, the corresponding weight is {q 1 ,q 2 ,q 3 ,…,q n},So

[0115]

[0116]

[0117] In order to verify the similarity of the event association graph, the embodiment of the present invention uses cosine similarity for determination. Assuming PGx = {pg_x1, pg_x2, ..., pg_xn}, PGy = {pg_y1, pg_y2, ..., pg_ym}, the similarity of the two vectors is determined as:

[0118]

[0119] Given a similarity threshold fa_g, if Cos(PGx,PGy) is greater than fa_g, the event association graphs of the two people are considered similar, and the mapping between the current log template x and the original log template y passes the workflow verification.

[0120] In summary, the workflow verification process is as follows:

[0121]

[0122]

[0123] 4. Test module

[0124] Furthermore, in order to verify the validity of the log mapping, the evolution log is detected based on the training results of the original log, and the log mapping is examined through the detection indicators. After the above operations, the log mapping table has been established, so that the mapping relationship between certain log events in the original log and the evolution log is known. In other words, after obtaining the event template of the evolution log, the evolution log can be converted into a log similar to the original log, and the detection is implemented based on the original training model to test the validity of the evolution log.

[0125] The log template of the evolution log is composed as follows Figure 6 As shown. The first part is the events that are the same as the original log, as shown by D_1 in the figure. The second part is the log template that can be converted into the original log through the log mapping table, as shown by D_2 in the figure. The third part is the part that is different from the log template of the original log and cannot be converted into the log template of the original log, as shown by D_3 in the figure. Ideally, D_1 in the evolved log should be as much as possible, D_2 is the evolution result, reflecting the software upgrade result, and D_3 should be as little as possible. However, sometimes this is inevitable. For example, a new log template is introduced during the software upgrade process.

[0126] For the above situation, we adopted two methods to generate logs similar to the original logs. The logs similar to the original logs used in the first category are composed of D_1 and D_2, and the logs similar to the original logs used in the second category are composed of D_1, D_2, and D_3. Using the first category of input is to test the accuracy of directly using the log mapping, but it cannot ensure the impact of D_3 on anomaly detection. Using the second category of input is to test the feasibility of directly replacing the original log with the entire evolved log.

[0127] Try to test the model used with the training model of the original log. For the convenience of discussion, the LSTM model is used by us. That is: using the original log as the input, training and detecting with LSTM, and obtaining indicators such as accuracy, recall, and F1. In theory, when using the evolved log, re-training should be carried out first, and then detection should be implemented. However, if there is a good mapping between the original log and the evolved log, then the evolved log can be converted into a log similar to the original log and directly implement detection, thus reducing unnecessary training.

[0128] We conduct a validity test of the log mapping with the log similar to the original log (D_1 + D_2). We use the log similar to the original log (D_1 + D_2) as the data input, and use the LSTM used for training and detecting the original log as the detection model to conduct anomaly detection, and count the accuracy, recall, and F1 score. Assume that the F1 score of the original log detection is C_F1, and the F1 score of the current log detection is C_F1'. If C_F1 - C_F1' < fa_c, then the log mapping table is considered feasible and accurate. We call this situation mapping accuracy. Generally speaking, the preset first-round threshold fa_c should not be greater than 2%.

[0129] We conduct a validity test for log replacement using the original-like log (D_1 + D_2 + D_3). We use the original-like log (D_1 + D_2 + D_3) as the data input, and the LSTM used for training and detection with the original log as the detection model to conduct anomaly detection, and count the accuracy, recall, and F1 score. If the detection is passed, it means that the quantity of D_3 is insufficient to cause problems such as a decrease in detection accuracy, so it can be ignored, and the evolved log can be directly used to replace the original log for log anomaly detection. Assume that the F1 score of the original log detection is C_F1, and the F1 score of the current log detection is C_F1'. If C_F1 - C_F1' < fa_cf, then it is considered feasible to use the evolved log to replace the original log. We call this situation replacement feasible. Generally speaking, the preset second-round threshold fa_cf should not be greater than 5%.

[0130] 5 Anomaly Detection Module

[0131] Furthermore, if in Section 6.5, the trial test result is replacement feasible, then the evolved log can be converted into the original-like log through the replacement of the D_2 part, and anomaly detection can be directly conducted. This process is the same as the existing log anomaly detection and will not be elaborated here.

[0132] If the trial test result is mapping accurate, then supplementary training is required to improve the anomaly detection ability. The training data includes D_1, D_2, and D_3. The trained model is based on the model that has been trained with the original log (note: not completely retrained). After necessary supplementary training, anomaly detection is then conducted. Compared with retraining, the supplementary training requires less data volume and fewer training rounds, which also utilizes the training results of the original log to a certain extent. After the supplementary training is completed, anomaly detection can be formally conducted. The specific anomaly detection process will not be elaborated here.

[0133] If the trial test result fails, then the training results of the original log cannot be utilized, and only complete retraining can be performed. This process is the same as the existing log anomaly detection and will not be elaborated here.

[0134] The above processing flow of the anomaly detection module can be summarized as follows:

[0135] When the log mapping table used is replacement feasible, the LSTM model does not need to be updated;

[0136] When the log mapping table used is mapping accurate, the LSTM model conducts supplementary training;

[0137] When the log mapping table used fails the second-round trial test, the LSTM model conducts retraining.

[0138] Such as Figure 7As shown, the embodiment of the present invention also provides an anomaly detection method for evolution logs, including:

[0139] Step 1: Get the original log and the evolution log, and generate the original log template word vector and the evolution log template word vector through semantic analysis. The original log is the log output by the old version of the software, and the evolution log is the log output by the upgraded software.

[0140] Step 2: Perform similarity analysis on the original log template word vector and the evolved log template word vector to generate a set of similar log template pairs;

[0141] Step 3: Use the workflow method to verify the set of similar log templates and obtain the log mapping table;

[0142] Step 4: Convert the evolution log into the original log based on the log mapping table;

[0143] Step 5: Use the LSTM model trained on the original log to detect the original log and obtain the detection result of the evolved log.

[0144] Furthermore, the original log and the evolved log are semantically analyzed to obtain the original log template word vector and the evolved log template word vector. The specific process is as follows:

[0145] Step 101: performing log template mining on the original log and the evolved log to obtain an original log template set and an evolved log template set, wherein the original log template set includes the core content of the original log text, and the evolved log template set includes the core content of the evolved log text;

[0146] Step 102: using the Glove algorithm to extract word vectors of the original log template set and the evolved log template set respectively;

[0147] Step 103: using the TFIDF algorithm to extract the word weights of the original log template set and the evolved log template set respectively;

[0148] Step 104: Generate an original log template word vector and an evolved log template word vector based on the outputs of the word vector generation unit and the word weight generation unit.

[0149] Furthermore, the original logs and the evolved logs are mined for log templates, including:

[0150] Step 1011: pre-processing the original log and the evolution log based on the regular expression;

[0151] Step 1012: mining the preprocessed original logs and the evolved logs using three log template algorithms to obtain three initial original log template sets and three initial evolved log template sets;

[0152] Step 1013: Based on the big tree voting, the three initial original log template sets and the three initial evolved log template sets are merged respectively to obtain the original log template set and the evolved log template set.

[0153] Furthermore, in step 1012, the three log template algorithms are the Drain algorithm, the LogMine algorithm, and the Spell algorithm.

[0154] Furthermore, step 2 specifically includes:

[0155] Step 201: respectively calculating the word vector set values ​​of the original log template and the evolved log template, and the original log template and the evolved log template whose word vector set value difference is less than the word vector threshold are judged as a preliminary similar log template pair;

[0156] Step 202: Calculate the sum of square differences of word vectors of corresponding words in similar log template pairs. The original log template and the evolved log template whose sum of square differences of word vectors of corresponding words is less than the log template pair threshold are determined to be similar log template pairs.

[0157] Furthermore, step 3 specifically includes:

[0158] Step 301: Select any mapping node pair in the set of similar log template pairs as the center to construct an original event association graph and an evolved event association graph;

[0159] Step 302: cutting the original event correlation graph and the evolved event correlation graph;

[0160] Step 303: converting the cropped original event correlation graph and the evolved event correlation graph into original event low-dimensional vectors and evolved event low-dimensional vectors based on a random walk method;

[0161] Step 304: Calculate the similarity between the original event low-dimensional vector and the evolved event low-dimensional vector, and generate a log mapping table.

[0162] Furthermore, the method also includes performing a trial test and an anomaly test on the final log mapping table, wherein the trial test is used to test the validity of the log mapping table, and the anomaly test is used to determine whether to update the LSTM model based on the result of the trial test.

[0163] Furthermore, the test inspection is used to inspect the validity of the log mapping table, including the first round inspection and the second round inspection;

[0164] The first round of inspection includes: using the original log of the sample class mapped by the log mapping table as the input data of the first round of inspection module, using the LSTM model in the trained LSTM module as the detection model, implementing anomaly detection, and calculating the accuracy, recall rate and F1 score. If the difference between the F1 score of the original log detection of the sample class and the F1 score of the original log detection is less than the preset first round threshold, the log mapping table is considered to be feasible for replacement and passes the first round of trial inspection;

[0165] The second round of inspection includes: deleting the independent content in the original log of the sample class that failed the first round of trial inspection and using it as the input data for the second round of inspection module, using the LSTM model in the trained LSTM module as the detection model, implementing anomaly detection, and calculating the accuracy, recall rate and F1 score. If the difference between the F1 score of the input data of the second round of inspection module and the F1 score of the original log detection is less than the preset second round threshold, the final log mapping table is considered to be accurately mapped and passes the second round of trial inspection; the independent content is the part of the evolution log that is inconsistent with the original log and cannot be mapped through the mapping table.

[0166] Furthermore, the anomaly test is used to determine whether to update the LSTM model based on the test results, including:

[0167] When the log mapping table used is replaceable, the LSTM model does not need to be updated;

[0168] When the log mapping table used is accurate, the LSTM model performs additional training;

[0169] When the used log mapping table fails the second round of trials, the LSTM model is retrained.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An anomaly detection system for evolving logs, characterized in that: It includes log template word vector generation module, similarity determination module, workflow verification module, log mapping module and trained LSTM module; The log template word vector generation module is used to generate an original log template word vector and an evolved log template word vector by semantic analysis of the original log and the evolved log respectively; The similarity determination module is used to generate a set of similar log template pairs by performing similarity analysis on the original log template word vector and the evolved log template word vector; The workflow verification module is used to verify the set of similar log template pairs by using a workflow method to obtain a log mapping table; The log mapping module is used to convert the evolution log into a log similar to the original log based on the log mapping table; The trained LSTM module is used to detect the original log of the class through the LSTM model trained by the original log to obtain the detection result of the evolution log.

2. The anomaly detection system for evolving logs according to claim 1, characterized in that: The log template word vector generation module includes a log template mining unit, a word vector generation unit, a word weight generation unit and a log template word vector generation unit; The log template mining unit is used to perform log template mining on the original log and the evolved log to obtain an original log template set and an evolved log template set, wherein the original log template set includes the core content of the original log text, and the evolved log template set includes the core content of the evolved log text; The word vector generating unit is used to respectively extract the word vectors of the original log template set and the evolved log template set by using the Glove algorithm; The word weight generating unit is used to extract the word weights of the original log template set and the evolved log template set respectively by using the TFIDF algorithm; The log template word vector generating unit is used to generate the original log template word vector and the evolved log template word vector based on the outputs of the word vector generating unit and the word weight generating unit.

3. The anomaly detection system for evolving logs according to claim 2, characterized in that: The log template mining unit includes a log preprocessing subunit, a log template mining subunit and a log module output subunit; Wherein, the log preprocessing subunit is used to preprocess the original log and the evolution log based on a regular expression; The log template mining subunit is used to mine the preprocessed original log and the evolved log using three log template algorithms to obtain three initial original log template sets and three initial evolved log template sets; The log module output subunit is used to merge the three initial original log template sets and the three initial evolved log template sets respectively based on big tree voting to obtain the original log template set and the evolved log template set.

4. The anomaly detection system for evolving logs according to claim 3, characterized in that: The three log template algorithms are Drain algorithm, LogMine algorithm and Spell algorithm.

5. The anomaly detection system for evolving logs according to claim 1, characterized in that: The similarity determination module is specifically used for: Calculating the word vector set values ​​of the original log template and the evolved log template respectively, and the original log template and the evolved log template whose difference of the word vector set values ​​is less than a preset word vector threshold are judged as a preliminary similar log template pair; The sum of square differences of word vectors of corresponding words in the similar log template pair is calculated, and the original log template and the evolved log template whose sum of square differences of corresponding word vectors is less than a preset log template pair threshold are determined to be similar log template pairs.

6. The anomaly detection system for evolving logs according to claim 1, characterized in that: The workflow verification module is specifically used for: Select any mapping node pair in the similar log template pair set as the center to construct the original event association graph and the evolved event association graph; Cutting the original event correlation graph and the evolved event correlation graph; Based on the random walk method, the pruned original event correlation graph and the evolved event correlation graph are converted into the original event low-dimensional vector and the evolved event low-dimensional vector; The similarity between the original event low-dimensional vector and the evolved event low-dimensional vector is calculated to generate the log mapping table.

7. The anomaly detection system for evolving logs according to claim 1, characterized in that: The system also includes a trial inspection module and an abnormality inspection module; The trial inspection module is used to inspect the validity of the log mapping table; the abnormality inspection module is used to determine whether to update the trained LSTM module according to the output of the trial inspection module.

8. The anomaly detection system for evolving logs according to claim 7, characterized in that: The test module is used to test the validity of the log mapping table, and specifically includes a first round test module and a second round test module; The first round inspection module is used to use the sample class original log mapped by the log mapping table as the input data of the first round inspection module, use the LSTM model in the trained LSTM module as the detection model, implement anomaly detection, and calculate the accuracy, recall rate and F1 score. If the difference between the F1 score of the sample class original log detection and the F1 score of the original log detection is less than the preset first round threshold, it is considered that the log mapping table is feasible to replace and passes the first round of trial inspection; The second-round inspection module is used to delete the independent content in the original log of the sample class that has not passed the first-round trial inspection, and use it as input data for the second-round inspection module. The LSTM model in the trained LSTM module is used as the detection model to implement anomaly detection, and to count the accuracy, recall rate and F1 score. If the difference between the F1 score of the input data of the second-round inspection module and the F1 score of the original log detection is less than a preset second-round threshold, it is considered that the final log mapping table is accurately mapped and passes the second-round trial inspection; wherein the independent content is the part of the evolution log that is inconsistent with the original log and cannot be mapped by the mapping table.

9. The anomaly detection system for evolving logs according to claim 8, characterized in that: The abnormality detection module is used to determine whether to update the trained LSTM module according to the output of the trial detection module, and is specifically used to: When the log mapping table used is replaceable, the trained LSTM module does not need to be updated; When the used log mapping table is accurately mapped, supplementary training is performed on the LSTM model in the trained LSTM module; When the used log mapping table fails the second round of test, the LSTM model in the trained LSTM module is retrained.

10. An anomaly detection method for evolving logs, characterized in that: include: Step 1: Obtain the original log and the evolution log, and generate the original log template word vector and the evolution log template word vector by semantic analysis of the original log and the evolution log respectively; The original log is the log output by the old version of the software; the evolution log is the log output by the upgraded software; Step 2: Performing similarity analysis on the original log template word vector and the evolved log template word vector to generate a set of similar log template pairs; Step 3: Using a workflow method, verify the set of similar log templates to obtain a log mapping table; Step 4: Convert the evolution log into a log similar to the original log based on the log mapping table; Step 5: Detect the class original log using the LSTM model trained with the original log to obtain the detection result of the evolution log.

Images