A log anomaly detection auxiliary decision-making method and system based on a knowledge graph and reinforcement learning

Abstract

The application discloses a log anomaly detection auxiliary decision-making method and system based on a knowledge graph and reinforcement learning, which comprises the following steps: constructing a log anomaly detection knowledge base based on a knowledge graph, and collecting existing log anomaly detection knowledge; extracting knowledge from existing literature and cases, and initially constructing a log anomaly detection knowledge base; optimizing the log anomaly detection knowledge base through a similar entity merging method; constructing a reasoning engine based on reinforcement learning, wherein the reasoning engine takes the log anomaly detection knowledge base as data support, takes a reinforcement learning algorithm as a core, and continuously adjusts and optimizes a recommended result in the process of generating a recommended strategy, implementing the strategy and feeding back the strategy, and meanwhile feeds back to the log anomaly detection knowledge base, drives evolution of the log anomaly detection base, and continuously optimizes and adapts to detection requirements. The application provides an auxiliary decision-making suggestion for log anomaly detection by constructing a knowledge base, reduces the difficulty of using log anomaly detection, and improves the usability and universality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of log anomaly detection, and particularly relates to a log anomaly detection auxiliary decision-making method and system based on a knowledge graph and reinforcement learning. BACKGROUND

[0002] Most of the existing log anomaly detection schemes are trying to optimize the indicators of log anomaly detection by introducing new algorithms, optimizing existing algorithms, adjusting algorithm inputs and other measures. There is no clear auxiliary decision-making solution for log anomaly detection. Multi-CAD [Xie X, Jin Z, Wang J, et al. Confidence guided anomaly detection model for anti-concept drift in dynamic logs [J]. Journal of Network and Computer Applications, 2020, 162: 102659] aims at the problem of concept drift, and examines the concept drift in the log by comprehensive analysis of the results of multiple algorithms, so as to determine which algorithm is more suitable for the processing of the current log. [Li G, Zhu P, Chen Z. Accelerating System Log Processing by Semi-supervised Learning: A Technical Report [J]. 2018] [He S, Zhu J, He P, et al. Experience Report: System Log Analysis for Anomaly Detection [C] / / 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 2016.] statistically analyzes various log anomaly detection algorithms, hoping to provide some experience accumulation for people to refer to when making decisions to use detection algorithms. The existing log preprocessing and anomaly detection all have various methods or strategies, and different algorithms have different performances in accuracy, recall rate and F1 score when processing different styles of logs. Various uncertain factors make people face the difficulty of decision-making when implementing log anomaly detection.

[0003] One of the problems of current log anomaly detection is that the log anomaly detection scheme does not have universality. Since the log anomaly detection scheme is too closely combined with the log, the detection scheme has good performance on the target data set. However, the diversity of the log makes it difficult to have universality of such excellent performance. Since there is no universal standard for log generation, software developers differ, software is frequently upgraded, and other reasons, which lead to the diversity of the log. The test results for specific experimental data are difficult to completely reproduce on other dynamic log data. When people choose a log anomaly detection scheme, they can only analyze and judge according to historical experience. This not only consumes a lot of manpower, but also is difficult to ensure the correctness of the decision. SUMMARY

[0004] In view of the above problems, the present application provides a log anomaly detection auxiliary decision-making method and system based on a knowledge graph and reinforcement learning, which hopes to guide users to complete log anomaly detection under the guidance of a log anomaly detection knowledge base with professional field knowledge, and avoid the decision-making difficulties of people in log anomaly detection. Based on the statistics, comparison and analysis of the existing log anomaly detection scheme, the present application extracts related knowledge to form entities, attributes and relationships, relies on the knowledge graph to build a log anomaly detection knowledge base, and uses a reinforcement learning algorithm to realize the generation and optimization of detection decision suggestions, and also drives the evolution of the log anomaly detection knowledge base itself according to the feedback, so as to continuously adapt to the dynamically changing and diverse log database. Overall, the workload of personnel is reduced, and the log anomaly detection scheme is self-adaptive to the target log data set.

[0005] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0006] The present application provides a log anomaly detection auxiliary decision-making method based on a knowledge graph and reinforcement learning, which comprises the following steps:

[0007] The log anomaly detection knowledge base is constructed based on the knowledge graph, and the existing log anomaly detection knowledge is collected; including: extracting knowledge from existing literature and cases to preliminarily construct a log anomaly detection knowledge base; optimizing the log anomaly detection knowledge base through a similar entity merging method;

[0008] The reasoning engine is constructed based on the reinforcement learning, the reasoning engine is supported by the log anomaly detection knowledge base, and the reinforcement learning algorithm is used as the core. Through the process of generating a recommended strategy, implementing the strategy and the feedback of the strategy, the recommended result is continuously adjusted and optimized, and the log anomaly detection knowledge base is also fed back to drive the evolution of the log anomaly detection library, so as to continuously optimize and adapt to the detection requirements.

[0009] Further, the step of extracting knowledge from existing literature and cases to preliminarily construct a log anomaly detection knowledge base comprises the following steps:

[0010] Based on the processing habits and processing effects of different logs based on regular expressions, the strategy entity of the pre-processing stage is extracted;

[0011] The commonly used log template mining algorithm is listed as the log template mining algorithm entity, the log template of the common log is counted around the log template mining result, and the log template mining entity is constructed accordingly; meanwhile, the relationship between the log template and the log template mining algorithm entity is established; the common type log is listed as an entity, and the attributes are used to record the indicators of the log template mining algorithm processing different logs;

[0012] Abnormal information that may exist in the log is found by using different log anomaly detection algorithms: the commonly used log anomaly detection algorithm entity is constructed, the characteristics of the entity are recorded using attributes, and stored in the graph database Neo4j; based on the abnormal detection algorithm evaluation index, the corresponding relationship and attribute are constructed.

[0013] Further, the log anomaly detection knowledge base is optimized by the similar entity merging method, which comprises:

[0014] Firstly, based on the similarity of the entities, the similar entities are screened out according to the predetermined threshold; then the structural similarity of the similar entities is determined from the screened similar entity set; based on the principle of relationship alignment, the knowledge of similar entities is fused.

[0015] Further, the entity similarity determination method comprises:

[0016] The knowledge base entity set is processed by word segmentation coding based on the BERT model trained in the general word, the cosine similarity is used to calculate the similarity between the entities, and all entities are traversed by matching one by one, when the similarity of two entities is higher than the threshold, they are considered as similar entities, and they are added to the similar entity set.

[0017] Further, the structural similarity determination method comprises:

[0018] The similar entity set is taken as the input, the similar entity set is extracted, and the similarity of the topological structure reflected in the knowledge graph is determined, wherein the features of the topological structure include the number of preprocessing methods, the number of templates, the number of regular expressions, the number of anomaly detection methods, and the average of index evaluation; the topological structure of the entity is converted into a feature vector by querying the knowledge graph, and then the similarity is calculated based on the cosine similarity principle; if it is higher than the threshold, it is judged that the two entities are similar, and the subsequent knowledge fusion is carried out.

[0019] Further, the knowledge fusion of similar entities comprises:

[0020] First, the TransE algorithm is used for relation representation learning, and the relationship related to the entity is vectorized; then, according to the cosine similarity, the similarity between the relationship vectors corresponding to the similar entities is calculated, and the relationship greater than the predetermined threshold is considered similar and is fused, and if the relationship is less than the predetermined threshold, it is considered dissimilar and is reserved or deleted by the expert for further confirmation.

[0021] Further, the construction process of the inference engine specifically includes:

[0022] Information is collected from the perspectives of log original text and log template, and verification is completed using cosine similarity; the knowledge base contains log template entities, and the knowledge base records the corresponding log template information according to the log type; n log items are randomly extracted from the target log original text, and after preprocessing, the similarity with the reserved log template in the knowledge base is determined; if the n log items are similar to the reserved log template of the same log type, it is considered that the target log belongs to the log type;

[0023] According to the actual situation, the algorithms and rules that do not meet the current detection task are filtered from the current strategy arrangement process to ensure that the final generated strategy implementation can be supported by the underlying algorithm library and rule library;

[0024] The user sets the preference of the recommended strategy of reasoning; the preference includes time efficiency, accuracy, scalability, interpretability and adaptability;

[0025] The candidate processing method and detection algorithm of the current log template are obtained through the log anomaly detection knowledge base, and the encoding is converted into a state matrix, so as to establish the mapping relationship under different concept layers;

[0026] During the reasoning process, the strategy is preferentially recommended according to the user preference, and the return is evaluated;

[0027] The strategy reasoning is realized based on the Q-learning algorithm.

[0028] Further, the strategy reasoning based on the Q-learning algorithm includes:

[0029] By receiving context information, the preprocessing mode, the coding selection mode matrix and the detection algorithm mode are matched by using the log anomaly detection knowledge base;

[0030] By executing the state transition formula, the current state is transferred to the next state; at the same time, the corresponding strategy is executed to obtain the return, and the cumulative return is calculated;

[0031] A voting mechanism is adopted to speed up the learning speed and make it quickly converge.

[0032] Another aspect of the present application provides a log anomaly detection auxiliary decision system based on a knowledge graph and reinforcement learning, comprising:

[0033] A knowledge base construction module is configured to construct a log anomaly detection knowledge base based on a knowledge graph, and to collect existing log anomaly detection knowledge, including extracting knowledge from existing literature and cases to preliminarily construct a log anomaly detection knowledge base, and optimizing the log anomaly detection knowledge base through a similar entity merging method.

[0034] A reasoning engine construction module is configured to construct a reasoning engine based on reinforcement learning, and the reasoning engine is data-supported by the log anomaly detection knowledge base and takes a reinforcement learning algorithm as a core, and through the process of generating a recommended strategy, implementing the strategy and strategy feedback, the recommended result is continuously adjusted and optimized, and at the same time, feedback is given to the log anomaly detection knowledge base to drive the evolution of the log anomaly detection library to continuously optimize and adapt to detection requirements.

[0035] Compared with the prior art, the present application has the beneficial effects that:

[0036] Based on the statistics, comparison and analysis of existing log anomaly detection schemes, the present application extracts relevant knowledge to form entities, attributes and relationships, constructs a log anomaly detection knowledge base relying on a knowledge graph, and uses a reinforcement learning algorithm to realize the generation and optimization of detection decision suggestions, and at the same time, drives the evolution of the log anomaly detection knowledge base itself according to feedback, so as to continuously adapt to dynamically changing and diverse log databases. The present application provides auxiliary decision suggestions for log anomaly detection by constructing a knowledge base, reduces the difficulty of using log anomaly detection, improves its ease of use and universality, and reduces the workload of personnel as a whole, so that the log anomaly detection scheme is self-adaptive to the target log data set. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 FIG. 1 is a schematic diagram of the principle of a log anomaly detection auxiliary decision method based on a knowledge graph and reinforcement learning according to an embodiment of the present application;

[0038] Figure 2 FIG. 2 is a flowchart of a log anomaly detection auxiliary decision method based on a knowledge graph and reinforcement learning according to an embodiment of the present application;

[0039] Figure 3 FIG. 3 is a schematic diagram of part of the entities and relationships extracted in the log preprocessing stage according to an embodiment of the present application;

[0040] Figure 4 FIG. 4 is a schematic diagram of part of the entities and relationships extracted in the log template mining stage according to an embodiment of the present application;

[0041] Figure 5 FIG. 5 is a schematic diagram of part of the entities and relationships extracted in the log anomaly detection stage according to an embodiment of the present application;

[0042] Figure 6 Rational engine for the embodiment of the present application;

[0043] Figure 7 Process flow for the embodiment of the present application;

[0044] Figure 8 Log data for simulating unknown templates for the embodiment of the present application;

[0045] Figure 9 Results of simulating matching similarity for the embodiment of the present application;

[0046] Figure 10 Regular expression corresponding to the Hadoop template for the embodiment of the present application;

[0047] Figure 11 Preprocessing method corresponding to the Hadoop template for the embodiment of the present application;

[0048] Figure 12 Index evaluation data for preprocessing the Hadoop template using Drain for the embodiment of the present application;

[0049] Figure 13 Detection method corresponding to the Hadoop template for the embodiment of the present application;

[0050] Figure 14 Encoding mode corresponding to each method for the embodiment of the present application;

[0051] Figure 15 Partial index data of the Hadoop corresponding method for the embodiment of the present application;

[0052] Figure 16 Optimal strategy example of the whole process of log anomaly detection for the embodiment of the present application. DETAILED DESCRIPTION

[0053] For the convenience of understanding, some of the terms appearing in the specific embodiments of the present application are explained as follows:

[0054] 1. Knowledge graph: The knowledge graph is a present theory that combines the theories and methods of mathematics, graphics, information visualization technology, information science, etc. with citation analysis, co-occurrence analysis, etc. and uses visualized graphs to display the core structure, development history, frontiers and overall knowledge architecture of a discipline to achieve the purpose of multi-disciplinary integration. It displays complex knowledge fields through data mining, information processing, knowledge measurement and graph drawing, and reveals the dynamic development law of the knowledge field, providing practical and valuable reference for discipline research.

[0055] 2. Reinforcement learning: Also known as relearning, evaluation learning or enhanced learning, it is one of the paradigms and methodologies of machine learning, used to describe and solve the problem of maximizing the reward or achieving a specific goal through learning strategy in the interaction process between the agent and the environment.

[0056] 3. Log anomaly detection knowledge base: The concept proposed by the present application is the collection and accumulation of log anomaly detection related knowledge, and is a knowledge system constructed with knowledge graph as the core, which can support log anomaly detection related auxiliary decision-making. More popularly, the log anomaly detection knowledge base is more like a "person" with background knowledge related to log anomaly detection, which is used to guide the development of log anomaly detection activities.

[0057] 4. TransE algorithm: TransE algorithm is a typical algorithm for early knowledge graph representation learning, which converts triples (h, l, t) of knowledge graph into vectors according to certain rules. That is: given the knowledge graph (h, l, t), where h is the head entity set, t is the tail entity set, and l is the relationship set. Then the entities and relationships are represented as vectors in their respective spaces (entity space and relationship space), so that the entity vector and relationship vector of each triple in the knowledge graph satisfy |h + l| ≈ t: that is, if the triple is correct, the tail entity vector should be more close to the sum of the head entity vector and the relationship vector; on the contrary, if the triple is wrong, the tail entity vector should be more far away from the sum of the head entity vector and the relationship vector.

[0058] 5. Log preprocessing: Log preprocessing is to remove or replace unnecessary components in the original log text. For example, existing log anomaly detection often takes event sequence as input, and information such as IP address is often ignored, so during log preprocessing, after identifying the IP address, replace the specific IP address with a special mark to reduce the difficulty and workload of subsequent processing.

[0059] 6. L2 norm or L1 norm: In the TransE algorithm, L2 and L1 are norm calculation methods. L2 norm is Euclidean norm, representing the square root of the sum of squares of each element of the vector; L1 norm is Manhattan norm, representing the sum of absolute values of each element of the vector. The distance between the sum of the head entity vector and the relationship vector and the tail entity vector is constrained by these two norm calculation methods.

[0060] 7. Hyperparameter margin: Hyperparameter margin represents the distance between reasonable vectors. When the difference between head entity vector h, relation vector r and tail entity vector t is less than or equal to margin, it is considered that the triple is embedded in the low-dimensional vector space in the correct way, otherwise it is necessary to update the vectors to make them closer to the target distance. Therefore, the application controls the performance and generalization ability of the model by adjusting the value of margin, so as to better adapt to different tasks and data sets. The application sets the value of margin to 1 when performing relation representation training, so that better generalization ability and training efficiency can be obtained while ensuring the accuracy of the model.

[0061] 8. Safe and stable threshold: refers to a protection mechanism that automatically stops the update of the strategy when the effect of the strategy cannot be effectively improved or exceeds the safe range. This can avoid the unstable state of the agent in the process of exploring new strategies.

[0062] 9. Stability factor: refers to the degree of punishment for bad decisions. Its main role in the Q-learning algorithm is to limit the step length of policy exploration, so that the policy is more stable in continuous adjustment.

[0063] 10. Punishment factor: refers to the negative feedback received by the agent when executing a bad decision. Its main role in the Q-learning algorithm is to reduce the difficulty of policy exploration and avoid local optimal solutions.

[0064] 11. Temperature: refers to the exploration degree of the control strategy. If the temperature is high, the strategy is more likely to accept a less good decision, and vice versa. Temperature controls the relative probability of sampling new observations or old observations, so as to better balance the exploration and utilization problem.

[0065] The application will be further explained in conjunction with the accompanying drawings and specific embodiments:

[0066] The working principle of the log anomaly detection auxiliary decision-making method based on knowledge graph and reinforcement learning of the application is as shown in Figure 1 The application constructs a log anomaly detection knowledge base based on a knowledge graph, and collects existing log anomaly detection knowledge. The construction of the log anomaly detection knowledge base is derived from existing log anomaly detection schemes, relies on existing literature and experts for processing, abstraction and integration, and converts these knowledge into knowledge graph content. The application constructs a reasoning engine based on reinforcement learning. The reasoning engine takes the log anomaly detection knowledge base as data support and takes the reinforcement learning algorithm as the core. Through the process of generating recommended strategies, implementing strategies and strategy feedback, the recommended results are continuously adjusted and optimized, and at the same time feedback to the log anomaly detection knowledge base, driving the evolution of the log anomaly detection library to continuously optimize and adapt to detection requirements.

[0067] To avoid redundancy, unless otherwise specified, the log anomaly detection knowledge base will be referred to as a knowledge base in the following text.

[0068] like Figure 2 As shown, this method consists of two parts: a knowledge base construction and a recommendation engine. The knowledge base, built based on a knowledge graph, is used to record and integrate existing log anomaly detection information. It mainly includes two stages: knowledge extraction and knowledge fusion. Knowledge extraction relies primarily on experts to extract knowledge from existing literature and cases to initially construct the log anomaly detection knowledge base. Knowledge fusion optimizes the knowledge base through methods such as merging similar entities. The inference engine, built based on reinforcement learning, sets up the inference environment through context settings and generates recommendation strategies through policy orchestration, supported by the log anomaly detection knowledge base. The knowledge base can also have other construction stages, and the inference engine can have other specific algorithms, but this invention only requires the content described above.

[0069] 1. Knowledge Base Construction

[0070] We build a knowledge base based on knowledge graphs. Starting with existing research on log anomaly detection, we will extract relevant entities and relationships, and utilize the Neo4j graph database to construct a knowledge graph for log anomaly detection, providing knowledge support for related decision-making. Given the specialized and unique nature of log anomaly detection, unlike general knowledge graphs, the construction of the knowledge base is divided into two stages: knowledge extraction and knowledge fusion. In the knowledge extraction stage, we mainly rely on experts and a large number of reference cases to extract relevant entities, relationships, and attributes. In the knowledge fusion stage, we eliminate ambiguous knowledge, merge identical entities, relationships, and attributes, and fill in missing knowledge based on similarity.

[0071] 1.1 Knowledge Extraction

[0072] Knowledge extraction utilizes experts to extract relevant entities, relationships, and attributes from existing log anomaly detection schemes. However, due to limitations in expert capabilities and the incomplete coverage of existing log anomaly detection schemes, the knowledge obtained through extraction cannot fully encompass log anomaly detection. Knowledge extraction is an open and continuous process; this invention cannot completely extract the knowledge involved in log anomaly detection. This invention only makes claims regarding the stated content.

[0073] 1.1.1 Knowledge Extraction in the Preprocessing Stage

[0074] Preprocessing is a necessary step to structure the original log information. In order to reduce the unstructured information of the log, some matched tokens, such as IP address and number, need to be deleted from the original log message. At the same time, regular expressions are used to match specific patterns to extract the required information, such as error code, request URL, etc. In this stage, the main concern is the preprocessing rule and the preprocessing strategy. The preprocessing rule mainly records how to process a certain component in the original log, and the preprocessing strategy reflects which preprocessing rule is suitable for processing a certain log.

[0075] We give the preprocessing rule entity defined in the present invention. The common log preprocessing method at present is to use regular expressions. Some typical regular expressions we counted are shown in Table 1. Based on Table 1, the preprocessing rule entity is given. In the process of forming knowledge, the present invention defines the log template as a templats type entity and the regular expression as a regular type entity, and the relationship is <log template-template_regular expression-regular expression>. The attribute of each regular expression explains the extraction meaning represented by the expression, which is convenient for operation and maintenance personnel to quickly understand and use.

[0076] Table 1 Common regular expressions

[0077]

[0078]

[0079] Based on the habit and effect of processing different logs by using regular expressions, the strategy entity of the preprocessing stage is extracted. In the log preprocessing process, there are common points in the regular expression extraction rules of most known templates, such as the extraction of date and priority, and at the same time, each log template has some unique attribute column information, which needs to be analyzed according to the specific situation and specific scene. Therefore, the knowledge base describes the relationship between the log and the preprocessing rule in real life by the relationship between entities, and records the strategy information of the preprocessing stage.

[0080] Figure 3 Some entities and relationships extracted in the log preprocessing stage are shown. The current version of the knowledge base records 22015 entity nodes, of which there are 11837 relationships, and there are 118 kinds of relationship model definitions, reflecting the methods and strategies of the common log preprocessing stage. Limited by human resources, the present invention cannot exhaust all the log preprocessing related contents. The present invention only analyzes and archives part of the common logs and the logs discussed more in the literature. The knowledge contained in the current knowledge base is only a subset of all knowledge. The extraction method and concept given by the present invention can support others to complete similar work to enrich the knowledge base.

[0081] 1.1.2 Knowledge extraction in log template mining stage

[0082] Log template mining is responsible for transforming pre-processed logs into event sequences or other sequences. At present, most log template mining algorithms are aimed at mining event sequences, because the events recorded in the logs are the most critical and direct information. However, with the continuous optimization of log anomaly detection algorithms, some log anomaly detection algorithms no longer only deal with event sequences, but also pay attention to other information recorded in the logs. The present invention only focuses on log template algorithms for mining event sequences for the time being.

[0083] Based on the statistics of Table 2, the log template mining algorithm entity is constructed, and the corresponding log template entity is constructed. Table 2 is a commonly used log template mining algorithm, which is mainly divided into two categories based on frequent item mining and clustering, a total of 13 algorithms, which are listed as log template mining algorithm entities in the present invention. Around the log template mining results, the log templates of common logs are counted, and the log template mining entity is constructed accordingly. At the same time, the relationship between the log template and the log template mining algorithm entity is established, such as "< log template mining algorithm -(mining) -> log template >".

[0084] Table 2 Commonly used log template mining algorithms

[0085] Serial number Method 1 Drain 2 SHISO 3 Logmine 4 K-means 5 DBSCAN 6 PCA 7 GMM 8 Mean-shift 9 HAC 10 GCD 11 Logram 12 FT-tree 13 Apriori

[0086] The algorithm evaluation index entity is given according to the historical experience of log template mining algorithms. The present invention lists the commonly used types of logs as entities, and uses attributes to record the indicators of different logs processed by log template mining algorithms, as shown in Table 1. Figure 4 The attributes of the indicators caused by different information sources may not be completely the same. In the early stage of knowledge extraction, the present invention uses the method of newly created attributes to distinguish different values of the same indicator. When a certain period is reached, based on the cumulative average method, the attribute values belonging to the same indicator are combined. For example: there is currently a Drain entity. The accuracy attribute = 0.87, after practice feedback, a new value 0.91 of the attribute is obtained, the knowledge base will newly set the accuracy temporary attribute 1 = 0.91, and when the knowledge fusion period is reached, the value of the accuracy attribute will be combined to 0.89, and the accuracy temporary attribute 1 will be deleted. In this way, the dynamically changing attribute values are gradually stabilized in the process of continuous practice and optimization.

[0087] 1.1.3 Knowledge extraction in anomaly detection stage

[0088] The log anomaly detection stage is to find possible abnormal information in the log by using different algorithms. The existing log detection algorithms are from a certain dimension, analyze a certain feature in the log, and judge normal or abnormal according to this feature. The early log anomaly detection algorithm is usually PCA algorithm, which is relatively simple and direct. With the further popularization of deep learning technology, such as LSTM, it is applied to log anomaly detection, thereby improving the efficiency of log anomaly detection. The present application focuses on the commonly used log anomaly detection algorithm, or the log anomaly detection algorithm mentioned in the literature cited many times, and collects their corresponding indicators.

[0089] Table 3 is the log anomaly detection algorithm concerned by the present application, and 23 log anomaly detection algorithm entities are constructed according to this, and the characteristics of them are recorded using attributes. For example: the entity of Deeplog algorithm is constructed, and its characteristics are semi-supervised algorithm, which can adopt online and offline two working modes. 23 log anomaly detection algorithms form 23 entities and attributes, and are stored in the graph database Neo4j.

[0090] Table 3 abnormal detection algorithm entity table

[0091] Serial number Method Remark 1 MLP Feedforward neural network, supervised, unsupervised, semi-supervised, online, offline 2 CNN Feedforward neural network, supervised, online, offline 3 RNN Neural network, supervised, unsupervised, semi-supervised, online, offline 4 LSTM Supervised, online, offline 5 Bi-LSTM Supervised, online, offline 6 PCA Unsupervised, online, offline 7 Deeplog Semi-supervised, online, offline 8 LogBERT Supervised, unsupervised, semi-supervised, online, offline 9 Autoencoders Unsupervised, online, offline 10 GAN Unsupervised, online, offline 11 SLCT Semi-supervised, online, offline 12 AEL Semi-supervised, online, offline 13 IPLOM Supervised, online, offline 14 LKE Supervised, unsupervised, semi-supervised, online, offline 15 LFA Unsupervised, online, offline 16 LogSig Unsupervised, online, offline 17 SHISO Unsupervised, online, offline 18 LogCluster Unsupervised, online, offline 19 LenMa Unsupervised, online, offline 20 LogMine Unsupervised, online, offline 21 Spell Supervised, online, offline 22 Drain Unsupervised, online, offline 23 MoLFI Semi-supervised, online, offline

[0092] Table 4 is the evaluation index of the anomaly detection algorithm concerned by the present application, and the corresponding relationship and attribute are constructed according to this. For example, HDFS log adopts LogSig for anomaly detection, and often adopts index AUC to judge the advantages and disadvantages of the algorithm, based on such analysis, the entity, relationship and attribute as shown in Figure 5 need to be explained: (1) since some log anomaly detection algorithms do not directly process the original text of the log, so from the path, it reflects what kind of log template mining algorithm the detection algorithm depends on; (2) when the detection algorithm processes multiple logs, the results of the same index are not completely consistent, so they need to be distinguished, as shown in Figure 5 .

[0093] Table 4 abnormal detection algorithm evaluation index

[0094] Serial number Index Remark 1 CE Cross-entropy 2 MSE Mean squared error 3 KL KL divergence 4 MAE Mean absolute error 5 ACC Accuracy 6 RMSE Root mean squared error 7 Pre Accuracy 8 Recall Recall rate 9 F1 An index that considers the accuracy and recall rate 10 AUPRC An index that evaluates the detection accuracy of the classifier for abnormal samples at different thresholds 11 AUC An index that evaluates the classification accuracy of the classifier for samples at different thresholds 12 TE Time efficiency 13 IPT Interpretability 14 LF Loss Function 15 CEL Cross-entropy loss function 16 RE Reconstruction error

[0095] 1.2 Knowledge fusion

[0096] The present application is based on knowledge fusion, which refers to integrating knowledge from different sources to form a more comprehensive and accurate knowledge system. Due to the problems such as the ability of analysts and the difference of data sources in the process of constructing the knowledge base, the knowledge base may have inconsistent knowledge and contradictory relationships. Knowledge fusion needs to be used to fuse similar ontologies. The attribute association of nodes or the heterogeneous relationship of entities can be used as the basis for processing, avoiding data loss caused by ambiguous naming, and at the same time, entity alignment and entity disambiguation can be performed to avoid low-quality knowledge caused by chaotic knowledge structure.

[0097] The present application adopts the following process to complete knowledge fusion. First, based on entity similarity, similar entities are screened out according to a predetermined threshold value; then, from the set of similar entities screened out, a structure similarity judgment is performed to determine whether the entities are similar; based on the principle of relationship alignment, knowledge fusion is performed on similar entities on the basis of confirming similar entities. The knowledge amount of the knowledge base is very large, and through entity similarity, similar nodes can be quickly screened out. Under the premise of entity similarity, structure similarity is used to further confirm the similarity degree, thereby reducing the computational complexity.

[0098] 1.2.1 Entity similarity determination

[0099] The entity similarity determination method is as follows. The knowledge base entity set is processed by word segmentation and coding based on the BERT model trained in the general word, and the cosine similarity is used to calculate the similarity between entities (cosine similarity is as formula 1). By matching one by one, all entities are traversed. When the similarity of two entities exceeds the threshold value (default 0.9), they are considered as similar entities and are added to the similar entity set. For example, due to human error, there are entities PCAs in the knowledge base, and there are also PCAs. In the knowledge fusion stage, similarity determination will be performed on them, and the above method will obtain two entity similarities of 0.965, which should be similar entities.

[0100] The calculation process of the cosine similarity of two vectors is as follows. Assuming that vector X=(x1,x2) and Y=(y1,y2), then the cosine similarity of vectors X and Y is calculated as formula 1.

[0101]

[0102] The entity similarity determination process is as follows.

[0103]

[0104] 1.2.2 Structure similarity determination

[0105] The determination method of structural similarity is as follows. The similar entity set S_M is input, the similar entity set is extracted, and the similarity of the topological structure reflected in the knowledge graph is determined. The topological features are set to 26 dimensions in the application, including the number of preprocessing methods, the number of templates, the number of regular expressions, the number of anomaly detection methods, the average of index evaluation, and the like. By querying the knowledge graph, the topological features of the entity are converted into a feature vector, and then the similarity is calculated based on the cosine similarity principle (the calculation formula is as shown in formula 2). If it is higher than the threshold value, it is determined that the two entities are indeed similar, and subsequent knowledge fusion is carried out. For example, still taking the example of 1.2.1, the entity "PCAs" and "PCA," are traversed to obtain two feature vectors [0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 12, 8, 14, 13, 14, 4, 11, 4, 11, 7, 0, 6, 0] and [1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 12, 8, 14, 13, 14, 3, 11, 5, 11, 8, 2, 3, 1] respectively. The structural similarity value is 0.9484 by calculation. The structural similarity is greater than the set threshold value, and it is confirmed that the two entities are similar.

[0106] The calculation process of the two 26-dimensional vectors based on the cosine similarity determination is as follows. Assuming that the vector X=(x1, x2,..., x 26 ) and Y=(y1, y2,..., y 26 ), then the cosine similarity of the vector X and Y is calculated as shown in formula 2.

[0107]

[0108] The structural similarity determination process is as follows.

[0109]

[0110] 1.2.3 Similar entity fusion

[0111] On the basis of confirming the similarity of the entities, the similar entities are fused. The fusion of the similar entities mainly considers whether the relationships between the similar entities are similar or whether they conflict, that is, the problem of relationship alignment. Because the two entities are similar, their relationships are most likely not completely the same. For example, the relationships corresponding to the similar entities derived from different literatures have a high probability of having certain differences in the events, coding indicators, and detection algorithm accuracy they reflect. In order to avoid the difficulties brought by such differences to the subsequent use and maintenance of the knowledge base, it is necessary to process them to realize the fusion of the similar entities.

[0112] This invention utilizes the TransE algorithm to align relationships from a set of similar entities, ultimately fusing similar entities. The process consists of two steps: First, the TransE algorithm is used to learn relationship representations, vectorizing entity-related relationships to more intuitively identify differences between them. Then, based on cosine similarity, the similarity between the relationship vectors corresponding to similar entities is calculated. Relationships with similarity greater than a predetermined threshold are considered similar and fused; relationships with similarity less than the predetermined threshold are considered dissimilar and await further expert confirmation for retention or deletion.

[0113] First, the relation representation learning process based on the TransE algorithm is given, which is the first step mentioned above.

[0114]

[0115] The process of relationship fusion is given, namely the second step mentioned above.

[0116]

[0117]

[0118] This invention sets the topological features to 26 dimensions, where the features include information such as the number of preprocessing methods, the number of templates, the number of regular expressions, the number of anomaly detection methods, the average evaluation index, the average evaluation index of clustering, and the average evaluation index of anomaly detection algorithms. The 26-dimensional features are shown below.

[0119] Table 5 26-dimensional features

[0120]

[0121] 2 Inference Engines

[0122] This invention uses reinforcement learning as its core to construct an inference engine. The inference engine is responsible for analyzing logs, establishing a mapping relationship between logs and the knowledge base, and constructing corresponding vectors; based on user-set preferences and environmental conventions, it constructs contextual information to prepare data for policy orchestration; based on the contextual information, and with the support of reinforcement learning principles and the knowledge base data, it implements policy orchestration; relying on evaluation feedback, including user evaluation of actual detection results, and through experience replay, it continuously accumulates and updates the content of the knowledge base to improve its completeness, accuracy, and precision.

[0123] The main components of an inference engine are as follows: Figure 6The reasoning engine is mainly divided into two parts: context setting and policy arrangement. The policy arrangement is the core component of the reasoning engine, and its guiding ideology is reinforcement learning, which optimizes the policy and adjusts the knowledge base through incentives and feedback. The context setting is the forerunner of the policy arrangement, responsible for the necessary data preparation for the policy arrangement, such as the user's reasoning preferences, and the conventions of the reasoning environment (such as limiting decisions to a number of algorithms).

[0124] 2.1 Context setting

[0125] The context setting mainly includes four steps: log analysis, environment convention, reasoning preference, and vectorization. Log analysis is used to analyze the log data submitted by the user, and its purpose is to determine the type of log to be processed, such as Hadoop log. If the user directly gives the log type, this step can be omitted. Environment convention is used to set the spatial range of reasoning, such as specifying that the detection algorithm set by the current reasoning does not include LSTM, so that LSTM is not included in the reasoning range in subsequent reasoning. Reasoning preference refers to the user's preference for the recommended strategy of reasoning, such as the user's desire to detect more accurately, but not performance optimization. Vectorization refers to the formation of all relevant data into data that can be used by the reasoning engine, which is a preparation for policy arrangement.

[0126] 2.1.1 Log analysis

[0127] Log analysis is to obtain the type information of the log. Log type is an important part of abnormal detection context information. If the Windows system log is mistaken for a Hadoop log, the subsequent decision will also be wrong. Log analysis is an optional step, and when the user explicitly gives the log type information, the log analysis step can be skipped. It can also be set as a mandatory step to verify whether the log type information given by the user is correct.

[0128] In order to verify the log type, the present invention collects information from the log original and the log template, and uses cosine similarity to complete the verification. The knowledge base contains log template entities. The knowledge base records the corresponding log template information according to the log type (software type and version). The present invention randomly extracts n log items (the present invention defaults n = 20) from the target log original, and after necessary preprocessing, it is compared with the log template reserved in the knowledge base to determine the similarity. If the n log items are similar to the log template reserved for the same log type, the log is considered to be of that log type.

[0129]

[0130]

[0131] 2.1.2 Environment convention

[0132] The environmental convention is to limit the policy search space. Log anomaly detection relies on the support of a number of algorithms and rules, but not every detection task has enough underlying support of algorithms or rules. Therefore, it is necessary to filter the algorithms and rules that do not meet the current detection task from the current policy arrangement process according to the actual situation, to ensure that the final generated policy implementation can be supported by the underlying algorithm library, rule library and the like.

[0133] 2.1.3 User preference

[0134] The user preference setting is to make the generated policy more in line with the user's needs. The present application provides five user preferences for the user by default: time efficiency, accuracy, scalability, explainability and adaptability. For existing log anomaly detection, it is often a mass of log data processing, so the present application defaults to time efficiency first, and then accuracy first.

[0135] 2.1.4 Vectorization

[0136] The candidate processing method and detection algorithm of the current log template are obtained from the knowledge base, and the encoding is converted into a state matrix, thereby establishing a mapping relationship under different concept layers.

[0137] When selecting the operation and maintenance processing strategy, the more auxiliary decision basis possessed is better, and building a complete information model is the starting point of the present application. The knowledge base established in advance provides decision basis for policy arrangement reasoning, and intelligent algorithms are introduced for policy arrangement, so that man-machine cooperation is achieved in the log anomaly detection process, and the pressure of manual operation and maintenance of administrators is reduced.

[0138] The triple AG=(S, V, E) model is used, S is a state node, and V is a set of evaluation index nodes.

[0139] The pre-processing mode pre(v i S j ) means that the index parameter obtained by using the i-th pre-processing method in the S j state is v i .

[0140] The encoding selection mode matrix encode k : The encoding selection matrix is a 01 matrix. If the corresponding encoding mode encode(v i S j ) is selected, then

[0141] The detection algorithm mode algorithm k : Similarly, a 01 matrix is used. If the corresponding encoding mode algorithm(v i S j ) is selected, then

[0142] Select mode degree N: represents all cases that can be used in this state, obtained by the following operation:

[0143]

[0144] 2.2 Strategy arrangement

[0145] 2.2.1 Benefit calculation

[0146] According to user preferences, the benefit calculation is also different when arranging strategies. In the process of reasoning, the strategy will be recommended according to the user preference, and the return on investment will be evaluated. The way of evaluating the return and updating the accumulated return and expected income will affect the learning efficiency and effect. The present application gives different weight indicators.

[0147] (1) Time efficiency

[0148] It is required to quickly process a large amount of log data, to detect and alarm in real time, and to assign rules as shown in Table 6.

[0149] Table 6 Time efficiency immediate return

[0150]

[0151]

[0152] The time efficiency immediate return is as follows:

[0153]

[0154] wherein is the return index, s represents the state, and s→s' represents the transition from the current state to the next state by executing the strategy.

[0155] (2) Accuracy

[0156] It can accurately detect abnormal behavior in logs, reduce false positive rate and false negative rate. The assignment rules are as shown in Table 7.

[0157] Table 7 Accuracy immediate return

[0158]

[0159] The accuracy immediate return is as follows:

[0160]

[0161] wherein is the return index, s represents the state, and s→s' represents the transition from the current state to the next state by executing the strategy.

[0162] (3) Scalability

[0163] It can adapt to different sizes of log data and different application scenarios, supporting horizontal and vertical expansion. The assignment rules are shown in Table 8.

[0164] Table 8 Scalability immediate return

[0165]

[0166]

[0167] The scalability immediate return is as follows:

[0168]

[0169] where is the return index, s represents the state, and s→s' represents the transition from the current state to the next state by executing the policy.

[0170] (4) Explainability

[0171] It can explain the results of anomaly detection, provide visual and monitoring interfaces, and facilitate user understanding and use. The assignment rules are shown in Table 9.

[0172] Table 9 Explainability immediate return

[0173]

[0174] The explainability immediate return is as follows:

[0175]

[0176] where is the return index, s represents the state, and s→s' represents the transition from the current state to the next state by executing the policy.

[0177] (5) Adaptability

[0178] It can adaptively learn and adjust the model to adapt to changes in data distribution and new attack methods. The assignment rules are shown in Table 10.

[0179] Table 10 Adaptability immediate return

[0180]

[0181] The adaptability immediate return is as follows:

[0182]

[0183] where For the return indicator, s represents the state, and s→s' represents the transition from the current state to the next state by executing the strategy.

[0184] (6) Example

[0185] The reasoning purpose setting bias is more inclined to accuracy and adaptability, and the requirements for time and interpretability are slightly lower. The voting weight value of the pretreatment method obtained by combining the indicators is shown in Table 11.

[0186] Table 11 Voting values of different reasoning purposes

[0187] rd1 rd2 rd3 rd4 rd5 rd6 0 0.6 0 0 0 0.4

[0188] By preference, the reward function income indicator weight is adjusted, and the expected cumulative return of the corresponding detection algorithm for each different purpose is calculated as follows:

[0189]

[0190] In the process of selection, speed is faster under the premise of ensuring accuracy to meet the conditions. In order to speed up the learning process under the condition of large-scale selection strategy, the application uses the reward and punishment mechanism to update the state transition income for sequence decision-making during online learning. After each step of exploration and utilization selection, the cumulative return and cumulative expected return are adjusted, and the calculation is as follows:

[0191]

[0192] vote γ (s, a) > θ, reward the response measure.

[0193] Q i (s, a) = μQ i (s, a), μ ≥ 1

[0194] Where μ is the reward factor.

[0195] vote γ (s, a) < θ, punish the response measure.

[0196] Q i (s, a) = vQ i (s, a), 0 ≤ v ≤ 1

[0197] Where v is the punishment factor.

[0198] 2.2.2 Scheduling algorithm

[0199] The application is based on Q-learning algorithm to realize policy reasoning. Under the support of data provided by the knowledge base, initial information under the current scene is obtained, candidate state policy and reasoning purpose preference are combined, and online learning mechanism is used for policy reasoning arrangement. The arrangement process mainly includes three links: state matching, state transition and online learning. The workflow is as shown in Figure 7

[0200] In the state matching link, by receiving context information, pre-processing mode pre(v i S j ), encoding selection mode matrix encode k and detection algorithm mode algorithm k are obtained by using the knowledge base matching. These results will provide an initial parameter environment for the following Q-learning algorithm, so as to better provide an initial state value for decision-making.

[0201] In the state transition link, by executing the state transition formula, the transition from the current state to the next state is realized. At the same time, the corresponding strategy is executed to obtain the return income, and the cumulative return income is calculated. The state transition formula can refer to the specific implementation in the pseudo code, and the state transition formula is as follows:

[0202]

[0203]

[0204] Among them a represents the execution action, am k represents the current state, represents the return income value from s→s′ state, γmax a′ Q i (am k′ , a′) is the maximum return income of the introduced reward and punishment factor, and α represents the stability factor.

[0205] In the online learning link, a voting mechanism is used to speed up the learning speed and make it quickly converge. The voting mechanism includes safety stability threshold, stability factor, punishment factor and temperature and other factors. Under the comprehensive action of these factors, the user demand can be learned and adapted faster, and more accurate and efficient service can be provided.

[0206] The arrangement algorithm flow is given.

[0207]

[0208]

[0209] ​The present application relates to two links have corresponding alternative algorithm. In the knowledge fusion stage, not only can the cosine similarity adopted by the present application be used to determine similarity, but also other similarity determination algorithms can be used. In the arrangement algorithm aspect, not only can the Q-Learning algorithm used by the present application be used, but also other reinforcement learning algorithms can be used.

[0210] 2.3 policy arrangement example

[0211] We take Hadoop logs as an example to test the present application. In order to simulate the influence of unknown templates, we add a column of attribute values PID to the obtained Hadoop, so that it cannot be completely matched with the Hadoop template.

[0212] The log data is as shown in the table. Figure 8

[0213] The cosine similarity is used to calculate the matching degree between templates, Figure 9 The matching result is as follows:

[0214] The most similar template of the current matching is the No. 23 template, and from the query in the database we can know that the most similar template at present is the Hadoop template. Next, the abnormal detection knowledge base of the present application is used to judge the state, so as to obtain the quantitative index of the current state, and provide a basis for the optimization of the strategy.

[0215] As can be seen from Figure 10 , the knowledge base provides four regular expressions, which can facilitate the extraction of events.

[0216] Search the knowledge base to find the optional strategy, and finally the following indexes can be determined:

[0217] As Figure 11 , there are 10 kinds of preprocessing methods that can be used based on the Hadoop template, including Drain, Logmine, SHISO, IPLOM, FT-tree, POP, Mean-shift, DBSCAN, GCD and K-means.

[0218] Among them, the indexes involved are SC, CH, DBI, RI, ARI, NMI and AMI. This time, online learning is carried out by using reinforcement learning, and the purpose skew is more inclined to accuracy and security, and the requirements for time and interpretability are slightly low, so by combining the indexes, the voting weight values of the preprocessing methods can be obtained as shown in the following table:

[0219] Table 12 voting values of different response purposes

[0220] rd1 rd2 rd3 rd4 rd5 0 0.6 0 0 0.4

[0221] ​The current index in the index database is queried, and the arithmetic average of the historical data of the multiple index unified templates is accumulated to obtain the pretreatment R value as [0.60482345, 0.81830932, 0.74391856, 0.91937483, 0.66123651, 0.67489423, 0.82190348, 0.86913485, 0.81263016, 0.92673906].

[0222] Figure 12 The knowledge base evaluation index data preprocessed by Drain for Hadoop is used.

[0223] Based on the Hadoop template, 22 kinds of anomaly detection methods can be used, as shown in Figure 13-15 .

[0224] The R table for the current detection algorithm is obtained as [0.60482345, 0.81830932, 0.74391856, 0.91937483, 0.66123651, 0.67489423, 0.82190348, 0.86913485, 0.81263016, 0.92673906, 0.89073261, 0.78059342, 0.69786024, 0.75772938, 0.86475933, 0.88291476, 0.76255733, 0.62192435, 0.90123456, 0.88324878, 0.66281766, 0.71347095].

[0225] We use the above R table and state and response purpose preference weight as input to perform online training of reinforcement learning, and the training times can be set to end convergence to obtain the final result, and we select 200 times of online training, compare the final cumulative expectation value by calculating the cumulative expectation of each strategy, and select the maximum expectation value as the optimal strategy result.

[0226] Under this test, a processing flow given by the application is to use a regular expression to preprocess the log, extract key elements, then use Drain to preprocess the log, including pattern extraction and clustering, use an embedding layer to encode the log, use Bi-LSTM to perform log anomaly detection, and log data not classified into any cluster can be clustered using GCD, thereby realizing automatic parsing and anomaly detection of logs, as shown in Figure 16 .

[0227] Another aspect of the application provides a log anomaly detection auxiliary decision system based on a knowledge graph and reinforcement learning, comprising:

[0228] A knowledge base construction module is configured to construct a log anomaly detection knowledge base based on a knowledge graph, and to collect existing log anomaly detection knowledge, including: extracting knowledge from existing literature and cases, and preliminarily constructing a log anomaly detection knowledge base; and optimizing the log anomaly detection knowledge base through a similar entity merging method.

[0229] A reasoning engine construction module is configured to construct a reasoning engine based on reinforcement learning, and the reasoning engine is data-supported by the log anomaly detection knowledge base and takes the reinforcement learning algorithm as a core, and through the process of generating a recommended strategy, implementing the strategy and strategy feedback, the recommended result is continuously adjusted and optimized, and meanwhile, the recommended result is fed back to the log anomaly detection knowledge base, and evolution of the log anomaly detection base is driven, so as to continuously optimize and adapt to detection requirements.

[0230] The above only shows the preferred embodiments of the present application, and it should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A log anomaly detection aided decision-making method based on a knowledge graph and reinforcement learning, characterized in that, Comprise: Based on the knowledge graph construction log anomaly detection knowledge base, collection of existing log anomaly detection knowledge; including: from existing literature and cases to extract knowledge, preliminary construction of log anomaly detection knowledge base; through the similar entity merging method optimization log anomaly detection knowledge base; Based on reinforcement learning to build reasoning engine, reasoning engine with log anomaly detection knowledge base for data support, with reinforcement learning algorithm as the core, through the generation of recommended strategy, implementation strategy and strategy feedback process, constantly adjust and optimize the recommended results, but also feedback to the log anomaly detection knowledge base, drive the evolution of log anomaly detection library, to optimize and adapt to the detection demand; The construction process of the reasoning engine specifically includes: From the log original text and log template two angles to collect information, and use cosine similarity to complete the verification; the knowledge base contains log template entity, and the knowledge base records the corresponding log template information according to the log type; randomly extract n log items from the target log original text, and determine the similarity between the preprocessed log items and the log template reserved in the knowledge base; if the n log items are similar to the log template reserved for the same log type, the target log is considered to belong to the log type; According to the actual, the algorithm and rule that does not meet the detection task this time will be filtered from the strategy arrangement process this time, to ensure that the final generated strategy implementation can be supported by the underlying algorithm library and rule library; The user sets the preference of the recommended strategy of reasoning; the preference includes time efficiency, accuracy, scalability, interpretability and adaptability; Through the log anomaly detection knowledge base, the candidate processing method and detection algorithm of the current log template are obtained, which are coded into a state matrix, thereby establishing the mapping relationship under different concept layers; In the process of reasoning, the strategy is recommended according to the user preference, and the return is evaluated; Based on Q-learning algorithm to realize strategy reasoning.

2. The log anomaly detection aided decision method based on knowledge graph and reinforcement learning according to claim 1, characterized in that, The preliminary construction of log anomaly detection knowledge base by extracting knowledge from existing literature and cases includes: Based on regular expression, the processing habit and processing effect of different logs are extracted to form the strategy entity in the preprocessing stage; The commonly used log template mining algorithm is listed as the log template mining algorithm entity, and the log template mining result is around the log template mining result. The log template of the common log is counted, and the log template mining entity is constructed accordingly. The relationship between the log template and the log template mining algorithm entity is established; the common type log is listed as an entity, and the attribute is used to record the index of the log template mining algorithm processing different logs; Different log anomaly detection algorithms are used to find the abnormal information that may exist in the log: the commonly used log anomaly detection algorithm entity is constructed, and the characteristics of the entity are recorded by using the attribute, and stored in the graph database Neo4j; based on the abnormal detection algorithm evaluation index, the corresponding relationship and attribute are constructed.

3. The log anomaly detection aided decision method based on knowledge graph and reinforcement learning according to claim 1, characterized in that, The optimization of log anomaly detection knowledge base by similar entity merging method includes: Firstly, similar entities are screened out based on entity similarity according to a predetermined threshold value; then, whether the entities are similar is determined based on structure similarity judgment from the screened similar entity set; and on the basis of confirming the similar entities, the similar entities are knowledge fused based on the principle of relationship alignment.

4. The log anomaly detection aided decision method based on knowledge graph and reinforcement learning according to claim 3, characterized in that, The entity similarity determination method comprises: The knowledge base entity set is processed by word segmentation coding based on the BERT model trained in the general word, the similarity between entities is calculated using cosine similarity, and all entities are traversed by matching two by two, when the similarity of two entities exceeds the threshold value, they are considered as similar entities and added to the similar entity set.

5. The log anomaly detection aided decision method based on knowledge graph and reinforcement learning according to claim 4, characterized in that, The structure similarity determination method comprises: The similar entity set is input, the topology structure of the similar entity set reflected in the knowledge graph is extracted, and similarity is determined, wherein the features of the topology structure include the number of preprocessing methods, the number of templates, the number of regular expressions, the number of anomaly detection methods, and the average of index evaluation, the topology structure of the entity is converted into a feature vector by querying the knowledge graph, and then the similarity is calculated based on the cosine similarity principle; if it is higher than the threshold value, it is determined that the two entities are similar, and subsequent knowledge fusion is performed.

6. The log anomaly detection aided decision-making method based on a knowledge graph and reinforcement learning according to claim 3, characterized in that, The knowledge fusion of the similar entities comprises: Firstly, the relationship representation learning is performed using the TransE algorithm to vectorize the relationship related to the entity; then, the similarity between the relationship vectors corresponding to the similar entities is calculated according to the cosine similarity, the relationship greater than the predetermined threshold value is considered as similar and is fused, and the relationship less than the predetermined threshold value is considered as dissimilar and is reserved or deleted by the expert for further confirmation.

7. The log anomaly detection aided decision-making method based on a knowledge graph and reinforcement learning according to claim 1, characterized in that, The strategy inference based on the Q-learning algorithm comprises: By receiving context information, the preprocessing mode, the coding selection mode matrix and the detection algorithm mode are matched by using the log anomaly detection knowledge base; By executing the state transition formula, the current state is transferred to the next state; at the same time, the corresponding strategy is executed to obtain the return reward, and the cumulative return reward is calculated; A voting mechanism is adopted to speed up the learning speed and make it quickly converge.

8. A knowledge graph and reinforcement learning based log anomaly detection aided decision system, characterized in that, It comprises: The knowledge base construction module is used to construct the log anomaly detection knowledge base based on the knowledge graph, and the existing log anomaly detection knowledge is collected; including: extracting knowledge from existing literature and cases to preliminarily construct the log anomaly detection knowledge base; optimizing the log anomaly detection knowledge base by the similar entity merging method; The reasoning engine construction module is used to construct the reasoning engine based on reinforcement learning, the reasoning engine takes the log anomaly detection knowledge base as data support and takes the reinforcement learning algorithm as the core, through the process of generating recommended strategies, implementing strategies and strategy feedback, the recommended results are constantly adjusted and optimized, and at the same time, feedback is given to the log anomaly detection knowledge base to drive the evolution of the log anomaly detection library to continuously optimize and adapt to the detection demand; The construction process of the reasoning engine comprises: Information is collected from log original text and log template, and verification is completed by using cosine similarity; the knowledge base contains log template entities, and the knowledge base records corresponding log template information according to log types; n log items are randomly extracted from the target log original text, preprocessed, and compared with the reserved log template in the knowledge base for similarity determination; if the n log items are similar to the reserved log template of the same log type, it is considered that the target log belongs to the log type; According to the actual situation, the algorithms and rules that do not meet the current detection task are filtered from the current strategy arrangement process to ensure that the finally generated strategy implementation can be supported by the underlying algorithm library and rule library; The user sets the preference of the recommended strategy for reasoning; the preference includes time efficiency, accuracy, scalability, interpretability and adaptability; Through the log anomaly detection knowledge base, the candidate processing method and detection algorithm of the current log template are obtained, which are coded into a state matrix, thereby establishing a mapping relationship under different concept layers; In the reasoning process, the strategy is preferentially recommended according to the user preference, and the return on investment is evaluated; The strategy reasoning is realized based on the Q-learning algorithm.

Images