Abnormal event prediction model acquisition method based on log analysis

Abstract

The invention discloses a log analysis-based abnormal event prediction model acquisition method, which belongs to the technical field of data processing, and comprises the following steps: S1, defining a corresponding rule according to target system characteristics, based on an NLP large model, extracting a mutual relation between process entities from first historical unstructured data in various log data of the target system, and further obtaining a process triple; s2, constructing a knowledge graph based on a process triple, a corresponding rule and first historical structured data in the various log data; and S3, verifying the association relationship in the knowledge graph based on the second historical unstructured data and the second historical structured data in the various log data, obtaining a verification result, and obtaining an abnormal event prediction model based on the verification result. The technical problem that the accuracy of anomaly prediction is low due to the fact that structural characteristics of log data are not considered is solved. And the accuracy of anomaly prediction is obviously improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and specifically to a method for obtaining an abnormal event prediction model based on log analysis. Background Art

[0002] With the increasing complexity of software and hardware systems, the requirements for their performance and stability are also constantly rising. In order to ensure the ability to quickly identify and respond to potential abnormal situations in the system, the existing technology mainly deeply analyzes system logs to achieve rapid detection of abnormal situations. For example, an abnormal log detection method, abnormal log detection device and storage medium with the patent number CN112269730A includes: obtaining a first log and a second log, where the first log is the log after receiving an alarm signal indicating log abnormality, and the second log is the log during the non-alarm period before receiving the alarm signal; extracting a first log template based on the first log and a second log template based on the second log; determining the log template that exists in the first log template and does not exist in the second log template as an abnormal log template; and determining the log in the first log corresponding to the abnormal log template as an abnormal log. The above solution can achieve rapid detection of abnormal logs in the logs generated by service operation, providing a guarantee for fault cause location. However, in the process of processing logs in the above solution, the structural characteristics of log data are not considered. In actual applications, log data often includes structured data and unstructured data. Structured data has a clear format and fields. If these characteristics are not fully utilized for targeted processing, it is difficult to mine the key information in structured data. For unstructured data, if the processing method is too general, it is also difficult to accurately capture the key information contained therein, resulting in the problem of low accuracy of abnormal prediction. Summary of the Invention

[0003] Aiming at the technical problem that the existing technology does not consider the structural characteristics of log data, resulting in low accuracy of abnormal prediction, the present invention provides a method for obtaining an abnormal event prediction model based on log analysis. By using the powerful language understanding ability of the NLP large model to extract process entities and mutual relationships from unstructured data, structured process triples suitable for constructing a knowledge graph are obtained. By constructing a knowledge graph with the structured process triples suitable for constructing a knowledge graph and structured data, while fully reflecting the key information and correlation in the structured process triples and structured data in the knowledge graph, the richness of the knowledge graph data is also ensured, solving the technical problem of low accuracy of abnormal prediction due to the failure to consider the structural characteristics of log data. Through a dual mechanism, the accuracy of abnormal prediction is significantly improved.

[0004] To solve the above technical problems, the present invention provides a method for obtaining an abnormal event prediction model based on log analysis, including the following steps: S1: Define corresponding rules according to the characteristics of the target system, extract the mutual relationships between process entities from the first historical unstructured data in various log data of the target system based on the NLP large model, and then obtain process triples; S2: Construct a knowledge graph based on the process triples, corresponding rules, and the first historical structured data in the various log data; S3: Verify the association relationships in the knowledge graph based on the second historical unstructured data and the second historical structured data in the various log data and obtain a verification result, and obtain an abnormal event prediction model based on the verification result.

[0005] After adopting the above technical solution, the present invention has the following advantages: Considering that the NLP large model has powerful language understanding capabilities and is suitable for processing unstructured data, and it is relatively complex to construct a knowledge graph through unstructured data. Therefore, the NLP large model is used to convert unstructured data into structured process triples suitable for constructing a knowledge graph, which improves the efficiency of constructing the knowledge graph and the richness of the data for constructing the knowledge graph. And the knowledge graph can fully reflect the key information and relevance in the structured process triples and structured data. Therefore, abnormal prediction through the knowledge graph significantly improves the efficiency and accuracy of abnormal event prediction; Construct a knowledge graph through the corresponding rules defined by the characteristics of the target system, meet the log data analysis requirements of corresponding systems in different industries, and improve the applicability and generality of abnormal event prediction; Solve the technical problem of low accuracy of abnormal prediction caused by not considering the structural characteristics of log data.

[0006] Preferably, in S1, the defining corresponding rules according to the characteristics of the target system includes: Respectively obtain keyword rules according to the prompts indicating abnormal processes and normal processes in the target system; Set the priority of the keyword rules based on the embedding principle, and obtain the fault level rules based on the priority.

[0007] Preferably, in S1, the extracting the mutual relationships between process entities from the first historical unstructured data in various log data of the target system based on the NLP large model, and then obtaining process triples includes: Define entity types and mutual relationship types based on the corresponding rules, and construct an identification model based on the NLP large model; Input the first historical unstructured data into the successfully trained identification model to obtain the mutual relationships between process entities, and then obtain initial triples; The semantic similarity of the initial triples is obtained based on the word vectors, and the initial triples are updated based on the semantic similarity to obtain the process triples.

[0008] In this solution, a recognition model is constructed through a large NLP model, so that the recognition model can convert unstructured data into structured process triples, providing a data basis for the construction of the knowledge graph. The initial triples are updated by obtaining semantic similarity through word vectors, and similar initial triples are eliminated, thereby reducing the amount of data and further improving the efficiency of anomaly prediction. At the same time, by obtaining structured process triples, a rich data basis is provided for the construction of the knowledge graph, which improves the accuracy and comprehensiveness of the constructed knowledge graph.

[0009] Preferably, the step of inputting the first historical unstructured data into a successfully trained recognition model to obtain the relationship between process entities comprises: The recognition model is trained based on the historical log data, entity types and relationship types in the target system. When the entity types of the training entities extracted from the historical log data and the relationship types between the training entities meet the preset conditions, it means that the recognition model training is successful.

[0010] In this solution, the recognition model is trained according to the entity types and relationship types defined by the corresponding rules defined in the target system, so that customers can select the target system according to their needs, and then conduct targeted training on the recognition model according to the characteristics of the selected target system to adapt to the actual usage scenarios, thereby further improving the versatility and adaptability of anomaly prediction. When the recognition model's recognition results meet the preset conditions, it means that the recognition model training is successful, thereby improving the accuracy of the recognition model.

[0011] Preferably, S2 includes: S21: The triple corresponding to the first historical structured data and the process triple form a final triple, the final triple is matched with the keyword rule, and the final triple that successfully matches is classified as the first initial abnormal triple; S22: classify the final triplet that fails to match as the initial normal triplet; S23: based on the 3σ algorithm, a second initial abnormal triplet is selected from the initial normal triplet to obtain a final normal triplet, and the first initial abnormal triplet and the second initial abnormal triplet form an abnormal triplet; S24: Constructing knowledge graph based on abnormal triples.

[0012] In this solution, the first initial abnormal triple is obtained by matching the keyword rules obtained according to the prompts indicating abnormal processes and normal processes in the target system with the final triple. While further ensuring the generality of abnormal prediction, it also initially eliminates the initial normal triples, further improving the efficiency of abnormal detection. The abnormal triples are obtained through the 3σ algorithm, further eliminating the final normal triples and further improving the detection efficiency. By converting unstructured data into structured process triples and combining them with structured data, and at the same time processing the obtained final triples to obtain abnormal triples, it improves the data richness of building the knowledge graph while improving the efficiency of abnormal detection and the accuracy of the knowledge graph.

[0013] Preferably, the S23 includes: S231: Obtain the mean, variance, and standard deviation of the initial normal data corresponding to the initial normal triples; S232: Determine the 3σ range with the mean as the benchmark and the standard deviation as the boundary; S233: The initial normal triples corresponding to the initial normal data within the 3σ range are the second initial abnormal triples.

[0014] Preferably, the S24 includes: S241: Compare the abnormal entities in the abnormal triples, combine the successfully compared abnormal triples to obtain a highly correlated combination, mark the highly correlated combination as highly correlated, and mark the abnormally compared triples as moderately correlated, thereby obtaining a correlation knowledge graph; S242: Match the final normal triples with the correlation knowledge graph to obtain a matching result, and update the correlation knowledge graph based on the matching result to obtain a knowledge graph.

[0015] In this solution, by combining the successfully compared abnormal triples, when an abnormal situation occurs, the root cause of the abnormal situation can be searched horizontally and vertically according to the combination order, and then processed in a timely manner, improving the timeliness of maintenance and the stability and security of the system. By matching the normal triples with the correlation knowledge graph and updating the correlation knowledge graph according to the matching result, it is convenient to discover the potential root cause of the abnormal situation, improving the comprehensiveness of abnormal situation detection and further improving the stability and security of the system. By making corresponding marks, it is convenient to make priority responses according to the marked situation in the future, thereby improving the scheduling ability of the system and the ability to avoid huge risks.

[0016] Preferably, the S24 further includes: Based on the fault level rule, the abnormal triples marked as highly relevant and moderately relevant in the correlation knowledge graph are associated with the alarm phenomena, and then the correlation knowledge graph is updated.

[0017] In this solution, the abnormal triples marked as highly relevant and moderately relevant are set with priorities through the fault level rule, the priorities are associated with the alarm phenomena, and then the correlation knowledge graph is updated. Alarms can be triggered according to the priorities, further improving the stability and security of the system, and further improving the scheduling ability and huge risk avoidance ability of the system.

[0018] Preferably, the S3 includes: S31: Match the second historical unstructured data and the second historical structured data with the association relationships in the knowledge graph to predict the root cause of the fault; S32: If the number of the second historical unstructured data or the second historical structured data whose root cause of the fault matches the actual cause is greater than the preset number, obtain the abnormal event prediction model based on the knowledge graph; otherwise, adjust the association relationships in the knowledge graph according to the second historical unstructured data and the second historical structured data whose root cause of the fault fails to match the actual cause to obtain the abnormal event prediction model.

[0019] Preferably, in S1, the various types of log data at least include operation log data, system output log data, middleware log data, and interface log data.

[0020] The beneficial effects of this solution: Considering that the NLP large model has powerful language understanding ability and is suitable for processing unstructured data, and it is relatively complex to construct a knowledge graph through unstructured data. Therefore, the NLP large model is used to convert the unstructured data into structured process triples suitable for constructing the knowledge graph, which improves the construction efficiency of the knowledge graph and the richness of the data for constructing the knowledge graph. The knowledge graph can fully reflect the key information and correlation in the structured process triples and the structured data. Therefore, abnormal prediction through the knowledge graph significantly improves the efficiency and accuracy of abnormal event prediction; The knowledge graph is constructed through the corresponding rules defined by the target system characteristics, which meets the log data analysis requirements of corresponding systems in different industries and improves the applicability and generality of abnormal event prediction; By combining the successfully matched abnormal triples, when an abnormal situation occurs, it is possible to search horizontally and vertically according to the combination order to obtain the root cause of the abnormal situation, and then process it in a timely manner, improving the timeliness of maintenance and the stability and security of the system. By matching the normal triples with the correlation knowledge graph and updating the correlation knowledge graph according to the matching results, it is convenient to discover the potential root cause of the abnormal situation, improving the comprehensiveness of abnormal situation detection and further enhancing the stability and security of the system. By making corresponding marks, it is convenient to make priority responses according to the marked situation later, thereby improving the scheduling ability and huge risk avoidance ability of the system; It solves the technical problem of low accuracy of abnormal prediction caused by not considering the structural characteristics of log data. Brief Description of the Drawings

[0021] Other features, objects, and advantages of the present invention will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings. The drawings are only for the purpose of showing the preferred embodiments and are not considered as limiting the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components.

[0022] Figure 1 It is a flowchart of the method for obtaining an abnormal event prediction model based on log analysis of the present invention. Detailed Embodiments

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only the best embodiments of the present invention and are only used to explain the present invention, without limiting the protection scope of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0024] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations (or steps) can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operations are completed, but there can also be additional steps not included in the drawings; the process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0025] Embodiment 1: As Figure 1 shown, the method for obtaining an abnormal event prediction model based on log analysis includes the following steps: S1: Define corresponding rules according to the characteristics of the target system, extract the mutual relationships between process entities from the first historical unstructured data in various log data of the target system based on the NLP large model, and then obtain process triples.

[0026] In S1, the defining of corresponding rules according to the characteristics of the target system includes: Obtain keyword rules respectively according to the prompts indicating abnormal processes and normal processes in the target system; Set the priority of the keyword rules based on the embedding principle, and obtain the fault level rules based on the priority.

[0027] In S1, the extracting of the mutual relationships between process entities from the first historical unstructured data in various log data of the target system based on the NLP large model, and then obtaining process triples includes: Define entity types and mutual relationship types based on the corresponding rules, and build an identification model based on the NLP large model; Input the first historical unstructured data into the successfully trained identification model to obtain the mutual relationships between process entities, and then obtain initial triples; Obtain the semantic similarity of the initial triples based on word vectors, and update the initial triples based on the semantic similarity to obtain process triples.

[0028] The inputting of the first historical unstructured data into the successfully trained identification model to obtain the mutual relationships between process entities includes: Train the identification model based on the historical log data, entity types and mutual relationship types in the target system. When the training entities extracted from the historical log data, the entity types to which the training entities belong, the mutual relationships between the training entities, and the mutual relationship types to which the mutual relationships between the training entities belong all meet the preset conditions, it indicates that the identification model is successfully trained.

[0029] In S1, the various log data at least include operation log data, system output log data, middleware log data, and interface log data.

[0030] In this embodiment, the structured data in various types of log data obtained from the target system, namely operation log data, system output log data, middleware log data, and interface log data, is directly placed into KAFKA, and the unstructured data is placed in a specified cache directory for temporary storage. This realizes a flexible configuration method that is general and easy to use. The collected various types of log data are classified according to profession, system, and manufacturer, tagged, and then the original various types of log data are stored in Elasticsearch, realizing various flexible queries and parses based on Elasticsearch, and improving the log query and analysis capabilities.

[0031] In this embodiment, the prompts for abnormal processes and abnormal processes are set according to the systems corresponding to different industries. The keyword rules are as follows: the prompt indicating the normal execution of a process in the log data is defined as normal, and the prompt indicating the abnormal execution of a process in the log data is defined as abnormal. At the same time, the abnormal context information when the prompt for abnormal process execution appears is also defined as abnormal. For example, when a user needs to query "the departure time of train number G1212", at this time, the user enters a train number that does not exist in the system, such as A1211. After the user enters A1211 for query, the prompt information of the system is "query failed". At this time, "query failed" is defined as abnormal. When the same user enters a train number that does not exist in the system multiple times for query, in addition to defining "query failed" as abnormal, it is also necessary to record the user's access times and user characteristics, and initially define this user as abnormal, so as to facilitate detection based on the defined abnormal situations, and improve the security and stability of the system.

[0032] In this embodiment, based on the embedding principle, the priority of the keyword rules is set as follows: If although the information required by the user is displayed after the user performs a certain operation, but a warning prompt appears before the display of the information, and the user performs other operations on the information display page, and at this time a warning prompt appears again, then the first warning prompt has a higher priority than the second warning prompt. By obtaining the fault level rules through the embedding principle, warnings that have a more significant impact on the system operation are given higher priorities, facilitating users to handle abnormal situations according to the priorities, and thus improving the stability and security of the system.

[0033] In this embodiment, the entity types and mutual relationship types are defined based on corresponding rules as follows: Entities are obtained from various types of log information in the target system, the mutual relationships between entities are set according to experience, and the entities are classified, thereby forming entity - mutual relationship - entity, and also including the corresponding rules for entity types and mutual relationship types. The entity types and mutual relationship types in the corresponding rules are the defined entity types and mutual relationship types. Obtaining entity types and mutual relationship types from various types of log information through experience improves the practicality of entity types and mutual relationship types. Obtaining the semantic similarity of the initial triples based on word vectors and updating the initial triples based on the semantic similarity to obtain process triples specifically means: Using measurement methods such as cosine similarity and Euclidean distance to calculate the semantic similarity between the initial triples. At the same time, more complex semantic matching models such as ConvKB can also be considered to calculate the scores of the initial triples, thereby reflecting their semantic similarity. Constructing an identification model through an NLP large model enables the identification model to convert unstructured data into structured process triples, providing a data basis for the construction of the knowledge graph. Updating the initial triples by obtaining semantic similarity based on word vectors, eliminating similar initial triples, thereby reducing the data volume, further improving the anomaly prediction efficiency. At the same time, by obtaining structured process triples, it provides a rich data basis for the construction of the knowledge graph, improving the accuracy and comprehensiveness of the constructed knowledge graph. When both the entity type to which the training entity extracted from the historical log data belongs and the mutual relationship type to which the mutual relationship between the training entities belongs meet the preset conditions, specifically: When both the entity type to which the training entity belongs and the mutual relationship type to which the mutual relationship between the training entities belongs match the actual entity type to which the training entity belongs and the actual mutual relationship type to which the mutual relationship between the training entities belongs, it means that the preset conditions are met. Through targeted training of the model, the accuracy of the identification model is improved, and thus the accuracy of the obtained mutual relationships between entities is improved.

[0034] S2: Construct a knowledge graph based on the process triples, the corresponding rules, and the first historical structured data in the various types of log data.

[0035] The S2 includes: S21: The triples corresponding to the first historical structured data and the process triples form the final triples. The final triples are matched with the keyword rules, and the final triples that match successfully are classified as the first initial abnormal triples; S22: The final triples that fail to match are classified as the initial normal triples; S23: Based on the 3σ algorithm, second initial abnormal triples are screened out from the initial normal triples to obtain the final normal triples. The first initial abnormal triples and the second initial abnormal triples form the abnormal triples; S24: Construct a knowledge graph based on abnormal triples.

[0036] The S23 includes: S231: Obtain the mean, variance, and standard deviation of the initial normal data corresponding to the initial normal triples. S232: Determine the 3σ range with the mean as the benchmark and the standard deviation as the boundary. The initial normal triples corresponding to the initial normal data within the 3σ range are the second initial abnormal triples.

[0037] The S24 includes: S241: Compare the abnormal entities in the abnormal triples, combine the successfully compared abnormal triples to obtain a highly correlated combination, mark the highly correlated combination as highly correlated, mark the abnormally compared triples that fail to match as moderately correlated, and then obtain a correlation knowledge graph. S242: Match the final normal triples with the correlation knowledge graph to obtain a matching result, and update the correlation knowledge graph based on the matching result to obtain a knowledge graph.

[0038] The S24 further includes: Associate the abnormal triples marked as highly correlated and moderately correlated in the correlation knowledge graph with the alarm phenomena based on the fault level rule, and then update the correlation knowledge graph.

[0039] In this embodiment, the final triples are matched with the keyword rules, and the final triples with successful matches are classified as the first initial abnormal triples. Specifically, for example, if the number of times the same user logs in to the system on the same day exceeds 5 times, it is abnormal; if it is less than or equal to 5 times, it is normal. At this time, the initial keyword rule is (user ID, number of logins is 5, system). If the final triples in the final triples are (ID1, 6, system) and (ID2, 4, system), then (ID1, 6, system) is the first initial abnormal triple, and (ID2, 4, system) is the initial normal triple. By matching the keyword rules obtained according to the prompts indicating abnormal processes and normal processes in the target system with the final triples to obtain the first initial abnormal triples, while further ensuring the generality of abnormal prediction, the initial normal triples are initially excluded, further improving the efficiency of abnormal detection. The 3σ range is an interval bounded by plus and minus three standard deviations based on the mean. By using the 3σ algorithm to obtain abnormal triples, the final normal triples are further excluded, further improving the detection efficiency. By converting unstructured data into structured process triples, combining them with structured data, and processing the obtained final triples to obtain abnormal triples, while improving the data richness of constructing the knowledge graph, the efficiency of abnormal detection and the accuracy of the knowledge graph are improved.

[0040] In this embodiment, the abnormal entities in the abnormal triples are compared. If the head abnormal entity in an abnormal triple is the same as the head abnormal entity in another abnormal triple, the two head abnormal entities are connected. If the head abnormal entity in an abnormal triple is the same as the tail abnormal entity in another abnormal triple, the head abnormal entity and the tail abnormal entity are connected. If the tail abnormal entity in an abnormal triple is the same as the tail abnormal entity in another abnormal triple, the two tail abnormal entities are connected, thus obtaining a highly correlated combination. Similarly, the method of matching the final normal triples with the correlation knowledge graph is the same as the above method. By combining the abnormal triples with successful comparisons, when an abnormal situation occurs, the root cause of the abnormal situation can be searched horizontally and vertically according to the combination order, and then processed in a timely manner, improving the timeliness of maintenance, the stability and security of the system. By matching the normal triples with the correlation knowledge graph and updating the correlation knowledge graph according to the matching results, it is convenient to discover the potential root cause of the abnormal situation, improving the comprehensiveness of abnormal situation detection, and further improving the stability and security of the system. By making corresponding marks, it is convenient to make priority responses according to the mark situation in the follow-up, thereby improving the scheduling ability and huge risk avoidance ability of the system.

[0041] In this embodiment, the abnormal triple with the strongest event correlation, that is, the event, is found through the fault level rule. Then, the correlation analysis is carried out between these events with the strongest correlation and the status data and alarm phenomena to construct the correlation relationship among alarms, status, and events. Through this correlation relationship, the occurrence of future alarms can be predicted. For example, if Event 1 and Event 2 occur and the device status parameters are close to the warning value, then an alarm may occur in the next period of time, and it can be predicted in advance. At the same time, the alarm can be triggered according to the priority, which further improves the stability and security of the system, and further improves the scheduling ability and huge risk avoidance ability of the system.

[0042] S3: Verify the correlation relationship in the knowledge graph based on the second historical unstructured data and the second historical structured data in the various types of log data, and obtain the verification result. Obtain the abnormal event prediction model based on the verification result.

[0043] The S3 includes: S31: Match the second historical unstructured data and the second historical structured data with the correlation relationship in the knowledge graph to predict the root cause of the fault occurrence; S32: If the number of the second historical unstructured data or the second historical structured data whose root cause of the fault occurrence matches the actual cause is greater than the preset number, obtain the abnormal event prediction model based on the knowledge graph. Otherwise, adjust the correlation relationship in the knowledge graph according to the second historical unstructured data and the second historical structured data whose root cause of the fault occurrence fails to match the actual cause to obtain the abnormal event prediction model.

[0044] In this embodiment, the preset number is flexibly set according to user requirements. If it is necessary to maximize the accuracy of the abnormal event prediction model, the preset number is set relatively large. The flexibility of the abnormal prediction is improved. When the number is less than the preset number, the data with failed matching is used to adjust the correlation relationship to obtain the abnormal event prediction model, which not only improves the accuracy of the abnormal event model but also improves the model acquisition efficiency.

[0045] The above specific implementation manners are the preferred implementation manners of the method for obtaining the abnormal event prediction model based on log analysis of the present invention, and do not limit the specific implementation scope of the present invention. The scope of the present invention includes but is not limited to this specific implementation manner. Any equivalent changes made according to the shape and structure of the present invention are within the protection scope of the present invention.

Claims

1. A method for obtaining an abnormal event prediction model based on log analysis, characterized in that, It includes the following steps: S1: Define corresponding rules according to the characteristics of the target system. Based on the NLP large model, extract the mutual relationships between process entities from the first historical unstructured data in various log data of the target system, and then obtain process triples; S2: Construct a knowledge graph based on the process triples, corresponding rules, and the first historical structured data in the various log data; S3: Verify the association relationships in the knowledge graph based on the second historical unstructured data and the second historical structured data in the various log data, and obtain the verification results. Obtain an abnormal event prediction model based on the verification results.

2. The method for obtaining an abnormal event prediction model based on log analysis according to claim 1, wherein, In S1, the defining of the corresponding rules according to the characteristics of the target system includes: Obtain keyword rules respectively according to the prompts indicating abnormal processes and normal processes in the target system; Set the priorities of the keyword rules based on the embedding principle, and obtain fault level rules based on the priorities.

3. The method for obtaining an abnormal event prediction model based on log analysis according to claim 1, wherein In S1, the extracting of the mutual relationships between process entities from the first historical unstructured data in the various log data of the target system based on the NLP large model, and then obtaining process triples includes: Define entity types and mutual relationship types based on the corresponding rules, and construct an identification model based on the NLP large model; Input the first historical unstructured data into the successfully trained identification model to obtain the mutual relationships between process entities, and then obtain initial triples; Obtain the semantic similarity of the initial triples based on word vectors, and update the initial triples based on the semantic similarity to obtain process triples.

4. The method for obtaining an abnormal event prediction model based on log analysis according to claim 3, wherein, The inputting of the first historical unstructured data into the successfully trained identification model to obtain the mutual relationships between process entities includes: Train the identification model based on the historical log data, entity types, and mutual relationship types in the target system. When both the entity type to which the training entity extracted from the historical log data belongs and the mutual relationship type to which the mutual relationship between the training entities belongs meet the preset conditions, it indicates that the identification model is successfully trained.

5. The method for obtaining an abnormal event prediction model based on log analysis according to claim 2, wherein The S2 includes: S21: The triples corresponding to the first historical structured data and the process triples form the final triples. Match the final triples with the keyword rules, and classify the successfully matched final triples as the first initial abnormal triples; S22: Classify the final triples with failed matches as initial normal triples; S23: Screen out the second initial abnormal triples from the initial normal triples based on the 3σ algorithm to obtain the final normal triples. The first initial abnormal triples and the second initial abnormal triples form abnormal triples; S24: Construct a knowledge graph based on the abnormal triples.

6. The method for obtaining an abnormal event prediction model based on log analysis according to claim 5, characterized in that, The S23 includes: S231: Obtain the mean, variance, and standard deviation of the initial normal data corresponding to the initial normal triples; S232: Determine the 3σ range with the mean as the benchmark and the standard deviation as the boundary; The initial normal triples corresponding to the initial normal data within the 3σ range are the second initial abnormal triples.

7. The method for obtaining an abnormal event prediction model based on log analysis according to claim 5, wherein The S24 includes: S241: Compare the abnormal entities in the abnormal triples, combine the successfully compared abnormal triples to obtain a highly correlated combination, mark the highly correlated combination as highly correlated, mark the abnormally compared triples as moderately correlated, and then obtain a correlation knowledge graph; S242: Match the final normal triples with the correlation knowledge graph to obtain a matching result, and update the correlation knowledge graph based on the matching result to obtain a knowledge graph.

8. The method for obtaining an abnormal event prediction model based on log analysis according to claim 7, wherein The S24 further includes: Associate the abnormal triples marked as highly correlated and moderately correlated in the correlation knowledge graph with the alarm phenomena based on the failure level rule, and then update the correlation knowledge graph.

9. The method for obtaining an abnormal event prediction model based on log analysis according to claim 1, wherein The S3 includes: S31: Match the second historical unstructured data and the second historical structured data with the association relationships in the knowledge graph to predict the root cause of the fault; S32: If the number of the second historical unstructured data or the second historical structured data whose root cause of the fault matches the actual cause is greater than the preset number, obtain an abnormal event prediction model based on the knowledge graph; otherwise, adjust the association relationships in the knowledge graph according to the second historical unstructured data and the second historical structured data whose root cause of the fault fails to match the actual cause to obtain an abnormal event prediction model.

10. The method for obtaining an abnormal event prediction model based on log analysis according to claim 1, wherein, In S1, the various types of log data at least include operation log data, system output log data, middleware log data, and interface log data.

Images