A cloud application fault location method and device based on knowledge analysis

Based on knowledge analysis methods, real -time obtaining and analyzing monitoring indicators of cloud applications solved the problem of slow and inaccurate fault positioning of cloud application failure, and achieved rapid and accurate fault positioning.

CN118740604BActive Publication Date: 2025-01-28NANJING UNIV OF FINANCE & ECONOMICS
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410966895.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2025-01-28
Estimated Expiration
2044-07-18

AI Technical Summary

Technical Problem

The fault location speed of cloud applications is slow and inaccurate, mainly due to the redundancy and dispersion of massive log data, and existing log analysis methods cannot effectively process large-scale data.

Method used

Adopt a method based on knowledge analysis, obtain monitoring indicators data of cloud applications in real time, detect abnormal monitoring indicators, determine the distribution sequence of the change point, obtain an abnormal time window, generate a fault representation vector, and match it to the preset vector fault library for positioning to position Failure position.

Benefits of technology

Realize fast and accurate fault positioning. Through real -time detection of abnormal monitoring indicators and changing points distributed sequences, accurately locate the time point of faults, reduce log failure time, and improve the efficiency and accuracy of fault positioning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118740604B_ABST
    Figure CN118740604B_ABST
Patent Text Reader

Abstract

The present invention discloses a cloud application fault location method and device based on knowledge analysis, which relate to the field of data processing technology. The method comprises the following steps: obtaining indicator data of a monitoring indicator, and for any monitoring indicator, if the indicator data of the monitoring indicator is not within a preset range, marking the monitoring indicator as an abnormal monitoring indicator; performing change point detection on the indicator data corresponding to the abnormal monitoring indicator in a time series to obtain a change point distribution sequence; determining an abnormal time window according to the change point distribution sequence; obtaining log information corresponding to the time window, generating a fault representation vector according to the log information, and determining a fault position according to the fault representation vector; determining the change point by real-time detection of the abnormal monitoring indicator can accurately locate the time point when the fault occurs, quickly determine the abnormal time window, reduce the time for log fault troubleshooting, and then match the fault representation vector with a vector in a preset vector fault library, thereby further improving the accuracy and efficiency of fault location.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and specifically relates to a cloud application fault locating method and device based on knowledge analysis. Background Art

[0002] With the development of society and the acceleration of digital transformation, cloud services have become the preferred solution for enterprises to build and deploy applications. Putting applications in the cloud brings many advantages, such as scalability, flexibility, high availability, data recovery, cross-platform, ease of use, security, privacy, cost-effectiveness, intelligence and diversification. However, with the rapid expansion of cloud services and the increasing complexity, fault location of cloud applications is also facing unprecedented challenges.

[0003] In cloud environments, application fault location usually relies on in-depth analysis of logs. Logs are important data that record various events and states during the operation of applications, and they provide valuable clues for fault location. However, as the scale of applications continues to grow, the amount of log data generated has exploded. These massive log data not only contain a lot of redundant and irrelevant information, but are also scattered across multiple different systems and components, making fault location extremely difficult.

[0004] The existing log analysis method analyzes all logs, which cannot meet the processing requirements of such large-scale data, resulting in slow and inaccurate fault location. Summary of the invention

[0005] The purpose of the present invention is to solve the problems of slow fault location and inaccurate location, and to propose a cloud application fault location method and device based on knowledge analysis.

[0006] In a first aspect of the present invention, a cloud application fault location method based on knowledge parsing is first proposed, and the method includes:

[0007] Obtain the indicator data of all monitoring indicators in cloud applications in real time to obtain the target data set;

[0008] For the target data set, for any monitoring indicator, if there is an indicator data of the monitoring indicator that is not within a preset range, the monitoring indicator is marked as an abnormal monitoring indicator;

[0009] For all abnormal monitoring indicators, for any abnormal monitoring indicator, change point detection is performed on the indicator data corresponding to the abnormal monitoring indicator in the time series to obtain a change point distribution sequence;

[0010] Determine the abnormal time window based on the distribution sequence of change points corresponding to all abnormal monitoring indicators;

[0011] Obtain log information corresponding to the time window, generate a fault representation vector according to the log information, and substitute the fault representation vector into a preset vector fault library to determine the fault location; the preset vector fault library stores the fault locations of all faults and the fault representation vector corresponding to each fault location.

[0012] Optionally, performing change point detection on the indicator data corresponding to the abnormal monitoring indicator in the time series to obtain a change point distribution sequence includes:

[0013] Performing data preprocessing on the indicator data corresponding to the abnormal monitoring indicator to obtain a preprocessed data set;

[0014] Substituting the preprocessed data set into the mean shift model, and evaluating each data in the preprocessed data set using the mean square error as the cost function to obtain a target evaluation set;

[0015] The target evaluation set is substituted into the hidden Markov model, and the change point distribution sequence is obtained using the Bayesian information criterion as a constraint condition.

[0016] Optionally, determining the abnormal time window according to the distribution sequence of change points corresponding to all abnormal monitoring indicators includes:

[0017] Obtaining the time index of each change point in the change point distribution sequence to obtain a time index set, and performing one-dimensional vector conversion on the indexes in the time index set to obtain a one-dimensional vector set;

[0018] Generate a probability density function corresponding to the one-dimensional vector set by a kernel density estimation method, and calculate a density estimation value of each change point in the one-dimensional vector set according to the probability density function to obtain a density estimation value set;

[0019] Determine the boundary change point by using the preset density value and the density estimation value set, and cut the time index set according to the time indexes corresponding to all the boundary change points to obtain a plurality of time index segments;

[0020] The time index segments in which the number of change points is greater than the preset number of changes are recorded as target segments, and the start time and end time of all target segments are obtained to obtain the abnormal time window.

[0021] Optionally, generating a fault representation vector according to the log information includes:

[0022] For the log information, perform entity extraction and event extraction on each piece of fault information, and establish an initial knowledge graph based on the extracted entities and events;

[0023] By combining different entities referring to the same entity in the initial knowledge graph through semantic analysis, a target knowledge graph is obtained;

[0024] The entities and relations in the target knowledge graph are embedded into the vector space, and a fault representation vector is generated through a preset algorithm.

[0025] Optionally, the preset algorithm for generating a fault representation vector includes:

[0026] Obtain the occurrence counts of all fault cases and all fault log templates in the historical data, and determine the reverse fault frequency of each fault log template according to the occurrence counts of all fault cases and all fault log templates;

[0027] For each fault case, calculate the frequency of all fault log templates appearing in the fault case, and obtain the probability set of each fault log template appearing in different fault cases;

[0028] determining a weight of each fault log template in each fault case according to a reverse fault frequency of each fault log template and a probability set of each fault log template;

[0029] Substitute all fault log templates into the knowledge graph embedding model to obtain the feature vector corresponding to each fault log template;

[0030] The fault representation vector of each fault case is determined according to the weight of each fault log template in each fault case and the feature vector corresponding to each fault log template.

[0031] In a second aspect of the present invention, a cloud application fault location device based on knowledge analysis is proposed, comprising:

[0032] The target data set generation module is used to obtain the indicator data of all monitoring indicators in the cloud application in real time to obtain the target data set;

[0033] An abnormal monitoring indicator determination module is used for marking any monitoring indicator as an abnormal monitoring indicator if the indicator data of the monitoring indicator is not within a preset range for the target data set;

[0034] A change point determination module is used to detect the change points of the indicator data corresponding to any abnormal monitoring indicator in the time series for all abnormal monitoring indicators to obtain a change point distribution sequence;

[0035] An abnormal time window determination module is used to determine the abnormal time window according to the distribution sequence of change points corresponding to all abnormal monitoring indicators;

[0036] A fault location determination module is used to obtain the log information corresponding to the time window, generate a fault representation vector according to the log information, and substitute the fault representation vector into a preset vector fault library to determine the fault location; the preset vector fault library stores the fault locations of all faults and the fault representation vector corresponding to each fault location.

[0037] Optionally, the change point determination module includes:

[0038] A data preprocessing module is used to perform data preprocessing on the indicator data corresponding to the abnormal monitoring indicator to obtain a preprocessed data set;

[0039] A data evaluation module, used for substituting the preprocessed data set into a mean shift model, and evaluating each data in the preprocessed data set using a mean square error as a cost function to obtain a target evaluation set;

[0040] The change point generation module is used to substitute the target evaluation set into the hidden Markov model and obtain the change point distribution sequence using the Bayesian information criterion as a constraint condition.

[0041] Optionally, the abnormal time window determination module includes:

[0042] A one-dimensional vector conversion module, used to obtain the time index of each change point in the change point distribution sequence to obtain a time index set, and perform one-dimensional vector conversion on the index in the time index set to obtain a one-dimensional vector set;

[0043] A density estimation module, used for generating a probability density function corresponding to the one-dimensional vector set by a kernel density estimation method, and calculating a density estimation value of each change point in the one-dimensional vector set according to the probability density function to obtain a density estimation value set;

[0044] A time index segment generating module, used to determine the boundary change point by using the preset density value and the density estimation value set, and to cut the time index set according to the time indexes corresponding to all the boundary change points to obtain a plurality of time index segments;

[0045] The abnormal time window generation module is used to record the time index segment in which the number of change points in the time index segment is greater than the preset transformation number as the target segment, and obtain the start time and end time of all target segments to obtain the abnormal time window.

[0046] Optionally, the fault location determination module includes:

[0047] An initial knowledge graph establishment module is used to perform entity extraction and event extraction on each piece of fault information according to the log information, and establish an initial knowledge graph based on the extracted entities and events;

[0048] A target knowledge graph generation module, used for merging different entities referring to the same entity in the initial knowledge graph through semantic analysis to obtain a target knowledge graph;

[0049] The fault representation vector generation module is used to embed the entities and relationships in the target knowledge graph into the vector space and generate a fault representation vector through a preset algorithm.

[0050] Optionally, the fault representation vector generation module includes:

[0051] A reverse fault frequency generation module is used to obtain the number of occurrences of all fault cases and all fault log templates in historical data, and determine the reverse fault frequency of each fault log template according to the number of occurrences of all fault cases and all fault log templates;

[0052] A fault log template occurrence probability determination module is used to calculate the frequency of occurrence of all fault log templates in each fault case, and obtain a probability set of each fault log template occurring in different fault cases;

[0053] A weight generation module, used for determining the weight of each fault log template in each fault case according to the reverse fault frequency of each fault log template and the probability set of each fault log template;

[0054] The feature vector generation module is used to substitute all fault log templates into the knowledge graph embedding model to obtain the feature vector corresponding to each fault log template;

[0055] The fault representation vector determination module is used to determine the fault representation vector of each fault case according to the weight of each fault log template in each fault case and the feature vector corresponding to each fault log template.

[0056] Beneficial effects of the present invention:

[0057] The present invention proposes a cloud application fault location method based on knowledge analysis, which obtains the target data set by real-time acquisition of indicator data for all monitoring indicators in the cloud application; for the target data set, for any monitoring indicator, if there is indicator data of the monitoring indicator that is not within a preset range, the monitoring indicator is marked as an abnormal monitoring indicator; for all abnormal monitoring indicators, for any abnormal monitoring indicator, change point detection is performed on the indicator data corresponding to the abnormal monitoring indicator in the time series to obtain a change point distribution sequence; the abnormal time window is determined according to the change point distribution sequence corresponding to all abnormal monitoring indicators; the log information corresponding to the time window is obtained, a fault representation vector is generated according to the log information, and the fault representation vector is substituted into a preset vector fault library to determine the fault location; by real-time detection of abnormal monitoring indicators to determine the change point, the time point when the fault occurs can be accurately located, the abnormal time window can be quickly determined, and the time for log fault troubleshooting can be reduced; and then the fault representation vector is matched with the vector in the preset vector fault library, thereby further improving the accuracy and efficiency of fault location. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The present invention will be further described below in conjunction with the accompanying drawings.

[0059] Figure 1 A flow chart of a cloud application fault location method based on knowledge parsing is provided for an embodiment of the present invention;

[0060] Figure 2 A structural schematic diagram of a cloud application fault locating device based on knowledge parsing is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the present invention, the description of "first", "second", etc. is only used for descriptive purposes, and cannot be understood as indicating or implying its relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0062] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0063] The embodiment of the present invention provides a cloud application fault location method based on knowledge analysis. Figure 1 , Figure 1 A flow chart of a cloud application fault location method based on knowledge analysis provided by an embodiment of the present invention. The method comprises the following steps:

[0064] S101, obtaining indicator data for all monitoring indicators in the cloud application in real time to obtain a target data set.

[0065] S102: For any monitoring indicator of the target data set, if the indicator data of the monitoring indicator is not within a preset range, the monitoring indicator is marked as an abnormal monitoring indicator.

[0066] S103, for all abnormal monitoring indicators, for any abnormal monitoring indicator, performing change point detection on the indicator data corresponding to the abnormal monitoring indicator in the time series to obtain a change point distribution sequence.

[0067] S104: Determine the abnormal time window according to the change point distribution sequence corresponding to all abnormal monitoring indicators.

[0068] S105, obtaining log information corresponding to the time window, generating a fault representation vector according to the log information, and substituting the fault representation vector into a preset vector fault library to determine the fault location.

[0069] The preset vector fault library stores the fault generation locations of all faults and the fault representation vector corresponding to each fault generation location.

[0070] A cloud application fault location method based on knowledge parsing provided in an embodiment of the present invention can accurately locate the time point when the fault occurs by determining the change point through real-time detection of abnormal monitoring indicators, quickly determine the abnormal time window, and reduce the time for log fault troubleshooting. The fault representation vector is then matched with the vector in the preset vector fault library, thereby further improving the accuracy and efficiency of fault location.

[0071] In one implementation, the monitoring indicators are one and / or more of CPU usage, memory usage, disk usage, network bandwidth, network latency, disk I / O performance, server load, process monitoring, security events, application performance, user requests, database performance, cache performance, error rate, etc.

[0072] In one implementation, the preset range of each monitoring indicator is determined by a technician.

[0073] In one implementation, by acquiring the indicator data of all monitoring indicators in real time, the system can capture abnormal conditions as soon as a fault occurs, which can greatly shorten the time for fault discovery and improve the efficiency of fault location.

[0074] In one implementation, a preset range is used to determine whether the monitoring indicator is abnormal, making the identification of abnormal indicators more accurate. At the same time, a change point detection algorithm is used to find the change point of the abnormal indicator in the time series, which helps to accurately locate the time point when the fault occurs and provides support for subsequent fault location.

[0075] In one embodiment, performing change point detection on the indicator data corresponding to the abnormal monitoring indicator in the time series to obtain a change point distribution sequence includes:

[0076] Performing data preprocessing on the indicator data corresponding to the abnormal monitoring indicator to obtain a preprocessed data set;

[0077] Substitute the preprocessed data set into the mean shift model, and use the mean square error as the cost function to evaluate each data in the preprocessed data set to obtain the target evaluation set;

[0078] The target evaluation set is substituted into the hidden Markov model, and the change point distribution sequence is obtained using the Bayesian information criterion as a constraint.

[0079] In one implementation, data preprocessing includes missing value processing, outlier processing, and data normalization.

[0080] In one implementation, the mean shift model can be used to cluster data points with similar characteristics into one category, providing a basis for subsequent data evaluation.

[0081] In one implementation, by combining the hidden Markov model and the Bayesian information criterion, the distribution sequence of change points of the target evaluation set can be quickly identified, further improving efficiency.

[0082] In one embodiment, determining the abnormal time window according to the distribution sequence of change points corresponding to all abnormal monitoring indicators includes:

[0083] Obtaining the time index of each change point in the change point distribution sequence to obtain a time index set, and performing one-dimensional vector conversion on the indexes in the time index set to obtain a one-dimensional vector set;

[0084] Generate a probability density function corresponding to a one-dimensional vector set by a kernel density estimation method, and calculate the density estimation value of each change point in the one-dimensional vector set according to the probability density function to obtain a density estimation value set;

[0085] Determine the boundary change point by presetting the density value and the density estimation value set, and cut the time index set according to the time indexes corresponding to all the boundary change points to obtain multiple time index segments;

[0086] The time index segments in which the number of change points is greater than the preset number of changes are recorded as target segments, and the start time and end time of all target segments are obtained to obtain the abnormal time window.

[0087] In one implementation, the preset density value and the preset number of changes are determined by a technician.

[0088] In one implementation, by determining the boundary change points and cutting the time index set based on these points, the time series can be clearly divided into multiple segments. Then, by identifying the segments with the number of change points exceeding the preset threshold as the target segments, the specific time period when the anomaly occurred can be directly located, providing a clear time range for subsequent log analysis.

[0089] In one implementation, the probability density function of a one-dimensional vector set is generated by a kernel density estimation method, and the density estimate of each change point is calculated based on this. This can more accurately quantify the significance and importance of the change point, and can better capture anomalies in complex data than simple threshold judgment.

[0090] In one implementation, by setting reasonable thresholds and cutting strategies, some non-substantial change points caused by noise or random fluctuations can be filtered out, thereby reducing the false alarm rate and improving the efficiency and accuracy of outlier detection.

[0091] In one embodiment, generating a fault representation vector according to log information includes:

[0092] For log information, perform entity extraction and event extraction on each fault information, and build an initial knowledge graph based on the extracted entities and events;

[0093] The target knowledge graph is obtained by merging different entities referring to the same entity in the initial knowledge graph through semantic analysis;

[0094] The entities and relationships between entities in the target knowledge graph are embedded into the vector space, and the fault representation vector is generated through a preset algorithm.

[0095] In one implementation, events may be error reports, warning messages, status changes, performance indicators, security events, transaction records, user behavior, system maintenance, configuration changes, hardware status, network activity, process and thread behavior, dependent service status, resource usage, audit logs, etc.

[0096] In one implementation, the entity can be a component name, service name, error code, timestamp, log level, operation name, user ID or session ID, IP address, request ID, metric name, status code, variable or parameter name, environment information, configuration information, etc.

[0097] In one implementation, the relationship between each entity is obtained through semantic analysis.

[0098] In one implementation, by extracting entities and events, key information in the log information can be structured to form an initial knowledge graph, thereby quickly retrieving and locating specific fault information, thereby improving the efficiency of information retrieval.

[0099] In one implementation, the initial knowledge graph may contain different expressions referring to the same entity, such as aliases, abbreviations, etc. Through semantic analysis, these different entities referring to the same entity can be merged to obtain a more accurate and concise target knowledge graph, which not only optimizes the representation of knowledge, but also improves the accuracy and consistency of the knowledge graph.

[0100] In one embodiment, the preset algorithm generates a fault representation vector including:

[0101] Obtain the occurrence counts of all fault cases and all fault log templates in the historical data, and determine the reverse fault frequency of each fault log template according to the occurrence counts of all fault cases and all fault log templates;

[0102] For each fault case, calculate the frequency of all fault log templates appearing in the fault case, and obtain the probability set of each fault log template appearing in different fault cases;

[0103] determining a weight of each fault log template in each fault case according to a reverse fault frequency of each fault log template and a probability set of each fault log template;

[0104] Substitute all fault log templates into the knowledge graph embedding model to obtain the feature vector corresponding to each fault log template;

[0105] The fault representation vector of each fault case is determined according to the weight of each fault log template in each fault case and the feature vector corresponding to each fault log template.

[0106] In one implementation, by calculating the probability set of the fault log template appearing in different fault cases, new fault logs can be more accurately classified into corresponding fault cases, thereby quickly identifying the fault type and improving the accuracy of fault location.

[0107] In one implementation, the formula Get the reverse fault frequency of each fault log template, where N is the total number of fault cases, n i is the number of occurrences of fault log template i, and w(i) is the reverse fault frequency of the i-th fault log template.

[0108] In one implementation, for each fault log template, the product of the reverse fault frequency of the fault log template and the probability of occurrence of the fault log template in the fault case is used as the weight of the fault log template in the fault case.

[0109] In one implementation, for each fault log template, the weight of the log template in the fault case is multiplied by the feature vector corresponding to each fault log template, and then the weighted vectors of all fault log templates are summed to obtain the fault representation vector of the fault case.

[0110] Based on the same inventive concept, the embodiment of the present invention also provides a cloud application fault location device based on knowledge analysis. Figure 2 , Figure 2 A schematic diagram of a cloud application fault location device based on knowledge analysis provided by an embodiment of the present invention includes:

[0111] The target data set generation module is used to obtain the indicator data of all monitoring indicators in the cloud application in real time to obtain the target data set;

[0112] The abnormal monitoring indicator determination module is used for marking any monitoring indicator as an abnormal monitoring indicator for a target data set if the indicator data of the monitoring indicator is not within a preset range;

[0113] A change point determination module is used to detect the change points of the indicator data corresponding to any abnormal monitoring indicator in the time series for all abnormal monitoring indicators to obtain a change point distribution sequence;

[0114] An abnormal time window determination module is used to determine the abnormal time window according to the distribution sequence of change points corresponding to all abnormal monitoring indicators;

[0115] The fault location determination module is used to obtain the log information corresponding to the time window, generate a fault representation vector based on the log information, and substitute the fault representation vector into a preset vector fault library to determine the fault location; the preset vector fault library stores the fault locations of all faults and the fault representation vector corresponding to each fault location.

[0116] A cloud application fault location device based on knowledge parsing provided by an embodiment of the present invention can accurately locate the time point when the fault occurs by determining the change point through real-time detection of abnormal monitoring indicators, quickly determine the abnormal time window, and reduce the time for log fault troubleshooting. The fault representation vector is then matched with the vector in the preset vector fault library, thereby further improving the accuracy and efficiency of fault location.

[0117] In one embodiment, the change point determination module includes:

[0118] A data preprocessing module is used to perform data preprocessing on the indicator data corresponding to the abnormal monitoring indicator to obtain a preprocessed data set;

[0119] A data evaluation module is used to substitute the preprocessed data set into the mean shift model, and evaluate each data in the preprocessed data set using the mean square error as the cost function to obtain a target evaluation set;

[0120] The change point generation module is used to substitute the target evaluation set into the hidden Markov model and obtain the change point distribution sequence using the Bayesian information criterion as a constraint condition.

[0121] In one embodiment, the abnormal time window determination module includes:

[0122] A one-dimensional vector conversion module is used to obtain the time index of each change point in the change point distribution sequence to obtain a time index set, and perform one-dimensional vector conversion on the index in the time index set to obtain a one-dimensional vector set;

[0123] A density estimation module is used to generate a probability density function corresponding to a one-dimensional vector set by a kernel density estimation method, and calculate a density estimation value of each change point in the one-dimensional vector set according to the probability density function to obtain a density estimation value set;

[0124] A time index segment generation module is used to determine the boundary change points by using a preset density value and a density estimation value set, and to cut the time index set according to the time indexes corresponding to all the boundary change points to obtain multiple time index segments;

[0125] The abnormal time window generation module is used to record the time index segment in which the number of change points in the time index segment is greater than the preset transformation number as the target segment, and obtain the start time and end time of all target segments to obtain the abnormal time window.

[0126] In one embodiment, the fault location determination module includes:

[0127] The initial knowledge graph establishment module is used to extract entities and events from each fault information in the log information, and establish the initial knowledge graph based on the extracted entities and events;

[0128] A target knowledge graph generation module is used to obtain a target knowledge graph by merging different entities referring to the same entity in the initial knowledge graph through semantic analysis;

[0129] The fault representation vector generation module is used to embed the entities and relationships in the target knowledge graph into the vector space and generate the fault representation vector through a preset algorithm.

[0130] In one embodiment, the fault representation vector generation module includes:

[0131] A reverse fault frequency generation module is used to obtain the number of occurrences of all fault cases and all fault log templates in historical data, and determine the reverse fault frequency of each fault log template according to the number of occurrences of all fault cases and all fault log templates;

[0132] A fault log template occurrence probability determination module is used to calculate the frequency of occurrence of all fault log templates in each fault case, and obtain a probability set of each fault log template occurring in different fault cases;

[0133] A weight generation module, used for determining the weight of each fault log template in each fault case according to the reverse fault frequency of each fault log template and the probability set of each fault log template;

[0134] The feature vector generation module is used to substitute all fault log templates into the knowledge graph embedding model to obtain the feature vector corresponding to each fault log template;

[0135] The fault representation vector determination module is used to determine the fault representation vector of each fault case according to the weight of each fault log template in each fault case and the feature vector corresponding to each fault log template.

[0136] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A cloud application fault location method based on knowledge analysis, characterized in that: The method comprises: Obtain the indicator data of all monitoring indicators in cloud applications in real time to obtain the target data set; For the target data set, for any monitoring indicator, if there is an indicator data of the monitoring indicator that is not within a preset range, the monitoring indicator is marked as an abnormal monitoring indicator; For all abnormal monitoring indicators, for any abnormal monitoring indicator, change point detection is performed on the indicator data corresponding to the abnormal monitoring indicator in the time series to obtain a change point distribution sequence; Determine the abnormal time window based on the distribution sequence of change points corresponding to all abnormal monitoring indicators; Obtaining log information corresponding to the time window, generating a fault representation vector according to the log information, and substituting the fault representation vector into a preset vector fault library to determine the fault location; The preset vector fault library stores the fault generation locations of all faults and the fault representation vector corresponding to each fault generation location; Generating a fault representation vector according to the log information includes: For the log information, perform entity extraction and event extraction on each piece of fault information, and establish an initial knowledge graph based on the extracted entities and events; By combining different entities referring to the same entity in the initial knowledge graph through semantic analysis, a target knowledge graph is obtained; Embed the entities and relations in the target knowledge graph into a vector space, and generate a fault representation vector through a preset algorithm; The preset algorithm generates the fault representation vector including: Obtain the occurrence counts of all fault cases and all fault log templates in the historical data, and determine the reverse fault frequency of each fault log template according to the occurrence counts of all fault cases and all fault log templates; For each fault case, calculate the frequency of all fault log templates appearing in the fault case, and obtain the probability set of each fault log template appearing in different fault cases; determining a weight of each fault log template in each fault case according to a reverse fault frequency of each fault log template and a probability set of each fault log template; Substitute all fault log templates into the knowledge graph embedding model to obtain the feature vector corresponding to each fault log template; Determine a fault representation vector for each fault case according to the weight of each fault log template in each fault case and the feature vector corresponding to each fault log template; By formula Get the reverse fault frequency of each fault log template, where N is the total number of fault cases, Fault log template The number of occurrences, For the Reverse fault frequency of each fault log template; For each fault log template, the product of the reverse fault frequency of the fault log template and the probability of occurrence of the fault log template in the fault case is used as the weight of the fault log template in the fault case; For each fault log template, the weight of the log template in the fault case is multiplied by the feature vector corresponding to each fault log template, and then the weighted vectors of all fault log templates are summed to obtain the fault representation vector of the fault case.

2. According to the cloud application fault location method based on knowledge analysis according to claim 1, it is characterized in that: The change point distribution sequence obtained by performing change point detection on the indicator data corresponding to the abnormal monitoring indicator in the time series includes: Performing data preprocessing on the indicator data corresponding to the abnormal monitoring indicator to obtain a preprocessed data set; Substituting the preprocessed data set into the mean shift model, and evaluating each data in the preprocessed data set using the mean square error as the cost function to obtain a target evaluation set; The target evaluation set is substituted into the hidden Markov model, and the change point distribution sequence is obtained using the Bayesian information criterion as a constraint condition.

3. According to the cloud application fault location method based on knowledge analysis according to claim 1, it is characterized in that: Determining the abnormal time window based on the distribution sequence of change points corresponding to all abnormal monitoring indicators includes: Obtaining the time index of each change point in the change point distribution sequence to obtain a time index set, and performing one-dimensional vector conversion on the indexes in the time index set to obtain a one-dimensional vector set; Generate a probability density function corresponding to the one-dimensional vector set by a kernel density estimation method, and calculate a density estimation value of each change point in the one-dimensional vector set according to the probability density function to obtain a density estimation value set; Determine the boundary change point by using the preset density value and the density estimation value set, and cut the time index set according to the time indexes corresponding to all the boundary change points to obtain a plurality of time index segments; The time index segments in which the number of change points is greater than the preset number of changes are recorded as target segments, and the start time and end time of all target segments are obtained to obtain the abnormal time window.

4. A cloud application fault location device based on knowledge analysis, characterized in that: The device comprises: The target data set generation module is used to obtain the indicator data of all monitoring indicators in the cloud application in real time to obtain the target data set; An abnormal monitoring indicator determination module is used for marking any monitoring indicator as an abnormal monitoring indicator if the indicator data of the monitoring indicator is not within a preset range for the target data set; A change point determination module is used to detect the change points of the indicator data corresponding to any abnormal monitoring indicator in the time series for all abnormal monitoring indicators to obtain a change point distribution sequence; An abnormal time window determination module is used to determine the abnormal time window according to the distribution sequence of change points corresponding to all abnormal monitoring indicators; A fault location determination module, used to obtain log information corresponding to the time window, generate a fault representation vector according to the log information, and substitute the fault representation vector into a preset vector fault library to determine the fault location; The preset vector fault library stores the fault generation locations of all faults and the fault representation vector corresponding to each fault generation location; The fault location determination module comprises: An initial knowledge graph establishment module is used to perform entity extraction and event extraction on each piece of fault information according to the log information, and establish an initial knowledge graph based on the extracted entities and events; A target knowledge graph generation module, used for merging different entities referring to the same entity in the initial knowledge graph through semantic analysis to obtain a target knowledge graph; A fault representation vector generation module is used to embed entities and relationships in the target knowledge graph into a vector space and generate a fault representation vector through a preset algorithm; The fault representation vector generation module comprises: A reverse fault frequency generation module is used to obtain the number of occurrences of all fault cases and all fault log templates in historical data, and determine the reverse fault frequency of each fault log template according to the number of occurrences of all fault cases and all fault log templates; A fault log template occurrence probability determination module is used to calculate the frequency of occurrence of all fault log templates in each fault case, and obtain a probability set of each fault log template occurring in different fault cases; A weight generation module, used for determining the weight of each fault log template in each fault case according to the reverse fault frequency of each fault log template and the probability set of each fault log template; The feature vector generation module is used to substitute all fault log templates into the knowledge graph embedding model to obtain the feature vector corresponding to each fault log template; A fault representation vector determination module, used to determine the fault representation vector of each fault case according to the weight of each fault log template in each fault case and the feature vector corresponding to each fault log template; By formula Get the reverse fault frequency of each fault log template, where N is the total number of fault cases, Fault log template The number of occurrences, For the Reverse fault frequency of each fault log template; For each fault log template, the product of the reverse fault frequency of the fault log template and the probability of occurrence of the fault log template in the fault case is used as the weight of the fault log template in the fault case; For each fault log template, the weight of the log template in the fault case is multiplied by the feature vector corresponding to each fault log template, and then the weighted vectors of all fault log templates are summed to obtain the fault representation vector of the fault case.

5. A cloud application fault location device based on knowledge analysis according to claim 4, characterized in that: The change point determination module comprises: A data preprocessing module is used to perform data preprocessing on the indicator data corresponding to the abnormal monitoring indicator to obtain a preprocessed data set; A data evaluation module, used for substituting the preprocessed data set into a mean shift model, and evaluating each data in the preprocessed data set using a mean square error as a cost function to obtain a target evaluation set; The change point generation module is used to substitute the target evaluation set into the hidden Markov model and obtain the change point distribution sequence using the Bayesian information criterion as a constraint condition.

6. A cloud application fault location device based on knowledge analysis according to claim 4, characterized in that: The abnormal time window determination module includes: A one-dimensional vector conversion module, used to obtain the time index of each change point in the change point distribution sequence to obtain a time index set, and perform one-dimensional vector conversion on the index in the time index set to obtain a one-dimensional vector set; A density estimation module, used for generating a probability density function corresponding to the one-dimensional vector set by a kernel density estimation method, and calculating a density estimation value of each change point in the one-dimensional vector set according to the probability density function to obtain a density estimation value set; A time index segment generating module, used to determine the boundary change point by using the preset density value and the density estimation value set, and to cut the time index set according to the time indexes corresponding to all the boundary change points to obtain a plurality of time index segments; The abnormal time window generation module is used to record the time index segment in which the number of change points in the time index segment is greater than the preset transformation number as the target segment, and obtain the start time and end time of all target segments to obtain the abnormal time window.

Citation Information

Patent Citations

  • Real-time log control system and method, cloud computing system and server

    CN108038049A

  • Fault root cause positioning method and device based on knowledge graph, equipment and medium

    CN114218403A

  • Micro-service fault detection method and device, storage medium and computer equipment

    CN115357418A